DHG 2017: Research priorities for international scholarship programmes

The response to my ‘World Café’ session on our book, International Scholarships in Higher Education, at the eighth Donor Harmonisation Group annual seminar was gratifying and stimulating. Two large and energetic groups participated in the sessions and many colleagues from agencies in Germany, the Netherlands, Norway, and Austria, to name a few, agreed to spread the word to colleagues who they felt would be very interested in our work.

The session had a dual purpose: to highlight that the book had been published, and to gather feedback on future research priorities to report back to colleagues in the academic and evaluation communities. From the two sessions, the major topics suggested were:

  • The economic impact of scholarships on the donor country economy – we did not delve into the detail in the World Café session, but this could be taken as either impacts during the scholarship, as a result of the programme investments, or long-term impacts on trade and business relationships (the latter was a recurring theme)
  • Impact on structures in society, rather than individual: how do you assess it? – Several chapters in our book address exactly this issue, because there is only a limited body of research that has drawn any convincing conclusions about organisational and institutional level impacts
  • Standalone scholarships or embedded in programmes: which is most effective and under what circumstances? – What are the benefits (and difficulties) of including academic scholarships, or other mobility arrangements, as actions within wider capacity building programmes?
  • What is the differential impact by modality of scholarship study, including lengths of study, local mobility (e.g. intra-regional) vs international mobility, and level of award (undergraduate, doctorate, etc.)
  • Effective strategies for successful outreach to target scholarships at the most relevant potential social change leaders, and not just those who have easier access to, and awareness of, application systems.
  • Similar to the last point, how to deal with the difficulties of documentation, including missing details from refugees (transcripts, schools certificates, etc), plagiarism in application forms, and forgery of documentation

A varied and interest list, with many familiar issues discussed at least over the last few years, in some cases much longer.

Unpicking research priorities

Documentation topics are largely administrative issues that require cooperation and knowledge sharing between implementing agencies, but not necessarily extensive research input. They are, however, pressing issues for the effectiveness of selection processes and thus have implications for impact more generally.

Other topics call for detailed comparative analysis if a meaningful answer is to be offered. The juxtaposition of standalone scholarships and embedded mobility was particularly prominent at the DHG conference this year. Several of the major European scholarship programmes have recently shifted away standalone scholarships to embedded mobility in wider capacity-building programmes. Nuffic, for instance, have administered the standalone Netherlands Fellowship Programme (NFP) for decades, but its 2017 replacement – the Orange Knowledge Programme – includes scholarships as part of a wider suite of actions, effectively combining the NFP scholarships with the NICHE capacity-building projects. Norway has taken a similar route in recent years, winding up the well known (and domestically popular) Quota Scheme scholarships in favour of mobility embedded within the wider NORHED programme.

Whether embedding scholarships will be a trend for more European donors in the future is not clear, but the fundamental research question – which approach works best, and in what circumstances? – cannot easily be answered by single-programme evaluations. That is also true of questions about the differential effectiveness of scholarship designs / modalities, the most widely discussed at DHG being about the type of mobility (intercontinental, regional, etc) and the length of study. Research on modalities is perhaps more straightforward than comparing programme impact for standalone and embedded mobility, but still relies on administrative or primary data that is largely not available within and between programmes. Clearly there is scope for collaboration in data collation, sharing, and cross-programme primary research.

Future research agenda

Attending conferences is always excellent for scoping out work underway. From this year’s DHG I was very pleased to hear about a study the University of Uppsala have nearly completed using long-term data on Swedish scholarship holders. The research was expected to be published at the end of 2017, with at least a summary – if not the entire report – in English. Two tracer studies of Swedish scholarship students are also currently underway. Likewise, the Danida Fellowship Centre is planning to launch a tracer study of Danish scholarship students, after a successful period of building administrative data on their alumni network. DAAD, who already invest heavily in monitoring and evaluation, including a variety of surveys instruments, have developed a new tracer survey with baseline, on-scholarship, and post-scholarship data collection points, similar in outlook to the CSC approach I helped design.

Beyond the new research from the agencies themselves, the research priorities discussed in my World Café session can offer some indication of the types of topics likely to be treated most seriously – and to have the most policy impact – among funders and administrators. We might not see a plethora of new projects launched based on this conference session, but existing or upcoming academic research could certainly improve its reach and relevance by considering whether any of these priority topics could be included. Feeding the topics back into the Scholarship Programme Research Network (SPRN) is the next step.

Book published – International scholarships in higher education

Autumn’s most exciting news has been Springer’s publication of our book, International scholarships in higher education: Pathways to social change, in hardback and e-book. Cover of book "International Scholarships in higher education"

The finished manuscript is the culmination of an enormous effort by 22 colleagues based all around the world. For the editors – Joan Dassin, Robin Marsh, and me – working on the text has been a stimulating experience and we have each made a multitude of connections with other researchers and specialists. The development of a rich network of connections between authors is one of the less visible impacts of working on a book project but can be one of the most profound: already collegiate relationships catalysed by participation in our edited text have generated new research collaborations and professional exchange.

Continue reading

DHG 2017 – Missive from Gatwick

On my way to the eighth Donor Harmonisation Group annual seminar – my fourth attendance – to catch up on higher education and international development projects and to talk about the new book, International Scholarships in Higher Education: Pathways to Social Change.

DHG is a forum of policymakers, funders, and administering agencies: the perfect place to ask some direct questions about how we can better bridge the work of the research community worldwide and the need for accessible, robust evidence in the scholarship design and implementation. At the top of the agenda for my ‘Word café’ session on the new book:

  • What are the current ‘hot topics’ in policymaking around scholarship programmes? How closely do they relate to topics in the book?
    What are the top research priorities from an administering agency and funder perspective?
  • And what kind of work is going to be most useful? – New primary research; More accessible syntheses of published work; Consultancy for donor-initiated projects (e.g. programme evaluations)? etc
  • How do we continue to make the links between administering agencies and academia, especially the detailed and extensive work of graduate students?
  • Is there genuine appetite to work with researchers and others to build partnerships and conduct research? If we created a list from today of staff who wanted to be contacted about participating, who would be on it?

I am particularly interested in that last point: is this an agenda that we can genuinely push with some home of progress? In the UK, the various departments of government have begun to set out ‘Areas of Research Interest’ with the intention of helping to guide academic research that aims at policy influence. I wonder how far it is possible to achieve this in the international scholarship space as well?

Launching the Scholarship Program Research Network

July 2017 saw the launch of the Scholarship Program Research Network (SPRN), an informal virtual community that aims to bring together individuals who study and evaluate scholarship programmes (both international and domestic).

The purpose of the SPRN is to connect individuals and organisations across sectoral and geographic divides, share resources, and facilitate opportunities for future exchange and collaboration.

Read the launch post on the ACU Measuring Success? Blog.

Join the SPRN LinkedIn Group.

View SPRN bibliography on Zotero or download a copy to your reference manager (or view as a MS Word doc) in our Google Drive folder.

ACU Measuring Success? Blog on longitudinal scholarship research

Re-post of my recent blog in the ACU’s Measuring Success? series. Original post here: ACU Measuring Success?

Tracking international scholarship outcomes: the CSC Longitudinal Research Framework

Matt Mawer, Rachel Day, and Shireen Quraishi (Commonwealth Scholarship Commission in the UK)

In stark contrast to only five years ago, much of the prominent new research on scholarship programme outcomes adopts a longitudinal model. Some examples have been posted within this blog series: see, for instance, essays by Anna Seifried and Cate Lawrence or Mirka Martel. A recent request for proposals from the MasterCard Foundation – architects of the Scholars Program – reinforces the point:

“The Scholars Program Longitudinal Cohort Study will be a central component of the Program’s learning partnership. It will seek to track post-Program Scholar outcomes and to understand the impact of Scholars on their families, communities, organizations, and societies.” (RFP 3, 4th March 2017)

The increased profile of longitudinal research within scholarship evaluation is a testament both to the difficulties presented by previous approaches and to the will of policymakers and administering agencies to grapple with those complexities and strengthen the evidence-base for grant making. Whether conducted within administering agencies or outsourced to expert consultants, contemporaneous research tracking scholarship alumni during and post-scholarship is fast becoming the ‘gold standard’ for assessing individual-level impacts.

Like others in this field, the Commonwealth Scholarship Commission in the UK (CSC) has now invested considerable time and energy into building an effective longitudinal study of mid-range scholarship outcomes (professional development, organisational change, etc.). Unlike most others in the field, however, the CSC Longitudinal Research Framework is designed to continue perpetually as a component of basic scholarship administration. We introduce the logic and methodology of the framework in our article. Continue reading

Article published: Approaches to analysing international scholarship programmes

To start the new year, the Journal of Studies in International Education has published my paper: “Approaches to Analyzing the Outcomes of International Scholarship Programs for Higher Education

The paper is based on research I conducted at the Commonwealth Scholarship Commission examining published literature on the outcomes of scholarship programmes and focuses on three issues: 1) the relationship between aims and outcomes, 2) difficulties with “attribution” and “contribution,” and 3) scholarship programs in comparison with their alternatives. You can also read the original report at the CSC website.

Continue reading

‘Measuring Success?’ blog – Editorial

The end of October 2016 marks the one-year anniversary of ‘Measuring Success?’: The Association of Commonwealth Universities’ blog series about research on scholarship outcomes. To celebrate a year of fascinating content from authors around the world, I have written a reflection with my co-editor Sian Julian:

‘Measuring Success?’ year one – The scene, the motifs, the actors

The blog series has been developed – or perhaps curated – without a strong editorial line. We have helped develop some themes, but have not guided the topics much beyond defining the basic parameter that posts had to be relevant to understanding the outcomes of scholarship programmes. Reflecting on the contributions across 2016, it is evident that the series has grown to be much more than simply a technical blog about methodology (if it even started as such!)

Measuring Success will continue into 2017: we already have a few pieces lined up and plan to start the new year with a mini-series of linked posts. As always, we are happy to receive ideas for content and volunteers to write for the blog: send me an email directly or use the email link on the blog landing page to get in touch.

Successes and complexities: the outcomes of UK Commonwealth Scholarships 1960-2012

In the Commonwealth Scholarship Commission’s latest report we present a detailed analysis of both the trajectories of individual Scholars through the labour market, and the catalytic impacts that have been generated by their activities. The report is available in abridged and full format from the Commonwealth Scholarship Commission website.

Since 2012, the Commonwealth Scholarship Commission in the UK has been conducting a cycle of evaluation surveys designed to generate insight into the experiences, achievements, and difficulties of our alumni. By mid-2015, we had sent a survey to 100% of the members of our alumni network for whom we had current contact details.

Almost 2,100 Scholars and Fellows responded to the survey, including participants from each scholarship programme operated by the CSC, residing in 84 countries, having studied over 100 academic disciplines, and having been hosted at over 300 UK institutions. The survey gathered responses from Scholars and Fellows who had held scholarships as far back as 1960 and in every subsequent year until 2012.

Findings from the survey are set out in the ‘Successes and complexities’ report, which extends and expands our previous interim report from late-2014.

Book commissioned on scholarship outcomes

Alongside my colleagues Prof Joan Dassin and Dr Robin Marsh, I am very pleased to be editing a new book which has been commissioned to be written across the course of 2016. The text will be called ‘International scholarships in higher education: Pathways for social change’ and will include contributions from authors based across the world and in organisations spanning government, academia, NGOs, and more.

An overview of the book contents is available here.

We are very excited to have been able to develop the edited collection from our meetings over the last 1-2 years with colleagues involved with international scholarships, in particular the workshop co-convened by Brandeis University and the Association of Commonwealth Universities last May in Boston.

Tracing scholarship outcomes at EAIE Glasgow 2015

The week of 15-18 September 2015 saw the 27th annual conference of the European Association for International Education (EAIE) held in Glasgow, UK. I joined a panel of colleagues from EP-Nuffic and DAAD to discuss ‘Tracing the outcomes of study abroad scholarships’ and the potential for deeper collaboration between agencies in future.

Evaluation timeline

A potted version of the paper is available here on the CSCUK website.