Evaluation methodology study published

‘A study of research methodology used in evaluations of international scholarship schemes for higher education’

Over the last six months we have been working on a scoping study examining the state of research practice in the sector: trends, ambiguities, and omissions. The aim has been both to inform our own evaluation practice and to update our colleagues around the world on how evaluation is being conducted.

The final report is now available on the CSCUK website.

The conclusions in brief:

  1. The majority of evaluation is ex-post: it traces alumni sometime after their scholarships.
  2. Methodology – separate from methods – is not discussed very much within the sector.
  3. Surveys are the dominant tools for data collection. Interviewing in-person is also common.
  4. Almost all evaluation is concerned with similar issues, such as completion rates, gains in knowledge and skills, return to home country, and employment trajectory post-scholarship.
  5. The practices and standards of data analysis are not always clear in evaluation, leaving ambiguities about how data is treated within the sector.
  6. Discussion of harmonisation, and perhaps harmonisation itself, is very limited and this in turn limits useful analyses of synergy and interference.

Although they focus on different elements of scholarship implementation (and, indeed, different types of scholarship), it might be useful to read the report in conjunction with the recent papers by Perna et al. (2014) deriving a typology for government scholarship schemes and the British Council and DAAD (2014) on outbound scholarship funding across a range of donor countries.