DELTAS Africa Learning Research Programme: Learning Report No.3 (Apr 2018 – Mar 2019)

Abstract

Background Development partners and research councils are increasingly investing in RCS initiatives in LMICs. There are few reported evaluations of RCS initiatives and no agreed evaluation metrics despite the large sums of public money invested. This paper aims to advance progress towards the establishment of a standardised set of outcome and impact indicators for use across RCS initiatives in LMIC contexts. Method A review of RCS indicators described in the published and grey literatures was undertaken. A systematic search in PubMed, Global Health, CINAHL Complete, IBSS, Google Advanced, BASE, Grey Literature and OpenGrey was completed along with a manual search for papers using reference checking and citation searching. RCS indicator descriptions were extracted from each publication/report, recorded in an Excel spreadsheet and sorted according to type (output, outcome or impact). All outcome and impact indicators were quality appraised and thematically analysed. Results We identified a total of 668 RCS indicators from across 32 publications/reports. Of these, 40% (265/668) were output indicators, 59.5% (400/668) outcome indicators and 0.5% (3/668) impact indicators. Thirty-four percent (225/668) measured individual research capacity, 38% (265/668) institutional research capacity and 21% (178/668) systemic research capacity. The 400 outcome indicators were coded into nine thematic categories, the most common of which were ‘research management and support’ (n=97), ‘skills/knowledge’ (n=62) and ‘collaboration activities’ (n=53). The three impact indicators were all systemic-level indicators and were all coded to a ‘health and wellbeing’ theme. Only 1% (6/403) of outcome and impact indicators met all four quality criteria. Conclusion Numerous RCS outcome indicators are present in the public and grey literature, although across a relatively limited range suggesting common interest in key focal areas. Very few impact indicators have been described and the quality of commonly described indicators, both outcome and impact, is poor.