Impact Assessment and Monitoring and Evaluation of Research Outcomes

Across all regions in which CCAFS works, Regional Program Leaders and Theme Leaders have always worked with partners to select and measure key indicators that can be used to monitor and evaluate progress towards outcomes and impacts. A globally common set of appropriate baseline indicators on agricultural productivity, rural livelihoods, and biogeophysical attributes have been collected at selected study sites, so that monitoring and ex post impact assessment can be carried out. These global indicators are supplemented by regional and sub-regional indicators deemed appropriate by partners and Regional Program Leaders. Care has been given to ensuring that indicators capture cross-scale impacts. The performance of CCAFS will be measured against ten-year outcomes and three-year Intermediate performance indicators.
 
The CCAFS baseline has been implemented across three levels – household, village and organisations. It collects indicators that describe current behaviour in relation to livelihood systems and farming practices in the CCAFS sites over time, as well as changes made to agriculture and natural resources management strategies in the recent past. Other indicators are helping CCAFS to understand the enabling environment that mediates these practices and behaviours (e.g., natural resource conditions, policies, institutions), as well as the provision of agricultural and climatic information at each site by the organizations that work there.  
 
The key aim of the CCAFS baseline is to provide snapshots of current behaviour at the sites using instruments that can be applied unchanged in all the CCAFS regions. The same households and communities will be revisited after roughly 5 years, and again in 10 years, to monitor what changes have occurred since the baseline was carried out.  The same survey is being carried out in very diverse locations across all of our target regions. To date, close to 2,100 households have been surveyed in 105 villages; 15 communities in 15 CCAFS sites participated in qualitative focus group discussions and over 150 organizations have been interviewed at these sites. This allows for valid and robust cross-site and cross-regional comparisons to be carried out. As a result, baselines are broad rather than deep; the intention is that complex relationships will be explored in further research in the same locations and through the use of secondary data.
 
The emphasis on being able to carry out cross-site comparisons has two costs. First, the baselines do include some site characterisation information, but typically not in sufficient detail for (say) farming systems studies: more information is being collected to complement the site characterisation information in the baselines. Second, the baselines do not contain all the information needed to be able to carry out mainstream ex-post impact assessment (EPIA).  Such studies are usually designed to evaluate specific technological or policy changes in a location and to attribute the changes to specific activities carried out by specific agents.  The CCAFS baseline meets the first objective of impact assessment well (tracking change over time), but does not allow us to attribute these changes to specific activities. The goal is not to attribute these changes to the program, but to be able to assess what kinds of changes have occurred and whether these changes are helping households adapt to, and mitigate, climate change.
 
Nevertheless, formal EPIAs will be carried out at several of the CCAFS sties in the next 2-3 years. CCAFS and partners are contractually required to carry out EPIAs on a regular timeframe, and these will build on the baseline data already collected, complemented with additional site- and intervention-specific data as needed.