Back to baselines: sharing lessons learned from measuring change

Monitoring research for development work requires a good baseline. CCAFS shares lessons learned from developing own baselines in new article. Photo: Sandeep Chetan
(view original)
Nov 26, 2014

by

Cecilia Schubert, Communications Officer (CCAFS)

Open communication, capacity building and establishing long-term partnerships are key lessons learned from CCAFS baseline work, shared in a newly released article.

As pressure on research for development programs to track and evaluate outcomes mounts, an increasing number of actors will have to find ways to implement baseline surveys as cost-effective and accurately as possible.

For organisations lacking previous experience with this type of work, there are a number of potential mishaps throughout the process that can affect the quality and timeliness of the results.

This is particularly true in complex programs where a baseline might need to cover very different social, economic, cultural and ecological contexts and involve a wide range of stakeholders.

The CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS) is now sharing lessons learned from developing an extensive baseline at household, village and organisational levels.

The article "Back to baselines: measuring change and sharing data" includes lessons learned on selecting useful indicators, building the capacity of surveyors and managing the collected data, and more that others can learn from and avoid making similar mistakes.

Setting up a baseline

It was decided very early on in the CCAFS program start up, that a baseline survey would be key in order to go back and track behavioural change of households, communities and organisations over time.

The plan is to revisit the research sites after roughly 5 years, and again in 10 years, to monitor what changes have occurred since the baselines were carried out.

Starting in South Asia, West Africa and East Africa, the program used the same baseline tools in 15 sites, selected because of their exposure and vulnerability to climatic changes.

For CCAFS sites, 2,095 households have been surveyed in 108 villages; 15 communities in 15 sites engaged in participatory community exercises; and over 150 organisations have been interviewed.

The work also assess gender differences by separately interviewing men and women.

Given the scale of the work, the baselines are broad rather than deep, with the intention hat complex relationships can be explored in further research in the same locations and through the use of secondary data.

The goal for the baseline work is not to attribute these changes to CCAFS. This as multiple partners and interventions are already underway in these sites. Instead ambition is to be able to assess what kinds of changes have occurred and whether these changes are helping households adapt to, and mitigate, climate change.

In seeking to catalyse new partnerships and strengthen existing ones, the ’contributionapproach, viewing CCAFS work as contributing to further development, is key, as attempts to attribute results to one particular actor or investment can be harmful to partnerships.

Harnessing support from local partners

The surveys were carried out by local partners such as national agricultural research services and universities, and in some sites by short-term consultants.

We strategically chose to implement baselines through local partners as their knowledge and understanding of the social and cultural contexts were critical. However, working with so many different teams, and in some sites with short-term consultants, proved to be quite the coordination challenge," says Wiebke Förch, lead author of the article and managing the baseline development.

Working with different types of actors to conduct the baselines proved to be a challenge as to gain similar results from the activities as well:
 

In South Asia the baselines were implemented by consultants, who in some cases had limited local knowledge of the specific sites and were not invested in becoming long-term research partners to CCAFS. This was also a factor affecting the analyses of the results,” she says. “This inevitably led to different models of baseline implementation, different capacities for site-level analysis and write up of reports.

One extra objective was to simultaneously build the capacity of local partners to carry out this type of work, which was a key activity in the process to secure good results.  

Baseline survey work by CCAFS was extensive and long. Lessons learned shared in new article.

Building the capacity of local surveyors in the baseline process is crucial to produce high-quality results. Photo: A.Eitzinger (CIAT)

Going broad, rather than deep

The goal for the baselines was to make cross-site comparisons, more so than getting a deeper understanding of the various sites and communities. Some partners didn't understood this particular objective, which proved to create some confusion among key stakeholders:

Open communication, both within the program itself and with partners, and being able to clearly articulate the trade-offs among depth, scope, and detail that inevitably take place, are critical and is something we learned through this process," says Wiebke Förch, also working as Science Officer for the Data and Tools team.

The baselines are now being implemented in CCAFS sites in Latin America and Southeast Asia. As a way to complete the surveys faster and more effectively, and enhance data quality, the teams are using something called the Open Data Kit.

The Open Data Kit enables users to create questionnaires online that are then downloaded to smart phones or tablets. This saves researcher a lot of time as they don’t have to transcribe questionnaires and answers, and data is safely stored and available to the whole team.

The efforts and time invested in the surveys have all been worth it, as through comparing the results, CCAFS will be able to better target project activities and evaluate efforts.

Key lessons learned while conducting baseline surveys for CCAFS

With open access to all data and materials — and more than 6,000 downloads from the project archive to date — other researchers and practitioners are already benefitting from this unique resource.


Explore resources:
- Download the article: Back to baselines: measuring change and sharing data. By Förch W, Kristjanson P, Cramer L, Barahona C, Thornton P 2014. Back to baselines: measuring change and sharing data. Agriculture & Food Security 2014, 3:13

- Download updated Baseline Working Paper, Global Summary of Baseline Household Survey Results. Version 2, Kristjanson P, Garlick C, Cramer L, Förch W, Thornton PK, Ngungu A. 2014. CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS). Copenhagen, Denmark.

- Learn more about how the baseline surveys were conducted
- Review all of our survey reports, resources and data on CCAFS Baseline page