Dec 9, 2014

CGIAR opens agricultural data to the world using Amazon Web Services

We need to find ways to give useful insights and information for farmers to better manage climate variability. Photo: UNDP
Datasets will be accessible online to spur future ‘climate-smart’ innovations for food and farming

Lima, Peru (10 December 2014) — CGIAR, a consortium of international agricultural research centers, will make its wealth of data more accessible and available for addressing critical food security and development challenges using the Amazon Web Services (AWS) Cloud.

By making this data open for worldwide public access, it will help researchers to solve critical issues such as reducing rural poverty, improving human health and nutrition, and sustainably managing the Earth’s natural resources.

The first datasets to move into the cloud are Global Circulation Models (GCM), presently the most important tool for representing future climate conditions. This data is available to any researcher interested in understanding how the climate will change in the next 100 years. Moving to the cloud will enable CGIAR to develop applications which let non-experts access information about current and future climates, by browsing and processing data on user-friendly maps. The vision is for farmers to eventually access information directly.

“Farmers need to plan for the short and long term, and climate change throws a major wrench in the works,” said Andy Jarvis, a senior scientist with the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS), which contributed the “CCAFS-Climate” dataset.

“Ten years ago we struggled to find the data to understand climate change and its implications for agriculture,” said Jarvis. “Today, we have so much data that we struggle to make sense of it and more importantly harness its power to give farmers real advice on how to manage climate more effectively.” Putting this data in the AWS Cloud could be the key to unlocking innovation.

The datasets were initially opened to participants at a Hackathon event held over 24 hours on the sidelines of the United Nations climate change conference in Lima, Peru. Hosted by CCAFS, it attracted some of the best and brightest software developers and computer programmers from Latin America and the Caribbean.

The Hackathon challenged these developers to synthesize available datasets and turn them into useful insights and information for farmers to better manage climate variability.

The winning innovation tackled a crucial problem: helping farmers more accurately predict when to plant their crops each season. Farmers traditionally depend on almanacs for predicting seasonal planting dates, but climate change has made these predictions unreliable. The prototype, created by Colombian team Geomelodicos, combines data on historical production and climate trends, historical planting dates with current climate trends and short-term weather forecasts, to generate more accurate information about optimal planting dates for different crops and locations. This information could one day be disseminated via SMS messaging, effectively replacing the traditional almanac.

Another winning innovation, from the Peruvian team Viasoluciones, tackles with water scarcity, a challenge for farmers around the world. Called Illapa, after the Quechua goddess of water, the solution could help farmers make better decisions about how much water to use for irrigating different crops. The prototype application combines climate data and information from a tool that directly senses a plant’s water use, to calculate water needs in real-time. This could be a life-saver for farmers in times of drought.

“In 24 hours the Hackathon demonstrated the kind of innovation that is possible when this data is made available to developers and researchers,” said Jamie Kinney, Senior Manager for Scientific Computing at Amazon Web Services, Inc. “AWS is committed to helping the scientific research community access the data needed to solve global climate issues.”

The UN climate change conference highlighted that practical solutions are urgently needed. “While negotiators have spent the last two weeks trying to hammer out a new political deal on climate change, we have demonstrated how science can also play an important role in addressing the climate challenge,” said Ana Maria Loboguerrero Rodriguez, who leads the CCAFS Latin America research program, based out of the CGIAR’s International Center for Tropical Agriculture (CIAT) in Colombia.

“The data revolution is upon us, and it has the ability to transform the agricultural sector,” said Frank Rijsberman, CEO of the CGIAR Consortium. “CGIAR centers have decades of data, which we are moving quickly to open up. Whereas in the past you may have needed stand-alone software and powerful computers to work with our data, the cloud removes these barriers. The data is open, and the platform is quick and powerful. In short, the potential is huge,” he said.

“We’re ‘shifting to the cloud’ in CGIAR,” said Michael Marus, ICT Manager of the CGIAR Consortium. “The cloud is flexible, cost-effective and powerful, offering all of the data-crunching capacity and storage that researchers may need to turn raw data into actionable knowledge for farmers. Imagine the possibilities when users will no longer be limited by their own Internet connection speed, processing power, or storage capabilities.”

“This is just the first step towards making scientific information about agriculture more relevant in the fight against climate change, especially for those most in need,” said Rijsberman. “There are over 800 million undernourished people in the world today – and this tool helps us to ensure that the full breadth of existing knowledge can be deployed to help those most at risk.”

For more information on AWS Public Data Sets  visit the Public Data Set Catalog and to access the GCM dataset, visit ccafs-climate.org.

Read more on the AWS blog: Earth Science on AWS with new CGIAR and Landsat Public Data Sets


CGIAR is a global partnership that unites organizations engaged in research for a food-secure future. CGIAR research is dedicated to reducing rural poverty, increasing food security, improving human health and nutrition, and ensuring more sustainable management of natural resources. It is carried out by the 15 centers that are members of the CGIAR Consortium in close collaboration with hundreds of partner organizations, including national and regional research institutes, civil society organizations, academia, and the private sector.

The CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS) is a strategic partnership of CGIAR and Future Earth, led by the International Center for Tropical Agriculture (CIAT). CCAFS brings together the world’s best researchers in agricultural science, development research, climate science and earth system science, to identify and address the most important interactions, synergies and tradeoffs between climate change, agriculture and food security.