Big data, big prospects: crunching data for farmers' climate adaptation
CIAT big data scientists analyse many years of rice production and climate data to solve the declining rice production output puzzle in Colombia.
Big data has been and is being applied to a wide range of human endeavours. From predicting the outcome of legal cases in the US Supreme Court to precision marketing by tracking individual retail habits and making purchase suggestions based on recognized patterns of preferences; to allocating police/security resources through the ability to predict where and when crimes/uprisings are likely to take place, predicting disease and pest outbreak, to unmasking voters’ biases and predicting the outcome of elections – ask the Obama Campaign team.
So when the scientists on the CIAT big data team decided to put many years of rice production data – in combination with the climate/meteorological data for those years – under the magnifying lens of big data, they understood exactly what they needed to do and what results they wanted to achieve. The rice production statistics of Colombian farmers have been on the downward trend since 2007. And from the outset, the team recognized that identifying previously unseen patterns in existing data could be the key to solving the puzzle.
The team also understood the enormity of the task they had cut out for themselves. For one, most of the data needed to accomplish their task was not immediately at their disposal; they would have to retrieve the data from various agencies. Since open access and data sharing is still in its infancy in many places, it would take some process of trust-building coupled with a good dose of patience to allay the genuine fears of the primary data owners and seek their understanding of what the project intended to achieve.
After many months of hard work and data crunching, their tenacity and patience paid off; hitherto concealed patterns began to appear. For the Espinal rice growing area of the country, the results revealed the average minimum temperature during the ripening stage as the most relevant climatic factor in terms of yield.
From further observations, the team realized that the Cimarron Barinas rice variety mostly planted in this region is highly sensitive to minimum temperature at the ripening stage and displays a negative relationship with increase in minimum temperature – at the ripening stage – that shows an interesting breaking point around 22⁰C. Therefore, it is not suitable for hot regions.
In the Saldaña region, the level of accumulated solar energy in irrigated rice plants was found to have a positive relationship with – and the most important impact on – yield. Although this connection has been known by rice scientists for decades, the novelty of the team’s results in this area, is the discovery of the specific stage where the impact of solar radiation on yield is most significant – that is during the ripening stage of the rice plants.
Exploring existing data further with big data analytics, the team found many other interesting details; from clustering climate sequences to reveal the most adapted varieties, to identifying the yield distribution of each variety in the different clusters. The team also built on this foundation to predict the likely events for the production period of June – August 2014 in the Villavicencio region and to make recommendations on the best varieties for adaptation.
The import of these results is multifarious and can be profound. For one, it will help farmers when making decisions that foster their adaptation to climate variability e.g. in adjusting the planting date for the ripening stage to coincide with the time of maximum solar energy accumulation in Saldaña, or adjusting the planting date to avoid the high temperatures during ripening in Espinal.
Also, it will accelerate agricultural research, in terms of time taken and money spent, by providing valuable feedback to breeders on the actual behaviour of plant materials in the field. It will likewise help identify the best inputs for a breeding program for new cultivars. Overall, it will significantly cut back the number of years required to breed new improved varieties and aid better climate adaptation of Colombian rice growers.
Already, farmers are beginning to reap the benefits of this research. In the north coast (Caribbean) rice growing areas, about 170 farmers planting between them 1800Ha of irrigated rice were able to avoid economic losses from following recommendations based on the seasonal forecast, generated by the principles of this big data analyses. As compared to their peers, who either did not have the information or followed the conventional way.
For this immediate impacts of the project and its many prospects for climate adaptation and yield impact for Colombian rice growers, the project recently won the UN Pulse Big Data Climate Challenge. Following a month long evaluation, the Advisory Board and Technical Committee of the competition selected the CIAT team as one of the two winners based on the unique innovation of the “project that uses big data to drive climate action around the world.”
And indeed, going forward, the CIAT team intends to embed soil and management variables into their model in order to increase its explanation and pattern-recognizing powers and to extend its usage to scientists and farmers in other countries. Currently, they are working on scaling up the initiative, starting in 2015, using a similar approach with rice growers in Peru and Nicaragua through a partnership with the Latin American Fund for Irrigated Rice (FLAR).
But in the meantime, as they crunch and reduce mountains of many years of data into a refreshing and increasingly useful data-stream, their results will make Colombian farmers more adaptive to climate variability and more competitive as the country enters into free trade agreements – with the US and other countries – in a progressively globalized world with its liberalized markets.