Small-scale farmers are highly threatened by climate change. Experts often base their interventions to support farmers to adapt to climate change on their own perception of farmers’ livelihood risks. However, if differences in risk perception between farmers and experts exist, these interventions might fail. Thus, for effective design and implementation of adaptation strategies for farmers, it is necessary to understand farmers’ perception and how it influences their decision-making. We analyze farmers’ and experts’ systemic view on climate change threats in relation to other agricultural livelihood risks and assess the differences between their perceptions. For Cauca, Colombia, we found that experts and farmers perceived climate-related and other livelihood risks differently. While farmers’ perceived risks were a failure in crop production and lack of access to health and educational services, experts, in contrast, perceived insecurity and the unreliable weather to be the highest risks for farmers. On barriers that prevent farmers from taking action against risks, experts perceived both external factors such as the national policy and internal factors such as the adaptive capacity of farmers to be the main barriers. Farmers ranked the lack of information, especially about weather and climate, as their main barrier to adapt. Effective policies aiming at climate change adaptation need to relate climate change risks to other production risks as farmers often perceive climate change in the context of other risks. Policymakers in climate change need to consider differences in risk perception.