Smallholders

What this leaderboard measures

The Smallholders track evaluates AI advisory systems against real questions asked by smallholder farmers, drawn from more than 3,100 farmer–expert question-and-answer pairs contributed by Digital Green, GAIA/IFPRI and Precision Development.

These scores are not comparable to the main track. The two tracks use different datasets and different sources. A model that scores well on technical agronomy questions may score considerably lower here — that difference is the finding, not an error.

Results

Why this track exists

Frontier models score 94–98% on technical agronomy questions in v0.5 and 65–81% on real smallholder farmer questions. Current systems handle textbook agronomy well and struggle with the practical, contextual questions farmers actually ask. For the more than 500 million farmers worldwide who may come to rely on these tools, that gap is the thing worth measuring.

Current scope and limits

Evaluation is text-based and in English. Support for Hindi, Odia, Amharic and audio-based question answering is on the roadmap, not in production. Stating this scope is required whenever these results are cited.

Download the data

• CSV: /data/smallholders-latest.csv
• JSON: /data/smallholders-latest.json