Frequently Asked Questions
What is AI AgriBench?
AI AgriBench is a public consortium led by the Center for Digital Agriculture at the University of Illinois Urbana-Champaign that develops expert-grounded benchmarks for evaluating AI question-answering systems used in production agriculture. Results are published on a public leaderboard at aiagribench.org.
Who runs AI AgriBench?
The Center for Digital Agriculture at the University of Illinois Urbana-Champaign leads the consortium. It spun out of the CropWizard project within AIFARMS, the National AI Institute for Food and Agriculture. A Steering Committee of academic, industry and nonprofit representatives governs it.
Why does an independent benchmark matter?
More than twenty commercial and open-source AI advisory chatbots are deployed in agriculture, plus general-purpose assistants farmers use anyway. Buyers have no standardised way to compare them and vendors have an incentive to self-report favourably. There is no Consumer Reports for agricultural AI.
How are models scored?
Every response is scored from 0 to 100 on four dimensions — accuracy, completeness, relevance and conciseness — using a calibrated LLM-as-judge framework with human expert validation. No model is ever used to judge itself.
Where do the questions come from?
The main track is built from more than 40,000 extension publications and research papers from US land-grant universities. The Smallholders track uses more than 3,100 real farmer–expert question-and-answer pairs from Digital Green, GAIA/IFPRI and Precision Development.
How many questions are in the benchmark?
45,172 synthetic question–answer pairs were generated, 951 went through expert review, and 416 were retained in the final v0.5 benchmark. Twenty-three agronomists conducted the review.
Why are there two leaderboards?
The main track measures technical agronomic understanding in a US, English-language context. The Smallholders track measures performance on real questions from smallholder farmers in India and Africa. They use different datasets and different sources, so scores are not comparable across the two.
What does the asterisk next to a score mean?
It marks results on the post-cutoff subset: questions drawn from documents published after 30 September 2024, used to check whether a model's performance depends on having seen the source material during pretraining.
What does v0.5 not cover?
Field-level decisions such as seed or hybrid selection, input product choices, and recommendations based on farm-specific data. Image-based question answering, data-driven recommendations and open-ended reasoning are planned for future versions.
Can I submit my model?
Yes. Proprietary and open-source systems are both welcome. See submit an AI system for evaluation.
Is the data available?
Yes. Each release publishes a CSV and a JSON at permanent URLs, linked from the leaderboard page and advertised in the page's structured data.
How often does the leaderboard update?
TODO (Committee): the update cadence has not been fixed. This is a communications decision as much as a technical one, because press cycles depend on it.
Does AI AgriBench certify or endorse AI tools?
No. AI AgriBench publishes scores against a defined benchmark. It does not certify, endorse or recommend any product. Certification is a long-term item on the roadmap, not something the consortium does today.
Are member companies scored differently?
No. Member companies' scores are reported exactly as non-member companies' scores are. Neutrality is the basis of the consortium's credibility.
Who funds AI AgriBench?
Funders to date are AIFARMS, the Center for Digital Agriculture, and the Gates Foundation. See governance for the conflict-of-interest policy.
How do I stay informed?
Join the mailing list at go.illinois.edu/aiagribench-form, or follow news.