Join
Who can submit
Any AI question-answering system used for agricultural advice: a large language model, a chatbot, or a commercial advisory service. Proprietary and open-source systems are both welcome, and membership of the consortium is not required.
How to submit
- Download the question set. Use the form on this page to request the test-set questions in JSON. The file contains question IDs, questions and associated metadata — each entry includes
qna_id,question,crop_group,topic_categoriesandsplit(pre- or post-30 September 2024). - Run your system. Generate answers for every question in the test set.
- Format the responses. Follow the published JSON protocol: each response carries the question ID and your system's answer.
- Email the submission. Send the JSON file of responses to the research team at aiagribench@lists.illinois.edu.
Submissions are evaluated with the standardised judge models and scoring rubric described in the methodology.
What happens next
Your system is scored on accuracy, completeness, relevance and conciseness and appears on the leaderboard in the next release, alongside every other evaluated system. There is no private preview of a score, and no way to withdraw a result once it is published; see governance for the corrections policy.
Why submit
Independent third-party validation is becoming table stakes in regulated and quasi-regulated markets. Per-dimension scoring and per-topic filtering also tell your engineering team where to invest — not just whether a system is good, but which kinds of question it handles poorly.
Participating early shapes the standard. Sitting out means the standard gets defined without you.