Real user conversations with chatbots, with pairwise human votes from the Arena.
Its licence doesn't allow Rubynox to redistribute it, so we never deliver it. Get it directly from LMSYS; we list it so you can see all your options in one place.
More in Language and AI benchmarks
Language and AI benchmarks · Stanford NLP
Questions about Wikipedia passages with answers marked in the text; v2 adds unanswerable questions.
Language and AI benchmarks · Tsinghua University (THUNLP)
Multi-turn synthetic dialogues used to fine-tune chat models.
Language and AI benchmarks · Google Research
Real Google search questions answered from Wikipedia pages.
Language and AI benchmarks · NYU, University of Washington and DeepMind
The general language understanding benchmark that shaped BERT-era models.
Language and AI benchmarks · NYU, Facebook AI, University of Washington and DeepMind
A harder successor to GLUE for reading comprehension and reasoning.
Language and AI benchmarks · Hendrycks et al. (UC Berkeley)
Multiple-choice exam questions from law to physics, the most cited LLM knowledge benchmark.