Top 12 Trending LLM Leaderboards: A Guide to Leading AI Models’ Evaluation
Here is a list of top 12 Trending LLM Leaderboards: A Guide to Leading AI Models’ Evaluation
Open LLM Leaderboard
With numerous LLMs and chatbots emerging weekly, it’s challenging to discern genuine advancements from hype. The Open LLM Leaderboard addresses this by using the Eleuther AI-Language Model Evaluation Harness to benchmark models across six tasks: AI2 Reasoning Challenge, HellaSwag, MMLU, TruthfulQA, Winogrande, and GSM8k. These benchmarks test various reasoning and general knowledge skills. Detailed numerical results and model specifics are available on Hugging Face.
[Dated: 02 June, 2024]
MTEB Leaderboard
Text embeddings are often evaluated on a limited set of datasets from a single task, failing to account for their applicability to other tasks like clustering or reranking. The Massive Text Embedding Benchmark (MTEB) addresses this issue by spanning eight embedding tasks across 58 datasets and 112 languages. Benchmarking 33 models, MTEB offers the most extensive evaluation of text embeddings.
[Dated: 02 June, 2024]
Big Code Models Leaderboard
Inspired by the 🤗 Open LLM Leaderboard, this leaderboard compares multilingual code generation models on the HumanEval and MultiPL-E benchmarks. HumanEval measures functional correctness with 164 Python problems, while MultiPL-E translates these problems into 18 languages.
[Dated: 02 June, 2024]
SEAL Leaderboards
The SEAL Leaderboards utilize Elo-scale rankings to compare model performance across datasets. Human evaluators rate model responses to prompts, with ratings determining which model wins, loses, or ties.
[Dated: 02 June, 2024]
Berkeley Function-Calling Leaderboard
The Berkeley Function-Calling Leaderboard (BFCL) evaluates LLMs on their ability to call functions and tools, a critical capability for powering applications like Langchain and AutoGPT.
[Dated: 02 June, 2024]
Occiglot Euro LLM Leaderboard
This is a copy of the Open LLM Leaderboard from Hugging Face with the extension of the translated benchmarks.
[Dated: 02 June, 2024]
LMSYS Chatbot Arena Leaderboard
LMSYS Chatbot Arena is a crowdsourced open platform for evaluating LLMs. With over 1,000,000 human pairwise comparisons, models are ranked using the Bradley-Terry model and displayed in Elo-scale.
[Dated: 02 June, 2024]
Artificial Analysis LLM Performance Leaderboard
Artificial Analysis benchmarks LLMs on serverless API endpoints, measuring quality and performance from a customer perspective.
[Dated: 02 June, 2024]
Open Medical LLM Leaderboard
The Open Medical LLM Leaderboard tracks, ranks, and evaluates LLMs on medical question-answering tasks. It assesses models using diverse medical datasets, covering medical aspects like clinical knowledge, anatomy, and genetics.
[Dated: 02 June, 2024]
Hughes Hallucination Evaluation Model (HHEM) Leaderboard
The Hughes Hallucination Evaluation Model (HHEM) Leaderboard evaluates the frequency of hallucinations in document summaries generated by LLMs.
[Dated: 02 June, 2024]
OpenVLM Leaderboard
This platform presents evaluation results of 63 Vision-Language Models (VLMs) using the OpenSource Framework VLMEvalKit.
[Dated: 02 June, 2024]
🤗 LLM-Perf Leaderboard 🏋️
The 🤗 LLM-Perf Leaderboard benchmarks LLMs in latency, throughput, memory, and energy consumption across various hardware and optimizations using Optimum-Benchmark.
[Dated: 02 June, 2024]