Comprehensive: Benchmarks - Ultimate Guide

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

munimv2

Nov 2

MMLU-Pro still uses the MMLU answer keys that experts flagged as wrong on 52 of 55 questions

#machinelearning #datascience #nlp #benchmarks

4 min read

munimv2

Nov 2

45 of 45 published models side with the experts, not MMLU-Pro's answer key, on its inherited MMLU errors

#machinelearning #datascience #nlp #benchmarks

4 min read

Max Quimby

Nov 2

Agent Swarms Still Can't Coordinate

#multiagent #aiagents #benchmarks #coordination

10 min read

munimv2

Nov 1

Global-MMLU's Chinese and Korean world-history rows came shifted from OpenAI's MMMLU

#machinelearning #datascience #nlp #benchmarks

4 min read

QuantID

Nov 1

Hugging Face Official Benchmarks: Daily Update 2026-11-02

#ai #benchmarks #huggingface #llm

4 min read

Tadna

Nov 1

OUKITEL C36 spec sheet: what the numbers say, and where they stop

#android #hardware #mobile #benchmarks

1 min read

QuantID

Oct 31

Hugging Face Official Benchmarks: Daily Update 2026-11-01

#ai #benchmarks #huggingface #llm

5 min read

munimv2

Oct 31

When an MMLU answer key is wrong, "pick the longest answer" points to the experts' fix

#machinelearning #datascience #nlp #benchmarks

3 min read

김민식/학생

Oct 31

NL2SQL: cómo leer un 83,80 en BIRD Mini-Dev con un evaluador de precisión de ejecución en Python

#ai #python #sql #benchmarks

9 min read

munimv2

Oct 31

On MedMCQA, "pick the longest answer" beats chance, and EXCEPT questions only blunt it

#machinelearning #datascience #nlp #benchmarks

4 min read

dbslice

Oct 30

How to Read a Database Sync Speed Number

#performance #benchmarks #datamigration #databasesync

2 min read

munimv2

Oct 30

MMLU-Pro's "pick the longest answer" shortcut survives translation into all 29 MMLU-ProX languages

#machinelearning #datascience #nlp #benchmarks

3 min read

munimv2

Oct 30

Ten of 45 models lose to "pick the longest answer" on 656 MMLU-Pro questions

#machinelearning #datascience #nlp #benchmarks

4 min read

munimv2

Oct 30

Models follow MMLU-Pro's longest-answer cue too, and three score below it

#machinelearning #datascience #nlp #benchmarks

4 min read

munimv2

Oct 30

Picking the longest answer scores 67% on MMLU-Pro's STEM-website biology and psychology questions

#machinelearning #datascience #nlp #benchmarks

3 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.