DEV Community

munimv2 profile picture

munimv2

404 bio not found

Joined Joined on  github website
Simplified Chinese in Traditional Chinese app translations: 1 in 4 apps ship it

Simplified Chinese in Traditional Chinese app translations: 1 in 4 apps ship it

Comments
3 min read

Want to connect with munimv2?

Create an account to connect with munimv2. You can also sign in below to proceed if you already have an account.

Already have an account? Sign in
45 of 45 published models side with the experts, not MMLU-Pro's answer key, on its inherited MMLU errors

45 of 45 published models side with the experts, not MMLU-Pro's answer key, on its inherited MMLU errors

Comments
4 min read
The code in AI programming hints that wasn't the student's: half of it ended up in their final submission

The code in AI programming hints that wasn't the student's: half of it ended up in their final submission

Comments
3 min read
About 30% of public PX4 drone logs have a GPS delay setting more than 50 ms off, and EKF2's own residuals agree

About 30% of public PX4 drone logs have a GPS delay setting more than 50 ms off, and EKF2's own residuals agree

Comments
8 min read
Once the token limit was raised, AI feedback changed English learners' words more often

Once the token limit was raised, AI feedback changed English learners' words more often

Comments
4 min read
MMLU-Pro still uses the MMLU answer keys that experts flagged as wrong on 52 of 55 questions

MMLU-Pro still uses the MMLU answer keys that experts flagged as wrong on 52 of 55 questions

Comments
4 min read
Russian letters in Ukrainian app translations: mostly not copy-paste

Russian letters in Ukrainian app translations: mostly not copy-paste

Comments
2 min read
I told two AI models to copy the student's words exactly. Their quotes did not get more exact

I told two AI models to copy the student's words exactly. Their quotes did not get more exact

Comments
2 min read
I ranked 5,054 DEV challenge entries before the winners were named. Here is how it went

I ranked 5,054 DEV challenge entries before the winners were named. Here is how it went

Comments
7 min read
XGBoost and LightGBM model files: what a CI gate should check before loading

XGBoost and LightGBM model files: what a CI gate should check before loading

Comments
2 min read
Global-MMLU's Chinese and Korean world-history rows came shifted from OpenAI's MMMLU

Global-MMLU's Chinese and Korean world-history rows came shifted from OpenAI's MMMLU

Comments
4 min read
My token limit hid a third of my data. Re-running 20 old pairs showed what was in it.

My token limit hid a third of my data. Re-running 20 old pairs showed what was in it.

Comments
3 min read
Eleven of Canada's open food-chemistry files checked against their reports: the totals match, the splits don't

Eleven of Canada's open food-chemistry files checked against their reports: the totals match, the splits don't

Comments
4 min read
Weblate didn't put them there: tracing 90 stray zero-width spaces in Android translations

Weblate didn't put them there: tracing 90 stray zero-width spaces in Android translations

1
Comments 1
3 min read
Kubernetes rarely leaves feature gates in beta, and its two records of GA releases agree 98% of the time

Kubernetes rarely leaves feature gates in beta, and its two records of GA releases agree 98% of the time

Comments
4 min read
When an MMLU answer key is wrong, "pick the longest answer" points to the experts' fix

When an MMLU answer key is wrong, "pick the longest answer" points to the experts' fix

Comments
3 min read
A Core ML model can name app code to run: what to check before you ship one

A Core ML model can name app code to run: what to check before you ship one

Comments
2 min read
On MedMCQA, "pick the longest answer" beats chance, and EXCEPT questions only blunt it

On MedMCQA, "pick the longest answer" beats chance, and EXCEPT questions only blunt it

Comments
4 min read
Once a Java JEP becomes a Candidate it almost always ships: a pre-registered census of 563 JEPs

Once a Java JEP becomes a Candidate it almost always ships: a pre-registered census of 563 JEPs

Comments
3 min read
I tested whether telling an AI the student is an English learner changes how it quotes them. My own rules said: not enough data.

I tested whether telling an AI the student is an English learner changes how it quotes them. My own rules said: not enough data.

Comments
3 min read
Can you take the bus to a hospital and be home the same day? 53,202 stops in Japan's open bus timetables

Can you take the bus to a hospital and be home the same day? 53,202 stops in Japan's open bus timetables

Comments
4 min read
AI feedback often bolds its corrections. My quote checker now reads Markdown

AI feedback often bolds its corrections. My quote checker now reads Markdown

Comments
3 min read
MMLU-Pro's "pick the longest answer" shortcut survives translation into all 29 MMLU-ProX languages

MMLU-Pro's "pick the longest answer" shortcut survives translation into all 29 MMLU-ProX languages

Comments
3 min read
MCP Inspector up to 1.0.2: two critical advisories that npm audit can't see yet

MCP Inspector up to 1.0.2: two critical advisories that npm audit can't see yet

Comments
4 min read
The doubled zero-width space in Android translations, and why it clusters after "EXIF"

The doubled zero-width space in Android translations, and why it clusters after "EXIF"

Comments
3 min read
Ten of 45 models lose to "pick the longest answer" on 656 MMLU-Pro questions

Ten of 45 models lose to "pick the longest answer" on 656 MMLU-Pro questions

Comments
4 min read
An ExecuTorch .pte has no pickle. Its risk is every size and offset

An ExecuTorch .pte has no pickle. Its risk is every size and offset

Comments
4 min read
W3C now publishes half as many Recommendations a year, and Python's Accepted-PEP backlog is almost all packaging

W3C now publishes half as many Recommendations a year, and Python's Accepted-PEP backlog is almost all packaging

Comments
4 min read
Models follow MMLU-Pro's longest-answer cue too, and three score below it

Models follow MMLU-Pro's longest-answer cue too, and three score below it

Comments
4 min read
In a .tflite file, the Flex op name is a label: read the NodeDef

In a .tflite file, the Flex op name is a label: read the NodeDef

Comments
3 min read
Picking the longest answer scores 67% on MMLU-Pro's STEM-website biology and psychology questions

Picking the longest answer scores 67% on MMLU-Pro's STEM-website biology and psychology questions

Comments
3 min read
Two ways to put numpy in MessagePack, and only one of them unpickles

Two ways to put numpy in MessagePack, and only one of them unpickles

Comments
3 min read
My quote filter hid three quarters of the quotes: checking AI feedback ranked 'more specific'

My quote filter hid three quarters of the quotes: checking AI feedback ranked 'more specific'

Comments
3 min read
Invisible characters in 2,392 Android apps' translations: Weblate's zero-width-space fix shows up in what ships

Invisible characters in 2,392 Android apps' translations: Weblate's zero-width-space fix shows up in what ships

Comments
3 min read
Canada's open baby-food data: a 1000x unit label, a 10x table slip, and lead vs the FDA's action levels

Canada's open baby-food data: a 1000x unit label, a 10x table slip, and lead vs the FDA's action levels

Comments
4 min read
Your model archive can carry a program: check members by their bytes

Your model archive can carry a program: check members by their bytes

Comments
3 min read
The IETF switched from replacing RFCs to patching them, but revises less overall: two pre-registered follow-ups

The IETF switched from replacing RFCs to patching them, but revises less overall: two pre-registered follow-ups

Comments
4 min read
MMLU-Pro fixed one answer shortcut and doubled another, and its new questions are why

MMLU-Pro fixed one answer shortcut and doubled another, and its new questions are why

Comments
4 min read
A SavedModel is a program: file ops your scanner may not list

A SavedModel is a program: file ops your scanner may not list

Comments
3 min read
A .nemo checkpoint is a tar file: two risks your model scanner may not open

A .nemo checkpoint is a tar file: two risks your model scanner may not open

Comments
3 min read
Pick the longest answer: 52.7% on MMLU international law, below chance on public relations

Pick the longest answer: 52.7% on MMLU international law, below chance on public relations

Comments 1
3 min read
Most RFCs are never replaced, and errata held for a document update mostly stay held: two pre-registered studies

Most RFCs are never replaced, and errata held for a document update mostly stay held: two pre-registered studies

Comments
3 min read
A compressed pickle is still a pickle: how joblib's compress= hides a payload from your model scanner

A compressed pickle is still a pickle: how joblib's compress= hides a payload from your model scanner

Comments
3 min read
MMLU's 'the answer is D' is an MMLU quirk: ARC, CommonsenseQA and HellaSwag don't share it

MMLU's 'the answer is D' is an MMLU quirk: ARC, CommonsenseQA and HellaSwag don't share it

Comments
3 min read
Six AI models graded student essays. Do they quote the student's words exactly? This time all three predictions held

Six AI models graded student essays. Do they quote the student's words exactly? This time all three predictions held

Comments
3 min read
What breaks when you reproduce Canada's open food-safety data

What breaks when you reproduce Canada's open food-safety data

Comments
4 min read
The PyTorch checkpoint container is itself a security signal: CVE-2025-32434 and the legacy load path

The PyTorch checkpoint container is itself a security signal: CVE-2025-32434 and the legacy load path

Comments
3 min read
EIA's September 2026 file still shows no U.S. coal unit retired in 2026

EIA's September 2026 file still shows no U.S. coal unit retired in 2026

Comments
1 min read
Keras 3's TorchModuleWrapper: one layer, two deserialization CVEs, and the static check most scanners skip

Keras 3's TorchModuleWrapper: one layer, two deserialization CVEs, and the static check most scanners skip

Comments
3 min read
"Always D" scores 46.7% on MMLU high-school statistics: where benchmark answers sit, by subject and by split

"Always D" scores 46.7% on MMLU high-school statistics: where benchmark answers sit, by subject and by split

Comments
3 min read
I pre-registered 28 guesses that popular PyPI packages have cleaner metadata. Only 9 held. Open dataset.

I pre-registered 28 guesses that popular PyPI packages have cleaner metadata. Only 9 held. Open dataset.

Comments
3 min read
Your self-hosted AI might be open to the whole network. Here's a 2-minute self-check.

Your self-hosted AI might be open to the whole network. Here's a 2-minute self-check.

Comments
3 min read
MMLU's "the answer is D" tilt is mostly an "all of the above" artifact

MMLU's "the answer is D" tilt is mostly an "all of the above" artifact

Comments
4 min read
When AI essay feedback misquotes a student, it is mostly in short quotes: both of my predictions reversed again

When AI essay feedback misquotes a student, it is mostly in short quotes: both of my predictions reversed again

Comments
3 min read
PyPI: 72% of popular packages' dependency lines are optional extras, yet they carry fewer required deps than the long tail

PyPI: 72% of popular packages' dependency lines are optional extras, yet they carry fewer required deps than the long tail

Comments
4 min read
Why does AI essay feedback change strong students' words more? Two causes, not one

Why does AI essay feedback change strong students' words more? Two causes, not one

Comments
3 min read
Why Japan's open bus timetables go dark on April 1: late files with nothing to fall back on

Why Japan's open bus timetables go dark on April 1: late files with nothing to fall back on

Comments
5 min read
Does OpenBookQA leak its answer? It's the longest option and the first — and neither shortcut generalizes

Does OpenBookQA leak its answer? It's the longest option and the first — and neither shortcut generalizes

Comments
3 min read
PyPI naming hygiene reversed: popular packages write dependency names more canonically than the long tail (4.8% vs 5.9%)

PyPI naming hygiene reversed: popular packages write dependency names more canonically than the long tail (4.8% vs 5.9%)

Comments
4 min read
All five predictions reversed: in AI essay feedback, short quotes and strong essays get the most changed quotes

All five predictions reversed: in AI essay feedback, short quotes and strong essays get the most changed quotes

Comments
3 min read
loading...