Skip to content

Align dashboard AUC with leaderboard methodology - #12

Open
mitre88 wants to merge 1 commit into
Infini-AI-Lab:mainfrom
mitre88:fix-dashboard-auc-methodology
Open

Align dashboard AUC with leaderboard methodology#12
mitre88 wants to merge 1 commit into
Infini-AI-Lab:mainfrom
mitre88:fix-dashboard-auc-methodology

Conversation

@mitre88

@mitre88 mitre88 commented May 22, 2026

Copy link
Copy Markdown

Summary

  • move AUC calculation into a small tested helper
  • calculate dashboard AUC as the paper-described left Riemann sum starting at op=2
  • stop accumulating once accuracy falls below 5% and use actual op spacing for non-unit strides
  • add focused regression tests for start-op, truncation, stride, and sorting behavior

Validation

  • python -m unittest discover -s tests -v
  • python -m py_compile gsm-infinite/metrics.py gsm-infinite/app.py tests/test_metrics.py
  • git diff --check

Context: issue #9 asks about the missing AUC truncation / op-stride behavior. This PR implements that dashboard-side behavior without touching the existing preprocess or answer-parsing PRs.

/claim https://algora.io/PrimeIntellect-ai/bounties/m4592Ap5jB1qm9yd

@mitre88

mitre88 commented May 22, 2026

Copy link
Copy Markdown
Author

/claim https://algora.io/PrimeIntellect-ai/bounties/m4592Ap5jB1qm9yd

Claiming after submitting the implemented and validated dashboard AUC fix. Validation is listed in the PR body.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant