Skip to content

Pull requests: huggingface/lighteval

New pull request New
117 Open 672 Closed
117 Open 672 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

fix(mmlu_pro): correct answer letters in prompt and choices
#1269 opened Jun 19, 2026 by vineethsaivs Loading…
1
feat: add static linter command for custom tasks
#1268 opened Jun 19, 2026 by Kesav2k04 Loading…
1
Keep trailing context space for answer-only bits-per-byte
#1264 opened Jun 16, 2026 by kashif Contributor Loading…
2
feat: add ID-based few-shot example selection (#634)
#1263 opened Jun 16, 2026 by sohammmmm10 Loading…
Fix KeyError: 'default' for bigbench tasks with nonexistent split
#1262 opened Jun 14, 2026 by nuthalapativarun Loading…
1
Document unconditioned_query requirement for PMI normalization
#1261 opened Jun 14, 2026 by nuthalapativarun Loading…
1
Fix non-deterministic cache hash for MetricGrouping tasks
#1260 opened Jun 14, 2026 by nuthalapativarun Loading…
1
Sync LightevalTaskConfig docstring with current fields
#1259 opened Jun 14, 2026 by nuthalapativarun Loading…
1
Fix broken default_tasks.py link in quicktour docs
#1258 opened Jun 14, 2026 by nuthalapativarun Loading…
Fix LCB grader failing solutions that read sys.stdin.buffer
#1257 opened Jun 13, 2026 by vineethsaivs Loading…
1
fix(lcb): expose sys.stdin.buffer so binary-stdin solutions are graded correctly
#1256 opened Jun 13, 2026 by avalyset Loading…
1
Add AIME26 task
#1254 opened Jun 11, 2026 by Vedant-Agarwal Loading…
1
1
Enhance model loading and error handling
#1251 opened Jun 3, 2026 by nandeanie Loading…
1
docs: fix reference links
#1250 opened May 31, 2026 by charles-openclaw Loading…
1
Add optional juryeval integration for LLM-as-Judge metrics
#1249 opened May 31, 2026 by py-ai-dev Loading…
Fix flatten_dict crash/wrong key for bare numpy array values
#1247 opened May 28, 2026 by Kymi808 Loading…
chore: enable Dependabot weekly GitHub Actions bumps dependabot
#1246 opened May 26, 2026 by hf-dependantbot-rollout Bot Loading…
1
Add ArxivRollBench tasks
#1245 opened May 24, 2026 by liangzid Loading…
feat: implement loglikelihood and loglikelihood_rolling for LiteLLMClient (closes #1093)
#1244 opened May 21, 2026 by ALI-AL-MARJANI Loading…
Fix callable type hint in parallelism helper
#1239 opened May 20, 2026 by GoparapukethaN Loading…
fix: guard choices[0] and message=None before content access in llm_as_judge
#1238 opened May 17, 2026 by qizwiz Loading…
docs: fix custom model examples
#1237 opened May 15, 2026 by MukundaKatta Loading…
1
fix: prevent IndexError in Doc.get_golds() for out-of-bounds gold_index
#1236 opened May 13, 2026 by AmSach Loading…
1
fix typo
#1235 opened May 8, 2026 by fpetrakov Loading…
fix: transpose references before passing to sacrebleu in CorpusLevelTranslationMetric
#1234 opened May 8, 2026 by jaydenC88 Loading…
2 tasks done
1
ProTip! Add no:assignee to see everything that’s not assigned.