All pull requests

Pull requests

• Update eval_instruct.py

  #156·neginraoof opened Feb 24, 2026Collaborator·

  ·
  1

  2

• Improved logging to support lm-eval>=0.4.8

  #154·hjenryin opened Jan 4, 2026·

  ·
  4

• Fix vLLM crash when max_gen_toks exceeds model context window

  #153·Ali-Elganzory opened Jan 3, 2026·4 tasks done·

  ·
  1

  5

• [FEATURE] Allow command-line control of namespace when using launch_simple

  #151·penfever opened Dec 24, 2025Contributor·

  ·

• Penfever patch 1

  #138·penfever opened Jul 6, 2025Contributor·

  ·

• Register OpenLM models at startup

  #133·reinhardh opened Jun 27, 2025Collaborator·

  ·

• LCB official scorer instead of skythoughts

  #130·slimfrkha opened Jun 13, 2025·

  ·
  1

• Optimize Evaluation Workflow for Better Batching and Model Reuse For benchmarks with n_repeat > 1

  #125·ihebchaa opened May 27, 2025·

  ·
  3

• update lm-eval-harness version

  #124·jannalulu opened May 12, 2025·

  ·
  1

• Fix: truncate model identifier in case model name is too long

  #120·younesbelkada opened May 2, 2025·

  ·
  1

• Jean/update curator

  #115·jmercat opened Apr 15, 2025Collaborator·

  ·

• Support for Big Bench Extra Hard (General-purpose reasoning eval)

  #92·Hritikbansal opened Mar 8, 2025Collaborator·

  ·

• Adding Arena Hard Auto

  #65·asuvarna31 opened Jan 28, 2025·

  ·
  4

• Multi-node

  #29·jmercat opened Nov 22, 2024Collaborator·

  ·
  2