All pull requests

Pull requests

• [feat][combination]AISBench支持多组合条件下批量用例的性能测试

  large_prno_issueno_review
  #633·yangcheng001 opened Oct 31, 2026·15 tasks·

  ·
  1

• [bugfix][api_models] guard empty choices in VLLMCustomAPI.get_prompt_logprobs

  no_review
  #622·lsyx6399 opened Oct 28, 2026·4 tasks done·

  ·
  1

  2

• [bugfix][api_models] raise on non-positive max_out_len instead of returning empty body

  no_review
  #620·lsyx6399 opened Oct 28, 2026·4 tasks done·

  ·
  1

  2

• [bugfix][ocrbench_v2] remove ipdb debug residue that hangs evaluation

  no_review
  #618·lsyx6399 opened Oct 28, 2026·3 tasks done·

  ·
  1

  2

• [bugfix] Parse token usage for TGI/Triton/MindIE backends

  bugfixno_review
  #615·lsyx6399 opened Oct 28, 2026·1 task done·

  ·
  1

  7

• [Bugfix] Skip host_ip/host_port validation when a custom url is given

  bugfixno_review
  #614·lsyx6399 opened Oct 28, 2026·1 task done·

  ·
  1

  3

• [docs] add contribution guide

  docsno_issuereview_opened
  #594·Libotry opened Oct 13, 2026Collaborator·3 of 15 tasks·

  ·
  32

• fix(livecodebench): support sys.stdout.buffer in output capture

  no_review
  #581·mantuoluozk opened Sep 29, 2026·

  ·
  1

  4

• fix(livecodebench): judge five v6 tasks by problem rules

  no_review
  #578·mantuoluozk opened Sep 29, 2026·

  ·
  1

  14

• add agent doc

  no_issueno_review
  #575·jschen069 opened Sep 24, 2026Collaborator·15 tasks·

  ·
  1

• fix: avoid unnecessary tokenizer loading in perf summary

  no_issueno_review
  #574·jschen069 opened Sep 23, 2026Collaborator·15 tasks·

  ·
  5

• [fix][datasets] Support \boxed{X} answer extraction in GPQA_Simple_Eval_postprocess

  bugfix
  #546·tunglambk opened Sep 15, 2026·4 of 15 tasks·

  ·
  1

  7

• Escape backslash in HLE judge prompt config

  no_issueno_review
  #543·MBK-fr opened Sep 14, 2026·

  ·
  1

• fix(livecodebench): expose .buffer on mocked stdin/stdout

  no_issuereview_opened
  #518·yqy003 opened Sep 6, 2026·15 tasks·

  ·
  11

• fix: skip id: lines in streaming response parsing

  bugfixno_issueno_review
  #481·navtrade opened Aug 21, 2026·1 of 9 tasks·

  ·
  1

  1

• 0819 fix itl

  no_issueno_review
  #479·jschen069 opened Aug 19, 2026Collaborator·15 tasks·

  ·
  1

• Add din_integration/: SWE-bench DinD evaluation pipeline integration

  large_prno_issueno_reviewother
  #456·Huafubing opened Aug 12, 2026Contributor·1 of 2 tasks·

  ·
  4

• feat: add aa_lcr unit test

  #420·jschen069 opened Jul 24, 2026Collaborator·15 tasks·

  ·
  1

• fix: add cross-platform file locking and background progress thread

  bugfix
  #414·jschen069 opened Jul 23, 2026Collaborator·1 of 15 tasks·

  ·
  9

• feat: add ifbench unit test

  test-cases
  #413·jschen069 opened Jul 23, 2026Collaborator·1 of 15 tasks·

  ·
  1

• [Feature][CI/CD] Support agent runtime env auto deploy

  feature
  #410·SJTUyh opened Jul 23, 2026Collaborator·2 of 15 tasks·

  ·
  1

  1

• 【WIP】 control Prefill con in aisbench

  #396·vasede opened Jul 14, 2026·15 tasks·

  ·
  5

• bugfix: Support for extracting answers expressed in thousand separato…

  bugfix
  #360·yejj710 opened Jun 24, 2026Contributor·1 of 9 tasks·

  ·
  2

• test ai

  #352·wenba0 opened Jun 23, 2026Contributor·15 tasks·

  ·
  3

• Update answer pattern for COT chat prompt

  #347·F0undLinks opened Jun 18, 2026·7 tasks·

  ·
  4