Skip to content

fix: preserve OpenAI parallel tool-result associations - #10174

Merged
tastelikefeet merged 1 commit into
modelscope:mainfrom
taking-lying-flat:fix/openai-tool-result-order
Sep 22, 2026
Merged

tastelikefeet merged 1 commit into
modelscope:mainfrom
taking-lying-flat:fix/openai-tool-result-order

Conversation

@taking-lying-flat

Copy link
Copy Markdown
Contributor

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

OpenAI parallel tool results can arrive in a different order from the assistant's tool_calls. The normalizer discards call IDs, and dataset message validation also removes tool_call_id. Templates such as Hermes then render only the result contents in arrival order, losing which call produced each result.

For calls lookup_value(A) followed by lookup_value(B), these distinct records currently produce the same canonical context:

Results in arrival order Current result order Fixed result order
B → 20, A → 10 20, 10 10, 20
A → 20, B → 10 20, 10 20, 10

Align each immediately following, complete tool-result group by call ID before converting the assistant calls. The shared normalizer covers both dataset preprocessing and direct Template.encode() inputs. Copy the message list before reordering and move whole result messages to preserve their metadata.

The change is confined to one production function (+14/-1). Reordering requires distinct non-null call IDs and exactly one matching result per call. Single-call, already ordered, and ID-less legacy inputs retain their behavior. Incomplete, ambiguous, or noncontiguous result groups retain their existing behavior and are outside this fix.

Experiment results

  • 54 tests and 2 subtests passed: 40 targeted local regressions and 14 existing provider, rejected-message, and serialized-message tests. The targeted validation scripts are outside this production-only patch.
  • Verified all six result-arrival permutations for three calls, native and serialized JSONL dataset loading, result metadata, repeated rounds, and legacy input compatibility.
  • With the actual Qwen3-0.6B tokenizer and Hermes template, the original normalizer produces identical input IDs for the two distinct associations above. The fix keeps them distinct, while changes in arrival order alone produce identical input IDs and training labels. Verified direct encoding and preprocessed inputs.
  • Changed-file pre-commit hooks and git diff --check passed. No full training or model-accuracy experiment was run.

Signed-off-by: taking-lying-flat <1615405@qq.com>
@tastelikefeet
tastelikefeet merged commit d85802d into modelscope:main Sep 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants