Skip to content

fix(template): truncate token type IDs with retained positions - #10176

Open
taking-lying-flat wants to merge 1 commit into
modelscope:mainfrom
taking-lying-flat:fix/truncate-token-type-ids
Open

taking-lying-flat wants to merge 1 commit into
modelscope:mainfrom
taking-lying-flat:fix/truncate-token-type-ids

Conversation

@taking-lying-flat

Copy link
Copy Markdown
Contributor

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

When an encoded multimodal sample exceeds max_length, Template._truncate() truncates input_ids, labels, loss scales, and mm_token_type_ids but leaves token_type_ids at its original length. For example, left-truncating a 12-token Gemma3-Vision sample to 8 tokens leaves 12 token types and moves the image tokens away from their original positions. The collator pads these fields independently, preserving the mismatch.

Truncate token_type_ids with the same retained-position mask as the other per-token fields. This also handles non-contiguous retained positions caused by protected image placeholders. The production change is three lines in the shared template implementation.

Extend the existing Gemma3 template tests with left/right truncation cases containing text around one or two images. Check the exact values of all five per-token fields and their shapes and values after collation, including non-contiguous retention cases.

Experiment results

  • The four new truncation cases fail on the original implementation with misaligned token types.
  • python -m pytest tests/general/test_gemma3_template.py tests/general/test_moss_vl.py -q: 14 tests and 4 subtests passed.
  • pre-commit run --files swift/template/base.py tests/general/test_gemma3_template.py: all applicable hooks passed.
  • git diff --check: passed.

Validation uses synthetic encoded samples through the real truncation and collator paths. Full-model forward and training were not run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant