Skip to content

Python: fix(filesystem): bound the literal storage-key branch by the segment cap - #9137

Closed
Winoren (Winoren) wants to merge 1 commit into
microsoft:mainfrom
Winoren:bound-literal-storage-key-segment
Closed

Winoren (Winoren) wants to merge 1 commit into
microsoft:mainfrom
Winoren:bound-literal-storage-key-segment

Conversation

@Winoren

Copy link
Copy Markdown

Fixes #9136

Problem

_storage_key_segment derives a single path segment that callers turn into a filename or directory name. The module defines _MAX_ENCODED_STORAGE_KEY_SEGMENT_LENGTH = 180 and documents the cap as existing "so the segment stays within filesystem name limits", and two of its three branches honor it. The literal branch does not:

if _is_literal_storage_key_segment_safe(value):                      # no length bound
    return value                                                       # 308 chars, verbatim
...
if len(encoded_segment) <= _MAX_ENCODED_STORAGE_KEY_SEGMENT_LENGTH:   # bounded
    return encoded_segment
...
return f"{encoded_prefix}{_DIGEST_SEGMENT_MARKER}{digest}"           # bounded

A literal-safe value can be of any length, so a long-but-otherwise-perfectly-safe identifier reaches the filesystem unchanged and fails there instead of being encoded.

FileHistoryProvider.save_messages is one concrete path. _sessions.py:2548-2549 appends .jsonl to the derived stem, and _sessions.py:2466 then opens it:

session_id len=   9  -> session-1.jsonl                OK
session_id len= 308  -> session-aaa...aaa.jsonl        OSError: [Errno 63] File name too long

The same applies to every other caller that appends its own suffix: _file_access.py:1915 (session directory name), _file_memory.py:328, _todo.py:354, _memory.py:753, and _history_provider.py:219-236.

Fix

Apply the bound the other two branches already use, so longer literal-safe values fall through to the encoded forms:

if _is_literal_storage_key_segment_safe(value) and len(value) <= _MAX_ENCODED_STORAGE_KEY_SEGMENT_LENGTH:
    return value

This reads as an oversight rather than a deliberate exception, which is why it is worth fixing at the derivation point instead of guarding at each call site.

No data migration

Only values in the 181-249 byte range change their on-disk name. Those names were already unwritable before this change, because the caller appends an extension, so no existing file can carry one.

Verification

  • Differential run over 435 observations (87 values x5 prefixes), comparing the pre-fix baseline against the fix: 340 identical. All 95 differences are literal-safe values with len >= 181 moving from literal to digest. No non-regex-driven behavioral change: the _is_literal_storage_key_segment_safe predicate itself is untouched (0 observations changed on that field), and the base32 and digest branches are unchanged.
  • Injectivity preserved under all five prefixes (~scope-, ~session-, ~todo-, ~memory-, ~access-): 87 distinct values still fold to 87 distinct segments, including after case folding. The segment is an isolation boundary, so this matters.
  • Real filesystem probes: over-cap segments went from 19 to 0, failed mkdir from 8 to 0, and failed open from 10 to 0.
  • Tests extend the existing test_filesystem.py (no new file): 42 passed. The five downstream caller suites (test_sessions, test_harness_file_access, test_harness_file_memory, test_harness_todo, test_harness_memory) also pass.
  • Test effectiveness checked by reverting the fix with the new tests in place: 17 of the new tests fail with the original OSError: [Errno 63] File name too long, and pass again once the fix is restored.

The new cases cover the length bound, the boundary (<= cap must stay verbatim so existing on-disk layouts keep working), the cap across all five prefixes, injectivity of the newly-encoded range, and a real open() on a derived name.

Note that the existing SAFE_IDENTIFIERS entry "x" * 200 is updated to sit exactly at the cap, since the fix makes its original expectation (verbatim) impossible.

Risk

Low. Values at or under the cap are byte-for-byte unchanged. The .NET side has no equivalent defect: FoundryAgentSessionKeyEncoder.BuildStorageKey hashes unconditionally and is bounded by construction.

_storage_key_segment defines _MAX_ENCODED_STORAGE_KEY_SEGMENT_LENGTH and both
the base32 and digest branches honor it, but the literal branch returned any
literal-safe value verbatim with no upper bound. A 308-character session id
therefore reached the filesystem unchanged and failed there with ENAMETOOLONG
once the caller appended its extension, turning a valid id into a failed write.

Apply the same bound to the literal branch so longer literal-safe values fall
through to the encoded forms like any other over-long value. No data migration
is affected: names in the newly-encoded range were already unwritable.
Copilot AI balanced review requested due to automatic review settings October 7, 2026 09:04
@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Oct 7, 2026
@github-actions github-actions Bot changed the title fix(filesystem): bound the literal storage-key branch by the segment cap Python: fix(filesystem): bound the literal storage-key branch by the segment cap Oct 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Valid existing storage names over 180 characters become unreachable without migration or legacy lookup support.

Review effort: Balanced
Findings: 1 High severity · 2 Low severity

Open (3)
What changed in this PR

Bounds literal storage-key segments to prevent filesystem name-length failures.

Changes:

  • Applies the 180-character cap to literal-safe identifiers.
  • Adds boundary, collision-resistance, prefix, and filesystem tests.
File Description
python/​packages/​core/​agent_framework/​_filesystem.py Bounds literal storage-key segments.
python/​packages/​core/​tests/​core/​test_filesystem.py Adds regression and boundary coverage.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

A single path segment containing no path separators.
"""
if _is_literal_storage_key_segment_safe(value):
if _is_literal_storage_key_segment_safe(value) and len(value) <= _MAX_ENCODED_STORAGE_KEY_SEGMENT_LENGTH:
Comment on lines +294 to +299
def test_length_bounded_literals_stay_injective() -> None:
"""Moving long literals into the encoded namespace must preserve injectivity.

The segment is an isolation boundary, so two distinct over-long identifiers
may not be folded together by the change -- not even after case folding.
"""
Comment on lines +310 to +313
A 308-character literal-safe session id previously produced a 308-character
file stem, which ``open()`` rejects with ``ENAMETOOLONG`` (errno 63) on
macOS, Linux and Windows alike -- turning a valid session id into a failed
write. Writing ``stem + extension`` is exactly what

This branch was successfully deployed

No deployments
github-app-auth — 6303746a Deployed Oct 7, 2026 by Winoren via team_check #6201
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bound the literal branch of _storage_key_segment by the segment length cap

3 participants