[Manohar Paturi (ManoharPaturi)]

Motivation & Context

In InMemoryAgentFileStore, writing to a path whose ancestor is already an existing file succeeded, as did writing to a path that already has descendant files in the store. This allowed Reports and Reports/q1.txt to coexist simultaneously, causing list_children to surface both a directory and a file under the exact same name. Real filesystems reject these operations with NotADirectoryError and IsADirectoryError.

This change aligns InMemoryAgentFileStore with FileSystemAgentFileStore by enforcing directory and file path invariants during write and create_directory.

This PR replaces #8687, which was closed while it still had outstanding review feedback about preserving the file-only delete contract. Per the close reason, it is being resubmitted here with that feedback addressed on top of the original fix.

Description & Review Guide

1. In write, verified that no ancestor path of the target is an existing file in self._files, raising NotADirectoryError if encountered.
2. In write, checked whether the target path is already an existing directory with descendants, raising IsADirectoryError if so.
3. In create_directory, ensured that neither the target path nor any ancestor is an existing file, raising FileExistsError if a collision is detected.
4. Both complexity bounds live in the shared _normalize_relative_path, which both backends already call: a normalized path is refused with OSError(ENAMETOOLONG) when it exceeds 255 segments or 4096 characters. That is the same error the OS raises for the filesystem backend on such paths, and it closes the unbounded ancestor prefix materialization flagged in review: an 8000 deep write previously retained 64 million characters of ancestor keys and now stores nothing.
5. Added regression tests in test_harness_file_access.py covering both collision orders and confirming existing file contents remain intact.

Changes addressing the review feedback from #8687:

5. delete() is now strictly file-only: a path that names a directory — explicitly created, holding stored descendants, or holding a nested explicit directory — returns False, exactly matching FileSystemAgentFileStore.delete. Backend substitution is therefore no longer observable through delete().
6. Session-scoped directory cleanup is preserved through a new AgentFileStore.delete_directory(path) method that releases an empty explicitly created directory. The in-memory store removes the explicit-dir entry; the filesystem store uses rmdir and returns False (instead of raising) when the path is missing, is a file, or still holds entries. The base class ships a non-breaking default implementation returning False, so third-party stores are unaffected until they opt in.
7. Tests: test_in_memory_store_delete_is_file_only asserts the False returns; test_in_memory_store_delete_directory_releases_empty_explicit_directory covers the lifecycle; test_filesystem_delete_is_file_only_and_delete_directory_matches_in_memory runs both backends side by side for parity on delete() and delete_directory().

The full harness suites (259 tests), Ruff check/format, and all five typing checkers (pyrefly, ty, zuban, mypy, pyright) pass locally.

Related Issue

Fixes #8567

The new AgentFileStore.delete_directory surface is tracked in #9666, which also carries the Python and .NET consistency discussion for the store contract.

[Manohar Paturi (ManoharPaturi)]

The Public API Compatibility check fails on every open PR in this repo right now (e.g. #9602, #9607, and all of this author's open PRs), so it looks like a repo-wide CI issue rather than something introduced by this change.

The job crashes inside griffe while loading the base (main) snapshot, before it compares the PR changes — the traceback points at the base copy of the package:

griffe._internal.exceptions.AliasResolutionError: Could not resolve alias
agent_framework._workflows._edge.EdgeGroup.builtin_type pointing at builtins.type
(in /tmp/python-api-base-.../python/packages/core/agent_framework/_workflows/_edge.py:315)

Example run: https://github.com/microsoft/agent-framework/actions/runs/39810016173/job/119933441343

None of the changes in this PR touch _workflows/_edge.py. Filed an issue for the CI failure; happy to help debug if useful.

[Bicheng (Kenneth) (BichengWang)]

Reviewed 4f2d38c43136bcbc03de6544f51642f7cb8951e4, including the feedback on #8687.

P2: Bound directory-prefix materialization (python/packages/core/agent_framework/_harness/_file_access.py:1096-1103). Slicing key[:idx] still copies every full ancestor prefix, so the helper takes quadratic space in path depth despite its linear-time docstring. write() then retains those strings in _dir_counts. The path normalizer has no length/depth bound. This leaves the deep-path concern from #8687 unresolved, with persistent memory growth from one small write.

Offline reproduction on macOS / CPython 3.13.16:

store = InMemoryAgentFileStore()
path = "/".join(["a"] * 8000 + ["f"])
await store.write(path, "x")
print(len(path), sum(len(k) for k in store._dir_counts))
# 16001 64000000

Depths 2,000 / 4,000 / 8,000 retain 4 / 16 / 64 million prefix characters; the final case retains 64,328,000 bytes of key strings, excluding the dictionary. Please bound accepted path complexity before allocating prefixes, or use an index that avoids storing every full prefix, and add regression coverage for that bound.

The two affected test files pass locally: 176 passed, 2 skipped. Six extracted collision/directory regressions fail on current main 4982e4c61ec81cc4f9ba0cbdb5594ed530dbbcfe and pass on this head. Ruff 0.17.1 lint and formatting pass for all three changed files. Validation was offline; the complete core suite and Windows behavior were not tested.

[Manohar Paturi (ManoharPaturi)]

Verified your reproduction first: on the previous head the 8000 deep write retained exactly 64,000,000 characters of ancestor keys in _dir_counts, matching your numbers. dacf006 takes your first option and bounds the accepted complexity before any prefix is allocated.

The shared normalizer now refuses a path beyond 255 segments or 4096 total characters with OSError ENAMETOOLONG, the same errno the OS layer returns for the filesystem backend on the same input, so both stores reject identically and nothing is materialized before the check runs. Your exact script now raises that error and leaves _files and _dir_counts empty, with zero characters retained.

Tests cover the rejection and the empty index afterward, a shallow path past the length bound, a 200 deep path that still works with its retention staying linear in the stored path, and a side by side parity case running both backends on both sides of the bound. The rejection and parity tests fail on the previous head. The two affected suites pass at 162 tests, ruff check and format are clean, and mypy passes on the changed file.

[Eduard van Valkenburg (eavanvalkenburg)]

this introduces a new public API, I would like to see a issue for that, since we want this to be consistent with dotnet. CC westey (@westey-m)

[Manohar Paturi (ManoharPaturi)]

Filed as #9666, covering the motivation (session scoped cleanup of explicit directories), the chosen shape (base default returning False so third party stores are unaffected until they opt in), the alternative rejected in the earlier review (overloading delete(), which makes backend substitution observable), and the open .NET consistency question for the store contract. westey (@westey-m) looped in there for that decision. The PR body now references it under Related Issue as the tracking issue for the new surface.