Repository navigation
feat: add passive CLI doctor diagnostics and on-demand agent - #380
Merged
Merged
Conversation
Add a bounded provenance collector with a thin Click command, an on-demand diagnostic agent, compatibility-preserving entry points, and installed-wheel safety coverage. Generated with Amplifier Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Generated with Amplifier Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a CLI-local diagnostic path that can collect useful evidence without activating the normal application runtime:
amplifier_app_cli/lib/diagnostics.py, exposed through the thin Click commandamplifier doctor [--format text|json] [--session EXACT_ID].app-cli:cli-doctoragent. It consults the existingamplifier-configskill only for configuration cases. It adds no duplicate skill, always-on instruction body, or replacement root.This separates passive evidence collection from configuration resolution, live probes, and repair, so a diagnosis request does not silently authorize environment changes.
Scope and compatibility
health.statusis alwaysnot_assessed; collectioncompletemeans collection completed, not that the environment is healthy.doctorbypasses normal startup for the published console, package-module entry, and legacymain:mainlauncher. A lightweight dispatcher and guarded, in-place ordinary initialization preserve TLS setup, home-ownership enforcement, shell completion, and existing global patch points. Preceding launcher arguments do not take the passive route.ModuleTypeattribute hooks on the existing module objects resolve collisions with the actual same-named imported children; they do not replace modules or introduce aliases.unsupported_yaml. This is a collection limit, not a claim that runtime settings are invalid or an instruction to repair them. Runtime settings parsing is unchanged.Breaking changes: none intended. The published console target moves to the lightweight dispatcher; the legacy
amplifier_app_cli.main:mainentry remains supported.Verification
Pre-publication results below are from isolated Linux ARM64, Python 3.11.14 environments with locked dependencies, not a claim about other platforms:
4801c9d7bdf7e60159e21234b22e0c50c0bb214e-m, legacy module-m, and an actual installer-generated legacymain:mainconsolegit diff --checkpassedThe full suite is not green. No new failed IDs were observed against that independently installed base. The matching failures were in offline provider-source normalization/resolution paths: 121 direct/captured failures plus one concurrency assertion after two provider-resolution thread errors. This comparison does not authorize ignoring a remote failure.
The installed-wheel checks cover ownership refusal and explicit override, argument errors/help, and exact-session selection. The safety probe records forbidden attempts even when application code swallows their exceptions, so refusal cannot be mistaken for successful passive collection.
The raw strict syscall harness remains false, not an all-process zero-write pass: it observed 3 console / 15 module standard-library bytecode-cache
EACCESwrite attempts, all before the first package access. Paths and counts matched the correspondingpython -c/python -mcontrols. No application-owned writes, network activity, extra subprocesses, or persistent changes were observed. This is bounded evidence, not universal passivity.One prior real-provider observation exercised actual delegation, the child's doctor Bash tool call, and loading the installed
amplifier-configskill. That observation kept effective/live state unknown and gave no health or repair verdict. It is not evidence of universal model reliability; the agent was unchanged afterward and no additional live-model trial is claimed.The only post-runtime-test changes were two inline lint annotations for intentional bootstrap imports in
tests/doctor_safety_probe.py; the tested and committed Python ASTs match exactly, and all other file hashes match the tested snapshot. No full Pyright result is claimed.An independent source/privacy review and separate Astra and Fable packet reviews found no demonstrated remaining publication blocker. Committed material is source and synthetic fixtures only: no actual diagnostic reports, private evidence links, or raw research logs are included.
To verify in an isolated development environment:
The added CI step builds and installs a non-editable wheel, then runs
tests/check_doctor_wheel.pywithtests/doctor_safety_probe.pyfrom outside the checkout, including the installer-generated legacy launcher.Remaining checks
Remote verification is complete for exact head
3b92f17fea1f4ce4c634b2f35c3be465950292eb: CI run 40064089760 passed all eight jobs (Ubuntu/macOS/Windows Python 3.11/3.12 unit tests and both POSIX integration jobs), including the Ubuntu Python 3.11 installed-wheel step. The CLA check passed. GitHub still requires review; the owner explicitly authorized an admin merge to bypass that review-policy gate. The historical local-baseline, interpreter-startup passivity, and live-agent coverage limitations above remain unchanged.Windows CI follow-up
Initial CI run 40063047913 failed on Windows Python 3.11/3.12 during pytest setup/teardown: auto-generated parameter IDs for oversized YAML payloads made
PYTEST_CURRENT_TESTexceed Windows' 32,767-character environment-variable limit. Commit3b92f17fea1f4ce4c634b2f35c3be465950292ebadds 15 short, explicit IDs totest_bounded_settingsintests/test_doctor.py. All 15 payloads and assertions are unchanged; there are no new skips, xfails, or exclusions, and no production-code changes. This is an additional test-only change after the runtime verification documented above. Whole-file AST equality holds after removing only the newidskeyword. A standalone pytest 9.0.3 metadata smoke using the actual decorator inputs passed all 15 cases withPYTEST_CURRENT_TESTbelow 1,024 characters; this is not a collector test or proof on Windows. Full remote CI for this exact new head has now passed all eight jobs in run 40064089760, including both Windows versions and the installed-wheel step; CLA also passed. The earlier local-baseline and passivity/model-coverage limitations still apply.