Skip to content

feat(doctor): check a solution against the one shape it has - #1218

Merged
antoinetoussaint-byte merged 2 commits into
mainfrom
issue-1215-doctor-solution
Oct 28, 2026
Merged

antoinetoussaint-byte merged 2 commits into
mainfrom
issue-1215-doctor-solution

Conversation

@antoinetoussaint-byte

Copy link
Copy Markdown
Contributor

Follow-up to #1216, which added codefly add solution. That verb generates the shape; nothing checked it.

codefly doctor solution [module] judges a solution against the shape add solution generates — read-only, stable diagnostic codes, versioned --json report. With no argument it judges every module in the workspace; a module shipping no solution manifest is reported as "not a solution" rather than as broken.

Classification: fix

A gap in the CLI, filled in the CLI. No core change.

Why

The shape was checkable only against somebody's survey of the solutions that happened to exist. That is how obin-ai/core-solutions reached nine solutions in four different states, two of them unable to receive a person capability at all with nothing failing to say so. A shape nothing can check is a convention, and a convention drifts.

Epic obin-ai/core-solutions#466's "done when" is "every solution's documents match what codefly add solution generates" — until now verifiable only by eyeball.

Reuse

Every document is read by the reader that already owns it — manifest.LoadConsumption, modulecontract.Load, composition.LoadPackageManifest, solutionhost.ParseFrontend, resources.LoadModuleFromDir. The checker adds no parsing of its own, so a rule it reports is a rule those readers enforce.

The command follows doctor workspace exactly: SilenceUsage, stable codes documented in Long, --json, exit 0/1.

Every rule is one that silently broke something

Two carry the weight:

code what it catches
owner_aware_missing The authority issues a person a capability at a solution only when the contract declares owner_aware: true, refusing by name otherwise. Without it the solution can act for nobody — and nothing fails at build time.
module_edge_declared A service-dependencies entry naming anything but the host's door renders a mesh allow rule straight to that module's service, past the authority's check.

A warning is not a failure: publishing the page's shell, a headless solution, a solution with no shell. Those are worth seeing and do not make the solution wrong.

Both host doors, because running it caught that I only knew one

The first draft flagged saas/frontend and saas/auth-gateway on lastlogin-go as illegal edges. They are not — core-solutions' own check_solution_mesh.py holds two doors:

HOST_DOOR = {("saas", "auth-gateway"), ("saas", "frontend")}
AUTHORITY_DOOR = {("authority", "hop"): "codefly.dev/authority"}

A saas host's gateway and frontend unconditionally; the authority's hop only in a manifest that also requires the authority interface, since without it an edge to the hop is a path to a module service like any other. The checker now holds the same table. Shipping the first draft would have told all seven conforming PRs to delete their legitimate door.

Verification

Run against all nine real solutions in obin-ai/core-solutions:

wiki               OK      proposals          OK
lastlogin-go       FAIL    releases           FAIL
summary            FAIL    robin-nest         FAIL
lastlogin-python   FAIL

That independently reproduces the gap ledger in #466, which was derived by hand — and confirms wiki and proposals as two independent instances of the shape rather than one reading of one file.

TestTheGeneratedShapeConforms pins the loop closed: what add solution generates passes doctor solution. If they ever disagree, one is wrong and a scaffolded solution is born failing.

go build ./..., go test ./pkg/solutionshape ./cmd pass, golangci-lint run cmd/... pkg/solutionshape/... 0 issues, docs/cli-reference.md regenerated.

What I did not verify

  • No solution was booted or rendered. The checker reads committed documents; it does not prove a conforming solution runs.
  • It does not cover what core-solutions' gates check beyond the shape — the solution-runtime-go pin, the boundary denylist, removed credential keys. Those stay with the consumer repository.
  • excel-extract and robin-builder were not judged: neither is present in the checkout I had.

MCP

Not exposed. It is a read-only diagnostic a person runs or a CI gate calls; doctor workspace is not exposed either.

Docs

docs/commands.md (the verb, both load-bearing rules, exit codes, --json shape), docs/cli-reference.md (regenerated), AGENTS.md (How-To Index).

🤖 Generated with Claude Code

antoinetoussaint-byte and others added 2 commits October 28, 2026 16:24
`codefly doctor solution [module]` judges a solution against the shape
`codefly add solution` generates, read-only, with stable diagnostic codes
and a versioned `--json` report.

The shape was previously checkable only against somebody's survey of the
solutions that happened to exist, which is how obin-ai/core-solutions
reached nine solutions in four different states — two of them unable to
receive a person capability at all, with nothing failing to say so. A
shape nothing can check is a convention, and a convention drifts.

Every rule is one that silently broke something. The two that cost the
most: a contract without `owner_aware: true`, which the authority refuses
by name so the solution can act for nobody; and a `service-dependencies`
entry naming a module, which renders a mesh allow rule past the
authority's check.

Both host doors are known, as core-solutions' own check_solution_mesh.py
holds them: a saas host's gateway and frontend unconditionally, and the
authority's hop only in a manifest that also requires the authority
interface. Running the first draft against the real solutions is what
caught that — it flagged lastlogin-go's legitimate saas door.

Verified against all nine solutions in obin-ai/core-solutions: wiki and
proposals conform; lastlogin-go, releases, summary, robin-nest and
lastlogin-python fail, reproducing the gap ledger in
obin-ai/core-solutions#466 that was derived by hand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Running the check across the real tree found two gaps in it.

`--dir` judges a module directory on its own. A consumer repository
gating `solutions/*/` has module directories and not necessarily a
workspace at each one: core-solutions' excel-extract is a real solution
with no workspace.codefly.yaml, which made it the single solution the
check could not judge at all.

`--require-solution` closes what --dir leaves open. A directory with no
solution manifest reads as "not a solution" and passes — right for a
library in the tree, wrong for a solution whose manifest is missing,
which is exactly robin-builder's state. A gate walking solutions/*/
means "every one of these is a solution", and now says so.

In --dir mode the no-solution-found guard does not apply: the caller
named the directory, so "it is not a solution" is an answer rather than
a vacuous pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@antoinetoussaint-byte

Copy link
Copy Markdown
Contributor Author

Pushed --dir and --require-solution, both found by running the check across the real tree rather than by design.

Using it on the six conforming PRs — it found three real gaps

PR solution verdict
#474 lastlogin-go OK
#475 robin-builder OK
#478 robin-nest OK
#476 lastlogin-python FAIL module_identity_missing
#479 summary FAIL module_identity_missing
#477 excel-extract could not be judged — see below

Both failures are the same real defect: the PR adds module.contract.codefly.yaml with a principal and owner_aware: true, and the backend does not declare module-identity: true. The contract and the identity only work as a pair — the authority takes the backend's projected credential as the actor of every Work Context it issues, and the CLI refuses an authority request from a module with no such service. Adding one without the other is worse than adding neither, because it looks done. Both authors have been told.

That is three PRs that would have merged looking conformant. It is the argument for this command.

Two gaps the tree run found in the command itself

--dir. #477 failed with no workspace found, and the cause was mine: the command resolved a workspace, and excel-extract is the only real solution in core-solutions with no workspace.codefly.yaml — so it was the one solution that could not be judged at all. A consumer repository gating solutions/*/ has module directories, not workspaces. With --dir, #477 reports OK (one warning), so the PR was fine and my tool was not.

--require-solution. With --dir alone the tree gate reported robin-builder OK on main — because it ships no solution manifest, so it reads as "not a solution" and passes. That is right for a library sitting in the tree and wrong for the one solution that most needs conforming. A gate walking solutions/*/ means "every one of these is a solution"; the flag says so.

In --dir mode the "no solution found, refusing a vacuous pass" guard no longer applies — the caller named the directory, so "it is not a solution" is an answer, not an empty run.

The gate, on core-solutions main today

for s in solutions/*/; do codefly doctor solution --dir "$s" --require-solution || exit 1; done
wiki OK   proposals OK
client-engagement FAIL   excel-extract FAIL   lastlogin-go FAIL
lastlogin-python FAIL     releases FAIL        robin-builder FAIL
robin-nest FAIL           summary FAIL

(client-engagement fails only under --require-solution; it is not a solution and is out of scope for the epic.)

Tests, lint and the regenerated reference are all green.

🤖 Generated with Claude Code

@antoinetoussaint-byte
antoinetoussaint-byte added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit a6bc762 Oct 28, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant