Skip to content

fix: honor native endpoint hosts in generated listeners - #210

Merged
antoinetoussaint-byte merged 5 commits into
codefly-dev:mainfrom
gael-obin:fix/native-listener-hosts
Oct 28, 2026
Merged

antoinetoussaint-byte merged 5 commits into
codefly-dev:mainfrom
gael-obin:fix/native-listener-hosts

Conversation

@gael-obin

@gael-obin gael-obin commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Three owner defects in the generated scaffold this agent owns, carried together because they share the generated adapters, the canonical base/ lock and one qualification of the built agent. Consumers adopt all of it through normal Sync.

1. Listeners discarded the resolved endpoint host

Generated gRPC, REST and Connect listeners discarded the hostname resolved by Codefly and listened on :port. A native service assigned a loopback endpoint therefore also accepted connections on other host interfaces.

The generated entrypoint carries the resolved endpoint host into the adapters for native/Nix execution. Listeners use net.JoinHostPort, and gateway/Connect self-dials follow that same address. Container/Kubernetes execution and absent runtime-context legacy behavior retain wildcard binding. Runtime context comes from the existing SDK carrier and Core parser. Existing loopback peer checks, no-proxy dialing and service interceptor chains are preserved. Main-only Sync for service-owned custom adapters retains its existing port-only contract and emits a warning; it does not assume new fields in handwritten adapters.

2. Native ownership was not bound to the authenticated launch identity

This carrier adopts Core's authenticated native workload writer and adds a maintained check of the actual built agent. Runtime registration, birth inspection and signals remain Core-owned; orphan selection and recovery remain CLI-owned. The test adapts service-go's proof to go-grpc Load-reported endpoints and Core network allocation.

An independent review found the first version of that test selected its victim by re-reading ProcessInfo().PID immediately before SignalProcess, without comparing it to a full identity captured at the original authenticated launch — Core binds the supplied identity to a kernel handle but cannot prove it is the agent the test started. The corrected test reads the manager-launched agent's Core tracked-group handle before any workload RPC (LookupProcessGroup / InspectOwnership), retains the recorded leader identity, and re-inspects that original PID immediately before the death effect, returning on any error or full-identity mismatch. A named stale_owner_refuses_delivery negative supplies a changed StartID and requires the pre-signal refusal. The recovery receipt's PID/BootID/StartID/Executable must match the retained owner (its documented owner PGID=0 is checked separately), and the sibling's final identity must equal its original authenticated capture.

The core pin moves to v0.19.10 in both go.mods and in the reusable-workflow ref, together, and the source builder selects codefly.dev/go:0.0.74. agent.codefly.yaml's version is untouched — that moves at release.

3. Connect JSON emitted nulls and defaults the protobuf profile omits

Core v0.19.10's runnable contract makes default and unset fields optional (runnable/project.go:126-145, with project_test.go:281-333 establishing the wire shape under protojson.MarshalOptions{}). The generated Connect bridge contradicted it: an absent message or timestamp came back as child:null / createdAt:null, with zero scalars and empty lists and maps materialized, so a consumer resolving "none of these" could not return an empty object.

Both generated Connect constructors now build the transcoder's codec through Vanguard v0.4.0's existing WithCodec + NewJSONCodec, with EmitUnpopulated, EmitDefaultValues, UseProtoNames, UseEnumNumbers and AllowPartial explicitly false; the codec's existing resolver and DiscardUnknown remain intact. Both target protobuf through the existing WithDefaultServiceOptions / WithTargetCodecs seam, and the dialled constructor no longer advertises JSON on the strength of a globally registered gRPC JSON codec — Vanguard would otherwise hand JSON bytes to its opaque protobuf forwarder. One small unexported option factory is shared by the two constructors. Present empty messages stay {}.

The promise is Connect JSON. The Connect-only self-hop's native HTTP/2 gRPC direct dispatch (connect_gen.go:77-86, which prevents recursive transcoding) and the separate REST gateway keep their own output; the READMEs say so.

Reuse and classification

All three are fixes at the owner that owns the behavior — no hack, no second mechanism, no consumer-side patch.

Reused: endpoint resolution and the SDK runtime-context carrier; Core's runtime-context classification, StartTrackedProcessGroup, LookupProcessGroup, InspectOwnership and identity-bound SignalProcess; the CLI's existing orphan preview/recovery receipt; Vanguard's WithCodec, NewJSONCodec and WithTargetCodecs; the existing prepared-GOWORK source-builder seam; the staged transactional Sync.

Deliberately not added: no new serializer or parser, no public option framework, no process-global production codec, no address resolver or proxy, no authority bypass, no registry, no alternate service implementation, no consumer schema workaround, and no backward-compatibility switch (explicitly excluded). Factory templates, canonical base/ and their documentation move in the same commits, under the existing parity locks.

Docs updated in this PR

docs/native-ownership.md (new — artifact selection, isolation, reproduction, limits), base/code/pkg/adapters/README.md and templates/factory/code/pkg/adapters/README.md.tmpl (the Connect JSON profile, and what REST and native direct-dispatch JSON do instead), AGENTS.md (the reusable CI pin).

Validation

  • Listener hosts, at a3e353a: full go test ./... -count=1 -timeout 8m passed, including Docker and native lifecycle tests; go build ./..., go vet ./... and go mod tidy -diff passed. Process-level entrypoint tests cover native/Nix, container/Kubernetes, absent/blank/unknown contexts, actual loopback versus non-loopback reachability, stop/rebind, Connect-only faces and custom-adapter compilation. Restoring the old wildcard statements makes the reachability regression fail. Built through codefly agent build --native-only, then synced two real consumers through sync service --local-agents: Documents and Robin hot-reloaded the generated source and now bind loopback, with normal authenticated read/chat/citation requests still succeeding while unauthorized and cross-tenant requests fail.
  • Native ownership, at clean 4b5673d: TestNativeAgentOwnershipLifecycle PASS, package 1.035s, against an agent freshly built from those exact bytes through the maintained owner command with a private OS HOME and CODEFLY_HOME (binary SHA256 469896db…, adjacent CycloneDX SBOM 3ad7a4d8… matching the executable digest and listing Core v0.19.10; metadata clean 4b5673d, darwin/arm64, CGO enabled). Covers stale-owner refusal, the actual authenticated-v2 writer, ordinary gRPC Load/Init/Start/Stop/restart, the identity-bound kill of the disposable owner, the unchanged CLI preview receipt admitted through --expect with group_dead=true and record_removed=true, and the independent sibling leader preserved and then stopped normally.
  • Connect JSON, at clean ed59cf3: the new base/code/pkg/adapters/connect_json_test.go builds a reflected protobuf descriptor with dynamicpb, registers a real unary gRPC method and exercises NewConnectServer over real loopback HTTP in three shapes — in-process, separate dialled gRPC and Connect-only dialled self-hop. It compares raw bytes decoded with encoding/json, never through typed protobuf decoding, which would hide absence and null. Cases: absent messages/timestamps, present empty parent, populated parent with absent children, implicit scalar defaults, empty map and list, custom JSON names, named enum, quoted signed/unsigned 64-bit extremes, Timestamp/Duration/Any resolution, explicit Value null, absent and present-zero wrappers, and unknown request-field discard. On the parent's production source the same test fails all three shapes.
  • Each half of that fix is independently necessary, by two separate reversals rather than one combined: restoring only EmitUnpopulated=true fails the absent/nested/default cases in all three shapes; restoring only the JSON target advertisement emits upstream null/default JSON in-process and gives both dialled shapes HTTP 500 invalid protobuf wire-format data, with the registered codec reporting 14 invocations. A separate isolated subprocess registers a real emitting gRPC JSON codec and reruns the suite: it passes with zero calls into that codec, so the profile cannot be bypassed from the global registry, and the parent process registry stays uncontaminated.
  • Regression gates at ed59cf3: 16 selected owner gates pass in 43.286s — factory/base adapters, docs and dependency locks, source message-bound/timeout/dispatch assertions, self-dial address assertions, the full base/code suite (31.33s, through TestBaseGeneratedServiceServesConfiguredRPCsOverConnect, covering listener host/reachability, transport policy, gRPC/Connect/gRPC-Web, limits, timeout, streaming, REST and shutdown) and the normally rendered Connect-only factory suite (11.78s). GOWORK=off generated-service go build -mod=readonly, base vet and the README parity lock pass.
  • The built agent at ed59cf3: agent build --native-only --local-agents passed from clean ed59cf3 with GOWORK=off, no ancestor go.work and private HOME/CODEFLY_HOME; binary SHA256 6870014b…, SBOM 358707345… matching the executable digest and identifying Core v0.19.10; metadata darwin/arm64, CGO_ENABLED=1, vcs.modified=false, exact revision.
  • Independent review: separate design, implementation and focused follow-up reviews inspected the changed source, the library mechanisms, both reversal patches and logs, and the consumer source. The follow-up found no blocking defect at ed59cf3, rehashed all 142 archived source files against git show ed59cf3:<path> with no difference, and confirmed design acceptance.

Limits — what is not qualified

  • The full suite, Docker lifecycle, CreateToRun/matrix and cross-packaging were last run green at a3e353a; they were not rerun on ed59cf3 (engine allocation belongs to another task). Opt-in packaging-image coverage (CODEFLY_PACKAGING_IMAGE_TEST=1) was not run at all.
  • Consumer adoption is unfinished: the Records pin-resolution service still selects go-grpc 0.1.67 and its generated adapter still has the old constructors. It must adopt this through the supported update service --agent-version then sync service, never a hand-edited adapter, and its "none of these" empty-response behavior must then be verified against the normally derived contract. That acceptance is not claimed here.
  • No Linux runtime, Nix, sandbox/principal policy, full CLI composition, publication or protected-environment claim. The 0.1.69 installation key on the local binary is a candidate, not a release; no release/OCI qualification is implied.
  • Local mixed container-to-native TCP forwarding was observed on OrbStack only; Linux bridge/host-gateway compatibility is not qualified. A deployment depending on direct access to a native service's host interface must supply a compatible resolved endpoint; this change adds no ingress or forwarding mechanism.
  • The new raw-wire fixture is unary; it is not an exhaustive streaming-JSON matrix, and it certifies serialization shape, not auth or domain behavior. Custom adapters remain service-owned, and other agent families are unaffected. Existing Core bare-IPv6 carrier parsing is out of scope (URL-form IPv6 is covered). The maintained build audit still reports the two unpatched upstream findings GO-2026-4883 and GO-2026-4887. This is not a fleet-wide network-isolation claim.

🤖 Generated with Claude Code

@antoinetoussaint-byte antoinetoussaint-byte added the working Claimed by a lazybox agent; cleared when the session ends. label Oct 28, 2026
@antoinetoussaint-byte antoinetoussaint-byte added working Claimed by a lazybox agent; cleared when the session ends. and removed working Claimed by a lazybox agent; cleared when the session ends. labels Oct 28, 2026
@antoinetoussaint-byte

antoinetoussaint-byte commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

lazybox has finished working on this. claude (Opus 5) held it from 2026-10-28 15:28 UTC until 2026-10-28 15:45 UTC. The working label has been removed; this comment is the record.

claim detail
Lazybox-Claim-Device: effacd542b611010656e
Lazybox-Claim-Session: 1ce8c525e5
Lazybox-Claim-Workspace: github-codefly-dev-service-go-grpc-210
Lazybox-Claim-Agent: claude
Lazybox-Claim-Model: Opus 5
Lazybox-Claim-Started: 2026-10-28T15:28:46Z
Lazybox-Claim-Heartbeat: 2026-10-28T15:45:03Z
Lazybox-Claim-Expires: 2026-10-28T16:44:00Z
Lazybox-Claim-Released: 2026-10-28T15:45:03Z

@antoinetoussaint-byte
antoinetoussaint-byte merged commit c645112 into codefly-dev:main Oct 28, 2026
3 checks passed
@antoinetoussaint-byte antoinetoussaint-byte removed the working Claimed by a lazybox agent; cleared when the session ends. label Oct 28, 2026
@gael-obin

Copy link
Copy Markdown
Contributor Author

Independent post-merge review — what landed is sound; consumer obligations are not discharged

Fresh-context review under the architecture-bouncer skill at ed59cf38 (scope ed319615..ed59cf38). This PR is already merged, so this is not a merge opinion — it is a record of what the change now obliges consumers to do, because several of those obligations are written nowhere.

The work itself holds up

The Connect JSON fix is real and was reproduced by reversal: Vanguard v0.4.0's default EmitUnpopulated:true (vanguard@v0.4.0/codec.go:99-104) genuinely contradicted Core's own runnable contract (runnable/project.go:126-137). Before/after on the same request:
{"child":null,"createdAt":null,"displayName":"","signedCount":"0",...,"labels":[],"attributes":{}} → {}.

The listener-host fallbacks are correct in source and in test — 9/9 runtime contexts pass, and the reversal fails 5. An empty or unbindable host fails loudly.

Obligations that are not written down

1. Nothing in the deployed compositions actually has this change. platform-obin:1092 and platform-nyl-services:32 pin go-grpc 0.1.69 as an absolute override (core/resources/agent_overrides.go:11-30), which also silently downgrades module-robin/.../host from its own 0.1.70 pin. Correction to the PR body: 0.1.69 is the merge base, not "a candidate" — this work shipped in released 0.1.70/0.1.71. Someone has to decide whether those overrides move.

2. A raw-JSON consumer the PR never names is affected. obin/module-runtime/.../pkg/runnable/transport.go calls every owner in Connect JSON mode with encoding/json. Any consumer reading createdAt: null or a zero scalar now gets an absent key.

3. The native process registry moved authenticated-v1 → authenticated-v2 (Core v0.19.8+, runners/base/pgid.go:85,186), and namespaces are never cross-swept. A workspace mixing new go-grpc with service-go 0.0.74 reaps and recovers only half its processes. This is not stated anywhere.

4. module-document-store/.../service.codefly.yaml:11-18 already carries scaffold generated from the unreleased ed59cf38 under version: 0.1.69 — a mismatch worth reconciling before the next sync.

5. The container→native case fails silently at the caller. Core's own Linux runner maps host.docker.internal to 172.17.0.1 (core/runners/dockerrun/docker_runner.go:683-686), so the Linux break is a source-established prediction, not an unknown. OBIN exposure is low — --runtime-context is one global flag per run and Kubernetes keeps wildcard binding — but the failure mode deserves a named error rather than an unreachable service.

Cleared, not a finding

main.go.tmpl imports resources/strings unguarded under CustomAdapters, which looks like a compile break; production formatStagedGo goimports prunes them (builder.go:329-367). Chased and cleared.

GO-2026-4883/4887 are Moby daemon-side; only docker/docker/api/*, client, errdefs and pkg/stdcopy are linked. Disclosure adequate, nothing exposed.

What was run

Full root go test at this head: EXIT 1 on two environment failures only — a port race in TestConnectOnlyFactoryServesEveryProtocol, and an OrbStack socket timeout in TestCreateToRunDocker. The canonical base/code suite is fully green, and both reversal checks reproduce the failures they should.

The question this leaves open

Who decides, and where is it recorded, that every OBIN module's Connect JSON wire shape changes on its next agent sync? That decision currently exists only in this PR body and one module's service-file comment. There is no OBIN-side record of which faces adopted it, which consumers were checked, or what the platform override should be — so each module adopts blind.

Related: codefly-dev/service-go#159 still pins Core v0.19.9, while this carrier took v0.19.10. The v0.19.9→v0.19.10 diff is the Linux fix that stops a live owner being reported absent, which is the predicate orphan recovery kills on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants