Skip to content

fix(gateway): default inbound auth to iam — authorizer is immutable after creation - #779

Merged
DerrickF merged 1 commit into
developfrom
fix/gateway-inbound-auth-fail-safe
Jul 28, 2026
Merged

fix(gateway): default inbound auth to iam — authorizer is immutable after creation#779
DerrickF merged 1 commit into
developfrom
fix/gateway-inbound-auth-fail-safe

Conversation

@DerrickF

Copy link
Copy Markdown
Contributor

Hotfix for #778. The Gateway authorizer migration cannot work in place, and the partial deploy left dev in a broken state.

What happened

PlatformStack failed deploying #778 to dev:

Authorizer type cannot be updated for an existing gateway
(Service: BedrockAgentCoreControl, Status Code: 400)
HandlerErrorCode: InvalidRequest

The stack rolled back cleanly. The Gateway kept AWS_IAM, READY, and both registered targets (arxiv, policy-search) — no infrastructure damage.

Why the pre-deploy checks missed it

Two signals said the change was safe. Both describe CloudFormation's plan, not the AgentCore service's validation:

Signal What it said Why it was wrong
CFN resource reference AuthorizerType"Update requires: No interruption" Describes CFN's update mode, not whether the service accepts the call
cdk diff (real change set) [~] modify-in-place, no replacement A change set predicts CFN's plan; it never invokes service-side validation

The general lesson for AgentCore resources: a change set is not a deploy test.

Two problems fixed, not one

1. Dev was about to break (the urgent one). The backend half of #778 deployed successfully while the infra half rolled back. That left the runtime with no AGENTCORE_GATEWAY_INBOUND_AUTH — and the agent's default was jwt. So the new agent sent bearer tokens to an AWS_IAM Gateway and 401'd every Gateway tool call. It didn't even degrade gracefully: GatewayAuthError only fires when there's no token, and here there was one.

Both defaults are now iam. An absent value means "behave like the Gateway that is actually deployed" — the only safe direction when the two halves can land independently. The agent is correct now even if the env var never lands.

2. The escape hatch didn't exist. CDK_GATEWAY_INBOUND_AUTH was only in config.ts — step 1 of the repo's 7-step config pattern. The documented rollback knob was unreachable from CI, which mattered once it became load-bearing. Now:

  • exported and validated (iam|jwt) in load-env.sh
  • passed as context (synth.sh / deploy.sh already use build_cdk_context_params)
  • carried in platform.yml's job-level env
  • backend.yml runs no CDK, so it needs nothing

Corrections to the record

Every place that asserted the opposite is fixed: the GatewayConfig doc, the construct's inline comment and class doc, the plan's Phase 0 and 1A, open decision #1, and the public environments.md page — now a :::danger box carrying the real error text and an explanation of why the pre-deploy signals mislead.

What this means for the migration

The single-Gateway endstate still holds. What changes is the mechanism: inboundAuth effectively sets the authorizer at Gateway creation. Reaching CUSTOM_JWT on an existing deployment requires:

  1. Create a second Gateway with CUSTOM_JWT (the existing one keeps serving)
  2. Re-register every target on it
  3. Repoint the agent via /{prefix}/gateway/id
  4. Verify, then delete the old Gateway

Step 2 is the hard part — targets are managed out-of-band by app-api's GatewayTargetService and by the mcp-servers repo, so it isn't CDK-only. That needs its own design; it's recorded in the plan rather than attempted here.

The plan's original "option B" (second Gateway) turns out not to be a preference but the only available mechanism.

Verification

  • Synth against dev now emits AuthorizerType: AWS_IAM, matching the live Gateway
  • cdk diff shows no authorizer change on the Gateway resource — the deploy will proceed
  • Runtime env var resolves to iam
  • tsc clean; 481 infra jest; 58 backend gateway + supply-chain tests (the latter guard load-env.sh)

After merge

platform.yml should deploy green, and backend.yml re-ships the agent with iam. Then confirm a Gateway tool (arxiv or policy-search) works in dev — that verifies SigV4 is restored end to end.

PlatformStack failed deploying #778 to dev:

    Authorizer type cannot be updated for an existing gateway
    (Service: BedrockAgentCoreControl, Status Code: 400)

AgentCore will not change a Gateway's authorizerType after creation. The stack
rolled back cleanly and the Gateway kept AWS_IAM, READY, and both targets — but
the migration as designed cannot work in place.

Neither pre-deploy check caught this. The CloudFormation resource reference
documents AuthorizerType as "Update requires: No interruption", and `cdk diff`
via a real change set reported an in-place [~] modify. Both describe CFN's
plan, not the AgentCore service's validation. A change set is not a deploy test.

Two failures to fix, not one:

1. The backend half shipped while the infra half rolled back, so the runtime
   had no AGENTCORE_GATEWAY_INBOUND_AUTH — and the agent's default was 'jwt'.
   That pointed the new agent at bearer auth against an AWS_IAM Gateway, 401ing
   every Gateway tool call. Both defaults are now 'iam': an absent value means
   "behave like the Gateway that is actually deployed", which is the only safe
   direction when the two halves can land independently.

2. CDK_GATEWAY_INBOUND_AUTH existed only in config.ts — step 1 of the repo's
   7-step config pattern. The documented escape hatch was unreachable from CI.
   Now exported and validated in load-env.sh, passed as context (synth.sh and
   deploy.sh already use build_cdk_context_params), and carried in platform.yml's
   job env. backend.yml runs no CDK, so it needs nothing.

Also corrects every place that asserted the opposite: the GatewayConfig doc, the
construct's inline comment and class doc, the plan's Phase 0/1A, and the public
environments.md page (now a :::danger: covering the real error and why the
pre-deploy signals mislead).

The single-Gateway endstate still holds. What changes is the mechanism: reaching
CUSTOM_JWT needs a new Gateway plus target re-registration and a cutover, not a
config flip. Targets are managed out-of-band by app-api's GatewayTargetService
and the mcp-servers repo, so that needs its own design — tracked in the plan.

Verified: synth against dev now emits AuthorizerType AWS_IAM matching the live
Gateway, and `cdk diff` shows no authorizer change, so the deploy proceeds.
tsc clean; infra 481 jest; 58 backend gateway + supply-chain tests.
@DerrickF
DerrickF merged commit bae196e into develop Jul 28, 2026
4 checks passed
@DerrickF
DerrickF deleted the fix/gateway-inbound-auth-fail-safe branch July 29, 2026 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant