Skip to content

Various fixes and improvements (#8361, #8367, DataBiosphere/azul-private#431, #8381, #8375) - #8372

Merged
achave11-ucsc merged 11 commits into
developfrom
issues/hannes-ucsc/8361-et-al-omnibus
Oct 22, 2026
Merged

achave11-ucsc merged 11 commits into
developfrom
issues/hannes-ucsc/8361-et-al-omnibus

Conversation

@hannes-ucsc

@hannes-ucsc hannes-ucsc commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Linked issues: #8361, #8367, DataBiosphere/azul-private#431, #8381, #8375

Checklist

Author

  • A01 PR is assigned to the author
  • A02 Status of PR is In progress
  • A03 PR is a draft
  • A04 Target branch is develop
  • A05 Name of PR branch matches issues/<GitHub handle of author>/<issue#>-<slug>
  • A06 PR is linked to all issues it (partially) resolves
  • A07 Status of linked issues is In progress
  • A08 PR description links to linked issues
  • A09 PR title matches1 that of a linked issue or comment in PR explains why they're different
  • A10 PR title references all linked issues
  • A11 For each linked issue, there is at least one commit whose title references that issue

1 when the issue title describes a problem, the corresponding PR
title is Fix: followed by the issue title

Author (partiality)

  • B01 Added p tag to titles of partial commits
  • B02 This PR is labeled partial or completely resolves all linked issues
  • B03 This PR partially resolves each of the linked issues or does not have the partial label

Author (reindex)

  • C01 Added r tag to commit title or the changes introduced by this PR will not require reindexing of any deployment
  • C02 This PR is labeled reindex:dev or the changes introduced by it will not require reindexing of dev
  • C03 This PR is labeled reindex:anvildev or the changes introduced by it will not require reindexing of anvildev
  • C04 This PR is labeled reindex:anvilprod or the changes introduced by it will not require reindexing of anvilprod
  • C05 This PR is labeled reindex:prod or the changes introduced by it will not require reindexing of prod
  • C06 This PR is labeled reindex:partial and its description documents the specific reindexing procedure for dev, anvildev, anvilprod and prod or requires a full reindex or carries none of the labels reindex:dev, reindex:anvildev, reindex:anvilprod and reindex:prod

Author (mirror)

  • D01 This PR is labeled mirror:dev or the changes introduced by it will not require mirroring of dev
  • D02 This PR is labeled mirror:anvildev or the changes introduced by it will not require mirroring of anvildev
  • D03 This PR is labeled mirror:anvilprod or the changes introduced by it will not require mirroring of anvilprod
  • D04 This PR is labeled mirror:prod or the changes introduced by it will not require mirroring of prod
  • D05 This PR is labeled mirror:partial and its description documents the specific mirroring procedure for dev, anvildev, anvilprod and prod or requires a full mirroring or carries none of the labels mirror:dev, mirror:anvildev, mirror:anvilprod and mirror:prod

Author (API changes)

  • E01 This PR and its linked issues are labeled API or this PR does not modify a REST API
  • E02 Added a (A) tag to commit title for backwards (in)compatible changes or this PR does not modify a REST API
  • E03 Updated REST API version number in app.py or this PR does not modify a REST API

Author (upgrading deployments)

  • F01 Ran make docker_images.json and committed the resulting changes or this PR does not modify azul_docker_images, or any other variables referenced in the definition of that variable
  • F02 Documented upgrading of deployments in UPGRADING.rst or this PR does not require upgrading deployments
  • F03 Added u tag to commit title or this PR does not require upgrading deployments
  • F04 This PR is labeled upgrade or does not require upgrading deployments
  • F05 This PR is labeled deploy:shared or does not modify docker_images.json, and does not require deploying the shared component for any other reason
  • F06 This PR is labeled deploy:gitlab or does not require deploying the gitlab component
  • F07 This PR is labeled deploy:runner or does not require deploying the runner image

Author (hotfixes)

  • G01 Added F tag to main commit title or this PR does not include permanent fix for a temporary hotfix
  • G02 Reverted the temporary hotfixes for any linked issues or the none of the stable branches (anvilprod and prod) have temporary hotfixes for any of the issues linked to this PR

Author (before every review)

  • H01 Rebased PR branch on develop, squashed fixups from prior reviews
  • H02 Ran make requirements_update or this PR does not modify pyproject.toml
  • H03 Added R tag to commit title or this PR does not modify uv.lock
  • H04 This PR is labeled reqs or does not modify uv.lock
  • H05 make integration_test passes in personal deployment or this PR does not modify functionality that could affect the IT outcome
  • H06 PR is awaiting requested review from a peer
  • H07 Status of PR is Review requested
  • H08 PR is assigned to only the peer and the author

Peer reviewer (after approval)

Note that after requesting changes, the PR must be assigned to only the author.

  • J01 Actually approved the PR
  • J02 PR is not a draft
  • J03 PR is awaiting requested review from system administrator
  • J04 Status of PR is Review requested
  • J05 PR is assigned to only the system administrator and the author

System administrator (after approval)

  • K01 Actually approved the PR
  • K02 Labeled linked issues as demo or no demo
  • K03 Commented on linked issues about demo expectations or all linked issues are labeled no demo
  • K04 Decided if PR can be labeled no sandbox
  • K05 A comment to this PR details the completed security design review
  • K06 PR title is appropriate as title of merge commit
  • K07 N reviews label is accurate
  • K08 Status of PR is Approved
  • K09 PR is assigned to only the operator and the author

Operator

  • L01 Checked reindex:… labels and r commit title tag
  • L02 Checked mirror:… labels
  • L03 Checked that demo expectations are clear or all linked issues are labeled no demo
  • L04 Squashed PR branch and rebased onto develop
  • L05 Sanity-checked history
  • L06 Pushed PR branch to GitHub

Operator (deploy .shared and .gitlab components)

  • M01 Ran _select dev.shared && CI_COMMIT_REF_NAME=develop make -C terraform/shared apply_keep_unused or this PR is not labeled deploy:shared
  • M02 Ran _select dev.gitlab && CI_COMMIT_REF_NAME=develop make -C terraform/gitlab apply(an error from _login_docker_gitlab is benign if the instance was stopped for backup) or this PR is not labeled deploy:gitlab
  • M03 Ran _select anvildev.shared && CI_COMMIT_REF_NAME=develop make -C terraform/shared apply_keep_unused or this PR is not labeled deploy:shared
  • M04 Ran _select anvildev.gitlab && CI_COMMIT_REF_NAME=develop make -C terraform/gitlab apply(an error from _login_docker_gitlab is benign if the instance was stopped for backup) or this PR is not labeled deploy:gitlab
  • M05 Checked the items in the next section or this PR is labeled deploy:gitlab
  • M06 PR is assigned to only the system administrator and the author or this PR is not labeled deploy:gitlab

System administrator (post-deploy of .gitlab component)

  • N01 Background migrations for dev.gitlab are complete or this PR is not labeled deploy:gitlab
  • N02 Background migrations for anvildev.gitlab are complete or this PR is not labeled deploy:gitlab
  • N03 PR is assigned to only the operator and the author

Operator (deploy runner image)

  • P01 Ran _select dev.gitlab && make -C terraform/gitlab/runner or this PR is not labeled deploy:runner
  • P02 Ran _select anvildev.gitlab && make -C terraform/gitlab/runner or this PR is not labeled deploy:runner

Operator (sandbox build)

  • Q01 Added sandbox label or PR is labeled no sandbox
  • Q02 Pushed PR branch to GitLab dev or PR is labeled no sandbox
  • Q03 Pushed PR branch to GitLab anvildev or PR is labeled no sandbox
  • Q04 Build passes in sandbox deployment or PR is labeled no sandbox
  • Q05 Build passes in anvilbox deployment or PR is labeled no sandbox
  • Q06 Reviewed build logs for anomalies in sandbox deployment or PR is labeled no sandbox
  • Q07 Reviewed build logs for anomalies in anvilbox deployment or PR is labeled no sandbox
  • Q08 Applied upgrade instructions from UPGRADING.rst to sandbox or this PR is not labeled upgrade, or upgrade instructions do not apply to sandbox
  • Q09 Applied upgrade instructions from UPGRADING.rst to anvilbox or this PR is not labeled upgrade, or upgrade instructions do not apply to anvilbox
  • Q10 In sandbox, deleted the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:dev label, or both
  • Q11 In anvilbox, deleted the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:anvildev label, or both
  • Q12 In sandbox, deindexed the sources sepcified in the notes or this PR is missing either the reindex:partial or the reindex:dev label, or both
  • Q13 In anvilbox, deindexed the sources sepcified in the notes or this PR is missing either the reindex:partial or the reindex:anvildev label, or both
  • Q14 In sandbox, indexed the sources specified in the notes or this PR is missing either the reindex:partial or the reindex:dev label, or both
  • Q15 In anvilbox, indexed the sources specified in the notes or this PR is missing either the reindex:partial or the reindex:anvildev label, or both
  • Q16 In sandbox, indexed the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:dev label, or both
  • Q17 In anvilbox, indexed the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:anvildev label, or both
  • Q18 Started full reindex in sandbox or this PR is not labeled reindex:dev or it is labeled reindex:partial
  • Q19 Started full reindex in anvilbox or this PR is not labeled reindex:anvildev or it is labeled reindex:partial
  • Q20 Checked for failures in sandbox or this PR is not labeled reindex:dev
  • Q21 Checked for failures in anvilbox or this PR is not labeled reindex:anvildev
  • Q22 Started mirroring in sandbox or this PR is not labeled mirror:dev
  • Q23 Started mirroring in anvilbox or this PR is not labeled mirror:anvildev
  • Q24 Checked for failures in sandbox or this PR is not labeled mirror:dev
  • Q25 Checked for failures in anvilbox or this PR is not labeled mirror:anvildev

Operator (merge the branch)

  • R01 All status checks passed and the PR is mergeable
  • R02 The title of the merge commit starts with the title of this PR
  • R03 Added PR # reference to merge commit title
  • R04 Collected commit title tags in merge commit title but only included p if the PR is also labeled partial
  • R05 Pushed merge commit to GitHub
  • R06 Status of PR is Merged lower
  • R07 Status of blocked issues is Triage or no issues are blocked on the linked issues

Operator (main build)

  • S01 Pushed merge commit to GitLab dev
  • S02 Pushed merge commit to GitLab anvildev
  • S03 Build passes on GitLab dev
  • S04 Reviewed build logs for anomalies on GitLab dev
  • S05 Build passes on GitLab anvildev
  • S06 Reviewed build logs for anomalies on GitLab anvildev
  • S07 Applied upgrade instructions from UPGRADING.rst to dev or this PR is not labeled upgrade, or upgrade instructions do not apply to dev
  • S08 Applied upgrade instructions from UPGRADING.rst to anvildev or this PR is not labeled upgrade, or upgrade instructions do not apply to anvildev
  • S09 Notified developers to apply upgrade instructions from UPGRADING.rst to their personal deployments or this PR is not labeled upgrade, or upgrade instructions do not apply to personal deployments
  • S10 Ran _select dev.shared && make -C terraform/shared apply or this PR is not labeled deploy:shared
  • S11 Ran _select anvildev.shared && make -C terraform/shared apply or this PR is not labeled deploy:shared
  • S12 Deleted PR branch from GitHub
  • S13 PR is assigned to only the operator
  • S14 Deleted PR branch from GitLab dev
  • S15 Deleted PR branch from GitLab anvildev
  • S16 Status of linked issues is Lower, or Triage, if PR is partial

Operator (reindex)

  • T01 In dev, deleted the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:dev label, or both
  • T02 In anvildev, deleted the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:anvildev label, or both
  • T03 In dev, deindexed the sources sepcified in the notes or this PR is missing either the reindex:partial or the reindex:dev label, or both
  • T04 In anvildev, deindexed the sources sepcified in the notes or this PR is missing either the reindex:partial or the reindex:anvildev label, or both
  • T05 In dev, indexed the sources specified in the notes or this PR is missing either the reindex:partial or the reindex:dev label, or both
  • T06 In anvildev, indexed the sources specified in the notes or this PR is missing either the reindex:partial or the reindex:anvildev label, or both
  • T07 In dev, indexed the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:dev label, or both
  • T08 In anvildev, indexed the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:anvildev label, or both
  • T09 Started full reindex in dev or this PR is not labeled reindex:dev or it is labeled reindex:partial
  • T10 Started full reindex in anvildev or this PR is not labeled reindex:anvildev or it is labeled reindex:partial
  • T11 Checked for, triaged and possibly requeued messages in both fail queues in dev or this PR is not labeled reindex:dev or it is labeled reindex:partial
  • T12 Checked for, triaged and possibly requeued messages in both fail queues in anvildev or this PR is not labeled reindex:anvildev or it is labeled reindex:partial
  • T13 Emptied fail queues in dev or this PR is not labeled reindex:dev or it is labeled reindex:partial
  • T14 Emptied fail queues in anvildev or this PR is not labeled reindex:anvildev or it is labeled reindex:partial
  • T15 Restarted the Data Browser pipeline for the ucsc/hca/dev branch on GitLab in dev, and it succeeded or this PR is not labeled reindex:dev
  • T16 Restarted the Data Browser pipeline for the ucsc/lungmap/dev branch on GitLab in dev, and it succeeded or this PR is not labeled reindex:dev
  • T17 Restarted deploy_browser job in the GitLab pipeline for this PR in dev, and it succeeded or this PR is not labeled reindex:dev
  • T18 Restarted the Data Browser pipeline for the ucsc/anvil/anvildev branch on GitLab in anvildev, and it succeeded or this PR is not labeled reindex:anvildev
  • T19 Restarted deploy_browser job in the GitLab pipeline for this PR in anvildev, and it succeeded or this PR is not labeled reindex:anvildev

Operator (mirroring)

  • U01 Started mirroring in dev or this PR is not labelled mirror:dev
  • U02 Started mirroring in anvildev or this PR is not labelled mirror:anvildev
  • U03 Checked for, triaged and possibly requeued messages in mirror fail queue in dev or this PR is not labelled mirror:dev
  • U04 Checked for, triaged and possibly requeued messages in mirror fail queue in anvildev or this PR is not labelled mirror:anvildev
  • U05 Emptied mirror fail queue in dev or this PR is not labelled mirror:dev
  • U06 Emptied mirror fail queue in anvildev or this PR is not labelled mirror:anvildev

Operator

  • V01 Propagated the upgrade, API, deploy:shared, deploy:gitlab, deploy:runner, reindex:partial, reindex:anvilprod, reindex:prod, mirror:partial, mirror:anvilprod and mirror:prod labels to any open promotion PRs or this PR carries none of these labels, or is not included in an open promotion PR
  • V02 Propagated any specific instructions related to those labels, from the description of this PR to that of any open promotion PRs or this PR carries none of those labels, or is not included in an open promotion PR
  • V03 PR is assigned to no one

Shorthand for review comments

  • L line is too long
  • W line wrapping is wrong
  • Q bad quotes
  • F other formatting problem

@hannes-ucsc hannes-ucsc self-assigned this Oct 6, 2026
@hannes-ucsc hannes-ucsc linked an issue Oct 6, 2026 that may be closed by this pull request
@hannes-ucsc hannes-ucsc changed the title Simplify switching between installed AWS CLI v2 versions (#8361) Various fixes and improvements (#8361) Oct 6, 2026
@hannes-ucsc hannes-ucsc linked an issue Oct 6, 2026 that may be closed by this pull request
@hannes-ucsc hannes-ucsc changed the title Various fixes and improvements (#8361) Various fixes and improvements (#8361, #8367) Oct 6, 2026
@hannes-ucsc

Copy link
Copy Markdown
Member Author

(Posted by Claude Code)

The title of this PR matches neither linked issue because the PR is an omnibus resolving both #8361 and #8367, whose titles are unrelated to each other. Each linked issue is resolved by the commit whose title references it.

@codecov

codecov Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.27778% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.45%. Comparing base (2a07043) to head (0a35fc8).

Files with missing lines Patch % Lines
src/azul/terra.py 0.00% 3 Missing ⚠️
test/integration_test.py 0.00% 3 Missing ⚠️
src/azul/oauth2.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8372      +/-   ##
===========================================
+ Coverage    85.42%   85.45%   +0.03%     
===========================================
  Files          171      171              
  Lines        26423    26479      +56     
===========================================
+ Hits         22571    22628      +57     
+ Misses        3852     3851       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coveralls

coveralls commented Oct 6, 2026 •

Copy link
Copy Markdown

Coverage Status

coverage: 85.524% (+0.04%) from 85.489% — issues/hannes-ucsc/8361-et-al-omnibus into develop

@hannes-ucsc hannes-ucsc changed the title Various fixes and improvements (#8361, #8367) Various fixes and improvements (#8361, #8367, DataBiosphere/azul-private#431) Oct 6, 2026
@hannes-ucsc
hannes-ucsc force-pushed the issues/hannes-ucsc/8361-et-al-omnibus branch 2 times, most recently from ce523cc to 1a672f5 Compare October 8, 2026 01:25
@hannes-ucsc hannes-ucsc changed the title Various fixes and improvements (#8361, #8367, DataBiosphere/azul-private#431) Simplify switching between installed AWS CLI v2 versions (#8361, #8367, DataBiosphere/azul-private#431) Oct 8, 2026
@hannes-ucsc hannes-ucsc changed the title Simplify switching between installed AWS CLI v2 versions (#8361, #8367, DataBiosphere/azul-private#431) Various fixes and improvements (#8361, #8367, DataBiosphere/azul-private#431) Oct 8, 2026
@hannes-ucsc
hannes-ucsc force-pushed the issues/hannes-ucsc/8361-et-al-omnibus branch from 1a672f5 to 192f8ab Compare October 8, 2026 03:56
@hannes-ucsc hannes-ucsc linked an issue Oct 8, 2026 that may be closed by this pull request
@hannes-ucsc hannes-ucsc changed the title Various fixes and improvements (#8361, #8367, DataBiosphere/azul-private#431) Various fixes and improvements (#8361, #8367, DataBiosphere/azul-private#431, #8381) Oct 8, 2026
@hannes-ucsc hannes-ucsc changed the title Various fixes and improvements (#8361, #8367, DataBiosphere/azul-private#431, #8381) Various fixes and improvements (#8361, #8367, DataBiosphere/azul-private#431, #8381, #8375) Oct 10, 2026
@hannes-ucsc
hannes-ucsc force-pushed the issues/hannes-ucsc/8361-et-al-omnibus branch 2 times, most recently from 762ee8a to c59dcf0 Compare October 14, 2026 19:06
@hannes-ucsc
hannes-ucsc force-pushed the issues/hannes-ucsc/8361-et-al-omnibus branch from c59dcf0 to 985464d Compare October 22, 2026 00:38
@hannes-ucsc

hannes-ucsc commented Oct 22, 2026 •

Copy link
Copy Markdown
Member Author

(Posted by Claude Code)

Security design review

  • Security design review completed; this PR does not …
    • … affect authentication; for example:
      • OAuth 2.0 with the application (API or Swagger UI)
      • Authentication of developers with Google Cloud APIs
      • Authentication of developers with AWS APIs
      • Authentication with a GitLab instance in the system
      • Password and 2FA authentication with GitHub
      • API access token authentication with GitHub
      • Authentication with Terra
    • … affect the permissions of internal users like access to
      • Cloud resources on AWS and GCP
      • GitLab repositories, projects and groups, administration
      • an EC2 instance via SSH
      • GitHub issues, pull requests, commits, commit statuses, wikis, repositories, organizations
    • … affect the permissions of external users like access to
      • TDR snapshots
    • … affect permissions of service or bot accounts
      • Cloud resources on AWS and GCP
    • … affect audit logging in the system, like
      • adding, removing or changing a log message that represents an auditable event
      • changing the routing of log messages through the system
    • … affect monitoring of the system
    • … introduce a new software dependency like
      • Python packages on PYPI
      • Command-line utilities
      • Docker images
      • Terraform providers
    • … add an interface that exposes sensitive or confidential data at the security boundary
    • … affect the encryption of data at rest
    • … require persistence of sensitive or confidential data that might require encryption at rest
    • … require unencrypted transmission of data within the security boundary
    • … affect the network security layer; for example by
      • modifying, adding or removing firewall rules
      • modifying, adding or removing security groups
      • changing or adding a port a service, proxy or load balancer listens on
  • Documentation on any unchecked boxes is provided in comments below

Authentication

All three changes are restrictive; the set of accepted credentials does not grow.

  • Unusable credentials. lambdas/service/app.py:268 and src/azul/auth.py:65 turn the last two BadRequestError outcomes into UnauthorizedError, with a constant message in place of the echoed header.

  • Per-controller APAT acceptance. src/azul/chalice.py:983 adds Controller.accepts_personal_access_token, default False, enforced with a 401 at src/azul/chalice.py:993. Only RepositoryController (src/azul/service/repository_controller.py:85) and UserController (src/azul/service/user_controller.py:72) opt in, so a controller added later fails closed. An APAT sent to /index/summary previously reached SAM as a bearer token.

  • Invalid access tokens. src/azul/oauth2.py:195 raises the new InvalidAccessTokenError for the authorization server's 400; any other non-200 still asserts into a 5xx. Both raise sites are handled, at src/azul/terra.py:681 and src/azul/service/user_controller.py:350.

Audit logging

Two WARNINGs are added, at src/azul/service/user_controller.py:364 and src/azul/terra.py:681, both interpolating the Authentication with %s and so redacted by src/azul/auth.py:73.

The removed header echo matters more. src/azul/chalice.py:521 logs response bodies under pattern-based redaction only, unlike the unconditional redaction of the Authorization header at src/azul/http.py:83, so a credential matching no known pattern — Basic …, or a token from another system — reached CloudWatch in clear text. test_unusable_authorization_header now asserts its absence.

Monitoring

An invalid access token on /user/token or /user/revoke used to produce a 500 and trip the 5xx alarm; it now produces a 401, with test_unexpected_token_info_response keeping the alarm for an unanticipated authorization-server response. A burst of invalid-token traffic therefore no longer raises one, and /user/token mints a 30-day APAT. A metric filter on authentication failures would keep a spike visible — follow-up, not an objection.

New software dependencies

GitHub actions. .github/workflows/ci.yml pins every action to a commit SHA, which upstream cannot repoint — a real supply-chain improvement. Several pins cross major versions in passing (actions/checkout v4 → v7.0.1, github/codeql-action v3 → v4.38.2, among others). scripts/upgrade_actions.py adopts the highest fully specified version tag and validates only the SHA's shape (scripts/upgrade_actions.py:155), so the safeguard is the operator following the compare URL it logs, as .github/ISSUE_TEMPLATE/upgrade.md calls for. Two smaller points: the codeql-action wrapper no longer self-updates, though its bundle is still fetched at run time; and the new .github/dependabot.yml entry is a thin net between upgrades, since advisories for actions are sparse.

awsclienv. README.md:197 recommends awsclienv for installing AWS CLI v2: a new command-line dependency, hosted under an individual account, unpinned, installing a tool that handles AWS credentials. make check_awscli verifies the version, not the payload; the tool is optional, macOS-only and confined to the home directory. Worth stating: whether it verifies AWS's published installer signature, and whether the repository is headed for DataBiosphere.

Items not flagged, with reasons

  • Permissions of external users. Unchanged; RepositoryController mediates TDR snapshot access, opts in, and behaves as before.

  • Routes that do not authenticate at all. An APAT sent to /index/catalogs, /health/… or /version is accepted and ignored, since Controller._authentication is consulted only by the user, repository, service index, manifest and indexer index controllers. Not a gap: those routes ignore every credential equally, so Controllers not supporting APATs don't reject them early #8367's failure mode cannot arise, and enforcing the check in AzulChaliceApp.__authenticate would 401 an APAT on /health/basic while still ignoring an access token there. DRS is not an unauthenticated data path either: its routes are gated on config.is_dss_enabled() (src/azul/service/drs_controller.py:110), unregistered when that is false (src/azul/chalice.py:374), and no deployment configures the dss plugin.

  • Network security layer. The IT's local server moves to an ephemeral port (test/integration_test.py:2407), still on 127.0.0.1 and alive only for the test.

  • Interface exposing sensitive data. None added; the only move in this direction is the removal of the credential echo.

Findings

Severities rate security impact, not effort: informational for conformance or consistency with no security consequence, low for a defect that degrades a security or compliance control without granting access, medium and up for anything that exposes a credential or widens access. Resolved findings carry the severity they had when posted. Nothing in this PR reaches medium; the pre-existing defect it closes — a non-matching credential reaching CloudWatch in clear text, under Audit logging above — would have.

  1. OpenSearchMapper can silently drop a domain from the FedRAMP inventory. Low; resolved. As first posted:

    src/azul/compliance/fedramp_inventory_service.py:168 claims AWS::Elasticsearch::Domain in order to keep DefaultMapper from emitting a generic row, but map emits nothing for that type. The correctness of the resulting inventory therefore depends on AWS Config reporting every domain under both types. If that ever stops being true — an older domain, a change on the AWS side — the domain disappears from the inventory entirely, with no error.

    The route taken is better than the one suggested at the time: _supported_resource_types names only AWS::OpenSearch::Domain (src/azul/compliance/fedramp_inventory_service.py:186) and the deprecated type is not discovered at all, so DefaultMapper never sees it. The new Mapper.validate_types hook (src/azul/compliance/fedramp_inventory_service.py:127) asserts that every supported type is discovered, catching a mapper gone dead, and OpenSearchMapper overrides it to assert the converse. For the record, that guards double counting rather than under-reporting, but documenting the assumption is a reasonable stopping point.

  2. 401 responses carry no WWW-Authenticate header. Informational; open. RFC 9110 §15.5.2 requires one. Azul sets that header nowhere outside src/azul/terra.py:319, where it is read from Terra rather than sent. This predates the PR, which widens the omission slightly by converting two former 400s into 401s. Low severity, and arguably a separate issue.

  3. /user/token answers an APAT with 400, not 401. Informational; resolved. As first posted:

    src/azul/service/user_controller.py:330 raises BadRequestError('Cannot exchange a personal access token for another'), and /user/revoke does the same. Since this PR is normalizing credential-related status codes, it is worth deciding whether these should remain 400 or become 403 — the credential is valid, the operation is not available to it, which is the textbook case for 403.

    Both endpoints now raise ForbiddenError, at src/azul/service/user_controller.py:343 and src/azul/service/user_controller.py:354, with the documented responses updated to match at src/azul/service/user_controller.py:200 and src/azul/service/user_controller.py:237, and both status assertions in test/service/test_user_controller.py changed to 403.

@hannes-ucsc
hannes-ucsc force-pushed the issues/hannes-ucsc/8361-et-al-omnibus branch from 985464d to 373aa7d Compare October 22, 2026 16:09
hannes-ucsc and others added 6 commits October 22, 2026 11:24
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ere/azul-private#431)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A version tag, even a fully specified one, can be moved by the project that
publishes the action, so a commit is the only reference that is fixed. Add a
script that rewrites every pin in the workflows to the commit behind the most
recent fully specified version tag, and a checklist item that runs it as part
of the biweekly upgrade. The next commit is the result of running it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The result of running `python scripts/upgrade_actions.py`. Besides replacing
each tag with the commit behind it, this updates `setup-uv` from v10.0.1 to
v10.2.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hannes-ucsc and others added 5 commits October 22, 2026 11:24
Routine version updates remain suppressed, as they are for `uv`; the pins are
maintained by `scripts/upgrade_actions.py` during the biweekly upgrade.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ser/revoke (#8375)

Both endpoints extend the defect to tokens issued to another client and to
users that never completed the authorization code flow.

A 400 from the `tokeninfo` endpoint is a condition we expect and know how to
report, so `token_info` raises a dedicated exception for it instead of
failing an assertion. A response we aren't prepared to interpret keeps its
assertion, and the 5xx it causes, since that mismatch crosses the boundary to
Google rather than indicating an inconsistency of our own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The message of the raised `UnauthorizedError` repeated the header, so the
response carried the client's own credentials back to it.

A bearer token of unexpected syntax yielded 400, at two sites, where RFC 6750
§3.1 classes a malformed token under `invalid_token`, to be answered with a
401.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The server bound port 8000, so a second, concurrent run of the IT failed in
`setUpClass` with `Address already in use`. That port is also the default of
`chalice local`, which `make -C lambdas/service local` runs, so a locally
served app collided with the IT just the same.

The operating system now picks the port, which is read back from the bound
socket, as `LocalAppTestCase` already does. The endpoint is therefore only
known once the server exists, leaving `url` a declaration. The `None` that
`server` and `server_thread` were initialized with was never observed, since
`tearDownClass` is skipped when `setUpClass` raises, so they become
declarations, too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The credential is valid and was authenticated; the operation is merely not
available to it. Both endpoints document the status now, as they already did
the 401.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@achave11-ucsc
achave11-ucsc force-pushed the issues/hannes-ucsc/8361-et-al-omnibus branch from 373aa7d to 0a35fc8 Compare October 22, 2026 18:26
@achave11-ucsc achave11-ucsc added the sandbox [process] Resolution is being verified in sandbox deployment label Oct 22, 2026
@achave11-ucsc
achave11-ucsc merged commit 2cfba11 into develop Oct 22, 2026
10 checks passed
@achave11-ucsc
achave11-ucsc deleted the issues/hannes-ucsc/8361-et-al-omnibus branch October 22, 2026 21:25
@achave11-ucsc achave11-ucsc removed their assignment Oct 22, 2026
@hannes-ucsc

hannes-ucsc commented Oct 23, 2026 •

Copy link
Copy Markdown
Member Author

Finding 2 is already tracked by #8385

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sandbox [process] Resolution is being verified in sandbox deployment

Projects

None yet

3 participants