Persist gateway state and default API keys in SQLite - #38
maiphucgiang wants to merge 4 commits into
Conversation
Load optional native .env settings and retain a generated default key across restarts, with one-time terminal disclosure. Migrate runtime snapshots into the control database, keep audit logs separate, and retire text log output.
Reviewer's GuideThis PR upgrades gateway runtime persistence to a validated schema-2 SQLite control database, adds secure optional dotenv/default-key startup resolution with one-time disclosure, retires text logging in favor of redacted SQLite auditing, and updates tests, WebUI source indicators, dependencies, and bilingual operational documentation. Sequence diagram for startup key resolution and one-time disclosuresequenceDiagram
participant CLI as CLI
participant Startup as startup.py
participant Control as control.sqlite3
participant Server as Uvicorn
actor Operator
CLI->>Startup: load_startup_env()
Startup->>Startup: resolve_startup_key()
alt explicit CLI, process environment, or .env key
Startup-->>CLI: use configured key
else no configured key
Startup->>Control: default_key(create=True)
Control-->>Startup: saved cb- key, pending announcement
Startup-->>CLI: configure generated key
end
CLI->>Server: run_server()
Server-->>Startup: listener started
opt generated key pending
Startup->>Control: claim_announcement(key)
Control-->>Startup: claim succeeds once
Startup-->>Operator: display key on interactive terminal
end
Flow diagram for validated legacy state migrationflowchart TD
Start[Gateway startup] --> Open[Open control.sqlite3]
Open --> Migrate[StateStore.migrate]
Migrate --> Check{Import marker exists?}
Check -- yes --> Next[Skip legacy source]
Check -- no --> Read[Read bounded legacy JSON]
Read --> Validate[Validate schema and reject credentials]
Validate -->|valid or absent| Transaction[Write state and import marker]
Validate -->|invalid or failure| Rollback[Rollback transaction and stop startup]
Transaction --> Next
Next --> Complete[Start runtime services]
File-Level Changes
Assessment against linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 1 security issue, and 1 other issue
Security issues:
- Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option. (link)
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="app/admin_auth.py" line_range="223-227" />
<code_context>
def _revoke(self):
"""Revoke the persisted snapshot; returns False when it could not be cleared."""
+ if self._store is not None:
+ try:
+ self._store.delete("sessions")
+ except (OSError, ValueError, sqlite3.Error) as error:
+ self._storage_failed(error)
+ return False
+ self._storage_ok()
+ return True
</code_context>
<issue_to_address>
**issue (broader_impact):** SQLite read failures are not handled consistently by the migrated runtime caches: `AdminAuth._restore` converts them into session revocation, but `CredentialCooldowns._load`, `UsageSnapshots._load`, and the corresponding catalog/ledger loaders allow `sqlite3.Error` to escape during object construction. A damaged or temporarily locked control database therefore aborts startup in some cache paths while silently discarding sessions in another, rather than applying one documented failure policy.
**Triggers:** When `control.sqlite3` is locked, corrupt, or otherwise raises a SQLite error while a runtime snapshot is loaded.
**Suggested fix:** Define and apply a single startup policy for control-state read failures—preferably fail startup for authoritative state and explicitly degrade only rebuildable caches, with diagnostics.
</issue_to_address>
### Comment 2
<location path="app/control_store.py" line_range="113" />
<code_context>
self._db.execute(f"PRAGMA user_version={self.SCHEMA_VERSION}")
</code_context>
<issue_to_address>
**security (python.sqlalchemy.security.sqlalchemy-execute-raw-query):** Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.
*Source: opengrep*
</issue_to_address>Sourcery assessment
Needs a human reviewer. 2 findings to address first, and this changes the authentication trust boundary by replacing an absent key with a persisted, shared default key and one-time announcement, so a mistake can expose or lock out management and API access from the moment it ships. Reverting the code does not undo generated keys, migrated state, or SQLite-authoritative runtime data, so recovery requires deliberate state handling.
Blocking findings: app/admin_auth.py:227, app/control_store.py:113
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Preserve authoritative SQLite state on read failures and leave failed session epochs inactive for retry. Publish CodeQL results against the checked-out pull request head and use a literal schema-version statement.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e61ae221f9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if key is None and config["host"] not in ("127.0.0.1", "::1", "localhost"): | ||
| raise ValueError("非回环监听请显式设置 API key；默认密钥仅在本地首次启动时生成") |
There was a problem hiding this comment.
When CODEBUDDY2API_KEY is genuinely unset, a non-loopback launch now raises here before converter.main() reaches its existing CODEBUDDY2API_ALLOW_OPEN_NOAUTH check. This breaks the shipped image's default direct-run configuration: the Dockerfile binds 0.0.0.0 and sets CODEBUDDY2API_ALLOW_OPEN_NOAUTH=true, but does not define CODEBUDDY2API_KEY, so the container exits during initialization. Account for the explicit unsafe opt-in here, or ensure the image supplies an explicit empty-key source.
Useful? React with 👍 / 👎.
Use the existing unsafe opt-in for unconfigured headless launches while keeping saved and externally supplied keys authoritative.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 22778a6c28
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if config["state_store"].claim_announcement(key): | ||
| terminal.write(f"\n默认 API key：{key}\n已保存到 control.sqlite3，仅显示这一次；管理登录与 API 请求共用。\n") | ||
| terminal.flush() |
There was a problem hiding this comment.
When the controlling terminal disappears or write()/flush() fails after claim_announcement() commits, the server exits but the database already records the key as announced. On the next launch default_key() therefore returns pending=False, so the gateway starts with a generated API key that was never fully disclosed and cannot be recovered through the UI or standard output. Handle terminal-write failures by invalidating/resetting this generated key or otherwise restoring a recoverable pending state before shutdown.
Useful? React with 👍 / 👎.
Commit the announcement marker after terminal flush under an exclusive transaction. Roll back failed writes, interrupted disclosures and failed commits so the saved key remains recoverable. Document possible redisclosure after a crash between terminal output and commit.
2 participants
Closes #37.
Changes
uv run converter.pyandpython3 converter.pywithout a required.env. Optionally load the current directory's.env, with explicit CLI > process environment >.env> saved SQLite values; show locked configuration sources in the WebUI.cb-default API key only when needed, save it privately in SQLite, and display it once through the controlling terminal (/dev/ttyor WindowsCONOUT$) after the listener starts, never through stdout/stderr. Restarts reuse it without printing it again; external overrides do not replace the saved default. Key material stays out of persistent logs and public settings snapshots.control.sqlite3. Import legacy JSON once in a validated transaction, including markers for absent files, and preserve account ownership, session revocation, trial reservations/backoff and in-memory routing caches..infofiles and audit data in the separatelogs.sqlite3. Retire--log/CODEBUDDY2API_LOGtext output with an explicit warning, add generated-key redaction, and synchronize concise English/Chinese documentation.python-dotenvpackage as a direct dependency.Verification
python3/uvlaunches with PTYs and loopback HTTP: first-key disclosure, restart/session survival, dotenv and CLI precedence, restored defaults without revived sessions, log secrecy, headless rejection and occupied-port recovery.Scope and rollback
/v1/*and/admin/*addresses and explicit-empty-key compatibility are preserved.CODEBUDDY2API_ALLOW_OPEN_NOAUTH=truealso permits unconfigured headless inference (as in the shipped image); management stays locked, and an existing key remains authoritative. First-time headless/non-loopback deployments should configure a key explicitly; protect the control database and its backups because they contain the saved default key and management sessions.Summary by Sourcery
Persist gateway state and securely manage default API keys through SQLite-backed startup, migration, logging, and configuration handling.
New Features:
.envloading and clear configuration precedence..envand generated defaults in the WebUI.Bug Fixes:
Enhancements:
.infofiles while consolidating operational state and strengthening SQLite transaction and failure handling.Build:
python-dotenvas a direct dependency and update locked dependency metadata.CI:
Documentation:
Tests: