Check SQLite readiness and qualify Windows startup

.github/workflows/readiness.yml

@@ -0,0 +1,65 @@
+name: Bounded readiness qualification
+
+on:
+  workflow_dispatch:
+  pull_request:
+    branches: [main, master]
+  push:
+    branches: [main, master]
+
+permissions:
+  contents: read
+
+jobs:
+  readiness:
+    name: Readiness / ${{ matrix.os }} / Node 22
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 8
+    defaults:
+      run:
+        shell: bash
+    steps:
+      - name: Initialize isolated runner environment
+        # runner context is available in step env, not job env.
+        env:
+          npm_config_cache: ${{ runner.temp }}/readiness-cache
+          HOME: ${{ runner.temp }}/readiness-home
+          TMPDIR: ${{ runner.temp }}
+          TMP: ${{ runner.temp }}
+          TEMP: ${{ runner.temp }}
+          TPC_RUNTIME_FIXTURE_ROOT: ${{ runner.temp }}/readiness-probes
+          READINESS_FIXTURE_PARENT: ${{ runner.temp }}/readiness-qualification
+        run: |
+          for name in npm_config_cache HOME TMPDIR TMP TEMP TPC_RUNTIME_FIXTURE_ROOT READINESS_FIXTURE_PARENT; do
+            printf '%s=%s\n' "$name" "${!name}" >> "$GITHUB_ENV"
+          done
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+      - name: Locked dependencies without lifecycle scripts
+        run: npm ci --ignore-scripts --no-audit --no-fund --fetch-retries=0 --fetch-timeout=20000
+      - name: Provision only sqlite3 via inspected package-supported N-API route
+        # No source-build fallback or other native packages. Missing prebuilt fails closed.
+        run: node -e 'const p=require("node:path");const r=require("node:child_process").spawnSync(process.execPath,[require.resolve("prebuild-install/bin.js"),"-r","napi"],{cwd:p.dirname(require.resolve("sqlite3/package.json")),stdio:"inherit",timeout:60000,killSignal:"SIGKILL"});process.exit(r.error||r.signal||r.status===null?1:r.status)'
+      - name: Offline runner regression probes
+        run: node --test --test-timeout=10000 scripts/readiness-ci.test.cjs
+      - name: Actual unchanged readiness contracts and native integration
+        # Write only sanitized runner output, not install logs, DBs or native files.
+        run: |
+          mkdir -p readiness-results
+          node scripts/readiness-ci.cjs "$READINESS_FIXTURE_PARENT" > readiness-results/readiness.log 2>&1
+      - name: Upload concise readiness result only
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: readiness-${{ matrix.os }}-node22
+          path: readiness-results/readiness.log
+          if-no-files-found: ignore
+          retention-days: 7

README.md

@@ -15,14 +15,18 @@ See [CHANGELOG.md](CHANGELOG.md) for detailed release notes.
 Run `npm test` (or `npm run test:unit`) to execute Jest API/integration tests.
 Run `npm run test:e2e` for Playwright UI tests.
 Run `npm run smoke:mcp` for deterministic MCP plan/thought lifecycle smoke validation.
+Run `npm run test:startup-contract` for dependency-free, injected offline startup/readiness tests.
 
 ### Production Security
 `/tpc.db` download is disabled in production by default.
 To explicitly allow it, set `EXPOSE_TPC_DB=true`.
 
 ### Operational Endpoints
 - `GET /health` returns basic liveness info.
-- `GET /ready` returns readiness info and DB path.
+- `GET /ready` returns readiness info and DB path after a successful database query, or 503 when unavailable.
+
+See the [startup/readiness contract](docs/startup-readiness.md) for initialization,
+failure behavior, test boundaries, and limitations.
 
 ### Data Contract (REST + MCP)
 - **IDs**: integer autoincrement IDs from SQLite for plans/thoughts.

docs/example-cache-contract.md

@@ -0,0 +1,31 @@
+# Tool example timestamp-cache contract
+
+`/tools/examples` validates the tool name before accessing a module-wide cache
+keyed by the exact tool name. It retains the first timestamp for that key; it
+still constructs the examples response on every request. There is no TTL,
+expiry, invalidation or whole-response caching guarantee.
+
+`routes/exampleTimestampCache.js` extracts the existing behavior into a factory
+with injectable cache, timestamp clock and delay. Production retains one
+module-wide instance (including across `createApp` calls), the same ISO timestamp
+payload and the existing artificial blocking delay: 40 ms on a miss, 1 ms on a
+hit. Separate factory calls have independent default maps. This is a testability
+seam, not a cache redesign or delay optimization.
+
+The previous cache ratio test ran after earlier tests had requested the same
+key, so its purported cold/hot comparison was normally hit/hit. Two tiny
+wall-clock HTTP samples cannot establish cache correctness or real-world speed.
+The isolated contracts in `tests/test_example_timestamp_cache.js` instead assert
+storage, timestamp retention as controlled time advances, distinct-key isolation,
+prewarmed hits, factory isolation and exact delay-branch selection. Run without
+dependencies using:
+
+```sh
+node --test tests/test_example_timestamp_cache.js
+```
+
+These contracts also run under the existing Jest test discovery. The performance
+suite retains HTTP timestamp-retention coverage; it makes no TTL claim. Other
+performance assertions are unchanged. Deterministic branch tests do not measure
+or promise real-world speed, throughput or latency. Any such requirement needs
+a separately defined controlled benchmark, not a relaxed ratio threshold.

docs/readiness-platform-ci.md

@@ -0,0 +1,100 @@
+# Bounded readiness qualification (candidate, not platform acceptance)
+
+`.github/workflows/readiness.yml` is separate from existing CI, which is unchanged.
+It proposes Ubuntu and native Windows hosted runners on Node 22, with finite
+8-minute jobs. No secrets, deployments, write permissions, `pull_request_target`,
+readiness skips or `continue-on-error` are used. Checkout does not retain credentials.
+
+## Installation boundary
+
+`npm ci --ignore-scripts` honors the existing lockfile, with lifecycle scripts
+(including other native dependencies and browser downloads) disabled. The only
+explicit native provisioning is the installed `prebuild-install/bin.js -r napi`,
+run in the installed sqlite3 package directory with a 60-second deadline. This is
+the first branch of sqlite3 5.1.7's inspected install script. It selects that
+package/version's upstream N-API prebuilt; there is no source-build fallback,
+ad-hoc executable, broad rebuild, or local binary copied into CI. Failure is a
+qualification failure, not permission to try unrelated packaging repairs.
+
+This upstream native asset is version-selected by the package but not independently
+checksum-pinned by package-lock.json. The runtime log records the loaded native
+SHA256, Node/OS/arch, Express, sqlite3 and SQLite versions. This is provenance,
+not independent supply-chain authentication. Existing local author receipts show
+a source build failed because Python lacked distutils, then this supported N-API
+route worked on Linux. That does not prove either hosted download will succeed.
+
+## Local isolated execution
+
+Use an absolute, private synthetic output directory, not a source/user data path:
+
+```sh
+env -i PATH=/opt/node-runtime/bin:/usr/bin:/bin \
+  HOME="$O/home" TMPDIR="$O/tmp" npm_config_cache="$O/cache" \
+  TPC_RUNTIME_FIXTURE_ROOT="$O/fixtures" \
+  timeout 35 node scripts/readiness-ci.cjs "$O/fixtures"
+
+env -i PATH=/opt/node-runtime/bin:/usr/bin:/bin \
+  HOME="$O/home" TMPDIR="$O/tmp" npm_config_cache="$O/cache" \
+  TPC_RUNTIME_FIXTURE_ROOT="$O/fixtures" \
+  timeout 15 node --test --test-timeout=10000 scripts/readiness-ci.test.cjs
+```
+
+The runner creates a unique synthetic root and a minimal child environment,
+including Windows system/path variables when present. It never inherits
+NODE_OPTIONS/NODE_PATH, credentials or CI/proxy variables. It runs only the two
+existing test files, byte-for-byte unchanged, in a Node child with a 25-second
+process deadline and 1 MiB output bound. Nonzero exits propagate; signals,
+spawn/output errors and deadlines fail closed. Parent finally removes its entire
+synthetic tree even after killing the child. This is not process-tree containment
+for arbitrary subprocesses, and hard termination of the parent cannot guarantee
+cleanup. Neither test starts MCP/provider children or the main server listener.
+
+The original integration harness uses a Windows-compatible directory junction,
+module-relative synthetic files, native SQLite and 127.0.0.1 ephemeral HTTP only.
+The six startup contracts and four native integration states are not weakened.
+The probe file tests allowlist behavior, actual child isolation, exit 7, a stalled
+child deadline, invalid relative parent rejection and success/failure cleanup.
+Additional regressions cover output overflow and signals, plus real bounded child
+stdout/stderr diagnostic redaction (including a spaces/Unicode synthetic parent).
+ANSI regressions cover static Windows roots and actual child stdout/stderr with
+exit 13 and cleanup; these fail against the previous sanitizer. Numeric/semicolon
+SGR sequences (`ESC [` followed by zero or more digits/semicolons and `m`) are
+recognized only as root boundaries, retained byte-for-byte in output. This narrow
+correction avoids treating the final `m` of a color prefix or the ESC of a reset
+as a path character; it does not strip colors or change subprocess handling.
+
+## Authorized-root handoff and acceptance
+
+Only after independent review and authorization, root may push the supplied
+branch or integrate its exact commit. This worker has not pushed or dispatched
+Actions. If using a non-default branch, ensure the workflow exists on the default
+branch before attempting manual `workflow_dispatch`, or use an authorized PR.
+Inspect the actual run and both distinct matrix jobs, not only workflow syntax:
+
+1. Confirm remote head equals the reviewed candidate SHA.
+2. Confirm locked install, sqlite3 provisioning, runner probes and native runtime
+   steps all succeed on **both** Ubuntu and Windows without skips.
+3. Download `readiness-ubuntu-latest-node22` and
+   `readiness-windows-latest-node22`. Inspect native provenance, 11 pass/0 fail,
+   no skips, uninitialized 503, initialized 200, closed-handle 503 and independent
+   health 200. Confirm synthetic root removal and compare exact tested SHA.
+4. If setup fails, inspect the Actions step output; no install log is uploaded.
+   An absent runtime artifact is not success. Artifacts contain only the sanitized
+   bounded test/provenance log, not DBs or binaries, and expire after seven days.
+   The runner redacts its known synthetic root in raw and forward-slash-normalized
+   forms, each also with doubled backslashes (Node inspection) or one JSON string
+   escaping layer. Descendant suffixes and unrelated diagnostic text are retained;
+   path-prefix/sibling lookalikes are not replaced. This is a finite literal-root
+   diagnostic filter, **not arbitrary secret or user-data scrubbing**. It does not
+   promise case-folded paths, mixed separator spellings, URL/HTML encodings,
+   repeated JSON encoding, truncated roots, or every inspection quoting/control
+   character convention. Other terminal sequences (including non-SGR CSI, OSC,
+   colon-form SGR and incomplete/malformed escapes) are not recognized framing.
+   Input must remain synthetic and environment-allowlisted; never upload arbitrary
+   program output on the assumption this filter makes it safe.
+
+Windows path/case-insensitive environment handling and junction support are
+static reasoning until the actual native Windows job succeeds. Local Linux
+verification is not Windows, hosted-CI, service, deployment or broad Jest proof.
+All existing approval/publication/ownership gates remain. Rollback is revert the
+workflow/runner/docs commit; production and dependency files are unchanged.

docs/readiness-runtime-validation.md

@@ -0,0 +1,56 @@
+# Offline readiness runtime validation
+
+This complements (does not replace) the six injected-boundary startup contract tests.
+It uses actual Express routing, the actual database module/migrations and native
+sqlite3 callback API. It never runs the default main entry point or calls providers.
+No production module or app API changes are needed.
+
+## Run
+
+Install the locked dependencies with lifecycle scripts disabled, then provision only
+sqlite3's inspected native binding (package-supported N-API prebuilt or a bounded
+source build). Do not run broad dependency lifecycle scripts. Use Node 22 or later
+compatible with the lockfile. Other native optional dependencies are not needed.
+
+Choose private, synthetic absolute output paths and run in an isolated environment:
+
+```sh
+env -i PATH="/path/to/node/bin:/usr/bin:/bin" \
+  HOME="$OUTPUT/home" TMPDIR="$OUTPUT/tmp" npm_config_cache="$OUTPUT/cache" \
+  NODE_ENV=test TPC_RUNTIME_FIXTURE_ROOT="$OUTPUT/fixtures" \
+  timeout 30 node --test scripts/readiness-runtime.test.cjs scripts/startup-contract.test.cjs
+```
+
+Create home/tmp/cache first. `TPC_RUNTIME_FIXTURE_ROOT` is a **test harness** setting,
+not a production database override. The database module has no supported data/path
+environment override. `initGlobalDB` uses its module-relative `data/tpc.db`; `initDB`
+only publishes the global handle when passed that exact path. Therefore the test
+copies only server.js, db/, routes/ and middleware/ byte-for-byte into a fresh
+fixture under the specified output, with a dependency symlink to this checkout's
+node_modules. It does not copy existing data, user JSON, tracked DBs or public assets.
+The copied server is required, never executed as main. Route imports create routers
+but do not initialize DBs, contact providers, or start listeners.
+
+`initGlobalDB(true)` performs real schema/index/FTS migration while skipping JSON
+imports. NODE_ENV=test suppresses backup creation. The test uses a real synthetic
+on-disk DB, not a mock or a misleading `:memory:` global handle. It checks:
+
+- uninitialized `/ready`: 503 with existing error shape;
+- `/health` before initialization: independent 200;
+- migrated database and real SELECT 1 callback: 200 with existing path response;
+- closed native handle: real SQLITE_MISUSE callback yields 503, health still 200.
+
+The only listener binds **127.0.0.1, ephemeral port 0** and is closed in finally;
+requests have a two-second timeout. The DB is closed, the fixture recursively removed,
+and removal asserted even after test failures. Use an outer finite process timeout
+as shown for protection against native initialization stalls. Run separate processes
+to obtain fresh module-global DB state.
+
+The existing default Jest suites are not claimed here. They use Supertest's default
+app binding (not this explicit loopback listener), and the stdio MCP contract spawns
+a separate MCP process. Running those unchanged exceeds this validation's listener/
+MCP containment. Playwright/e2e and performance/provider coverage are also excluded.
+No Windows or hosted CI evidence is implied. Authorized root should arrange independent
+review of the exact committed test diff and, if accepted, public-repository Windows
+Actions execution under existing ownership/approval gates. No remote write is needed
+for this local validation leaf.

docs/startup-readiness.md

@@ -0,0 +1,59 @@
+# Startup and readiness contract
+
+Run the dependency-free offline contract suite with Node's built-in test runner:
+
+```sh
+node --test scripts/startup-contract.test.cjs
+# equivalent: npm run test:startup-contract
+```
+
+These tests execute the actual `server.js` source in a fresh VM context, injecting
+Express, routers and the database boundary. They do not open sockets, load SQLite,
+read application data, run MCP, or use external services. They are not an Express,
+SQLite, HTTP integration, or Windows qualification test. The normal Jest suite
+remains separate (`npm test`) and requires installed dependencies.
+
+## Initialization
+
+- Importing `server.js` constructs and exports the global Express app but does not
+  initialize its database or call `listen`.
+- Running it as the main module awaits `initGlobalDB()` before `listen(PORT)`.
+  The promise represents backup preparation, opening SQLite and completion of
+  `performMigration`, not merely opening the file. Schema creation, alterations,
+  indexes, FTS triggers/rebuild, and baseline migration bookkeeping are awaited.
+- Historical JSON imports are **best effort**: their errors are caught and logged
+  inside `performMigration`. Initialization resolving does not certify all legacy
+  JSON was imported. `skipMigration=true` skips JSON import, not schema setup.
+- `getDB()` throws until global initialization has installed the handle.
+- Initialization rejection is logged and prevents the main module from listening.
+  This is **not** a clean shutdown guarantee: the current entry point does not set
+  a nonzero exit status, and migration failure after opening SQLite does not close
+  that handle. Those boundaries are not changed by the readiness repair.
+
+## Health versus readiness
+
+- `/health` is liveness only and returns 200 without a database check.
+- `/ready` returns 200 with `{ ready: true, db: <path> }` only after `getDB()`
+  succeeds and SQLite completes `SELECT 1` successfully. Missing initialization,
+  a closed database, or another query error returns 503 with `ready: false` and
+  the existing error-message field. Each request checks again; readiness is not
+  latched after first success.
+- This is point-in-time **connection/query readiness**, not proof of schema
+  integrity, writeability, imported records, or future query success. There is no
+  timeout: a query whose callback never arrives leaves the request pending.
+- The isolated `createApp()` factory waits for its own database before returning
+  and injects it on requests, but does not mount `/health` or `/ready`. These
+  endpoints describe the exported global app only.
+
+## Regression and scope
+
+Previously `/ready` only joined a path and returned success, even before global
+initialization or after a database query failure. The offline regression suite
+reproduces that behavior on the original source and checks the repaired result,
+including waiting for the query callback. Startup ordering and rejection tests
+also preserve the existing no-listen-before-initialization behavior.
+
+No bind address, port default, database permissions, schema, authentication,
+startup failure handling or production configuration was changed. Do not run the
+application against real data to exercise this suite. Revert the isolated commit
+to roll back; no data migration or runtime cleanup is needed for these tests.

package.json

@@ -5,6 +5,8 @@
   "scripts": {
     "test": "npm run test:unit",
     "test:unit": "cross-env NODE_ENV=test jest --runInBand",
+    "test:startup-contract": "node --test scripts/startup-contract.test.cjs",
+    "test:readiness-runtime": "node --test scripts/readiness-runtime.test.cjs",
     "test:e2e": "playwright test",
     "mcp": "node mcp-server.js",
     "smoke:mcp": "node scripts/mcp-smoke.js",

routes/exampleTimestampCache.js

@@ -0,0 +1,22 @@
+function sleepMs(ms) {
+  const start = Date.now();
+  while (Date.now() - start < ms) {}
+}
+
+// Timestamp retention and artificial delay selection, not a response cache or TTL.
+// Each factory owns fresh state unless a cache is explicitly supplied.
+function createExampleTimestampCache({
+  cache = new Map(),
+  now = () => new Date().toISOString(),
+  delay = sleepMs
+} = {}) {
+  return function getTimestamp(toolName) {
+    const cacheHit = cache.has(toolName);
+    const timestamp = cache.get(toolName) || now();
+    cache.set(toolName, timestamp);
+    delay(cacheHit ? 1 : 40);
+    return timestamp;
+  };
+}
+
+module.exports = { createExampleTimestampCache, sleepMs };