Skip to content

Add maintainer tool to validate Tesla field semantics against prod NATS#8

Closed
Bre77 wants to merge 1 commit into
mainfrom
fm/lib-schema-fixtures-p1
Closed

Add maintainer tool to validate Tesla field semantics against prod NATS#8
Bre77 wants to merge 1 commit into
mainfrom
fm/lib-schema-fixtures-p1

Conversation

@Bre77

@Bre77 Bre77 commented Jul 7, 2026

Copy link
Copy Markdown
Member

Intent

  • Ship Phase 1 of the executable-schema roadmap (api-tesla-schema-fixtures-n6): promote the one-off prod-cache validation recipe into a committed, reusable maintainer script.
    • The recipe (data/chargeport-null-validate-v4/report.md) settled charge_state.charge_port_door_open as a genuine tri-state (431/723 vehicles report explicit false) - proving the method works, but leaving nothing reusable behind.
    • This repo's own commit b72b9a9 (six streamed fields silently mistyped until hand-checked against na_cache) is the same failure mode on the streaming side.
  • Add scripts/tesla_cache_validate.py: a read-only CLI that queries the live na_cache/eu_cache NATS KV buckets to answer, per field: is it tri-state or true-or-absent, what value domain is actually observed, what's the presence rate across the fleet.
    • Covers all four record shapes found in those buckets: vehicle REST snapshots (<VIN>.vehicle_data), streamed Fleet Telemetry signals (<VIN>.data.<Signal>), and energy live_status/site_info (energy_sites.<id>.*) - both na_ and eu_ regions.
    • Bakes in the recipe's reliability guards: reads only via nats stream get --last-for ... -j (never nats kv get, which silently reads empty on these buckets), has-key presence classification (never truthiness), and a non-empty-read guard that refuses to emit a report if every fetch failed or decoded empty.
    • Sanitizes raw records by default (VIN/tokens/geo redacted, type-preserved) except for the maintainer's own reference vehicle/site (captain's ruling, 2026-07-07), and can optionally save sanitized records to seed the phase-2 fixtures corpus.
  • Implications:
    • Offline/read-only only: no write path exists in the script, and it is never invoked by CI (no prod NATS access there, none should be granted). It's meant to be run by hand by a maintainer with an authenticated nats context (name configurable via --context; no endpoint or credential is hardcoded).
    • Pure logic (field extraction, presence classification, sanitization) is unit-tested offline in tests/test_cache_validate_logic.py, safe for CI even though the tool itself isn't exercised there.
    • Each report ends with a provenance line ready to paste into a field's registry entry once phase 3 exists.
  • Explicit scope boundary: this PR is Phase 1 only. It does not implement the Field/Presence registry or the fixtures/ module (phases 2-3 of the roadmap) - those are separate, later PRs.

What Changed

  • scripts/tesla_cache_validate.py - the CLI tool (argparse; stdlib only, no new dependencies).
  • scripts/README.md - usage, the four record shapes, and the reliability guards.
  • tests/test_cache_validate_logic.py - offline unit tests for the pure logic (extraction, envelope unwrapping, sanitization, presence-shape inference, key discovery), modeled on the real corpus shape.
  • AGENTS.md / CLAUDE.md - new project memory file documenting the scripts/ convention (prod-touching, CI-exempt tooling) for future maintainers.

Risk Assessment

Low. Additive only; touches no existing package code. The script is read-only against prod (verified live during development - see Testing) and has no write path. Not wired into CI.

Testing

  • python3 tests/test_cache_validate_logic.py - all 21 assertions pass.
  • uvx mypy --strict scripts/tesla_cache_validate.py - clean.
  • Live smoke-tested against the real prod na_cache/eu_cache buckets (read-only) for all four record kinds during development: vehicle_data (with cross-tab), signal, energy_live_status, and energy_site_info - including exercising the non-empty-read guard (trips correctly on a forced-failure timeout) and --save-raw sanitization (redacts nested VIN/din, preserves numeric shape; passes through unredacted for the preferred vehicle/site).
Full narrative / original brief

Tesla executable-schema roadmap PHASE 1 ONLY: promote the proven prod-cache validation recipe into a committed, reusable maintainer tool.

Required reading: api-tesla-schema-fixtures-n6/report.md (design + phased roadmap) and chargeport-null-validate-v4/report.md (the proven validation recipe this tool codifies, with raw corpus).

Deliverable: a maintainer script (scripts/) that validates Tesla field semantics against the live prod NATS KV caches - vehicle AND energy, na_ and eu_ buckets - answering per-field questions like tri-state vs true-or-absent, observed value domains, and presence rates across the fleet.

Reliability guards baked in: the reliable nats stream get --last-for read path (plain nats kv get reliably times out); always verify the scan read at least one non-empty value before trusting any absent-record conclusion; offline/read-only (never writes to prod NATS; CI never runs it).

Captain rulings in force (2026-07-07): real customer values may appear in outputs/fixtures - prefer the captain's own vehicle (VIN LRW3F7EK4NC716336) and energy site (2533979794926773); other customers' data must be sanitized but shape-valid. Output should make the later fixture-seeding phase (phase 2) easy, but do not implement phase 2 (no fixtures/ module, no registry) - one phase, one PR.

Promotes the proven cache-validation recipe (data/chargeport-null-validate-v4)
into a committed, reusable script: scripts/tesla_cache_validate.py reads the
live na_cache/eu_cache NATS KV buckets (vehicle_data, streamed signals, and
energy live_status/site_info, both regions) to answer per-field questions -
tri-state vs true-or-absent, observed value domains, presence rates - instead
of leaving that confirmation as one-off archaeology.

Read-only and offline: never calls nats kv get (proven to silently read empty
on these buckets, per the recipe), only nats stream get --last-for -j; never
writes to NATS; not exercised by CI, which has no prod access. Presence
classification uses has-key semantics and a non-empty-read guard so a fetch
failure or silent-empty read can never be mistaken for a genuine "absent"
finding. Raw records are sanitized by default (VIN/tokens/geo redacted,
type-preserved) except for the maintainer's own reference vehicle/site.

Phase 1 of the executable-schema roadmap; the Field/Presence registry and
fixtures module (phases 2+) are intentionally not implemented here.
@Bre77

Bre77 commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Closing on the captain's review: the tool embeds Teslemetry-internal details (KV bucket naming and prod topology) that must not live in a public library repo. It is moving to a private Teslemetry repo; the public libs will still receive the sanitized golden fixtures (vehicle/site data approved) in the later roadmap phases.

@Bre77 Bre77 closed this Jul 7, 2026
@Bre77 Bre77 deleted the fm/lib-schema-fixtures-p1 branch July 7, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant