From c1b59892cf25efb2facbb0c7e1b067c4cb5a4ef3 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 29 Oct 2026 08:03:53 +0000 Subject: [PATCH 01/20] docs(specs): name every spec in lowercase kebab case Each spec file is now `.md` (or `-.md` for the OBC binary formats), without a `_Spec` or `-spec` suffix: the directory says it is a contract. Every reference in code comments, configs, guides and READMEs follows the new names. | Old | New | | --- | --- | | FLAT_Store_Format.md | flat-store-format.md | | FLAT_Store_Protocol.md | flat-store-protocol.md | | OBCA_Spec.md | obca-assembly.md | | OBCC_Spec.md | obcc-catalog.md | | OBCM_Spec.md | obcm-map.md | | OBCR_Spec.md | obcr-route.md | | OBCT_Spec.md | obct-terrain.md | | OBCU_Spec.md | obcu-update.md | | Ride_Archive_Contract.md | ride-archive-receipts.md | | Ride_Archive_Metadata.md | ride-archive-metadata.md | | obc-ble-interface-spec.md | ble-interface.md | Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DTHu3LHSGY5CY9j2XmscLi --- .github/workflows/bake.yml | 4 +- .github/workflows/deploy-site.yml | 2 +- builder/presets/skins/default.json | 2 +- builder/presets/skins/dusk.json | 2 +- builder/web/src/lib/catalog/manifest.ts | 2 +- builder/web/src/lib/firmware/obcu.ts | 2 +- builder/web/src/lib/usb/protocol.ts | 2 +- builder/web/tests/browser/trailview/box.sh | 2 +- companion-ios/OBCProtocol.md | 4 +- .../OBCKit/Sources/OBCDomain/BikeType.swift | 2 +- .../Sources/OBCDomain/OBCProtocol.swift | 2 +- .../OBCKit/Sources/OBCDomain/Ride.swift | 2 +- .../Sources/OBCDomain/RideZoneLimits.swift | 2 +- .../OBCKit/Sources/OBCDomain/Waypoint.swift | 2 +- .../Sources/OBCDomain/WaypointCategory.swift | 2 +- .../Sources/OBCTransport/BLE/GATT.swift | 2 +- .../Codecs/RouteObjectCodec.swift | 2 +- .../OBCTransport/Codecs/TripObjectCodec.swift | 2 +- .../OBCTransport/Firmware/FirmwareImage.swift | 2 +- .../OBCUI/Detail/RouteDetailModel.swift | 2 +- .../OBCTransportTests/BikeTypeTests.swift | 2 +- .../OBCTransportTests/TripDatesTests.swift | 2 +- companion-ios/README.md | 2 +- docs/content/software/companion-link.md | 8 +- docs/content/software/firmware-updates.md | 8 +- docs/content/software/formats.md | 12 +-- docs/content/software/packer-routing.md | 4 +- docs/content/software/terrain.md | 8 +- docs/content/software/trips.md | 6 +- docs/content/software/ui.md | 2 +- firmware/README.md | 4 +- firmware/obc-app/src/trip.rs | 2 +- firmware/obc-boot/README.md | 2 +- firmware/obc-boot/memory.x | 8 +- firmware/obc-dfu/keys/README.md | 2 +- firmware/obc-dfu/src/armer.rs | 2 +- firmware/obc-dfu/src/blobstage.rs | 2 +- firmware/obc-dfu/src/engine.rs | 2 +- firmware/obc-dfu/src/image.rs | 2 +- firmware/obc-dfu/src/sig.rs | 2 +- firmware/obc-dfu/src/state.rs | 2 +- firmware/obc-dfu/tests/cases/boot_state.rs | 2 +- firmware/obc-formats/src/bike.rs | 4 +- firmware/obc-formats/src/obcm.rs | 2 +- firmware/obc-formats/src/obcr.rs | 2 +- firmware/obc-formats/src/ride.rs | 2 +- firmware/obc-formats/src/trip_progress.rs | 4 +- firmware/obc-fw-nrf54l/README.md | 4 +- firmware/obc-fw-nrf54l/build.rs | 6 +- .../obc-fw-nrf54l/src/bin/flat_store_bench.rs | 2 +- firmware/obc-link/src/flat/engine.rs | 4 +- firmware/obc-link/src/flat/ids.rs | 4 +- firmware/obc-link/src/flat/mod.rs | 6 +- firmware/obc-link/src/flat/vectors.rs | 2 +- firmware/obc-link/src/flat/wire.rs | 2 +- firmware/obc-link/src/lib.rs | 2 +- firmware/obc-link/tests/flat_break_matrix.rs | 2 +- firmware/obc-link/tests/flat_engine.rs | 2 +- firmware/obc-link/tests/flat_harness/mod.rs | 4 +- firmware/obc-route/src/eta.rs | 2 +- firmware/obc-route/src/lib.rs | 2 +- firmware/obc-route/src/reader.rs | 4 +- firmware/obc-route/src/symbol.rs | 2 +- firmware/obc-route/src/trip.rs | 2 +- firmware/obc-route/tests/cases/format.rs | 2 +- firmware/obc-route/tests/cases/waypoints.rs | 2 +- firmware/obc-route/tests/common/mod.rs | 2 +- firmware/obc-storage/src/flat/bitmap.rs | 2 +- firmware/obc-storage/src/flat/catalog.rs | 2 +- firmware/obc-storage/src/flat/journal.rs | 2 +- firmware/obc-storage/src/flat/layout.rs | 2 +- firmware/obc-storage/src/flat/seam.rs | 4 +- firmware/obc-storage/src/flat/store.rs | 2 +- firmware/obc-storage/src/flat/superblock.rs | 2 +- firmware/obc-storage/src/flat/vectors.rs | 6 +- firmware/tools/ui_frames/environments.py | 8 +- host/obc-bake/assets/README.md | 2 +- host/obc-bake/src/cells.rs | 6 +- host/obc-bake/src/cut.rs | 2 +- host/obc-bake/src/guard.rs | 12 +-- host/obc-bake/src/main.rs | 6 +- host/obc-bake/src/source.rs | 2 +- host/obc-bake/src/terrain.rs | 4 +- host/obc-bake/src/verify.rs | 14 +-- host/obc-dem/Cargo.toml | 4 +- host/obc-dem/reference/README.md | 4 +- host/obc-dem/tests/assets.rs | 4 +- host/obc-map-core/src/grid.rs | 4 +- host/obc-mkimage/src/main.rs | 2 +- host/obc-pack/schema/catalog.schema.json | 40 ++++---- host/obc-pack/src/catalog.rs | 6 +- host/obc-pack/src/catalog/cells.rs | 10 +- host/obc-pack/src/catalog/landmarks.rs | 6 +- host/obc-pack/src/catalog/model.rs | 36 +++---- host/obc-pack/src/catalog/regions.rs | 6 +- host/obc-pack/src/catalog/schema.rs | 20 ++-- host/obc-pack/src/catalog/terrain.rs | 14 +-- host/obc-pack/src/catalog/tests.rs | 2 +- host/obc-pack/src/landmarks/README.md | 6 +- host/obc-places/src/metadata.rs | 2 +- host/obc-places/src/routing.rs | 2 +- host/obc-vectors/Cargo.toml | 2 +- host/obc-vectors/src/lib.rs | 12 +-- host/obc-vectors/tests/vectors.rs | 2 +- host/obcm-assemble/Cargo.toml | 2 +- host/obcm-assemble/src/emit.rs | 2 +- host/obcm-assemble/src/lib.rs | 2 +- host/obcm-assemble/src/nav.rs | 4 +- host/obcm-assemble/src/verify.rs | 2 +- host/obcm-testkit/src/lib.rs | 2 +- sim/flat-device/src/lib.rs | 2 +- sim/host-core/src/flat_store/tests.rs | 2 +- ...ble-interface-spec.md => ble-interface.md} | 20 ++-- ...T_Store_Format.md => flat-store-format.md} | 10 +- ...ore_Protocol.md => flat-store-protocol.md} | 24 ++--- specs/obc-data.md | 8 +- specs/{OBCA_Spec.md => obca-assembly.md} | 96 +++++++++---------- specs/{OBCC_Spec.md => obcc-catalog.md} | 6 +- specs/{OBCM_Spec.md => obcm-map.md} | 10 +- specs/{OBCR_Spec.md => obcr-route.md} | 8 +- specs/{OBCT_Spec.md => obct-terrain.md} | 24 ++--- specs/{OBCU_Spec.md => obcu-update.md} | 10 +- ...e_Metadata.md => ride-archive-metadata.md} | 2 +- ...e_Contract.md => ride-archive-receipts.md} | 10 +- specs/vectors/README.md | 14 +-- specs/vectors/eta.csv | 2 +- specs/vectors/manifest.json | 4 +- specs/wikimedia-content.md | 2 +- tools/fixture_catalog.py | 2 +- tools/justfile | 2 +- tools/testing/suites.toml | 2 +- 131 files changed, 369 insertions(+), 369 deletions(-) rename specs/{obc-ble-interface-spec.md => ble-interface.md} (97%) rename specs/{FLAT_Store_Format.md => flat-store-format.md} (99%) rename specs/{FLAT_Store_Protocol.md => flat-store-protocol.md} (98%) rename specs/{OBCA_Spec.md => obca-assembly.md} (90%) rename specs/{OBCC_Spec.md => obcc-catalog.md} (99%) rename specs/{OBCM_Spec.md => obcm-map.md} (99%) rename specs/{OBCR_Spec.md => obcr-route.md} (98%) rename specs/{OBCT_Spec.md => obct-terrain.md} (96%) rename specs/{OBCU_Spec.md => obcu-update.md} (98%) rename specs/{Ride_Archive_Metadata.md => ride-archive-metadata.md} (98%) rename specs/{Ride_Archive_Contract.md => ride-archive-receipts.md} (95%) diff --git a/.github/workflows/bake.yml b/.github/workflows/bake.yml index 81e7c5b4f3..c16686dd6e 100644 --- a/.github/workflows/bake.yml +++ b/.github/workflows/bake.yml @@ -31,7 +31,7 @@ name: Bake # Publishing needs R2 credentials, which live in repository secrets and are only # passed on an explicit `workflow_dispatch` with `publish: true`. A scheduled run # bakes and reports; it never uploads. Content first, root last, is the publisher's -# own invariant (`OBCC_Spec.md` §11) and is enforced in `obc-bake`, not +# own invariant (`obcc-catalog.md` §11) and is enforced in `obc-bake`, not # here — this file only decides *whether* to publish. on: @@ -81,7 +81,7 @@ env: europe/germany/saarland jobs: - # ── The mandatory re-bake guard (OBCC_Spec.md §10) ──────────────────────────────── + # ── The mandatory re-bake guard (obcc-catalog.md §10) ──────────────────────────────── # # Three mechanisms enforce "an OBCM bump invalidates every baked artifact". Two are # in the repository and already exist: the generator refuses a mixed tree, and diff --git a/.github/workflows/deploy-site.yml b/.github/workflows/deploy-site.yml index d0e215f494..8e8dd3a0db 100644 --- a/.github/workflows/deploy-site.yml +++ b/.github/workflows/deploy-site.yml @@ -37,7 +37,7 @@ name: Deploy site # only change by redeploying the site, which is exactly what #905 says a fresh bake # must not require. So the manifest lives **with the artifacts**, on whatever object # storage the bakery (B1 #898) publishes to, where its cache policy is that host's -# to set (OBCC_Spec.md §11 states the ≤ 60 s expectation). +# to set (obcc-catalog.md §11 states the ≤ 60 s expectation). # # 2. **No cross-origin isolation, ever, on this host.** Setting # `Cross-Origin-Opener-Policy: same-origin` + `Cross-Origin-Embedder-Policy: diff --git a/builder/presets/skins/default.json b/builder/presets/skins/default.json index 5246ee2906..80f12c4ab6 100644 --- a/builder/presets/skins/default.json +++ b/builder/presets/skins/default.json @@ -12,7 +12,7 @@ "#55AA55", "#AAAAAA" ], - "skin_note": "Presentation only. Every feature type and every style id comes from `schema.json` and is referenced by the feature headers of every baked chunk (OBCM_Spec.md §5.2), so a skin may restate the presentation values in the style record and nothing else — the catalog generator refuses one that adds, drops, reorders or renumbers a feature type (OBCC_Spec.md §5). The values below are the schema's own, which is what makes this the default look; `min_lod`, the LOD ladder, the merge passes and the routing profiles are schema data and are deliberately absent." + "skin_note": "Presentation only. Every feature type and every style id comes from `schema.json` and is referenced by the feature headers of every baked chunk (obcm-map.md §5.2), so a skin may restate the presentation values in the style record and nothing else — the catalog generator refuses one that adds, drops, reorders or renumbers a feature type (obcc-catalog.md §5). The values below are the schema's own, which is what makes this the default look; `min_lod`, the LOD ladder, the merge passes and the routing profiles are schema data and are deliberately absent." }, "features": { "highway": { diff --git a/builder/presets/skins/dusk.json b/builder/presets/skins/dusk.json index 55bb76e156..05b178e230 100644 --- a/builder/presets/skins/dusk.json +++ b/builder/presets/skins/dusk.json @@ -12,7 +12,7 @@ "#005500", "#AAAAAA" ], - "skin_note": "Presentation only, same rules as `default.json` (OBCC_Spec.md §5). The night inversion of the schema's field-guide look (epic #1016 §4): the cream paper becomes black ground, big fills go dark (deep-blue water, forest green, olive fields, a navy town lift), and everything a rider follows stays light — the warm through-road ramp is kept verbatim from the day skin because deep orange and amber already read on black, local streets drop from white to grey so a town grid does not glare, tracks lift from brown to tan, cycleways from indigo to light violet. Rail keeps its day values: black dashes on a grey base read as a grey dashed line on black ground with no edit at all. Buildings are dark grey — the quietest mark that still exists on plain black ground (a town without mapped landuse, e.g. central Monaco, would swallow black buildings whole). Every value lands exactly on the panel's RGB222 grid (4 levels/channel: 0/85/170/255) and every pair meant to differ occupies a distinct bucket; the catalog tests assert both. One more rule this document obeys deliberately: the bake merges features whose *schema* styles render identically (obc-pack's merge_fills/merge_lines retag merged geometry to one canonical style id), so feature types the schema merges — {unclassified, residential}, {service, pedestrian}, {drain, ditch}, {wood, forest}, {bare_rock, scree, shingle}, the grass family, the built-up landuse family — are restated with identical values here: a skin that split them would style only the canonical id's share of the merged geometry." + "skin_note": "Presentation only, same rules as `default.json` (obcc-catalog.md §5). The night inversion of the schema's field-guide look (epic #1016 §4): the cream paper becomes black ground, big fills go dark (deep-blue water, forest green, olive fields, a navy town lift), and everything a rider follows stays light — the warm through-road ramp is kept verbatim from the day skin because deep orange and amber already read on black, local streets drop from white to grey so a town grid does not glare, tracks lift from brown to tan, cycleways from indigo to light violet. Rail keeps its day values: black dashes on a grey base read as a grey dashed line on black ground with no edit at all. Buildings are dark grey — the quietest mark that still exists on plain black ground (a town without mapped landuse, e.g. central Monaco, would swallow black buildings whole). Every value lands exactly on the panel's RGB222 grid (4 levels/channel: 0/85/170/255) and every pair meant to differ occupies a distinct bucket; the catalog tests assert both. One more rule this document obeys deliberately: the bake merges features whose *schema* styles render identically (obc-pack's merge_fills/merge_lines retag merged geometry to one canonical style id), so feature types the schema merges — {unclassified, residential}, {service, pedestrian}, {drain, ditch}, {wood, forest}, {bare_rock, scree, shingle}, the grass family, the built-up landuse family — are restated with identical values here: a skin that split them would style only the canonical id's share of the merged geometry." }, "features": { "highway": { diff --git a/builder/web/src/lib/catalog/manifest.ts b/builder/web/src/lib/catalog/manifest.ts index 56e63e54dd..4d6de963bc 100644 --- a/builder/web/src/lib/catalog/manifest.ts +++ b/builder/web/src/lib/catalog/manifest.ts @@ -1,5 +1,5 @@ // The cell catalog root: one schema, the skins, the named regions, and the pins on -// the per-band cell indices. Normative in `OBCC_Spec.md`, produced by +// the per-band cell indices. Normative in `obcc-catalog.md`, produced by // `obc-pack catalog` / `obc-bake`. // // This takes a *string* because a document is read whole and parsed as one JSON diff --git a/builder/web/src/lib/firmware/obcu.ts b/builder/web/src/lib/firmware/obcu.ts index 0d13b668db..22cd1159c1 100644 --- a/builder/web/src/lib/firmware/obcu.ts +++ b/builder/web/src/lib/firmware/obcu.ts @@ -32,7 +32,7 @@ const OBCU_SIG_SCHEME_ED25519 = 1; /** Bytes of the Ed25519 signature trailer. */ export const OBCU_SIG_LEN = 64; -/** The device's whole application slot (`obc_dfu::MAX_IMAGE_LEN`, `OBCU_Spec.md` 1.1). */ +/** The device's whole application slot (`obc_dfu::MAX_IMAGE_LEN`, `obcu-update.md` 1.1). */ const OBCU_MAX_IMAGE_LEN = 2_023_424; const MAGIC = 0x4f424355; // "OBCU" diff --git a/builder/web/src/lib/usb/protocol.ts b/builder/web/src/lib/usb/protocol.ts index 637795beeb..2a69473d16 100644 --- a/builder/web/src/lib/usb/protocol.ts +++ b/builder/web/src/lib/usb/protocol.ts @@ -2,7 +2,7 @@ * Protocol v4's shared vocabulary in TypeScript: the opcode, kind, flag and error tables, the * decoded response shapes, and the few byte helpers the client and its parsers share. * - * `specs/FLAT_Store_Protocol.md` is the sole authority. The wire codec itself is the Rust + * `specs/flat-store-protocol.md` is the sole authority. The wire codec itself is the Rust * `obc_link::flat` client, which the browser runs through the wasm bridge. * * Every `u64` is a `bigint`, never a `number`: `Number` loses integers above 2^53, and a codec that diff --git a/builder/web/tests/browser/trailview/box.sh b/builder/web/tests/browser/trailview/box.sh index 747641fd99..8c58e19be2 100755 --- a/builder/web/tests/browser/trailview/box.sh +++ b/builder/web/tests/browser/trailview/box.sh @@ -94,7 +94,7 @@ else: for tx in range(x(w), x(e) + 1): for ty in range(y(n), y(s) + 1): blocks.setdefault(f"10-{tx >> 2}-{ty >> 2}", []).append(f"12-{tx}-{ty}=ch") - elif kind == "terrain": # specs/OBCT_Spec.md: origin -2^28 µdeg, tiles of 2^16 µdeg, 4 × 4 to a block + elif kind == "terrain": # specs/obct-terrain.md: origin -2^28 µdeg, tiles of 2^16 µdeg, 4 × 4 to a block t = lambda deg: (round(deg * 1e6) + 2**28) >> 16 for ti in range(t(s), t(n) + 1): for tj in range(t(w), t(e) + 1): diff --git a/companion-ios/OBCProtocol.md b/companion-ios/OBCProtocol.md index 87a7d0ca52..00cd6e241e 100644 --- a/companion-ios/OBCProtocol.md +++ b/companion-ios/OBCProtocol.md @@ -1,9 +1,9 @@ # OBC protocol — iOS implementation notes The normative wire contract is -[`../specs/obc-ble-interface-spec.md`](../specs/obc-ble-interface-spec.md). It owns every UUID, +[`../specs/ble-interface.md`](../specs/ble-interface.md). It owns every UUID, byte layout, state transition and security rule, and it wins over anything here. The ride archive -rules are in [`../specs/Ride_Archive_Contract.md`](../specs/Ride_Archive_Contract.md). Shared +rules are in [`../specs/ride-archive-receipts.md`](../specs/ride-archive-receipts.md). Shared fixtures under `specs/vectors/` pin the codecs byte for byte. This file records only the choices an iOS contributor needs while reading the implementation. diff --git a/companion-ios/Packages/OBCKit/Sources/OBCDomain/BikeType.swift b/companion-ios/Packages/OBCKit/Sources/OBCDomain/BikeType.swift index 6515c5a5be..183cb462bc 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCDomain/BikeType.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCDomain/BikeType.swift @@ -1,6 +1,6 @@ import Foundation -/// The four fixed bike types of `specs/OBCR_Spec.md` §1.2. The raw value is the OBCR header byte +/// The four fixed bike types of `specs/obcr-route.md` §1.2. The raw value is the OBCR header byte /// and the map's routing-profile index. public enum BikeType: UInt8, CaseIterable, Sendable { case road = 0 diff --git a/companion-ios/Packages/OBCKit/Sources/OBCDomain/OBCProtocol.swift b/companion-ios/Packages/OBCKit/Sources/OBCDomain/OBCProtocol.swift index dd548971ed..f553e4bd34 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCDomain/OBCProtocol.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCDomain/OBCProtocol.swift @@ -1,6 +1,6 @@ import Foundation -/// The frozen wire-protocol surface, in code form. `specs/obc-ble-interface-spec.md` is +/// The frozen wire-protocol surface, in code form. `specs/ble-interface.md` is /// canonical: if this and the firmware disagree, the firmware wins and this is corrected. public enum OBCProtocol { /// The `protocol_version` this app build is written against. The connect path compares the diff --git a/companion-ios/Packages/OBCKit/Sources/OBCDomain/Ride.swift b/companion-ios/Packages/OBCKit/Sources/OBCDomain/Ride.swift index 0a39c8aeb4..45731558e5 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCDomain/Ride.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCDomain/Ride.swift @@ -239,7 +239,7 @@ public struct RideSummary: Identifiable, Equatable, Sendable { } } -/// The trip day a ride started on (`specs/obc-ble-interface-spec.md` §7.2). +/// The trip day a ride started on (`specs/ble-interface.md` §7.2). public struct RideTrip: Equatable, Sendable { /// The trip key of the trip object; never 0. public var key: UInt64 diff --git a/companion-ios/Packages/OBCKit/Sources/OBCDomain/RideZoneLimits.swift b/companion-ios/Packages/OBCKit/Sources/OBCDomain/RideZoneLimits.swift index d9d283d5aa..6d854b05ab 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCDomain/RideZoneLimits.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCDomain/RideZoneLimits.swift @@ -1,7 +1,7 @@ import Foundation /// The rider's effort limits that were in force when a ride started -/// (`specs/obc-ble-interface-spec.md` §7.2). A ride keeps them when the rider's settings change +/// (`specs/ble-interface.md` §7.2). A ride keeps them when the rider's settings change /// later. Nil is not set on the device at ride time, and that metric then has no zones. public struct RideZoneLimits: Equatable, Sendable, Codable { public let maxHeartRate: Int? diff --git a/companion-ios/Packages/OBCKit/Sources/OBCDomain/Waypoint.swift b/companion-ios/Packages/OBCKit/Sources/OBCDomain/Waypoint.swift index 5af1d416ee..49bfb4f147 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCDomain/Waypoint.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCDomain/Waypoint.swift @@ -15,7 +15,7 @@ public struct Waypoint: Identifiable, Equatable, Sendable { public var category: WaypointCategory? /// Signed lateral distance from the route line, in metres: positive is right of the /// direction of travel, negative is left, `0` is on-route. Fixed at import by the same - /// projection that yields `distanceAlongMeters` (`OBCR_Spec.md`), because a riding device + /// projection that yields `distanceAlongMeters` (`obcr-route.md`), because a riding device /// cannot re-measure it from the decimated geometry it keeps. public var lateralOffsetMeters: Double public var provenance: WaypointProvenance? diff --git a/companion-ios/Packages/OBCKit/Sources/OBCDomain/WaypointCategory.swift b/companion-ios/Packages/OBCKit/Sources/OBCDomain/WaypointCategory.swift index bfdbecbb56..b0468e2a10 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCDomain/WaypointCategory.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCDomain/WaypointCategory.swift @@ -4,7 +4,7 @@ import Foundation /// map already uses, reused verbatim so one icon language covers both sources. /// /// The raw values are the stable wire ids stored in an OBCR waypoint record's category byte -/// (`OBCR_Spec.md`). `0` is not a case: it means generic, which this type models as `nil`. +/// (`obcr-route.md`). `0` is not a case: it means generic, which this type models as `nil`. public enum WaypointCategory: UInt8, CaseIterable, Sendable { case water = 1 case campsite = 2 diff --git a/companion-ios/Packages/OBCKit/Sources/OBCTransport/BLE/GATT.swift b/companion-ios/Packages/OBCKit/Sources/OBCTransport/BLE/GATT.swift index b20f4f4b1c..464b46c4c2 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCTransport/BLE/GATT.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCTransport/BLE/GATT.swift @@ -2,7 +2,7 @@ @preconcurrency import CoreBluetooth /// The GATT service and characteristic map `BLETransport` discovers: the control plane of -/// `obc-ble-interface-spec.md`. +/// `ble-interface.md`. /// /// The OBC Control UUIDs use the random base `3C92XXXX-9916-4EBA-ABC2-342FE08F6B10`, where the /// 16-bit `XXXX` block selects the entity (`0000` is the service, `000N` characteristic N). A diff --git a/companion-ios/Packages/OBCKit/Sources/OBCTransport/Codecs/RouteObjectCodec.swift b/companion-ios/Packages/OBCKit/Sources/OBCTransport/Codecs/RouteObjectCodec.swift index 3e870a95e3..4b27222b6f 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCTransport/Codecs/RouteObjectCodec.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCTransport/Codecs/RouteObjectCodec.swift @@ -3,7 +3,7 @@ import OBCDomain import OBCHost /// The Rust OBCR v5 encoder and the Swift decoder. Stored geometry owns the route totals. -/// Missing elevation and incoming graph validity stay explicit. The exact bytes are in specs/OBCR_Spec.md, and shared +/// Missing elevation and incoming graph validity stay explicit. The exact bytes are in specs/obcr-route.md, and shared /// Rust-produced vectors pin the reader. public enum RouteObjectCodec { // MARK: Format constants diff --git a/companion-ios/Packages/OBCKit/Sources/OBCTransport/Codecs/TripObjectCodec.swift b/companion-ios/Packages/OBCKit/Sources/OBCTransport/Codecs/TripObjectCodec.swift index 6f6a8a7eab..95cf914acb 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCTransport/Codecs/TripObjectCodec.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCTransport/Codecs/TripObjectCodec.swift @@ -5,7 +5,7 @@ import OBCDomain /// reconcile download. A trip object references one route per day by its device object id and never /// carries route bytes. /// -/// The layout is little-endian (`obc-ble-interface-spec.md` §7.7): a fixed 64-byte header, then one +/// The layout is little-endian (`ble-interface.md` §7.7): a fixed 64-byte header, then one /// 16-byte record per day in ride order. The header holds the version, the day count, a /// length-prefixed, zero-padded name of at most 48 bytes, the start date and the trip key. /// diff --git a/companion-ios/Packages/OBCKit/Sources/OBCTransport/Firmware/FirmwareImage.swift b/companion-ios/Packages/OBCKit/Sources/OBCTransport/Firmware/FirmwareImage.swift index abeee7fe54..b1bfde6149 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCTransport/Firmware/FirmwareImage.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCTransport/Firmware/FirmwareImage.swift @@ -33,7 +33,7 @@ public struct OBCUHeader: Equatable, Sendable { /// Bytes of the header covered by the header CRC, which is everything but the CRC. public static let headerCRCLength = 60 /// The device's whole application slot, and so the largest raw image it can flash - /// (`obc_dfu::MAX_IMAGE_LEN`, `OBCU_Spec.md` 1.1). An announced object past it is rejected at + /// (`obc_dfu::MAX_IMAGE_LEN`, `obcu-update.md` 1.1). An announced object past it is rejected at /// announce, so the app refuses it here first. public static let maxImageLength: UInt32 = 2_023_424 diff --git a/companion-ios/Packages/OBCKit/Sources/OBCUI/Detail/RouteDetailModel.swift b/companion-ios/Packages/OBCKit/Sources/OBCUI/Detail/RouteDetailModel.swift index dd75a4ed3a..8e775202c4 100644 --- a/companion-ios/Packages/OBCKit/Sources/OBCUI/Detail/RouteDetailModel.swift +++ b/companion-ios/Packages/OBCKit/Sources/OBCUI/Detail/RouteDetailModel.swift @@ -72,7 +72,7 @@ public final class RouteDetailModel { /// False for a ride whose points carry no elevation, or that has no points and no climb, so /// its climb and descent read as unknown, not 0. private let hasElevation: Bool - /// The `OBCR_Spec.md` §1.2 estimate. Distance and climb hold the upload's header figures, so it + /// The `obcr-route.md` §1.2 estimate. Distance and climb hold the upload's header figures, so it /// is the estimate the device shows for this route and type. private var estimatedDuration: TimeInterval? { if case .tracked = dressing { return nil } diff --git a/companion-ios/Packages/OBCKit/Tests/OBCTransportTests/BikeTypeTests.swift b/companion-ios/Packages/OBCKit/Tests/OBCTransportTests/BikeTypeTests.swift index fe65544386..abb7811b77 100644 --- a/companion-ios/Packages/OBCKit/Tests/OBCTransportTests/BikeTypeTests.swift +++ b/companion-ios/Packages/OBCKit/Tests/OBCTransportTests/BikeTypeTests.swift @@ -3,7 +3,7 @@ import Testing import OBCDomain import OBCTransport -/// The `OBCR_Spec.md` §1.2 estimate against the shared vector file, and the bike-type byte in +/// The `obcr-route.md` §1.2 estimate against the shared vector file, and the bike-type byte in /// the OBCR header. struct BikeTypeTests { private static let vectorsDir = URL(fileURLWithPath: #filePath) diff --git a/companion-ios/Packages/OBCKit/Tests/OBCTransportTests/TripDatesTests.swift b/companion-ios/Packages/OBCKit/Tests/OBCTransportTests/TripDatesTests.swift index 141d0d97d7..3dbd9d7ab1 100644 --- a/companion-ios/Packages/OBCKit/Tests/OBCTransportTests/TripDatesTests.swift +++ b/companion-ios/Packages/OBCKit/Tests/OBCTransportTests/TripDatesTests.swift @@ -2,7 +2,7 @@ import Testing import Foundation import OBCDomain -/// Dates follow the rides (`obc-ble-interface-spec.md` §7.7 "Day dates"). +/// Dates follow the rides (`ble-interface.md` §7.7 "Day dates"). struct TripDatesTests { private func trip(days: Int, start: Int?) -> Trip { let files = (0.. (u64, u64) { match self { BikeType::Road => (22, 16), diff --git a/firmware/obc-formats/src/obcm.rs b/firmware/obc-formats/src/obcm.rs index 394efcd2ff..4c5f4de64e 100644 --- a/firmware/obc-formats/src/obcm.rs +++ b/firmware/obc-formats/src/obcm.rs @@ -1,4 +1,4 @@ -//! OBCM map-format constants from `OBCM_Spec.md`. +//! OBCM map-format constants from `obcm-map.md`. use crate::io::{validate_prefix, DecodeError}; diff --git a/firmware/obc-formats/src/obcr.rs b/firmware/obc-formats/src/obcr.rs index bda22014a0..27b9dde9e4 100644 --- a/firmware/obc-formats/src/obcr.rs +++ b/firmware/obc-formats/src/obcr.rs @@ -1,4 +1,4 @@ -//! OBCR route-format constants from `OBCR_Spec.md`. +//! OBCR route-format constants from `obcr-route.md`. use crate::io::{validate_prefix, DecodeError}; diff --git a/firmware/obc-formats/src/ride.rs b/firmware/obc-formats/src/ride.rs index 7ef7b81491..461a4f7e27 100644 --- a/firmware/obc-formats/src/ride.rs +++ b/firmware/obc-formats/src/ride.rs @@ -69,7 +69,7 @@ impl Default for Name { } } -/// The trip day a ride started on (`obc-ble-interface-spec.md` §7.7 names the trip key). +/// The trip day a ride started on (`ble-interface.md` §7.7 names the trip key). #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct TripRef { /// Nonzero, so an `Option` costs no tag. diff --git a/firmware/obc-formats/src/trip_progress.rs b/firmware/obc-formats/src/trip_progress.rs index de924ba1e1..a5e06b1b8f 100644 --- a/firmware/obc-formats/src/trip_progress.rs +++ b/firmware/obc-formats/src/trip_progress.rs @@ -1,5 +1,5 @@ -//! The device's trip progress records (`obc-ble-interface-spec.md` §7.7). The Metadata singleton -//! holds them; `Ride_Archive_Metadata.md` has the bytes. +//! The device's trip progress records (`ble-interface.md` §7.7). The Metadata singleton +//! holds them; `ride-archive-metadata.md` has the bytes. use crate::io::{put_u32, rd_u16, rd_u32}; diff --git a/firmware/obc-fw-nrf54l/README.md b/firmware/obc-fw-nrf54l/README.md index 717d48838d..24ad34d03d 100644 --- a/firmware/obc-fw-nrf54l/README.md +++ b/firmware/obc-fw-nrf54l/README.md @@ -158,8 +158,8 @@ recovers on an MCU reset. See the ## BLE -See the [BLE contract](../../specs/obc-ble-interface-spec.md) and -[object protocol](../../specs/FLAT_Store_Protocol.md). +See the [BLE contract](../../specs/ble-interface.md) and +[object protocol](../../specs/flat-store-protocol.md). - Use the internal LF RC clock with MPSL calibration. `ExternalXtal` connections fail with HCI `0x3E` unless the DK's `OSCILLATORS` INTCAP registers are set diff --git a/firmware/obc-fw-nrf54l/build.rs b/firmware/obc-fw-nrf54l/build.rs index 9a05d0db0d..6f59974e61 100644 --- a/firmware/obc-fw-nrf54l/build.rs +++ b/firmware/obc-fw-nrf54l/build.rs @@ -130,8 +130,8 @@ fn flpr_memory_x() -> String { MEMORY {{ FLASH : ORIGIN = {APP_SLOT_BASE:#010X}, LENGTH = {app_len:#X} /* app slot ({app_kb}K) above the 32K obc-boot (#617) */ - SEMMC_STAGE : ORIGIN = {SEMMC_STAGE_BASE:#010X}, LENGTH = {stage_kb}K /* staged sEMMC blob — armer→boot handoff (#1158, OBCU_Spec.md §3) */ - BOOT_STATE : ORIGIN = {BOOT_STATE_BASE:#010X}, LENGTH = 4K /* DFU boot-state handoff page (#617, OBCU_Spec.md §2) */ + SEMMC_STAGE : ORIGIN = {SEMMC_STAGE_BASE:#010X}, LENGTH = {stage_kb}K /* staged sEMMC blob — armer→boot handoff (#1158, obcu-update.md §3) */ + BOOT_STATE : ORIGIN = {BOOT_STATE_BASE:#010X}, LENGTH = 4K /* DFU boot-state handoff page (#617, obcu-update.md §2) */ SETTINGS : ORIGIN = {SETTINGS_BASE:#010X}, LENGTH = 4K /* persistent settings page (#193) — top of RRAM */ RAM : ORIGIN = {SRAM_BASE:#010X}, LENGTH = {ram_kb}K /* M33 .data/.bss/stack */ /* Reserved for the FLPR (not linked by the M33; see the generated flpr.ld): @@ -144,7 +144,7 @@ PROVIDE(__settings_base = ORIGIN(SETTINGS)); /* Base of the carved boot-state page: the armer's write target. */ PROVIDE(__boot_state_base = ORIGIN(BOOT_STATE)); /* Base of the blob-stage carve, where the armer copies the sEMMC image for the - bootloader's Install/Rollback card bring-up (OBCU_Spec.md §3). */ + bootloader's Install/Rollback card bring-up (obcu-update.md §3). */ PROVIDE(__semmc_stage_base = ORIGIN(SEMMC_STAGE)); /* Base of the app slot, where the armer's rollback snapshot reads the running image from (memory-mapped; RRAM is XIP-readable). The linker map stays the only address authority. */ diff --git a/firmware/obc-fw-nrf54l/src/bin/flat_store_bench.rs b/firmware/obc-fw-nrf54l/src/bin/flat_store_bench.rs index 97f431f8af..5a50d507b7 100644 --- a/firmware/obc-fw-nrf54l/src/bin/flat_store_bench.rs +++ b/firmware/obc-fw-nrf54l/src/bin/flat_store_bench.rs @@ -2,7 +2,7 @@ //! //! cargo run --release --bin flat_store_bench //! -//! `FLAT_Store_Format.md` states five figures that only the board can check: a first boot "well +//! `flat-store-format.md` states five figures that only the board can check: a first boot "well //! under a second", a mount "about 100 ms", a commit "about 15-20 ms at a few hundred entries", a //! read path that is arithmetic rather than a chain walk, and a resident cost of the 8 KiB free //! bitmap plus the ride tail. This binary measures each of them against a card in the slot. It is diff --git a/firmware/obc-link/src/flat/engine.rs b/firmware/obc-link/src/flat/engine.rs index 1a6aca231d..43183de548 100644 --- a/firmware/obc-link/src/flat/engine.rs +++ b/firmware/obc-link/src/flat/engine.rs @@ -1,4 +1,4 @@ -//! The transfer engine for `FLAT_Store_Protocol.md`: one state machine, both links. +//! The transfer engine for `flat-store-protocol.md`: one state machine, both links. //! //! It owns no transport and no medium. A record comes in, a [`Reaction`] names the bytes to send, //! and every card access goes through [`Store`]. An adapter hands the engine whole records and then @@ -1367,7 +1367,7 @@ mod tests { assert_eq!(commit_refusal(StoreError::Invalid, 0).code, ErrorCode::Internal); } - /// The detail is what a client's retry policy reads, and `FLAT_Store_Protocol.md` freezes it. + /// The detail is what a client's retry policy reads, and `flat-store-protocol.md` freezes it. #[test] fn a_full_hold_table_is_busy_with_the_holds_detail() { assert_eq!(open_refusal(StoreError::Busy), Refusal::new(ErrorCode::Busy, detail::busy::HOLDS)); diff --git a/firmware/obc-link/src/flat/ids.rs b/firmware/obc-link/src/flat/ids.rs index 98a50672c5..23aa663abd 100644 --- a/firmware/obc-link/src/flat/ids.rs +++ b/firmware/obc-link/src/flat/ids.rs @@ -1,7 +1,7 @@ //! The vocabulary both sides of the engine speak: three identities, the kind table, the entry //! flags, a display name, and the metadata half of a catalog entry. //! -//! `FLAT_Store_Format.md` is the authority for every value here. The types are declared in this +//! `flat-store-format.md` is the authority for every value here. The types are declared in this //! crate rather than imported from the store, because the dependency runs the other way (see //! [`super::store`]), and the binder's pinning test holds the two definitions equal. @@ -40,7 +40,7 @@ impl Revision { } } -/// The kind table of `FLAT_Store_Format.md`, carried unchanged on the wire. +/// The kind table of `flat-store-format.md`, carried unchanged on the wire. #[derive(Debug, Clone, Copy, PartialEq, Eq)] #[repr(u16)] pub enum ObjectKind { diff --git a/firmware/obc-link/src/flat/mod.rs b/firmware/obc-link/src/flat/mod.rs index 2da5363a38..1f77c08364 100644 --- a/firmware/obc-link/src/flat/mod.rs +++ b/firmware/obc-link/src/flat/mod.rs @@ -1,14 +1,14 @@ //! The wire the flat store speaks, and the one engine that speaks it. //! -//! [`FLAT_Store_Protocol.md`] is the normative contract and [`FLAT_Store_Format.md`] is what the +//! [`flat-store-protocol.md`] is the normative contract and [`flat-store-format.md`] is what the //! card holds. Nothing here is negotiated: every version is a fixed fact and every message is a //! fixed layout. //! //! [`ids`] is the vocabulary, [`wire`] is the bytes, [`store`] is the seam declared from the side //! that consumes it, and [`engine`] sits on both. //! -//! [`FLAT_Store_Protocol.md`]: ../../../../specs/FLAT_Store_Protocol.md -//! [`FLAT_Store_Format.md`]: ../../../../specs/FLAT_Store_Format.md +//! [`flat-store-protocol.md`]: ../../../../specs/flat-store-protocol.md +//! [`flat-store-format.md`]: ../../../../specs/flat-store-format.md pub mod engine; pub mod ids; pub mod records; diff --git a/firmware/obc-link/src/flat/vectors.rs b/firmware/obc-link/src/flat/vectors.rs index 07754206c5..968ce43573 100644 --- a/firmware/obc-link/src/flat/vectors.rs +++ b/firmware/obc-link/src/flat/vectors.rs @@ -1,6 +1,6 @@ //! The spec-derived fixture producer for `specs/vectors/flat-store-v4/`. //! -//! Every byte below is laid down by hand, at the offset `FLAT_Store_Protocol.md` states, through the +//! Every byte below is laid down by hand, at the offset `flat-store-protocol.md` states, through the //! [`raw`] helpers and never through [`super::wire`]. A golden vector a codec produced proves only //! that the codec agrees with itself; here the two sides share the specification alone, and the //! tests in [`tests`] close the loop in both directions. diff --git a/firmware/obc-link/src/flat/wire.rs b/firmware/obc-link/src/flat/wire.rs index 18bb0086b5..8bfe0e33cc 100644 --- a/firmware/obc-link/src/flat/wire.rs +++ b/firmware/obc-link/src/flat/wire.rs @@ -1,7 +1,7 @@ //! The protocol's bytes: the control frame, the nine request bodies, the response bodies, the //! stream frame and the error body. //! -//! `FLAT_Store_Protocol.md` is the sole authority and every offset below is transcribed from its +//! `flat-store-protocol.md` is the sole authority and every offset below is transcribed from its //! tables. Decoding is total: an input is either a typed message or a typed [`Refusal`], and //! nothing here panics on hostile bytes. Encoding writes into a caller-provided slice and reports //! the length; nothing is allocated. diff --git a/firmware/obc-link/src/lib.rs b/firmware/obc-link/src/lib.rs index 64d34c5af7..b32e8f0a25 100644 --- a/firmware/obc-link/src/lib.rs +++ b/firmware/obc-link/src/lib.rs @@ -1,6 +1,6 @@ //! The protocol-v4 codec, device engine and optional host client. //! -//! [`flat`] implements the [`FLAT store protocol`](../../../specs/FLAT_Store_Protocol.md): +//! [`flat`] implements the [`FLAT store protocol`](../../../specs/flat-store-protocol.md): //! control and stream records, the store seam, and one transport-free transfer engine. //! The device codec and engine are allocation-free. Host-only fixture production is enabled by //! `std`; `client` enables the transport-free host client with owned records and catalog results. diff --git a/firmware/obc-link/tests/flat_break_matrix.rs b/firmware/obc-link/tests/flat_break_matrix.rs index 1cd234c022..9bb9f7fc76 100644 --- a/firmware/obc-link/tests/flat_break_matrix.rs +++ b/firmware/obc-link/tests/flat_break_matrix.rs @@ -1,6 +1,6 @@ //! The break matrix: every opcode flow, cut at every protocol step boundary. //! -//! It discharges one obligation of `FLAT_Store_Protocol.md`: any break before the commit leaves the +//! It discharges one obligation of `flat-store-protocol.md`: any break before the commit leaves the //! card as if nothing happened. Each flow is run once to the end to establish the card after it, //! then run again for every prefix of its records with the link torn down at that point. Three //! things must then hold: diff --git a/firmware/obc-link/tests/flat_engine.rs b/firmware/obc-link/tests/flat_engine.rs index 699d0bb828..19b274bf61 100644 --- a/firmware/obc-link/tests/flat_engine.rs +++ b/firmware/obc-link/tests/flat_engine.rs @@ -1,4 +1,4 @@ -//! The engine against a real flat store: one behaviour per rule of `FLAT_Store_Protocol.md`. +//! The engine against a real flat store: one behaviour per rule of `flat-store-protocol.md`. //! //! The card underneath is the sim-backed [`FlatStore`](obc_storage::flat::FlatStore), so a claim //! about the catalog here is a claim about the bytes on a card. diff --git a/firmware/obc-link/tests/flat_harness/mod.rs b/firmware/obc-link/tests/flat_harness/mod.rs index 3e55805149..c80bfd0f92 100644 --- a/firmware/obc-link/tests/flat_harness/mod.rs +++ b/firmware/obc-link/tests/flat_harness/mod.rs @@ -5,7 +5,7 @@ //! The device is the assembly the browser's tests run on too, so a behaviour proved here is proved //! against the same engine and the same card. The harness is what only a Rust test wants: a pump //! that collects every record of one exchange into a [`Wire`], store probes, and request bytes -//! written at the offsets `FLAT_Store_Protocol.md` states. The device never encodes a request, so +//! written at the offsets `flat-store-protocol.md` states. The device never encodes a request, so //! writing those bytes by hand keeps the tests honest about what a phone would send. #![allow(dead_code)] @@ -300,7 +300,7 @@ impl obc_formats::io::ByteSink for VecSink { use obc_storage::flat::seam::Store; -/// Request bytes, written at the offsets `FLAT_Store_Protocol.md` states. +/// Request bytes, written at the offsets `flat-store-protocol.md` states. pub mod client { use super::*; diff --git a/firmware/obc-route/src/eta.rs b/firmware/obc-route/src/eta.rs index bcf5001df2..754c0f8ea0 100644 --- a/firmware/obc-route/src/eta.rs +++ b/firmware/obc-route/src/eta.rs @@ -1,6 +1,6 @@ //! Time to go along a route: the ride's ETA and TIME TO GO tiles. //! -//! The estimate itself is [`BikeType::ride_time_s`], the contract table of `OBCR_Spec.md` §1.2 +//! The estimate itself is [`BikeType::ride_time_s`], the contract table of `obcr-route.md` §1.2 //! that the phone shares. This module only feeds it the remaining distance and the remaining //! ascent from [`Profile::ascent_between_m`], the curve the elevation profile already builds. diff --git a/firmware/obc-route/src/lib.rs b/firmware/obc-route/src/lib.rs index 783d7fda73..8d391d6274 100644 --- a/firmware/obc-route/src/lib.rs +++ b/firmware/obc-route/src/lib.rs @@ -2,7 +2,7 @@ //! //! `no_std`, so the same code runs in the desktop simulator and in the nRF firmware. A route is a //! single ordered polyline with per-point elevation plus precomputed ride stats. The on-disk -//! layout is specified in `OBCR_Spec.md`. +//! layout is specified in `obcr-route.md`. //! //! The elevation dead-band that every module here integrates through lives in [`obc_elevation`], //! not in this crate. One home for the hysteresis keeps the packer's ascent, the profile's climb diff --git a/firmware/obc-route/src/reader.rs b/firmware/obc-route/src/reader.rs index 61978ab1dd..58d0962f92 100644 --- a/firmware/obc-route/src/reader.rs +++ b/firmware/obc-route/src/reader.rs @@ -70,7 +70,7 @@ pub struct RoutePosition { } /// One chunk's index entry: its bbox, the absolute anchor it decodes from, and the cumulative -/// stats at its first point. See `OBCR_Spec.md`. +/// stats at its first point. See `obcr-route.md`. #[derive(Debug, Clone, Copy)] pub struct ChunkMeta { pub bbox: BBox, @@ -873,7 +873,7 @@ pub(crate) fn for_each_stored_point( /// One stored route waypoint as it sits on disk, every field included. The ride geometry path /// skips the section; [`RouteReader::load_waypoints`] distils the named ones into the resident -/// [`Waypoints`] table the UI reads. See `OBCR_Spec.md`. +/// [`Waypoints`] table the UI reads. See `obcr-route.md`. #[derive(Debug, Clone, PartialEq, Eq)] pub struct Waypoint { /// Distance from the route start to this waypoint's position, meters. diff --git a/firmware/obc-route/src/symbol.rs b/firmware/obc-route/src/symbol.rs index 21e33e3b5b..1b93485358 100644 --- a/firmware/obc-route/src/symbol.rs +++ b/firmware/obc-route/src/symbol.rs @@ -1,4 +1,4 @@ -//! The canonical GPX symbol to [`PoiCategory`] mapping. `OBCR_Spec.md` mirrors it row for row. +//! The canonical GPX symbol to [`PoiCategory`] mapping. `obcr-route.md` mirrors it row for row. //! //! A GPX `` carries its icon as freeform text in `` or ``. There is no registry, //! so this table is a curation read off real exports, not a standard. diff --git a/firmware/obc-route/src/trip.rs b/firmware/obc-route/src/trip.rs index 97b3b6d7ec..935d7cf543 100644 --- a/firmware/obc-route/src/trip.rs +++ b/firmware/obc-route/src/trip.rs @@ -1,6 +1,6 @@ //! The trip object: the phone-authored metadata object that describes a trip's days. A trip //! references one route object per day, in ride order, and holds no route bytes. The layout is in -//! `obc-ble-interface-spec.md` §7.7 and pinned by `specs/vectors/trip-v3.bin`. +//! `ble-interface.md` §7.7 and pinned by `specs/vectors/trip-v3.bin`. //! //! The object length is fully determined by the header: `64 + 16·day_count` bytes. A decoder //! rejects a payload of any other length with [`Error::BadOffset`], which is also the torn-write diff --git a/firmware/obc-route/tests/cases/format.rs b/firmware/obc-route/tests/cases/format.rs index e9eea0a492..d49f891020 100644 --- a/firmware/obc-route/tests/cases/format.rs +++ b/firmware/obc-route/tests/cases/format.rs @@ -1,7 +1,7 @@ //! Format-contract tests for the OBCR reader. //! //! Each test builds a synthetic `.obcr` byte buffer with the shared handwritten builder, which -//! mirrors `OBCR_Spec.md`, then asserts the reader parses it back. Hand-emitting the bytes instead +//! mirrors `obcr-route.md`, then asserts the reader parses it back. Hand-emitting the bytes instead //! of going through the converter pins the reader to the spec independently. use core::cell::Cell; diff --git a/firmware/obc-route/tests/cases/waypoints.rs b/firmware/obc-route/tests/cases/waypoints.rs index b2b54f2109..558d558817 100644 --- a/firmware/obc-route/tests/cases/waypoints.rs +++ b/firmware/obc-route/tests/cases/waypoints.rs @@ -1,4 +1,4 @@ -//! OBCR waypoint-section tests: the hand-built format contract from `OBCR_Spec.md`, the converter's +//! OBCR waypoint-section tests: the hand-built format contract from `obcr-route.md`, the converter's //! `` emission (category from ``/``, signed lateral offset), and the storage-only //! guarantee that a waypoint-bearing route rides identically to the same route without waypoints. diff --git a/firmware/obc-route/tests/common/mod.rs b/firmware/obc-route/tests/common/mod.rs index ca0bdb5dbe..848573c737 100644 --- a/firmware/obc-route/tests/common/mod.rs +++ b/firmware/obc-route/tests/common/mod.rs @@ -58,7 +58,7 @@ pub fn route_points(obcr: &[u8]) -> Vec { // The hand-rolled OBCR writer, the format tests' independent oracle. It deliberately does not go // through `gpx_to_obcr`: the converter decides chunk boundaries, decimation and totals itself, so a // fixture built with it could only prove that the reader agrees with the writer. Emitting the bytes -// here against `OBCR_Spec.md` pins both sides to the spec, and lets a test place points in a chosen +// here against `obcr-route.md` pins both sides to the spec, and lets a test place points in a chosen // chunk with chosen cumulative distances, or lie about a field on purpose. Only the fields the // reader reads are populated. diff --git a/firmware/obc-storage/src/flat/bitmap.rs b/firmware/obc-storage/src/flat/bitmap.rs index 0f0e80793a..5316cb716f 100644 --- a/firmware/obc-storage/src/flat/bitmap.rs +++ b/firmware/obc-storage/src/flat/bitmap.rs @@ -1,4 +1,4 @@ -//! The free-extent bitmap (`FLAT_Store_Format.md`): the complement of the catalog, recomputed at +//! The free-extent bitmap (`flat-store-format.md`): the complement of the catalog, recomputed at //! mount. //! //! There is no free list on the card and nothing to reconcile — an extent is free exactly when no diff --git a/firmware/obc-storage/src/flat/catalog.rs b/firmware/obc-storage/src/flat/catalog.rs index 9d3ff94784..86a35aebfb 100644 --- a/firmware/obc-storage/src/flat/catalog.rs +++ b/firmware/obc-storage/src/flat/catalog.rs @@ -1,4 +1,4 @@ -//! The catalog (`FLAT_Store_Format.md`): the header, the 128-byte entry, the gate sector, and the +//! The catalog (`flat-store-format.md`): the header, the 128-byte entry, the gate sector, and the //! structural rules an entry array must satisfy. //! //! The catalog is the store. It names every object that exists and which extents each one occupies, diff --git a/firmware/obc-storage/src/flat/journal.rs b/firmware/obc-storage/src/flat/journal.rs index 63b3937ecb..4ed9dbfa9b 100644 --- a/firmware/obc-storage/src/flat/journal.rs +++ b/firmware/obc-storage/src/flat/journal.rs @@ -1,4 +1,4 @@ -//! The ride journal (`FLAT_Store_Format.md`): the one place in this format where bytes become +//! The ride journal (`flat-store-format.md`): the one place in this format where bytes become //! durable without a commit. //! //! Each slot's tail is one full program page. Its header lives in a separate, page-isolated record and diff --git a/firmware/obc-storage/src/flat/layout.rs b/firmware/obc-storage/src/flat/layout.rs index d324eef8ba..f50f2fac28 100644 --- a/firmware/obc-storage/src/flat/layout.rs +++ b/firmware/obc-storage/src/flat/layout.rs @@ -1,5 +1,5 @@ //! Card geometry, the extent area, and the address arithmetic that is the whole read path -//! (`FLAT_Store_Format.md`). +//! (`flat-store-format.md`). //! //! Every address is arithmetic on these constants and on the one value that is not a constant: the //! extent size, which initialization scales to the card and the superblock records. [`Geometry`] is diff --git a/firmware/obc-storage/src/flat/seam.rs b/firmware/obc-storage/src/flat/seam.rs index 13560f0a62..b2e4b16806 100644 --- a/firmware/obc-storage/src/flat/seam.rs +++ b/firmware/obc-storage/src/flat/seam.rs @@ -1,5 +1,5 @@ //! The store seam: the five operations, the two members beside them, and the whole vocabulary they -//! speak (`FLAT_Store_Protocol.md`). +//! speak (`flat-store-protocol.md`). //! //! Nothing here names a block, an extent, an LBA, a path or a filename. [`EntryMeta`] is the metadata //! half of a catalog entry and [`Allocation`] is an opaque token that stands for reserved extents. @@ -26,7 +26,7 @@ impl ObjectId { pub const NONE: ObjectId = ObjectId(0); } -/// `FLAT_Store_Format.md` is the sole authority for these values. +/// `flat-store-format.md` is the sole authority for these values. #[derive(Debug, Clone, Copy, PartialEq, Eq)] #[repr(u16)] pub enum ObjectKind { diff --git a/firmware/obc-storage/src/flat/store.rs b/firmware/obc-storage/src/flat/store.rs index 82047aa690..eac191f944 100644 --- a/firmware/obc-storage/src/flat/store.rs +++ b/firmware/obc-storage/src/flat/store.rs @@ -1,5 +1,5 @@ //! The store: mount, initialization, the alternating catalog commit, the ride journal's write half -//! and the seam operations (`FLAT_Store_Format.md`, `FLAT_Store_Protocol.md`). The entry array stays +//! and the seam operations (`flat-store-format.md`, `flat-store-protocol.md`). The entry array stays //! on the card; resident state is the 8 KiB free bitmap and a few rows. //! //! Every operation takes `&self`, mutators included, and the resident state that moves lives behind diff --git a/firmware/obc-storage/src/flat/superblock.rs b/firmware/obc-storage/src/flat/superblock.rs index 4687ae9802..303269d758 100644 --- a/firmware/obc-storage/src/flat/superblock.rs +++ b/firmware/obc-storage/src/flat/superblock.rs @@ -1,4 +1,4 @@ -//! The superblock (`FLAT_Store_Format.md`): written once by initialization and never again. +//! The superblock (`flat-store-format.md`): written once by initialization and never again. //! //! Nothing in normal operation updates it, which is why it carries no gate and no sequence. Two //! copies of identical bytes exist so that one bad block does not make the card unreadable. diff --git a/firmware/obc-storage/src/flat/vectors.rs b/firmware/obc-storage/src/flat/vectors.rs index e08dc7bfc3..f7afba079b 100644 --- a/firmware/obc-storage/src/flat/vectors.rs +++ b/firmware/obc-storage/src/flat/vectors.rs @@ -1,7 +1,7 @@ //! Every test vector in both frozen specs, pinned byte for byte. //! -//! `FLAT_Store_Format.md`'s vectors are the on-card ones, and this crate encodes them, so they are -//! pinned against the encoder's own output. `FLAT_Store_Protocol.md`'s frames belong to the engine; +//! `flat-store-format.md`'s vectors are the on-card ones, and this crate encodes them, so they are +//! pinned against the encoder's own output. `flat-store-protocol.md`'s frames belong to the engine; //! what is pinned of them here is the half this contract owns — they carry the format's objects and //! offsets, and the two documents agree about them or one of the two is wrong. //! @@ -247,7 +247,7 @@ fn capacity_table() { assert_eq!(TAIL_CAPACITY, 16_384); } -/// `FLAT_Store_Protocol.md`'s `LIST` response carries the two objects. Its 88-byte entries are the +/// `flat-store-protocol.md`'s `LIST` response carries the two objects. Its 88-byte entries are the /// format entries' metadata, and this is the seam between the two specs. #[test] fn wire_list_entries_carry_the_same_metadata() { diff --git a/firmware/tools/ui_frames/environments.py b/firmware/tools/ui_frames/environments.py index a6387f2ff6..9079bbfd85 100644 --- a/firmware/tools/ui_frames/environments.py +++ b/firmware/tools/ui_frames/environments.py @@ -61,7 +61,7 @@ def plain_route(stage: Stage) -> Staging: def _ride(samples, name, start, distance_m, moving_s, climb_m, descent_m, hr, trip): - """A ride object as `specs/obc-ble-interface-spec.md` §7.2 lays it out: 20-byte samples, then + """A ride object as `specs/ble-interface.md` §7.2 lays it out: 20-byte samples, then the 154-byte footer. `samples` are `(lon, lat, ele)`; `hr` is one heart rate per sample or `None`, and a ride with heart rate records a 185 bpm max HR and a 250 W FTP; `trip` is `(key, day index, day count, name)` or `None`. @@ -136,7 +136,7 @@ def tracks(stage: Stage) -> Staging: def _shift_route_lon(route: bytearray, dlon: int) -> None: - """Move a waypoint-less OBCR east by `dlon` µdeg (`specs/OBCR_Spec.md`: the header start and + """Move a waypoint-less OBCR east by `dlon` µdeg (`specs/obcr-route.md`: the header start and bbox, and each chunk meta's bbox and anchor). Points are deltas from the anchor, so they follow. """ for off in (8, 16, 24): @@ -149,7 +149,7 @@ def _shift_route_lon(route: bytearray, dlon: int) -> None: def _rename_route(route: bytearray, name: str) -> None: - """Rewrite an OBCR header's name (`specs/OBCR_Spec.md` §1: Name Len at 6, Name at 64).""" + """Rewrite an OBCR header's name (`specs/obcr-route.md` §1: Name Len at 6, Name at 64).""" raw = name.encode() route[6] = len(raw) route[64:112] = raw.ljust(48, b"\0") @@ -181,7 +181,7 @@ def trips(stage: Stage) -> Staging: def trip_week(stage: Stage) -> Staging: """A seven-day trip, "Alpen Traverse Nord", on seven renamed copies of the plain vector route, starting Monday 2025-09-29. Its day list scrolls, so the title bar shows the counter. The trip - object is written here as `specs/obc-ble-interface-spec.md` §7.7 lays it out; the simulator + object is written here as `specs/ble-interface.md` §7.7 lays it out; the simulator maps each day's route index to the imported route. """ where = stage.dir("trip-week") diff --git a/host/obc-bake/assets/README.md b/host/obc-bake/assets/README.md index 7259cff12a..045223a4ad 100644 --- a/host/obc-bake/assets/README.md +++ b/host/obc-bake/assets/README.md @@ -30,7 +30,7 @@ fixtures/build-map-package.sh terrain ``` It is baked at the real v1 posting (`2^9` µdeg) with a `2^16` cell, so the 1 × 2 cell rectangle is -65 576 B rather than a 2 MiB v1 cell that is mostly outside the crop. `OBCT_Spec.md` §1.3 makes +65 576 B rather than a 2 MiB v1 cell that is mostly outside the crop. `obct-terrain.md` §1.3 makes both header data and §4.5 requires a reader to accept the pairing. There is no terrain in the map, deliberately: the Rhine plain has nothing to show in a 1.2 km diff --git a/host/obc-bake/src/cells.rs b/host/obc-bake/src/cells.rs index d96f205efe..dcaafb0baf 100644 --- a/host/obc-bake/src/cells.rs +++ b/host/obc-bake/src/cells.rs @@ -1,5 +1,5 @@ -//! The cell bake: named regions in, an [`OBCA`](../../../specs/OBCA_Spec.md) cell store plus an -//! [`OBCC`](../../../specs/OBCC_Spec.md) catalog out. +//! The cell bake: named regions in, an [`OBCA`](../../../specs/obca-assembly.md) cell store plus an +//! [`OBCC`](../../../specs/obcc-catalog.md) catalog out. //! //! `data/regions/` stays the curation surface — a region is still one reviewable file — but it names //! a selection rather than an artifact: the set of grid cells its coverage polygon touches, per @@ -893,7 +893,7 @@ impl CellBakery<'_> { if !existing.sidecar.partial && dest.is_file() { progress.warn(format!( " {} [{}]: kept the canonical cell — this run's sources ({}) do not cover its square, \ - and a partial cell must never replace a covering bake (OBCA_Spec.md §3.7). Co-bake {} \ + and a partial cell must never replace a covering bake (obca-assembly.md §3.7). Co-bake {} \ to refresh it.", artifact.id, artifact.band, diff --git a/host/obc-bake/src/cut.rs b/host/obc-bake/src/cut.rs index 7e6bb0c721..e8081b2cb5 100644 --- a/host/obc-bake/src/cut.rs +++ b/host/obc-bake/src/cut.rs @@ -1,5 +1,5 @@ //! The cell cutter: one ingested extract in, the cell artifacts of every band it touches out -//! (`OBCA_Spec.md`). +//! (`obca-assembly.md`). //! //! A cell artifact is an ordinary OBCM file a device could open on its own. What makes it a cell is //! a set of constraints, and every one of them is here. The header bbox is the grid square and not diff --git a/host/obc-bake/src/guard.rs b/host/obc-bake/src/guard.rs index d145ceba5e..3f62ab5ce0 100644 --- a/host/obc-bake/src/guard.rs +++ b/host/obc-bake/src/guard.rs @@ -43,7 +43,7 @@ impl GuardOutcome { let mut s = format!( "obcm-version guard: FAILED — this build writes OBCM v{expected}, but {} of {cells} \ published cells are a different version.\n\nAn OBCM bump invalidates every baked cell \ - (OBCC_Spec.md §10): every assembly from the catalog is unreadable to this firmware until the bakery \ + (obcc-catalog.md §10): every assembly from the catalog is unreadable to this firmware until the bakery \ re-runs. Re-bake and re-publish:\n\n obc-bake bake --out --force\n obc-bake \ publish --base-url --target r2\n\nStale schemas:\n", found.len() @@ -158,7 +158,7 @@ impl CellStoreOutcome { let _ = writeln!( s, " {} cell(s) are `partial` — co-bake the neighbouring extract to complete them \ - (OBCA_Spec.md §3.7):", + (obca-assembly.md §3.7):", self.partial.len() ); for id in self.partial.iter().take(10) { @@ -335,7 +335,7 @@ pub fn check_cell_store(tree: &std::path::Path) -> Result --force ", name_revision(network_terrain_revision), doc.revision diff --git a/host/obc-bake/src/main.rs b/host/obc-bake/src/main.rs index 28f4f24f99..a2455404c9 100644 --- a/host/obc-bake/src/main.rs +++ b/host/obc-bake/src/main.rs @@ -62,7 +62,7 @@ usage: obc-bake terrain [REGION…] --sources DIR [flags] Bake the curated coverage's OBCT terrain cells into the tree's terrain band. Terrain has its OWN revision track: this never re-bakes an OBCM cell, and a - schema bump never re-bakes a terrain cell (OBCC_Spec.md §13). + schema bump never re-bakes a terrain cell (obcc-catalog.md §13). --out TREE output tree (default: ./obc-bake) --sources DIR source DEM GeoTIFFs (default: fetched into /dem) --dataset-id ID source dataset (default: copernicus-glo-30) @@ -73,9 +73,9 @@ usage: --reference DIR reference archive mirror: `index.json` plus the tiles of the box, from host/obc-dem/reference/ingest.py. Where it covers a crest, the baked samples carry the finer model's height - (OBCT_Spec.md §9), each cell records the sources it used, and + (obct-terrain.md §9), each cell records the sources it used, and their credits reach the catalog. A reference change is a - terrain revision bump (OBCC_Spec.md §13.2), which + terrain revision bump (obcc-catalog.md §13.2), which re-stamps every cell and re-bakes only the cells the changed archive tiles reach. A cell the mirror is short of tiles for is REFUSED, with the --bbox to mirror: it diff --git a/host/obc-bake/src/source.rs b/host/obc-bake/src/source.rs index 07b1582783..215cffc5dc 100644 --- a/host/obc-bake/src/source.rs +++ b/host/obc-bake/src/source.rs @@ -152,7 +152,7 @@ impl ExtractSource for LocalExtracts { } Err(format!( "no coverage polygon for `{}` — looked for {} and {}. A cell bake needs it: it is what decides which \ - cells the region selects and whether a border cell is canonical (OBCA_Spec.md §3.7)", + cells the region selects and whether a border cell is canonical (obca-assembly.md §3.7)", region.id, nested.display(), flat.display() diff --git a/host/obc-bake/src/terrain.rs b/host/obc-bake/src/terrain.rs index 54bfce4e12..4dc82f4933 100644 --- a/host/obc-bake/src/terrain.rs +++ b/host/obc-bake/src/terrain.rs @@ -196,11 +196,11 @@ fn validate_doc(doc: &TerrainDoc) -> Result<(), String> { return Err("`dataset_id` and `dataset_version` must be non-empty — they are half the lockstep key".into()); } if doc.attribution.trim().is_empty() { - return Err("`attribution` must be non-empty — the credit travels with the data (OBCC_Spec.md §13.5)".into()); + return Err("`attribution` must be non-empty — the credit travels with the data (obcc-catalog.md §13.5)".into()); } obc_formats::obct::cell_samples_log2(doc.posting_log2, doc.cell_log2).ok_or_else(|| { format!( - "posting 2^{} µdeg with cell 2^{} µdeg is not a pairing OBCT permits (OBCT_Spec.md §1.3)", + "posting 2^{} µdeg with cell 2^{} µdeg is not a pairing OBCT permits (obct-terrain.md §1.3)", doc.posting_log2, doc.cell_log2 ) })?; diff --git a/host/obc-bake/src/verify.rs b/host/obc-bake/src/verify.rs index 31e77f283d..dbd1e28665 100644 --- a/host/obc-bake/src/verify.rs +++ b/host/obc-bake/src/verify.rs @@ -63,7 +63,7 @@ pub fn verify_cell(path: &Path, square: (i64, i64, i64, i64)) -> Result Result Result Result Result Result( if sha != sha256 { problems.push(format!( "{rel}: sha256 {sha}, root pinned {sha256} — a satellite that does not match its pin MUST be rejected \ - and the root retained (OBCC_Spec.md §9)" + and the root retained (obcc-catalog.md §9)" )); return None; } diff --git a/host/obc-dem/Cargo.toml b/host/obc-dem/Cargo.toml index 5f56332a5e..8d00039c32 100644 --- a/host/obc-dem/Cargo.toml +++ b/host/obc-dem/Cargo.toml @@ -3,7 +3,7 @@ name = "obc-dem" version = "0.1.0" edition = "2021" publish = false # internal host tool — not published to crates.io -description = "Copernicus GLO-30 GeoTIFF -> OBCT terrain cells (obc-dem fetch / bake). The producer side of specs/OBCT_Spec.md." +description = "Copernicus GLO-30 GeoTIFF -> OBCT terrain cells (obc-dem fetch / bake). The producer side of specs/obct-terrain.md." # The **container writer is this crate's wasm-reachable half**. `obcm-assemble` places published # cells into a terrain shard (EL4, #1072) and compiles for `wasm32-unknown-unknown`, so it takes this @@ -40,7 +40,7 @@ required-features = ["terrarium"] # exists to prevent — this crate writes the bytes that module names. obc-formats = { workspace = true } # The **shared** lattice arithmetic and the **shared** reader. A baker that re-derived "which cell -# owns sample i" from the prose would be a second implementation of the one thing `OBCT_Spec.md` §5 +# owns sample i" from the prose would be a second implementation of the one thing `obct-terrain.md` §5 # says two implementations must agree on — so this crate does the shifts with `obc_elevation::grid` # and every test reads its own output back through `obc_elevation::TerrainReader`. Depending on a # `no_std` foundation crate from a host tool is the allowed direction (`dependency_rules.json`). diff --git a/host/obc-dem/reference/README.md b/host/obc-dem/reference/README.md index d80ec0e198..84902b801a 100644 --- a/host/obc-dem/reference/README.md +++ b/host/obc-dem/reference/README.md @@ -2,7 +2,7 @@ Copernicus GLO-30 is the native elevation source, but 30 m cannot hold a rock tower. The archive holds finer national DTM heights on the OBCT lattice, and the bakery lifts a crest to them. See -`specs/OBCT_Spec.md` §9 for the lift rule. +`specs/obct-terrain.md` §9 for the lift rule. `obc data source prepare terrain` fills the archive from the national DTMs, never a bake. @@ -145,7 +145,7 @@ nothing into it**; unpacked members go into the work directory under the digest The attribution of every source a map's cells read must travel with that map, beside the Copernicus attribution the native heights carry. `index.json` states it per source; the terrain bakery records per cell which sources its lifts came from, and the catalog's terrain block lists -each of them once in `references` (`OBCC_Spec.md` §13.1). +each of them once in `references` (`obcc-catalog.md` §13.1). ## A local archive diff --git a/host/obc-dem/tests/assets.rs b/host/obc-dem/tests/assets.rs index 0481e8671f..a518835aa5 100644 --- a/host/obc-dem/tests/assets.rs +++ b/host/obc-dem/tests/assets.rs @@ -13,7 +13,7 @@ use obc_elevation::{TerrainReader, TileCache, DEFAULT_TILE_SLOTS}; use obc_formats::io::SliceSource; /// The pairing both sidecars are baked at: the **real** v1 posting with a 2^16 cell, so a map-sized -/// box is a few hundred KB instead of four 2 MiB blocks mostly outside the map. `OBCT_Spec.md` §1.3 +/// box is a few hundred KB instead of four 2 MiB blocks mostly outside the map. `obct-terrain.md` §1.3 /// makes both header data for this reason and §4.5 requires a reader to accept the pairing. const POSTING_LOG2: u8 = 9; const CELL_LOG2: u8 = 16; @@ -74,7 +74,7 @@ fn the_grimsel_sidecar_covers_its_map_and_reads_as_the_grimsel() { /// The skin-preview sidecar: the Rhine plain around Teningen, where the interesting property is that /// the terrain is nearly flat and low — the opposite of the alpine one, and a check that a 2-cell -/// container is as valid as a wide one (`OBCT_Spec.md` §4.1). +/// container is as valid as a wide one (`obct-terrain.md` §4.1). #[test] fn the_teningen_sidecar_covers_the_preview_camera() { let bytes = asset("host/obc-bake/assets/teningen-preview.obcd"); diff --git a/host/obc-map-core/src/grid.rs b/host/obc-map-core/src/grid.rs index 7b4ef6fd89..ccd137db3d 100644 --- a/host/obc-map-core/src/grid.rs +++ b/host/obc-map-core/src/grid.rs @@ -1,7 +1,7 @@ //! The OBCA cell grid: the fixed global µdeg lattice, cell identity, the per-band cell-size table, //! and the integer arithmetic the cell cutter cuts with. //! -//! Everything here is normative in [`OBCA_Spec.md`](../../../specs/OBCA_Spec.md) and deliberately +//! Everything here is normative in [`obca-assembly.md`](../../../specs/obca-assembly.md) and deliberately //! integer-only: the grid exists so that an OBCM quadtree's floor-midpoint subdivision lands exactly //! on cell boundaries, and a single rounding step in the wrong direction would break that. No float //! ever reaches a coordinate computed in this module. @@ -222,7 +222,7 @@ impl BandRole { } } -/// One band: a named class of cell content with one cell size. The JSON shape is `OBCC_Spec.md`'s +/// One band: a named class of cell content with one cell size. The JSON shape is `obcc-catalog.md`'s /// `bands` entry verbatim, so a bakery can hand the catalog's own schema straight to the cutter. #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] pub struct Band { diff --git a/host/obc-mkimage/src/main.rs b/host/obc-mkimage/src/main.rs index 6dac2f02d5..c6adade3ce 100644 --- a/host/obc-mkimage/src/main.rs +++ b/host/obc-mkimage/src/main.rs @@ -14,7 +14,7 @@ //! same step); `sign` attaches or replaces the Ed25519 trailer on an already-wrapped container; //! `inspect` decodes, verifies both CRCs **and** the signature, and exits non-zero on any failure — //! it is the release workflow's gate. See `firmware/README.md` (§Firmware update images) for how the -//! raw `.bin` is produced, and `specs/OBCU_Spec.md` §1 for the bytes. +//! raw `.bin` is produced, and `specs/obcu-update.md` §1 for the bytes. use std::io::Write; use std::path::{Component, Path, PathBuf}; diff --git a/host/obc-pack/schema/catalog.schema.json b/host/obc-pack/schema/catalog.schema.json index d40303d273..ac9790a481 100644 --- a/host/obc-pack/schema/catalog.schema.json +++ b/host/obc-pack/schema/catalog.schema.json @@ -1,7 +1,7 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", "title": "OpenBikeComputer map catalog (schema_version 3)", - "description": "The schema_version 3 map catalog: one schema, a set of skins, named cell-set regions with drawable boundaries, and digest-pinned per-band cell indices. A consumer prices a selection from this document alone — total bytes and bytes per band, which is the per-file projection a volume set needs — and verifies each satellite against the `bytes` + `sha256` pinned here. The satellite documents are `$defs/CellIndexDocument`, `$defs/RegionCellsDocument` and `$defs/TerrainIndexDocument`. The optional `terrain` block is a second artifact class on its own dataset and lattice identity (§13). Normative contract: OBCC_Spec.md, with OBCA_Spec.md for the grid, bands, and assembly and OBCT_Spec.md for the raster.", + "description": "The schema_version 3 map catalog: one schema, a set of skins, named cell-set regions with drawable boundaries, and digest-pinned per-band cell indices. A consumer prices a selection from this document alone — total bytes and bytes per band, which is the per-file projection a volume set needs — and verifies each satellite against the `bytes` + `sha256` pinned here. The satellite documents are `$defs/CellIndexDocument`, `$defs/RegionCellsDocument` and `$defs/TerrainIndexDocument`. The optional `terrain` block is a second artifact class on its own dataset and lattice identity (§13). Normative contract: obcc-catalog.md, with obca-assembly.md for the grid, bands, and assembly and obct-terrain.md for the raster.", "type": "object", "properties": { "schema_version": { @@ -175,11 +175,11 @@ "minItems": 1 }, "routing": { - "description": "Routing facts baked into the cells. The island-prune threshold is schema data: two cells pruned at different thresholds do not assemble into a graph with consistent semantics (`OBCA_Spec.md` §3.5).", + "description": "Routing facts baked into the cells. The island-prune threshold is schema data: two cells pruned at different thresholds do not assemble into a graph with consistent semantics (`obca-assembly.md` §3.5).", "$ref": "#/$defs/RoutingEntry" }, "chunk_size": { - "description": "The per-LOD chunk capacity bound the cells were written with (`OBCM_Spec.md` §3). An assembler re-checks every copied offset pair against it.", + "description": "The per-LOD chunk capacity bound the cells were written with (`obcm-map.md` §3). An assembler re-checks every copied offset pair against it.", "type": "integer", "format": "uint32", "minimum": 0 @@ -200,7 +200,7 @@ ] }, "GridEntry": { - "description": "`OBCA_Spec.md` §1.1's constants. Published so a consumer computing a cell's square or an assembly bbox never hard-codes a power of two.", + "description": "`obca-assembly.md` §1.1's constants. Published so a consumer computing a cell's square or an assembly bbox never hard-codes a power of two.", "type": "object", "properties": { "origin_udeg": { @@ -232,7 +232,7 @@ "minimum": 0 }, "max_mpp": { - "description": "Meters-per-pixel upper bound; `null` for the `+inf` coarsest level (`OBCM_Spec.md` §3).", + "description": "Meters-per-pixel upper bound; `null` for the `+inf` coarsest level (`obcm-map.md` §3).", "type": [ "number", "null" @@ -299,19 +299,19 @@ "description": "A non-geometry OBCM section a band may carry.", "oneOf": [ { - "description": "The navigation graph (`OBCM_Spec.md` §8).", + "description": "The navigation graph (`obcm-map.md` §8).", "type": "string", "const": "nav" }, { - "description": "POIs and the hours pool (`OBCM_Spec.md` §7).", + "description": "POIs and the hours pool (`obcm-map.md` §7).", "type": "string", "const": "poi" } ] }, "BandRole": { - "description": "Which file of a volume set a band's content assembles into (`OBCA_Spec.md`).", + "description": "Which file of a volume set a band's content assembles into (`obca-assembly.md`).", "oneOf": [ { "description": "The one unsplittable file: nav graph, POIs, style table. Carries no LOD, because its headroom under `4 GiB − 1` is the design's hard limit.", @@ -335,7 +335,7 @@ "type": "object", "properties": { "id": { - "description": "1-based style id, as referenced by feature headers (`OBCM_Spec.md` §2, §5.2).", + "description": "1-based style id, as referenced by feature headers (`obcm-map.md` §2, §5.2).", "type": "integer", "format": "uint8", "minimum": 1, @@ -357,13 +357,13 @@ "type": "object", "properties": { "min_component_edges": { - "description": "Island-prune threshold, applied at **assembly** over the merged graph (`OBCA_Spec.md` §3.5/§4.6) — a cell bake prunes only strictly interior components.", + "description": "Island-prune threshold, applied at **assembly** over the merged graph (`obca-assembly.md` §3.5/§4.6) — a cell bake prunes only strictly interior components.", "type": "integer", "format": "uint32", "minimum": 0 }, "profiles": { - "description": "Profile names in `OBCM_Spec.md` §8.6 table order.", + "description": "Profile names in `obcm-map.md` §8.6 table order.", "type": "array", "items": { "type": "string" @@ -398,7 +398,7 @@ "minimum": 0 }, "marker_color": { - "description": "RGB565 user-position marker color (`OBCM_Spec.md` §1).", + "description": "RGB565 user-position marker color (`obcm-map.md` §1).", "type": "integer", "format": "uint16", "minimum": 0, @@ -434,7 +434,7 @@ ] }, "SkinStyle": { - "description": "One feature type's presentation values — the bytes of a style record a skin owns (colors, weight, z, priority and the `Flags` bits). The **id** is not here: it belongs to the schema, and a skin may not renumber it (`OBCA_Spec.md` §4.7).", + "description": "One feature type's presentation values — the bytes of a style record a skin owns (colors, weight, z, priority and the `Flags` bits). The **id** is not here: it belongs to the schema, and a skin may not renumber it (`obca-assembly.md` §4.7).", "type": "object", "properties": { "feature_type": { @@ -764,7 +764,7 @@ ] }, "TerrainEntry": { - "description": "The terrain artifact class (OBCC_Spec.md §13): an OBCT raster on the OBCA grid. Dataset identity, dataset version, posting size and cell size define its lockstep. Terrain has no map schema binding. Network layer receipts identify the terrain bytes sampled while the bake integrates ascents.", + "description": "The terrain artifact class (obcc-catalog.md §13): an OBCT raster on the OBCA grid. Dataset identity, dataset version, posting size and cell size define its lockstep. Terrain has no map schema binding. Network layer receipts identify the terrain bytes sampled while the bake integrates ascents.", "type": "object", "properties": { "dataset_id": { @@ -778,7 +778,7 @@ "minLength": 1 }, "posting_log2": { - "description": "`log2(P)` of the sample lattice, µdeg (`OBCT_Spec.md` §1.1).", + "description": "`log2(P)` of the sample lattice, µdeg (`obct-terrain.md` §1.1).", "type": "integer", "format": "uint8", "minimum": 4, @@ -891,7 +891,7 @@ ] }, "LandmarkEntry": { - "description": "The catalog's landmark block: the class's credit, and one entry per compiled region (`OBCC_Spec.md` §14).", + "description": "The catalog's landmark block: the class's credit, and one entry per compiled region (`obcc-catalog.md` §14).", "type": "object", "properties": { "attribution": { @@ -1000,11 +1000,11 @@ ] }, "CellEntry": { - "description": "One published cell. There is deliberately no bbox: a cell's coverage is exactly its grid square, which the `id` determines to the microdegree (OBCA_Spec.md §1.3), and the generator verifies the artifact's own OBCM header bbox against it at bake time. A stored copy could only agree (redundant) or disagree (a lie).", + "description": "One published cell. There is deliberately no bbox: a cell's coverage is exactly its grid square, which the `id` determines to the microdegree (obca-assembly.md §1.3), and the generator verifies the artifact's own OBCM header bbox against it at bake time. A stored copy could only agree (redundant) or disagree (a lie).", "type": "object", "properties": { "id": { - "description": "Canonical cell id, `//` (`OBCA_Spec.md` §1.3).", + "description": "Canonical cell id, `//` (`obca-assembly.md` §1.3).", "type": "string", "pattern": "^\\d{1,2}/\\d{4,}/\\d{4,}$" }, @@ -1030,7 +1030,7 @@ "minItems": 1 }, "partial": { - "description": "`true` iff those sources do not fully cover the cell's square (`OBCA_Spec.md` §3.7). A consumer MUST NOT present a partial cell as canonical coverage.", + "description": "`true` iff those sources do not fully cover the cell's square (`obca-assembly.md` §3.7). A consumer MUST NOT present a partial cell as canonical coverage.", "type": "boolean" } }, @@ -1287,7 +1287,7 @@ ] }, "TerrainEmptyRun": { - "description": "An inclusive row run of terrain cells that are all `NODATA` — open ocean, which [`obc-dem`](https://github.com/timohueser/OpenBikeComputer) does not write an object for at all (`OBCT_Spec.md` §4.3 makes an absent cell and an all-void one answer identically). The catalog says so instead, so a hole is still distinguishable from a coverage gap.", + "description": "An inclusive row run of terrain cells that are all `NODATA` — open ocean, which [`obc-dem`](https://github.com/timohueser/OpenBikeComputer) does not write an object for at all (`obct-terrain.md` §4.3 makes an absent cell and an all-void one answer identically). The catalog says so instead, so a hole is still distinguishable from a coverage gap.", "type": "object", "properties": { "start": { diff --git a/host/obc-pack/src/catalog.rs b/host/obc-pack/src/catalog.rs index 9f150f0c35..51dd7403eb 100644 --- a/host/obc-pack/src/catalog.rs +++ b/host/obc-pack/src/catalog.rs @@ -1,7 +1,7 @@ //! The cell catalog: a cell store, one schema, a set of skins, and named regions that are -//! selections rather than artifacts. [`OBCC_Spec.md`](../../../../specs/OBCC_Spec.md) is normative +//! selections rather than artifacts. [`obcc-catalog.md`](../../../../specs/obcc-catalog.md) is normative //! and this module is its only sanctioned producer; the grid, band and cell semantics it publishes -//! are [`OBCA_Spec.md`](../../../../specs/OBCA_Spec.md). +//! are [`obca-assembly.md`](../../../../specs/obca-assembly.md). //! //! Cells are the artifacts. A cell's coverage is exactly its grid square, so a cell entry carries no //! bbox: the id determines the square to the microdegree, and this generator verifies that the @@ -259,7 +259,7 @@ pub fn generate(tree: &Path, opts: &CatalogOptions) -> Result warnings.push(format!( "the network band was baked against terrain revision {baked}, but this catalog publishes terrain \ revision {} — the router's baked ascents and the published raster are two different surfaces. Re-bake \ - the network band against the current terrain (OBCC_Spec.md §13.4).", + the network band against the current terrain (obcc-catalog.md §13.4).", t )), (Some(t), None) => warnings.push(format!( diff --git a/host/obc-pack/src/catalog/cells.rs b/host/obc-pack/src/catalog/cells.rs index c84c381121..1ee25088e0 100644 --- a/host/obc-pack/src/catalog/cells.rs +++ b/host/obc-pack/src/catalog/cells.rs @@ -213,7 +213,7 @@ fn read_cell_row( if sidecar.schema_revision != schema.revision.expect("a tree schema declares its revision") { return Err(format!( "{}: cell was baked at schema revision {} but `{SCHEMA_DOC}` is revision {}. A schema-revision bump \ - invalidates every cell (OBCA_Spec.md §6.3) — re-bake, or publish the revision the cells actually \ + invalidates every cell (obca-assembly.md §6.3) — re-bake, or publish the revision the cells actually \ carry. There is no mixed-revision catalog.", path.display(), sidecar.schema_revision, @@ -228,7 +228,7 @@ fn read_cell_row( if header.version != OBCM_VERSION { return Err(format!( "{}: cell is OBCM v{} but this obc-pack writes v{OBCM_VERSION}. An OBCM format bump invalidates every \ - baked cell (OBCC_Spec.md §10) — re-bake the store with the current packer.", + baked cell (obcc-catalog.md §10) — re-bake the store with the current packer.", path.display(), header.version )); @@ -250,7 +250,7 @@ fn read_cell_row( let name = |v: Option| v.map_or("none".to_string(), |r| r.to_string()); return Err(format!( "{}: cell was baked against terrain revision {} but the store was baked against {}. A cell store \ - samples one terrain revision or none (OBCC_Spec.md §13.4) — re-bake the store.", + samples one terrain revision or none (obcc-catalog.md §13.4) — re-bake the store.", path.display(), name(sidecar.terrain_revision), name(*have) @@ -269,7 +269,7 @@ fn read_cell_row( if got != want { return Err(format!( "{}: cell `{id}`'s header bbox is {got:?} but its grid square is {want:?}. A cell's bbox MUST be \ - exactly its square (OBCA_Spec.md §3.1) — that is what lets an assembler copy its chunk bytes \ + exactly its square (obca-assembly.md §3.1) — that is what lets an assembler copy its chunk bytes \ verbatim, and it is why the catalog stores no bbox for a cell.", path.display() )); @@ -312,7 +312,7 @@ fn read_cell_sidecar(path: &Path) -> Result { fn validate_sources(path: &Path, sources: &mut [CellSource]) -> Result<(), String> { if sources.is_empty() { return Err(format!( - "{}: `sources` is empty — a cell must record what it was baked from (OBCA_Spec.md §3.7)", + "{}: `sources` is empty — a cell must record what it was baked from (obca-assembly.md §3.7)", path.display() )); } diff --git a/host/obc-pack/src/catalog/landmarks.rs b/host/obc-pack/src/catalog/landmarks.rs index 4bfba1d100..828879db88 100644 --- a/host/obc-pack/src/catalog/landmarks.rs +++ b/host/obc-pack/src/catalog/landmarks.rs @@ -3,7 +3,7 @@ //! The third artifact class, read on its own terms the way terrain is. Nothing a consumer //! downloads: a cell already carries the landmark sections cut from these artifacts, so the //! published copies are the provenance and the licence record behind that content -//! (`OBCC_Spec.md` §14). +//! (`obcc-catalog.md` §14). use std::collections::BTreeSet; use std::path::{Path, PathBuf}; @@ -68,7 +68,7 @@ pub(super) fn read_landmarks( if !regions.contains(id.as_str()) { return Err(format!( "{}: a landmark artifact for `{id}`, which this catalog publishes no region for — a landmark \ - artifact is compiled from one region's boundary and belongs to it (OBCC_Spec.md §14)", + artifact is compiled from one region's boundary and belongs to it (obcc-catalog.md §14)", dir.display() )); } @@ -121,7 +121,7 @@ fn read_artifact(id: &str, dir: &Path, base_url: &str) -> Result, /// Routing facts baked into the cells. The island-prune threshold is schema data: /// two cells pruned at different thresholds do not assemble into a graph with - /// consistent semantics (`OBCA_Spec.md` §3.5). + /// consistent semantics (`obca-assembly.md` §3.5). pub routing: RoutingEntry, - /// The per-LOD chunk capacity bound the cells were written with (`OBCM_Spec.md` + /// The per-LOD chunk capacity bound the cells were written with (`obcm-map.md` /// §3). An assembler re-checks every copied offset pair against it. pub chunk_size: u32, } -/// `OBCA_Spec.md` §1.1's constants. Published so a consumer computing a cell's square +/// `obca-assembly.md` §1.1's constants. Published so a consumer computing a cell's square /// or an assembly bbox never hard-codes a power of two. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] pub struct GridEntry { @@ -150,14 +150,14 @@ pub struct LodEntry { /// Ladder index, `0` = coarsest. pub index: u32, /// Meters-per-pixel upper bound; `null` for the `+inf` coarsest level - /// (`OBCM_Spec.md` §3). + /// (`obcm-map.md` §3). pub max_mpp: Option, /// The band whose cells carry this LOD. pub band: String, } /// Which file of a volume set a band's content assembles into -/// (`OBCA_Spec.md`). +/// (`obca-assembly.md`). #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize, JsonSchema)] #[serde(rename_all = "lowercase")] pub enum BandRole { @@ -175,9 +175,9 @@ pub enum BandRole { #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize, JsonSchema)] #[serde(rename_all = "lowercase")] pub enum BandSection { - /// The navigation graph (`OBCM_Spec.md` §8). + /// The navigation graph (`obcm-map.md` §8). Nav, - /// POIs and the hours pool (`OBCM_Spec.md` §7). + /// POIs and the hours pool (`obcm-map.md` §7). Poi, } @@ -199,7 +199,7 @@ pub struct BandEntry { /// One feature type's canonical style id — the assignment baked into every chunk. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] pub struct StyleAssignment { - /// 1-based style id, as referenced by feature headers (`OBCM_Spec.md` §2, §5.2). + /// 1-based style id, as referenced by feature headers (`obcm-map.md` §2, §5.2). pub id: u8, /// `.`, e.g. `highway.primary`. pub feature_type: String, @@ -209,10 +209,10 @@ pub struct StyleAssignment { #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] pub struct RoutingEntry { /// Island-prune threshold, applied at **assembly** over the merged graph - /// (`OBCA_Spec.md` §3.5/§4.6) — a cell bake prunes only strictly interior + /// (`obca-assembly.md` §3.5/§4.6) — a cell bake prunes only strictly interior /// components. pub min_component_edges: u32, - /// Profile names in `OBCM_Spec.md` §8.6 table order. + /// Profile names in `obcm-map.md` §8.6 table order. pub profiles: Vec, } @@ -225,7 +225,7 @@ pub struct SkinEntry { pub description: String, /// The skin's content version. No cell carries a skin. pub version: u32, - /// RGB565 user-position marker color (`OBCM_Spec.md` §1). + /// RGB565 user-position marker color (`obcm-map.md` §1). pub marker_color: u16, /// One entry per `schema.styles` feature type, in schema id order. A skin that /// misses one would ship a map with an invisible layer; one that names a feature @@ -247,7 +247,7 @@ pub struct SkinPreview { /// One feature type's presentation values — the bytes of a style record a skin owns (colors, /// weight, z, priority and the `Flags` bits). The **id** is not here: it belongs to the schema, and -/// a skin may not renumber it (`OBCA_Spec.md` §4.7). +/// a skin may not renumber it (`obca-assembly.md` §4.7). #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] pub struct SkinStyle { pub feature_type: String, @@ -365,7 +365,7 @@ pub struct KnownEmptyRun { /// the generator verifies the artifact's header against it (§8). #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] pub struct CellEntry { - /// Canonical cell id, `//` (`OBCA_Spec.md` §1.3). + /// Canonical cell id, `//` (`obca-assembly.md` §1.3). pub id: String, pub bytes: u64, pub sha256: String, @@ -373,7 +373,7 @@ pub struct CellEntry { /// Every source extract this cell was baked from, sorted by `extract_id`. pub sources: Vec, /// `true` iff those sources do not fully cover the cell's square - /// (`OBCA_Spec.md` §3.7). A consumer MUST NOT present a partial cell as canonical + /// (`obca-assembly.md` §3.7). A consumer MUST NOT present a partial cell as canonical /// coverage. pub partial: bool, } @@ -405,7 +405,7 @@ pub struct RegionCellsDocument { } /// The catalog's terrain block: what the raster is, at what resolution, and the one -/// pinned index that lists its cells (`OBCC_Spec.md` §13.1). +/// pinned index that lists its cells (`obcc-catalog.md` §13.1). /// /// `dataset_version`, `posting_log2` and `cell_log2` bind the terrain index to the root. Nothing about the /// OBCM store appears here, and nothing about terrain appears in [`SchemaEntry`] — @@ -417,7 +417,7 @@ pub struct TerrainEntry { /// That dataset's release identity, e.g. `2021-1`. Opaque to a consumer: it is /// compared for equality, never parsed. pub dataset_version: String, - /// `log2(P)` of the sample lattice, µdeg (`OBCT_Spec.md` §1.1). + /// `log2(P)` of the sample lattice, µdeg (`obct-terrain.md` §1.1). pub posting_log2: u8, /// `log2(S)` of the terrain cell, µdeg. Independent of any band's `cell_log2`. pub cell_log2: u8, @@ -496,7 +496,7 @@ pub struct TerrainCellEntry { /// An inclusive row run of terrain cells that are all `NODATA` — open ocean, which /// [`obc-dem`](https://github.com/timohueser/OpenBikeComputer) does not write an object -/// for at all (`OBCT_Spec.md` §4.3 makes an absent cell and an all-void one answer +/// for at all (`obct-terrain.md` §4.3 makes an absent cell and an all-void one answer /// identically). The catalog says so instead, so a hole is still distinguishable from /// a coverage gap. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] @@ -517,7 +517,7 @@ pub struct RegionTerrain { } /// The catalog's landmark block: the class's credit, and one entry per compiled region -/// (`OBCC_Spec.md` §14). +/// (`obcc-catalog.md` §14). #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] pub struct LandmarkEntry { /// The class's required credit, verbatim, for the same reason the terrain block carries diff --git a/host/obc-pack/src/catalog/regions.rs b/host/obc-pack/src/catalog/regions.rs index 5194ce790e..1b076f2bee 100644 --- a/host/obc-pack/src/catalog/regions.rs +++ b/host/obc-pack/src/catalog/regions.rs @@ -170,7 +170,7 @@ fn read_region( index.get(&canonical).map_err(|e| format!("{}: {e}", doc_path.display()))?.ok_or_else(|| { format!( "{}: band `{}` names cell `{canonical}`, which is not published — either bake it or drop it from \ - the selection (OBCC_Spec.md §6)", + the selection (obcc-catalog.md §6)", doc_path.display(), band.id ) @@ -276,7 +276,7 @@ fn read_region_terrain( if !doc.terrain.is_empty() { return Err(format!( "{}: the region selects {} terrain cell(s) but the tree publishes no terrain — either bake it or drop \ - the selection (OBCC_Spec.md §13.3)", + the selection (obcc-catalog.md §13.3)", doc_path.display(), doc.terrain.len() )); @@ -307,7 +307,7 @@ fn read_region_terrain( None => { return Err(format!( "{}: terrain cell `{canonical}` is not published — either bake it or drop it from the selection \ - (OBCC_Spec.md §13.3)", + (obcc-catalog.md §13.3)", doc_path.display() )) } diff --git a/host/obc-pack/src/catalog/schema.rs b/host/obc-pack/src/catalog/schema.rs index 2b368d2e40..bbe09025c5 100644 --- a/host/obc-pack/src/catalog/schema.rs +++ b/host/obc-pack/src/catalog/schema.rs @@ -72,7 +72,7 @@ const CATALOG_DESCRIPTION: &str = verifies each satellite against the `bytes` + `sha256` pinned here. The satellite documents are \ `$defs/CellIndexDocument`, `$defs/RegionCellsDocument` and `$defs/TerrainIndexDocument`. The optional \ `terrain` block is a second artifact class on its own dataset and lattice identity (§13). Normative contract: \ - OBCC_Spec.md, with OBCA_Spec.md for the grid, bands, and assembly and OBCT_Spec.md for the raster."; + obcc-catalog.md, with obca-assembly.md for the grid, bands, and assembly and obct-terrain.md for the raster."; const BOUNDARY_DESCRIPTION: &str = "A region's simplified outline, in integer microdegrees, `[lat, lon]` per point. Presentation only: it \ @@ -80,14 +80,14 @@ const BOUNDARY_DESCRIPTION: &str = input bbox. The coverage outline a builder draws for a live selection is a different object — the union \ of the selected cells' squares, computed client-side — and is deliberately not in the catalog."; -const TERRAIN_DESCRIPTION: &str = "The terrain artifact class (OBCC_Spec.md §13): an OBCT raster on the OBCA grid. \ +const TERRAIN_DESCRIPTION: &str = "The terrain artifact class (obcc-catalog.md §13): an OBCT raster on the OBCA grid. \ Dataset identity, dataset version, posting size and cell size define its lockstep. \ Terrain has no map schema binding. Network layer receipts identify the terrain bytes \ sampled while the bake integrates ascents."; const CELL_DESCRIPTION: &str = "One published cell. There is deliberately no bbox: a cell's coverage is exactly its grid square, which \ - the `id` determines to the microdegree (OBCA_Spec.md §1.3), and the generator verifies the artifact's own \ + the `id` determines to the microdegree (obca-assembly.md §1.3), and the generator verifies the artifact's own \ OBCM header bbox against it at bake time. A stored copy could only agree (redundant) or disagree (a lie)."; /// Generate the catalog documents' JSON Schema from the same serde model the generator writes and @@ -403,7 +403,7 @@ fn check_band_table(bands: &[BandDoc], lod_count: usize, path: &Path) -> Result< if band.id == TERRAIN_DIR { return Err(format!( "{}: `{TERRAIN_DIR}` is reserved — `cells/{TERRAIN_DIR}/` holds the terrain artifact class, which is \ - on its own revision track and is not a band (OBCC_Spec.md §13.1)", + on its own revision track and is not a band (obcc-catalog.md §13.1)", at() )); } @@ -435,7 +435,7 @@ fn check_band_table(bands: &[BandDoc], lod_count: usize, path: &Path) -> Result< if let Some(other) = lod_owner.insert(lod, &band.id) { return Err(format!( "{}: LOD {lod} is in both band `{other}` and band `{}` — it would be written twice \ - (OBCA_Spec.md §1.2)", + (obca-assembly.md §1.2)", at(), band.id )); @@ -507,7 +507,7 @@ fn check_band_table(bands: &[BandDoc], lod_count: usize, path: &Path) -> Result< let missing: Vec = (0..lod_count).filter(|l| !lod_owner.contains_key(&(*l as u32))).collect(); if !missing.is_empty() { return Err(format!( - "{}: ladder LOD(s) {missing:?} are in no band — a map would be blank at that zoom (OBCA_Spec.md §1.2)", + "{}: ladder LOD(s) {missing:?} are in no band — a map would be blank at that zoom (obca-assembly.md §1.2)", at() )); } @@ -526,7 +526,7 @@ fn ladder(config: &Config, bands: &[BandEntry], path: &Path) -> Result {} (0, Some(mpp)) => { return Err(format!( - "{}: ladder LOD 0 has max_mpp {mpp} — the coarsest level is +inf (`null`), OBCM_Spec.md §3", + "{}: ladder LOD 0 has max_mpp {mpp} — the coarsest level is +inf (`null`), obcm-map.md §3", path.display() )) } @@ -658,7 +658,7 @@ pub(super) fn check_skin_document(json: &str, at: &str) -> Result<(), String> { Err(format!( "{at}: a skin is presentation only, and this one carries schema key(s): {}. A skin is stamped onto cells \ already cut at the schema's ladder, tolerances, merge passes and routing table, so these would have no \ - effect — changing any of them is a new schema revision and a re-bake (OBCC_Spec.md §5). Remove them; a \ + effect — changing any of them is a new schema revision and a re-bake (obcc-catalog.md §5). Remove them; a \ style record may carry {}.", offenders.join(", "), SKIN_STYLE_KEYS.iter().map(|k| format!("`{k}`")).collect::>().join(", ") @@ -680,7 +680,7 @@ fn check_skin_ids(want: &BTreeMap, have: &BTreeMap) -> R if !missing.is_empty() { return Err(format!( "this skin has no style for {}. A missing style would ship a map with an invisible layer \ - (OBCC_Spec.md §5).", + (obcc-catalog.md §5).", joined(missing.into_iter()) )); } @@ -692,7 +692,7 @@ fn check_skin_ids(want: &BTreeMap, have: &BTreeMap) -> R if !renumbered.is_empty() { return Err(format!( "a skin MUST NOT renumber style ids — every feature header in every baked chunk references them \ - (OBCA_Spec.md §4.7): {}", + (obca-assembly.md §4.7): {}", renumbered.join("; ") )); } diff --git a/host/obc-pack/src/catalog/terrain.rs b/host/obc-pack/src/catalog/terrain.rs index 8f2cacc022..bcf489ab9d 100644 --- a/host/obc-pack/src/catalog/terrain.rs +++ b/host/obc-pack/src/catalog/terrain.rs @@ -107,7 +107,7 @@ pub(super) fn read_terrain(tree: &Path, base_url: &str) -> Result Result Result Result` against the archive they were baked \ from, which is where the wording lives.", dir.display() @@ -281,7 +281,7 @@ fn read_terrain_row( if sidecar.terrain_revision != doc.revision { return Err(format!( "{}: terrain cell was baked at terrain revision {} but `{TERRAIN_DOC}` is revision {}. Terrain is \ - lockstep within its own track (OBCC_Spec.md §13.2) — re-bake, or publish the revision the cells \ + lockstep within its own track (obcc-catalog.md §13.2) — re-bake, or publish the revision the cells \ actually carry.", path.display(), sidecar.terrain_revision, @@ -305,7 +305,7 @@ fn read_terrain_row( if (header.posting_log2, header.cell_log2) != (doc.posting_log2, doc.cell_log2) { return Err(format!( "{}: cell is posting 2^{} / cell 2^{} but `{TERRAIN_DOC}` declares 2^{} / 2^{} — one lattice per \ - terrain revision (OBCC_Spec.md §13.2)", + terrain revision (obcc-catalog.md §13.2)", path.display(), header.posting_log2, header.cell_log2, @@ -315,7 +315,7 @@ fn read_terrain_row( } if (header.rows, header.cols) != (1, 1) { return Err(format!( - "{}: a published terrain cell is a container whose rectangle is 1 × 1 (OBCT_Spec.md §4.1), but this \ + "{}: a published terrain cell is a container whose rectangle is 1 × 1 (obct-terrain.md §4.1), but this \ one is {} × {} — that is a shard, not a cell", path.display(), header.rows, diff --git a/host/obc-pack/src/catalog/tests.rs b/host/obc-pack/src/catalog/tests.rs index b25c1e5ebd..73c66e36ed 100644 --- a/host/obc-pack/src/catalog/tests.rs +++ b/host/obc-pack/src/catalog/tests.rs @@ -1766,7 +1766,7 @@ fn the_shipped_schema_and_skin_generate_the_current_catalog() { "coarse", "coarse", "coarse", "coarse", "coarse", "mid", "mid", "mid", "mid", "fine", "fine", "fine", "fine", "fine" ], - "OBCA_Spec.md §1.5's band table partitions the shipped ladder" + "obca-assembly.md §1.5's band table partitions the shipped ladder" ); let skin = &g.root.skins[0]; assert_eq!(skin.id, "default"); diff --git a/host/obc-pack/src/landmarks/README.md b/host/obc-pack/src/landmarks/README.md index dad66cebca..2f47f3fb36 100644 --- a/host/obc-pack/src/landmarks/README.md +++ b/host/obc-pack/src/landmarks/README.md @@ -46,7 +46,7 @@ map-wide language argument. | Photo requests | Every signal is metadata, so a capture holds metadata for every candidate and no image inputs. `--photo-requests` names the two the compiler would use: the second covers a rejection only the bytes can prove. Without that flag the field is never written, and a candidate with no bytes is one the compiler cannot use. | | Counts | Captured sites, candidates, usable text, photos, raw photo bytes. A null approach count means the OSM approach join has not run. | | Records | Sorted by QID: category 1–16, display coordinate, default language, fallback-source QIDs, and every usable language variant with at most four text pages and its own attribution. Colocated QIDs stay separate records. | -| Attribution | Article and photo source, revision, licence and complete notice values, at most 8 KiB per asset. The map serializer derives the device credit from them (`credit.rs`, `OBCM_Spec.md` §9.2). | +| Attribution | Article and photo source, revision, licence and complete notice values, at most 8 KiB per asset. The map serializer derives the device credit from them (`credit.rs`, `obcm-map.md` §9.2). | | Omissions | QID, asset and reason. A rejected photo leaves usable text available. | ### In the bake tree @@ -56,11 +56,11 @@ map-wide language argument. | `landmarks//` | One compiled artifact per curated region, written by `obc bake landmarks`. | | `/content.json` + `.rgb222` | The artifact itself. The cell bake cuts every cell of every region whose coverage reaches it. | | `/landmarks.json` | The declaration: region, recipe version, the digests the capture ran under, and one digest over every other file of the directory. | -| catalogue root, `landmarks` | Each artifact's counts, languages, digest and licence URLs (`OBCC_Spec.md` §14). `obc bake verify` re-computes the digest; `obc bake publish` refuses a directory with no declaration. | +| catalogue root, `landmarks` | Each artifact's counts, languages, digest and licence URLs (`obcc-catalog.md` §14). `obc bake verify` re-computes the digest; `obc bake publish` refuses a directory with no declaration. | `obc data` compiles the same content in `device/landmark-content` and `device/peak-content`, and writes one artifact per network cell in `device/landmarks/` and `device/peaks/` -(`OBCC_Spec.md` §14.3, `specs/obc-data.md`). +(`obcc-catalog.md` §14.3, `specs/obc-data.md`). ### Local language fallback diff --git a/host/obc-places/src/metadata.rs b/host/obc-places/src/metadata.rs index e53a94122f..d50a86127b 100644 --- a/host/obc-places/src/metadata.rs +++ b/host/obc-places/src/metadata.rs @@ -2,7 +2,7 @@ //! category/subtype table, normalize names for the device font, and collapse OSM double-mapping. //! //! The shared category table is canonical and append-only: ids are stable, mirrored in firmware, and pinned -//! normatively in `OBCM_Spec.md`. Subtype `0` is reserved, and `0xFF` is the end-of-chunk sentinel. +//! normatively in `obcm-map.md`. Subtype `0` is reserved, and `0xFF` is the end-of-chunk sentinel. //! First match in table order wins, the same convention as the config's style map. //! //! This stage is deliberately config-free: the tag mapping is hardcoded, so packing the same extract diff --git a/host/obc-places/src/routing.rs b/host/obc-places/src/routing.rs index 37e131ed90..1525e2f65c 100644 --- a/host/obc-places/src/routing.rs +++ b/host/obc-places/src/routing.rs @@ -40,7 +40,7 @@ fn surface_class(surface: Option<&str>) -> u8 { /// Classify a way into its packed way-kind byte, or `None` if the way is not routable. /// /// The byte is `kind = (surface_class << 5) | highway_class`, from the canonical [`highway_class`] -/// and [`surface_class`] tables that `OBCM_Spec.md` mirrors. The device never sees raw tags: a +/// and [`surface_class`] tables that `obcm-map.md` mirrors. The device never sees raw tags: a /// routing profile weights edges purely off this byte. /// /// A way is not routable when any hard-exclude applies: `highway=motorway|motorway_link`; diff --git a/host/obc-vectors/Cargo.toml b/host/obc-vectors/Cargo.toml index 19d090c037..b443c06bab 100644 --- a/host/obc-vectors/Cargo.toml +++ b/host/obc-vectors/Cargo.toml @@ -3,7 +3,7 @@ name = "obc-vectors" version = "0.1.0" edition = "2021" publish = false # test-only: builds + verifies the shared specs/vectors/ fixtures -description = "S0 wire-protocol shared test vectors (obc-ble-interface-spec.md): builders and contract tests for the binary fixtures in specs/vectors/, consumed by both cargo test and the companion app's swift test" +description = "S0 wire-protocol shared test vectors (ble-interface.md): builders and contract tests for the binary fixtures in specs/vectors/, consumed by both cargo test and the companion app's swift test" [dependencies] obc-formats = { workspace = true } diff --git a/host/obc-vectors/src/lib.rs b/host/obc-vectors/src/lib.rs index 79cd3c95ff..87d63ee85d 100644 --- a/host/obc-vectors/src/lib.rs +++ b/host/obc-vectors/src/lib.rs @@ -1,7 +1,7 @@ //! Builders for the shared S0 wire-protocol test vectors (`specs/vectors/`). //! //! Each function constructs one fixture directly from the spec text -//! (`obc-ble-interface-spec.md` / `OBCR_Spec.md`), independently of the production +//! (`ble-interface.md` / `obcr-route.md`), independently of the production //! codecs on either side. The checked-in fixture files are these builders' output; //! `tests/vectors.rs` asserts they haven't drifted. //! @@ -231,7 +231,7 @@ pub fn ride_v6() -> Vec { v } -// OBCT terrain (OBCT_Spec.md) +// OBCT terrain (obct-terrain.md) /// The terrain fixture's posting and cell size as `log2(µdeg)`. The posting is the v1 value /// (`2^9`); the **cell is deliberately not** — a v1 `2^19` cell is 2 MiB of raster, and the whole @@ -272,7 +272,7 @@ pub fn terrain_coord(base_cell: u32, d: u32) -> i32 { (-(1i64 << 28) + ((base_sample + d as i64) << TERRAIN_POSTING_LOG2)) as i32 } -/// A full **OBCT terrain shard** (`OBCT_Spec.md` §4): the 32-byte header, the row-major `uint32` +/// A full **OBCT terrain shard** (`obct-terrain.md` §4): the 32-byte header, the row-major `uint32` /// offset directory over the 2 × 2 cell rectangle (`0` = the absent cell), then the three present /// cell blocks — each 2 × 2 tiles of 16 × 16 `int16` metres, row-major with rows advancing latitude. /// @@ -386,7 +386,7 @@ pub fn update_raw_image() -> Vec { v } -/// A full **OBCU update container** (`OBCU_Spec.md` §1, `UPDATE.BIN`): the fixed +/// A full **OBCU update container** (`obcu-update.md` §1, `UPDATE.BIN`): the fixed /// 64-byte header (magic `OBCU`, version 1, raw-image length + CRC-32, NUL-padded /// `fw_version`, header CRC-32 over bytes `0..60`) followed by [`update_raw_image`]. /// Built straight from the spec's field table — independent of the `obc-dfu` @@ -412,7 +412,7 @@ pub fn update_container_v1() -> Vec { v } -/// A full signed OBCU v2 update container (`OBCU_Spec.md` §1): the same +/// A full signed OBCU v2 update container (`obcu-update.md` §1): the same /// 64-byte header table as [`update_container_v1`] — *including* `header_version` still `1`, the /// flash-once-bootloader guarantee — with v1's reserved bytes `48..52` now carrying the /// signature-scheme marker (`sig_scheme` = 1 · `sig_len` = 64), followed by [`update_raw_image`] @@ -560,7 +560,7 @@ pub fn all() -> Vec<(&'static str, Vec)> { // finished ride-v6 object; headerless sample arrays are not accepted as rides. ("track-log.obct", track_log()), ("track-export.gpx", track_export_gpx()), - // The OBCT terrain shard (`OBCT_Spec.md`): a 2 × 2 cell rectangle with a hole + // The OBCT terrain shard (`obct-terrain.md`): a 2 × 2 cell rectangle with a hole // and a NODATA sample, over a plane. Not a wire layout — a *storage* one, like // `track-log.obct` beside it — and it is here because three implementations will sample it // (the device, the `obc-dem` baker's cross-check, and eventually the browser), and the diff --git a/host/obc-vectors/tests/vectors.rs b/host/obc-vectors/tests/vectors.rs index d943ff50ac..00150615ab 100644 --- a/host/obc-vectors/tests/vectors.rs +++ b/host/obc-vectors/tests/vectors.rs @@ -289,7 +289,7 @@ fn trip_vectors_are_self_consistent() { assert_eq!(days, TRIP_DAYS); } -/// The OBCT terrain shard (`OBCT_Spec.md`): the checked-in bytes parse through the production +/// The OBCT terrain shard (`obct-terrain.md`): the checked-in bytes parse through the production /// `obc-elevation` reader, and the three sampling rules that a second implementation is most likely /// to get wrong — the cross-cell fetch, the coverage clamp and `NODATA` propagation — produce the /// numbers the spec's worked examples state. diff --git a/host/obcm-assemble/Cargo.toml b/host/obcm-assemble/Cargo.toml index 30195b7528..89f5602ae2 100644 --- a/host/obcm-assemble/Cargo.toml +++ b/host/obcm-assemble/Cargo.toml @@ -32,7 +32,7 @@ obc-tree-walk = { workspace = true } # is on for `MapCache::new_boxed`: the ~277 KB cache must not land on a small (wasm) stack. obc-reader = { workspace = true, features = ["alloc"] } obc-map-scene = { workspace = true } -# The **one** OBCT container writer in the tree (`OBCT_Spec.md` §4): a published terrain cell and an +# The **one** OBCT container writer in the tree (`obct-terrain.md` §4): a published terrain cell and an # assembled terrain shard come out of the same code, which is what makes "a cell is a 1 × 1 shard" a # fact. `default-features = false` drops obc-dem's GeoTIFF decoder and HTTP client — neither of which # compiles for wasm32 and neither of which an assembler has any use for. diff --git a/host/obcm-assemble/src/emit.rs b/host/obcm-assemble/src/emit.rs index 4a272a0143..03f6b12522 100644 --- a/host/obcm-assemble/src/emit.rs +++ b/host/obcm-assemble/src/emit.rs @@ -26,7 +26,7 @@ use crate::{Error, Result}; /// The hard per-file ceiling: the smaller of the two walls a written file has to clear. /// -/// 1. The format wall — `OBCM_Spec.md`'s addressable interior at this engine's [`SCALE`], +/// 1. The format wall — `obcm-map.md`'s addressable interior at this engine's [`SCALE`], /// `2^32 units × U`, which at `U = 16` is 64 GiB. /// 2. The readable wall — how far [`obc_formats::io::ByteSource`], the tree's one read interface, /// can address. diff --git a/host/obcm-assemble/src/lib.rs b/host/obcm-assemble/src/lib.rs index df94cd0cae..ad44c25cb5 100644 --- a/host/obcm-assemble/src/lib.rs +++ b/host/obcm-assemble/src/lib.rs @@ -1,6 +1,6 @@ //! `obcm-assemble` — the cell assembly engine: baked OBCA grid cells in, one `.obcm` out. //! -//! The contract this crate implements is [`OBCA_Spec.md`](../../../specs/OBCA_Spec.md). In one +//! The contract this crate implements is [`obca-assembly.md`](../../../specs/obca-assembly.md). In one //! paragraph: because the cell grid is a power-of-two µdeg lattice and an OBCM quadtree subdivides //! its header bbox at integer floor-midpoints, a grid-aligned assembly bbox subdivides onto cell //! boundaries — so at one depth the assembly's quadtree nodes are the cells, and each cell's diff --git a/host/obcm-assemble/src/nav.rs b/host/obcm-assemble/src/nav.rs index 35710f5003..1946ce90ce 100644 --- a/host/obcm-assemble/src/nav.rs +++ b/host/obcm-assemble/src/nav.rs @@ -1,4 +1,4 @@ -//! Merging the navigation graph (`OBCA_Spec.md`). +//! Merging the navigation graph (`obca-assembly.md`). //! //! Node ids are file-local and an `Edge Id` addresses one cell's own pool, so nothing survives //! concatenation: the graph is read back out of every `network` cell, unified at the seams, pruned, @@ -1505,7 +1505,7 @@ fn fnv(bytes: &[u8]) -> u64 { /// `[directory][filler][profile table][alignment run][node index][filler][node chunks][edge pool] /// [alignment run][snap index][filler][snap chunks]`. /// -/// Every one of those gaps is `0xFF`: one fill byte, one rule (`OBCM_Spec.md`) — a gap is `0xFF` +/// Every one of those gaps is `0xFF`: one fill byte, one rule (`obcm-map.md`) — a gap is `0xFF` /// and a reserved field is `0`. An alignment run is a gap no offset reaches, so it takes the gap's /// byte. /// diff --git a/host/obcm-assemble/src/verify.rs b/host/obcm-assemble/src/verify.rs index b238666416..eebe0835cf 100644 --- a/host/obcm-assemble/src/verify.rs +++ b/host/obcm-assemble/src/verify.rs @@ -1,4 +1,4 @@ -//! The verify pass (`OBCA_Spec.md`): read the finished output back through the real reader — the +//! The verify pass (`obca-assembly.md`): read the finished output back through the real reader — the //! same crate the firmware runs — before anything is handed anywhere. //! //! This is a precondition of handing a map anywhere, not an optional extra: a catalog artifact was diff --git a/host/obcm-testkit/src/lib.rs b/host/obcm-testkit/src/lib.rs index 508f9ae7a0..c81368b2a6 100644 --- a/host/obcm-testkit/src/lib.rs +++ b/host/obcm-testkit/src/lib.rs @@ -2,7 +2,7 @@ //! tests. //! //! Both crates synthesise `.obcm` byte buffers by hand rather than checking in a binary fixture, so -//! the Rust reader stays pinned to `OBCM_Spec.md`: if either drifts, the format tests break. This +//! the Rust reader stays pinned to `obcm-map.md`: if either drifts, the format tests break. This //! crate is the single source, so a format bump edits the layout once. //! //! [`build_file`] and [`build_priority_tree`] write empty POI and nav sections so the reader diff --git a/sim/flat-device/src/lib.rs b/sim/flat-device/src/lib.rs index 2408e39177..980f1b06b7 100644 --- a/sim/flat-device/src/lib.rs +++ b/sim/flat-device/src/lib.rs @@ -49,7 +49,7 @@ pub use card::Card; pub use json::{catalog_json, trace_json}; pub use sim::{SimDevice, SimOptions}; -/// `FLAT_Store_Format.md` §2: the fixed region is 2 MiB and the extent area starts on the block +/// `flat-store-format.md` §2: the fixed region is 2 MiB and the extent area starts on the block /// after it. pub const EXTENT_AREA: u64 = 4_096; /// §6: one extent, in blocks. A card this size is well under 64 GiB, so §8 gives it the 1 MiB diff --git a/sim/host-core/src/flat_store/tests.rs b/sim/host-core/src/flat_store/tests.rs index ccfc32d3bb..4727b94e9e 100644 --- a/sim/host-core/src/flat_store/tests.rs +++ b/sim/host-core/src/flat_store/tests.rs @@ -189,7 +189,7 @@ fn exhausted_counter_mount_remains_readable_and_refuses_writes() { owner.import(ObjectKind::Route, None, &mut &ROUTE[..], ROUTE.len() as u64, DisplayName::default()).unwrap(); drop(owner); - // FLAT_Store_Format §§5.1–5.4: the first commit publishes copy B. Construct + // flat-store-format.md §§5.1–5.4: the first commit publishes copy B. Construct // an exhausted, checksummed catalog because the writer refuses counter wrap. use std::io::{Seek, SeekFrom}; let mut file = std::fs::OpenOptions::new().read(true).write(true).open(&path).unwrap(); diff --git a/specs/obc-ble-interface-spec.md b/specs/ble-interface.md similarity index 97% rename from specs/obc-ble-interface-spec.md rename to specs/ble-interface.md index 5766c220d4..c28036fda0 100644 --- a/specs/obc-ble-interface-spec.md +++ b/specs/ble-interface.md @@ -5,7 +5,7 @@ Control service, its live characteristics, and the payload layouts of the object characteristics carry. Pairing and encryption are §8. The QR link that starts pairing is §9. Object transfers are **not** here. They are protocol v4, whose normative contract is -[`FLAT_Store_Protocol.md`](FLAT_Store_Protocol.md); §5.1 of that document binds it to the +[`flat-store-protocol.md`](flat-store-protocol.md); §5.1 of that document binds it to the `protocolVersion`, `psm` and `objectControl` characteristics below. All multi-byte integers are **little-endian**. Shared binary test vectors pinning these layouts @@ -25,7 +25,7 @@ live in [`specs/vectors/`](vectors/) and are read by both `cargo test` and `swif in this preference order: 1. the **installed image's OBCU version string, verbatim** — the `fw_version` field of the container - the device installed ([`OBCU_Spec.md`](OBCU_Spec.md) §1). For a released build that is the + the device installed ([`obcu-update.md`](obcu-update.md) §1). For a released build that is the release tag, e.g. `v1.3.0`; 2. otherwise the **build's bare git short hash**, e.g. `ca9b336` — a device flashed over SWD, which has never installed a container and has no version to report. @@ -38,7 +38,7 @@ MUST NOT offer an automatic update. A development device stays on whatever its o gets back onto the release track through the manual install path. The value is at most 32 bytes (the OBCU `fw_version` field width). This characteristic and the USB -device-information read ([`FLAT_Store_Protocol.md`](FLAT_Store_Protocol.md) §5.2.1) always carry +device-information read ([`flat-store-protocol.md`](flat-store-protocol.md) §5.2.1) always carry identical bytes. ### 3.2 Battery Service — `0x180F` (SIG) @@ -62,7 +62,7 @@ Base UUID (random; **not** derived from the SIG base): the 16-bit block `XXXX` i | `3C920008-9916-4EBA-ABC2-342FE08F6B10` | `protocolVersion` | read, open | `u16` = `4`, the protocol-v4 major | | `3C920009-9916-4EBA-ABC2-342FE08F6B10` | `objectControl` | write + indicate | the protocol-v4 control channel | -The last three are bound by [`FLAT_Store_Protocol.md`](FLAT_Store_Protocol.md) §5.1, which owns +The last three are bound by [`flat-store-protocol.md`](flat-store-protocol.md) §5.1, which owns their byte rules. `3C920003`, `3C920005` and `3C920006` are retired and **MUST NOT** be reused. A retired UUID never @@ -107,7 +107,7 @@ support it". Commands `1` (`deleteObject`) and `2` (`ackRides`) are retired: object removal is protocol v4 `REMOVE` and ride possession is `ARCHIVE_RIDE` -([`FLAT_Store_Protocol.md`](FLAT_Store_Protocol.md) §3.7, §3.12). The device answers them +([`flat-store-protocol.md`](flat-store-protocol.md) §3.7, §3.12). The device answers them `unknown command`. **Next free command: `6`.** **`installFw`.** The command only *requests*; it never waits for the rider and never installs on its @@ -129,7 +129,7 @@ truth about the same fact, so the device accepts and the confirm card reports wh encrypted link (§8). **Installing** is gated on physical confirmation at the device. `installFw` therefore never arms and never reboots on its own. Image authenticity is separate and is checked on every delivery path: the armer verifies the container's Ed25519 signature, and refuses an unsigned -container, before an install can be armed ([`OBCU_Spec.md`](OBCU_Spec.md) §1.3, §1.4). +container, before an install can be armed ([`obcu-update.md`](obcu-update.md) §1.3, §1.4). **`forgetBond`.** The bonded phone asks the device to clear its side of the bond, so an app-side "Forget device" does not leave the pair wedged under the reject-when-bonded posture (§8). The device @@ -165,8 +165,8 @@ immediately after encryption. A timestamp the device stamps later in the session ## 7. Object layouts A route object's payload is exactly the bytes of an OBCR file -([`OBCR_Spec.md`](OBCR_Spec.md)); the device stores and serves it verbatim. An update package's -payload is exactly the bytes of an OBCU container ([`OBCU_Spec.md`](OBCU_Spec.md) §1). +([`obcr-route.md`](obcr-route.md)); the device stores and serves it verbatim. An update package's +payload is exactly the bytes of an OBCU container ([`obcu-update.md`](obcu-update.md) §1). ### 7.2 `ride` — ride object v6 @@ -229,7 +229,7 @@ The final 154 bytes are the summary footer: while the ride records and is not paused. The total is rounded down to whole kJ. A ride without power data writes the absent value, so `0` is a ride with power data and no work. - **Bike type.** The bike type that is current when the ride starts - ([`OBCR_Spec.md`](OBCR_Spec.md) §1.2). The phone can change its own copy. The device copy does + ([`obcr-route.md`](obcr-route.md) §1.2). The phone can change its own copy. The device copy does not change. - **Trip.** A ride is on a trip when it starts on a day of a stored trip (§7.7). The footer holds the trip key, the index of that day and the trip's day count. The rider sees Day 1 for day index 0. A reader requires `day index < day count`. @@ -333,7 +333,7 @@ any other length, which also rejects a torn write. The device keeps one progress record per trip key. The record never crosses the wire. The device writes it at Finish of a ride on a trip day, and keeps it in the ride-archive Metadata object -([`Ride_Archive_Metadata.md`](Ride_Archive_Metadata.md)) with the navigator checkpoint. +([`ride-archive-metadata.md`](ride-archive-metadata.md)) with the navigator checkpoint. - **Start record.** The fresh start of a ride on a trip day, and the continuation of that ride after a reset, write a start record for the trip: position day 0, the route ObjectId of day 0, diff --git a/specs/FLAT_Store_Format.md b/specs/flat-store-format.md similarity index 99% rename from specs/FLAT_Store_Format.md rename to specs/flat-store-format.md index 50b21e0732..3b8b2201e3 100644 --- a/specs/FLAT_Store_Format.md +++ b/specs/flat-store-format.md @@ -2,7 +2,7 @@ - Status: **normative** for the flat card store - Format version: 1 -- Seam and wire contract: [`FLAT_Store_Protocol.md`](FLAT_Store_Protocol.md) +- Seam and wire contract: [`flat-store-protocol.md`](flat-store-protocol.md) This document is the byte contract for the raw SD card. There is no partition table and no filesystem: the card is five fixed regions and an array of equal extents, and every address the @@ -108,11 +108,11 @@ same numbers and defines no others. | 1 | route | OBCR payload | | 2 | trip | the days of a trip, one route each | | 3 | ride | produced by the device; the one growing object (§7) | -| 5 | map shard | one OBCM object, carrying its terrain inside it (`OBCM_Spec.md` §1.3) | +| 5 | map shard | one OBCM object, carrying its terrain inside it (`obcm-map.md` §1.3) | | 6 | map set manifest | **retired.** No producer writes this kind; the value is not reissued | | 7 | update package | OBCU image | | 8 | firmware rollback reserve | extents owned by the store, bytes written by the update path (§5.3) | -| 9 | metadata | device-owned ride archive proofs; [contract](Ride_Archive_Metadata.md) | +| 9 | metadata | device-owned ride archive proofs; [contract](ride-archive-metadata.md) | ## 4. Superblock @@ -284,7 +284,7 @@ Flags: Bits `4..15` are zero. The acceptance flag is set by an `Amend` with unchanged ObjectId, revision, payload length, and CRC in the checkpoint transaction described in -[Ride archive metadata](Ride_Archive_Metadata.md). A fresh payload cannot carry this flag. +[Ride archive metadata](ride-archive-metadata.md). A fresh payload cannot carry this flag. Catalog copying preserves it. Clearing the checkpoint does not clear acceptance. **Display name** is what a menu shows. It is UTF-8, at most 48 bytes, and the store does not @@ -424,7 +424,7 @@ normative part and the time is not. One thing that reads like mount is deliberately outside it: reconciling an update that armed before the last reboot, which may remove an orphaned rollback reserve -([`FLAT_Store_Protocol.md`](FLAT_Store_Protocol.md) §4). That is device policy running against a +([`flat-store-protocol.md`](flat-store-protocol.md) §4). That is device policy running against a mounted store, on the ordinary commit path, and it is not part of bringing the card up. ### 5.7 Vectors diff --git a/specs/FLAT_Store_Protocol.md b/specs/flat-store-protocol.md similarity index 98% rename from specs/FLAT_Store_Protocol.md rename to specs/flat-store-protocol.md index 80fcf8ffba..54cd62ebc5 100644 --- a/specs/FLAT_Store_Protocol.md +++ b/specs/flat-store-protocol.md @@ -1,7 +1,7 @@ # Flat store seam and protocol v4 - Status: **normative** for the flat store's public seam and for wire major **4** -- On-card contract: [`FLAT_Store_Format.md`](FLAT_Store_Format.md) +- On-card contract: [`flat-store-format.md`](flat-store-format.md) §2 is the seam between the store and everything above it inside the firmware. §3 is what crosses the link to a phone, a cable client, or the simulator. §5 binds §3 to BLE and USB. @@ -17,7 +17,7 @@ and covers whole object payloads, never frames. Frame integrity belongs to the l CRC and retransmission, USB packet CRC and retry). `ObjectId`, `Revision`, `StoreId`, object kinds and entry flags are defined by -[`FLAT_Store_Format.md`](FLAT_Store_Format.md) §3 and carry the same values here. +[`flat-store-format.md`](flat-store-format.md) §3 and carry the same values here. ## 2. The store seam @@ -84,7 +84,7 @@ pub trait Store { fn next_object_id(&self) -> ObjectId; /// The ride exception, and the only way bytes become durable without a commit. Performs both - /// halves of `FLAT_Store_Format.md` §7.2: reconstruct the store-owned 16 KiB tail snapshot from + /// halves of `flat-store-format.md` §7.2: reconstruct the store-owned 16 KiB tail snapshot from /// its previous slot plus the bounded append, and gate any whole payload-page rollover. fn journal(&self, checkpoint: RideCheckpoint) -> Result<(), StoreError>; } @@ -124,7 +124,7 @@ pub struct RideCheckpoint<'a> { pub resume: &'a [u8; RIDE_RESUME_LEN], } -/// Why a mounted store refuses writes, or refuses everything (`FLAT_Store_Format.md` §5.6). The +/// Why a mounted store refuses writes, or refuses everything (`flat-store-format.md` §5.6). The /// `readOnly` details of §3.9 are chosen from it: the two exhausted cases still serve reads, and /// `CardTooSmall` — the card is not the card the superblock describes — reads out as `unformatted`. pub enum Mode { @@ -201,7 +201,7 @@ a ride journals. An implementation carries whatever interior mutability that nee covers and for how many commands. State must never be held across an `await`, across a callback, or across another seam call, carve-out or not. 2. **`close` on an object another reader still holds is a runtime refusal, not a teardown.** The - reader refcount of `FLAT_Store_Format.md` §6.2 decides it: such a `close` spends a count and + reader refcount of `flat-store-format.md` §6.2 decides it: such a `close` spends a count and returns, and the extents come back only when the last reader lets go. The guarantee is kept at runtime — *refused*, never *silent*. @@ -318,7 +318,7 @@ Request, 32 bytes: The cursor is the **pair**, and the page resumes strictly after it: the catalog is keyed by `(ObjectId, Revision)` and an object may hold two entries, the retained one sorting first -(`FLAT_Store_Format.md` §5.3). +(`flat-store-format.md` §5.3). Response payload is a 24-byte prefix followed by `n` entries: @@ -455,7 +455,7 @@ A `PUT` naming an entry that carries `RECORDING` or `RESERVED` is refused `inval replaces: those kinds are produced by the device, and a client that could overwrite a ride mid-recording or a rollback reserve mid-update would be writing where the store and the bootloader already are. Metadata writes require device policy admission under -[Ride_Archive_Metadata.md](Ride_Archive_Metadata.md); remote writes cannot establish archive proof. +[ride-archive-metadata.md](ride-archive-metadata.md); remote writes cannot establish archive proof. One commit publishes the new head and removes both the displaced head and any retained revision. @@ -560,7 +560,7 @@ request is well formed and can succeed against the same device a moment later. `readOnly` is the wire face of a store that cannot be written: no usable catalog, an exhausted revision space, or a card §5.6 step 1 classified as **not a flat store** -(`FLAT_Store_Format.md` §3, §5.6). Every opcode returns it in the unformatted and unreadable cases, +(`flat-store-format.md` §3, §5.6). Every opcode returns it in the unformatted and unreadable cases, including the reads, because there is nothing to read; only the exhausted case still serves reads. `FORMAT` is the sole exception: it is the explicit destructive recovery operation in §3.10. @@ -615,7 +615,7 @@ the recovery transport down. ### 3.11 Vectors -`StoreId` and the two objects are those of [`FLAT_Store_Format.md`](FLAT_Store_Format.md) §5.7. +`StoreId` and the two objects are those of [`flat-store-format.md`](flat-store-format.md) §5.7. `PUT` creating the route, `RequestId 0x00002A01`, 100 bytes on the wire: @@ -667,7 +667,7 @@ A complete `LIST` response, `RequestId 0x00002A02`, both entries, no further pag The client MUST send this request only after its archive persistence barriers complete, or after it revalidates an existing durable archive. GET completion, a UI flag and deletion history are not -archive proof. See [the archive contract](Ride_Archive_Contract.md). +archive proof. See [the archive contract](ride-archive-receipts.md). **Request — 44 bytes** @@ -733,8 +733,8 @@ Request, 16 bytes: package `ObjectId u64`, expected `Revision u64`. The device t allocate, only write where it is told. 3. **Writes the boot handoff**: both extent lists, resolved to absolute 512-byte block runs, into the RRAM boot-state page, then reads it back and verifies. The page format and the bootloader's - decision logic are [`OBCU_Spec.md`](OBCU_Spec.md) §2; its `Extent { start_block, blocks }` is - exactly what `FLAT_Store_Format.md` §6.1 computes. + decision logic are [`obcu-update.md`](obcu-update.md) §2; its `Extent { start_block, blocks }` is + exactly what `flat-store-format.md` §6.1 computes. 4. **Answers and drains**: the response, 16 bytes — rollback `ObjectId u64`, new catalog commit sequence `u64` — must reach the transport before the reboot, or the adapter's bounded drain timeout must expire. diff --git a/specs/obc-data.md b/specs/obc-data.md index 59731210c8..d4d4257200 100644 --- a/specs/obc-data.md +++ b/specs/obc-data.md @@ -1120,7 +1120,7 @@ and the step list reads files such as a `.poly` and the GLO-30 tile list, each a [Versions](#versions)). `copernicus-glo-30` and its tile list are `manual`. A leaf reads the archive tiles of its terrain cells: those of the windows that the crest rule of -`OBCT_Spec.md` §9 reads, which have a halo of two postings. A model whose `extent` meets one of +`obct-terrain.md` §9 reads, which have a halo of two postings. A model whose `extent` meets one of those tiles in the version of `terrain` that the target reads (see [Prepared sources](#prepared-sources)): the step list reads its manifest, and the leaf reads the blocks `18//.tif` of the manifest that hold those tiles and its `index.json`. So a machine with an empty store plans a @@ -1134,11 +1134,11 @@ the region; the version that adds it is a new input of the leaves that read it. | --- | --- | --- | --- | | `device/region-osm` | The selected PBF of each `device/source/`; only for multiple areas | None | `osm.pbf`: the available-snapshot union | | `device/osm` | The PBF of one `device/source/`, or `device/region-osm` | `leaves`: `[i, j]` of each leaf | `osm/-.osm.pbf`: the `osmium extract --strategy smart --set-bounds --generator obc-data` of the square of the leaf and one µdeg around it, without the header options of the input, such as its replication fields. Each Osmium run writes at most two extracts: the rectangle around the leaves is halved along its longer side, and each half is cut from the extract of its parent, until each part is one leaf. Each leaf has the bytes of the direct extract. The metrics name the version (`osmium`) | -| `device/map/-` | `device/osm`, the file of the leaf; `land-polygons`; `device/terrain/-`, which the contours, the nav graph and the POIs read | `leaf`: `[23, i, j]`; `cells`: for each band of the recommended band table (`OBCA_Spec.md`), `[ci, cj]` of each cell of the band in the leaf that the outline touches; `partial_cells`: the ids of those cells that the source coverage does not hold | `cells///.obcm` for each cell with content or with partial coverage; `cells//empty.json`: the ids of the other cells of the band. A cell has the bytes that one cut of the whole leaf with all bands writes, with `builder/presets/schema.json` and without landmarks or peaks. The step decodes the OSM of the leaf once | +| `device/map/-` | `device/osm`, the file of the leaf; `land-polygons`; `device/terrain/-`, which the contours, the nav graph and the POIs read | `leaf`: `[23, i, j]`; `cells`: for each band of the recommended band table (`obca-assembly.md`), `[ci, cj]` of each cell of the band in the leaf that the outline touches; `partial_cells`: the ids of those cells that the source coverage does not hold | `cells///.obcm` for each cell with content or with partial coverage; `cells//empty.json`: the ids of the other cells of the band. A cell has the bytes that one cut of the whole leaf with all bands writes, with `builder/presets/schema.json` and without landmarks or peaks. The step decodes the OSM of the leaf once | | `device/reference/-` | `terrain`, without params: the blocks of those tiles in its version and `index.json`; only when the version has one of them | `source`: `terrain`; `tiles`: the ids `/` of the archive tiles that the terrain cells of the leaf read | `reference/`: the reference archive (`host/obc-dem/reference/README.md`) of those tiles that the index names, each cut out of its block, and the `index.json` of the version cut to them, with `tiles` and the `sha256` of the pixels of each tile. A Python step with the group `terrain-reference` | -| `device/terrain/-` | `copernicus-glo-30`, `tile=` of each tile that the square of a cell reaches and that `copernicus-glo-30-tiles` names. A square without a tile is sea. A leaf without a tile reads no snapshot. `device/reference/-` when the leaf has one | `posting_log2` and `cell_log2` of OBCT v1; `cells`: `[ci, cj]` of each terrain cell in the leaf that the outline touches | `terrain//.obcd` for each cell with a height (`OBCC_Spec.md` §13), the bytes that `obc-bake terrain --reference` writes from the same tiles and archive; `metadata/empty.json`: the ids of the cells without a height; `metadata/credits.json`, when a cell reads a national model: `key`, `product`, `attribution` and `licence` of each model that a cell reads, as the reference archive states them | +| `device/terrain/-` | `copernicus-glo-30`, `tile=` of each tile that the square of a cell reaches and that `copernicus-glo-30-tiles` names. A square without a tile is sea. A leaf without a tile reads no snapshot. `device/reference/-` when the leaf has one | `posting_log2` and `cell_log2` of OBCT v1; `cells`: `[ci, cj]` of each terrain cell in the leaf that the outline touches | `terrain//.obcd` for each cell with a height (`obcc-catalog.md` §13), the bytes that `obc-bake terrain --reference` writes from the same tiles and archive; `metadata/empty.json`: the ids of the cells without a height; `metadata/credits.json`, when a cell reads a national model: `key`, `product`, `attribution` and `licence` of each model that a cell reads, as the reference archive states them | | `device/landmark-content[/]`, `device/peak-content[/]` | `wiki`, without params: `common.tar.zst` and the shards of the `2^20` µdeg cells that the `.poly` of the area touches, of its manifest; regional `geofabrik-extracts` and `geofabrik-poly` | None | `landmarks/content.json` or `peaks/peaks.json` and device photos. The metrics count the facts that the version lacks, by kind (`absent`), and the run log names them. No client reads the layer: the device cell layers publish the final assets | -| `device/landmarks/-`, `device/peaks/-` | Every area's landmark content and `device/osm`, the file of the leaf; or every area's peak content | `cell_log2`: 18; `cells`: `[ci, cj]` of each network cell of the leaf: the `network` list of `cells` of `device/map/-` | `landmarks//.bin` or `peaks//.bin` for each cell that owns content (`OBCC_Spec.md` §14.3). A landmark joins the OSM objects of the leaf that name it | +| `device/landmarks/-`, `device/peaks/-` | Every area's landmark content and `device/osm`, the file of the leaf; or every area's peak content | `cell_log2`: 18; `cells`: `[ci, cj]` of each network cell of the leaf: the `network` list of `cells` of `device/map/-` | `landmarks//.bin` or `peaks//.bin` for each cell that owns content (`obcc-catalog.md` §14.3). A landmark joins the OSM objects of the leaf that name it | ``, ``, `` and `` have four digits or more, as in a cell id. diff --git a/specs/OBCA_Spec.md b/specs/obca-assembly.md similarity index 90% rename from specs/OBCA_Spec.md rename to specs/obca-assembly.md index a34e9ca38e..d43cbb98e8 100644 --- a/specs/OBCA_Spec.md +++ b/specs/obca-assembly.md @@ -6,17 +6,17 @@ chunk bytes verbatim (§2), what a baked **cell** is (§3), and what an **assemb must do (§4). OBCA introduces **no new OBCM version and changes no OBCM semantics**. Every cell is an ordinary -[OBCM](OBCM_Spec.md) file that today's reader parses unchanged. What OBCA adds is a set of +[OBCM](obcm-map.md) file that today's reader parses unchanged. What OBCA adds is a set of *constraints* on how those files are produced. This document is normative. The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be interpreted as in RFC 2119. -Related contracts: [`OBCM_Spec.md`](OBCM_Spec.md) is the byte format of every cell and -assembly; [`OBCC_Spec.md`](OBCC_Spec.md) is the catalog manifest that publishes cells and names -the selectable regions; [`OBCT_Spec.md`](OBCT_Spec.md) is the terrain raster, a second artifact +Related contracts: [`obcm-map.md`](obcm-map.md) is the byte format of every cell and +assembly; [`obcc-catalog.md`](obcc-catalog.md) is the catalog manifest that publishes cells and names +the selectable regions; [`obct-terrain.md`](obct-terrain.md) is the terrain raster, a second artifact class on **this same grid** (§1) with its own revision track, carried **inside** the assembled map -(`OBCM_Spec.md` §1.3). +(`obcm-map.md` §1.3). --- @@ -48,7 +48,7 @@ divides `GRID_ORIGIN` and `WORLD_SIDE`. Three properties of this definition are load-bearing: - **Square in µdeg, not in metres.** An OBCM quadtree halves latitude and longitude *together* - (`OBCM_Spec.md` §4), so a node can only ever coincide with a cell if the cell is square in the + (`obcm-map.md` §4), so a node can only ever coincide with a cell if the cell is square in the quadtree's own units. Cells are therefore taller than wide on the ground — at 47°N a `2^18` cell is ≈ 29 km × 20 km. Producers MUST NOT "correct" this with per-latitude sizes; it would break §2. @@ -86,9 +86,9 @@ with `i` and `j` zero-padded to `max(4, decimal_width(WORLD_SIDE / S − 1))` di size at or above `2^16`, which is every band the v1 table uses, and wider for the smaller sizes §1.1 still permits. Producers MUST widen rather than truncate. Catalog entries (§6) and object paths use this id, so a cell's URL is derivable from its **band** and its id plus a base URL — the band, -because two bands may share a cell size ([`OBCC_Spec.md` §2](OBCC_Spec.md)) — while its +because two bands may share a cell size ([`obcc-catalog.md` §2](obcc-catalog.md)) — while its **square is derivable from its id alone**, which is why a catalog cell entry carries no bbox -([`OBCC_Spec.md` §8](OBCC_Spec.md)). +([`obcc-catalog.md` §8](obcc-catalog.md)). Implementations MAY use the packed convenience key `(log2(S) << 24) | (i << 12) | j`, valid for `S ≥ 2^18`. It is not normative and MUST NOT appear in any published artifact. @@ -144,7 +144,7 @@ that is **grid-aligned**: `A_lat ≡ A_lon ≡ GRID_ORIGIN (mod S_MAX)`, with `2 `S_MAX`-aligned, so the box stays tight to the selection.) > **Theorem.** For a quadtree built over a grid-aligned assembly bbox per -> [`OBCM_Spec.md` §4](OBCM_Spec.md), and for every band size `S = 2^s` in the table, the nodes at +> [`obcm-map.md` §4](obcm-map.md), and for every band size `S = 2^s` in the table, the nodes at > depth `d = n − s` are **exactly** the cells of size `S` that tile the assembly bbox — same > minimum corner, same span, to the microdegree. @@ -159,7 +159,7 @@ Two corollaries that implementations lean on: A cell artifact's own header bbox **is** its cell (§3.1), so the cell file's root node is that cell and its subtree is the same subdivision the assembly performs at that position — identical node bboxes at every level, to the microdegree. A feature's anchor is stored relative to its **leaf's** -minimum corner (`OBCM_Spec.md` §5.2) and its deltas are relative to the previous vertex, so every +minimum corner (`obcm-map.md` §5.2) and its deltas are relative to the previous vertex, so every byte of a chunk decodes to the same absolute geometry in the cell file and in the assembly. §4.3 states what is therefore copied and what must be rebuilt. @@ -179,11 +179,11 @@ A baked cell MUST be a complete, valid OBCM file of the catalog's OBCM version, pack computes, because §2 needs the box to be the grid square; - it MUST write the **complete ladder** in its LOD table — one entry per schema LOD, in ladder order, with each entry's `Max Meters/Pixel` taken from the schema (so LOD 0 is `+inf` and the - sequence is strictly decreasing, exactly as `OBCM_Spec.md` §3 requires). LODs outside the cell's + sequence is strictly decreasing, exactly as `obcm-map.md` §3 requires). LODs outside the cell's band are written **empty**: `Index Node Count = 0`, `Chunk Count = 0`, and the single-`0`-entry - offset table `OBCM_Spec.md` §5.1 mandates for an empty region. A reader walks an empty LOD and + offset table `obcm-map.md` §5.1 mandates for an empty region. A reader walks an empty LOD and finds nothing, so a cell of any band stays a legal, openable map; -- the POI section and the nav section MUST be present, per `OBCM_Spec.md` §7/§8, and MUST be +- the POI section and the nav section MUST be present, per `obcm-map.md` §7/§8, and MUST be **empty** unless the cell's band carries them; - both style tables MUST be the schema digest's **canonical table** (§6.2) — right ids, right count, right order, placeholder values — and both marker colors the schema's placeholder. @@ -269,7 +269,7 @@ follows, and the computation MUST be used verbatim by both neighbours: classify junction-ness from the **source snapshot's** way set, not from the ways that survive inside the cell, and MUST NOT rely on any *interior* synthetic node coinciding with anything. Interior synthetic nodes are minted by the packer's own edge splits at a midpoint index of the edge -*as that run sees it* (`nav.rs::split_edge`, plus the serializer's `OBCM_Spec.md` §8.4 chunk-fit and +*as that run sees it* (`nav.rs::split_edge`, plus the serializer's `obcm-map.md` §8.4 chunk-fit and span splits), so two runs over a different set of ways place them metres apart. **Exact-coordinate unification only — an epsilon snap is forbidden.** At a cell seam, genuinely @@ -278,7 +278,7 @@ distinct nodes and invent turns. Producers and assemblers MUST use exact integer NOT offer a tolerance knob. **Wire limits survive.** Unifying two junctions unions their adjacency and recomputes nothing, so -`OBCM_Spec.md` §8.3's degree cap of 24, its `int16` neighbour deltas, and its `uint16` `Cost M` all +`obcm-map.md` §8.3's degree cap of 24, its `int16` neighbour deltas, and its `uint16` `Cost M` all hold. Boundary junctions are ordinary degree-2 nodes. An assembler MUST nevertheless re-check the cap (§4.8) rather than assume it. @@ -351,7 +351,7 @@ an implicit metadata source. ## 4. The assembly contract An **assembly** is **one** OBCM file built from catalog cells for one selection and two authored skins, -carrying its terrain raster in `OBCM_Spec.md` §1.3's region when the selection has elevation. This +carrying its terrain raster in `obcm-map.md` §1.3's region when the selection has elevation. This section defines what an assembler does. ### 4.1 Inputs and preconditions @@ -363,7 +363,7 @@ known-empty identities (§3.8). It MUST refuse to proceed if: - the cells do not all carry the same OBCM version, or that version is not the one it writes; or - the cells do not all belong to the same schema digest; or - two cells disagree on the style table's ordered ids, or on the - `OBCM_Spec.md` §8.6 profile table; or + `obcm-map.md` §8.6 profile table; or - any selected cell is missing, and the caller has not accepted the resulting hole; or - any selected cell is `partial`, and the caller has not accepted the reduced coverage. @@ -372,14 +372,14 @@ so a selection with holes is well-formed by construction. Known-empty cells also leaves but are explicit canonical coverage, not holes. What is not legal is a *silent* hole. **The assembly's size is known before anything is downloaded.** A cell's `bytes` is published per -cell ([`OBCC_Spec.md` §8](OBCC_Spec.md)) and, for a named region, per band in the catalog root -(`bytes_by_band`, [`OBCC_Spec.md` §6](OBCC_Spec.md)), and the remaining overheads do not depend on +cell ([`obcc-catalog.md` §8](obcc-catalog.md)) and, for a named region, per band in the catalog root +(`bytes_by_band`, [`obcc-catalog.md` §6](obcc-catalog.md)), and the remaining overheads do not depend on content: the header, the schema's style table, the LOD table, the POI and nav directories, and the per-LOD offset tables, whose sizes follow from the cell count. Therefore: - A consumer **MUST** project the file before the download and **MUST refuse the selection** if the projection exceeds the per-file wall, which is **whichever is smaller**: the interior the file's - `Offset Scale` covers ([`OBCM_Spec.md` §1.1](OBCM_Spec.md)), and what the consumer's read seam + `Offset Scale` covers ([`obcm-map.md` §1.1](obcm-map.md)), and what the consumer's read seam addresses. It MUST NOT begin fetching an assembly it cannot legally write **or read**. Both walls are stated by reference rather than as literals, because a number copied into this section would go stale against either. @@ -394,9 +394,9 @@ The terrain raster is projected the same way. Its size is `32 + 4 · rows · cols + present · T² · 512` — a header, a directory over the assembly rectangle, and one fixed-size block per square that has a published object — and every term is known from the catalog: the rectangle from the assembly bbox and the terrain block's `cell_log2`, the block length -from the `posting_log2`/`cell_log2` pairing (`OBCT_Spec.md` §3.2), and which squares are present +from the `posting_log2`/`cell_log2` pairing (`obct-terrain.md` §3.2), and which squares are present from the pinned terrain index. A consumer MUST show the raster's bytes as their own figure rather -than folding them into the map's ([`OBCC_Spec.md` §13.3](OBCC_Spec.md)). +than folding them into the map's ([`obcc-catalog.md` §13.3](obcc-catalog.md)). ### 4.2 Choosing the assembly bbox @@ -431,8 +431,8 @@ overhang: the cells out there do not exist, so their leaves are empty. It MUST N | **Style tables** | Rebuilt from the skins (§4.7) — same ids, same count, same order; values replaced. | | **POI section + hours pool** | Rebuilt (§4.5). | | **Nav section** (directory, node tree, node chunks, edge pool) | Rebuilt (§4.6). | -| **Profile table** (`OBCM_Spec.md` §8.6) | Copied from the cells after checking every cell agrees; it is schema data. | -| **Terrain cell blocks** ([`OBCT_Spec.md`](OBCT_Spec.md) §3.2) | **Verbatim**, into a fresh directory over the assembly rectangle. Terrain assembly is *placement*, not grafting: the lattice is global and half-open, so two neighbouring cells already agree about every sample and there is nothing to relocate, re-index or unify. | +| **Profile table** (`obcm-map.md` §8.6) | Copied from the cells after checking every cell agrees; it is schema data. | +| **Terrain cell blocks** ([`obct-terrain.md`](obct-terrain.md) §3.2) | **Verbatim**, into a fresh directory over the assembly rectangle. Terrain assembly is *placement*, not grafting: the lattice is global and half-open, so two neighbouring cells already agree about every sample and there is nothing to relocate, re-index or unify. | An assembler MUST NOT decode a geometry chunk in the normal path. Decoding is for verification (§4.8). @@ -442,14 +442,14 @@ square tiles it exactly, because the assembly corner is congruent to `GRID_ORIGI and therefore modulo the terrain cell too. An assembler MUST refuse a terrain `cell_log2` larger than the assembly's `span_log2` rather than overhang the box or grow it (§4.2 forbids growing it). Squares the selection covers but the catalog does not publish an object for — canonically void -ocean ([`OBCC_Spec.md` §13.6](OBCC_Spec.md)), or ground outside the dataset — are directory `0`. +ocean ([`obcc-catalog.md` §13.6](obcc-catalog.md)), or ground outside the dataset — are directory `0`. ### 4.4 Grafting geometry, per LOD For each ladder LOD `L`, with band size `S = 2^s` and cell depth `d = n − s`: 1. Order the band's present cells by their depth-`d` node index (the BFS order of - `OBCM_Spec.md` §4), so output order is deterministic and independent of fetch order. + `obcm-map.md` §4), so output order is deterministic and independent of fetch order. 2. Emit the fresh upper tree for depths `0..d`, reserving a slot for every depth-`d` position and writing `0x7FFFFFFF` (empty leaf) where the cell is absent. A depth-`d` position whose cell is present takes that cell's **root node** — which may itself be a leaf or a branch, relocated as @@ -457,7 +457,7 @@ For each ladder LOD `L`, with band size `S = 2^s` and cell depth `d = n − s`: 3. Append each present cell's relocated subtree, then its offset-table entries, then its chunk bytes. 4. `Chunk Size` for the LOD is the schema's value; the assembler MUST verify every copied offset - pair still spans at most that (`OBCM_Spec.md` §5.1) — a cell that violated it would poison the + pair still spans at most that (`obcm-map.md` §5.1) — a cell that violated it would poison the assembly. A cell whose LOD `L` region is empty contributes an empty leaf, exactly like an absent cell. @@ -466,13 +466,13 @@ A cell whose LOD `L` region is empty contributes an empty leaf, exactly like an 1. Collect every POI record from every `network`-band cell. Records are 64 bytes with **absolute** coordinates, so they need no relocation. -2. Deduplicate by source identity (`OBCM_Spec.md` §7.3). §3.6 gives each POI exactly one cell, so a +2. Deduplicate by source identity (`obcm-map.md` §7.3). §3.6 gives each POI exactly one cell, so a duplicate is dropped and SHOULD be reported. 3. Rebuild the hours pool: collect each source blob, deduplicate the 29-byte blobs, and remap every service record's `HoursRef` to the new index. `0xFFFF` stays `0xFFFF`. Summit records (subtype 19) keep their signed elevation trailer; they do not reference the hours pool. 4. Re-bin each category into a fresh quadtree over the **assembly** bbox and re-chunk at the - directory's shared `Chunk Size`, per `OBCM_Spec.md` §7.1–§7.3. + directory's shared `Chunk Size`, per `obcm-map.md` §7.1–§7.3. 5. Order records within a chunk by source identity so the output is deterministic. Preserve the explicit approach metadata. If a spatial leaf exceeds capacity, fail the assembly instead of omitting service identities. @@ -482,7 +482,7 @@ The pool count MUST not exceed `0xFFFE` distinct blobs, because `HoursRef` is a ### 4.5.1 Peak article associations -Core cells carry the separate `OBCM_Spec.md` §10 collection. Select associations by the summit +Core cells carry the separate `obcm-map.md` §10 collection. Select associations by the summit SourceIds retained in the merged POI set. Keep each selected summit's article without a geographic content filter. Union summit associations, reject conflicting identities, deduplicate article records, and relocate all indexes and guarded content references. Apply §10.4's deterministic @@ -492,8 +492,8 @@ content precedence and bounded source-copy checks. Landmark queries remain separ This is the most involved rebuild, and its order matters. -1. **Read the serialized node set.** Walk each `network` cell's `OBCM_Spec.md` §8 node quadtree through a real - reader and collect junction records, keyed by `Node Id` (`OBCM_Spec.md` §8.2's bin-packing +1. **Read the serialized node set.** Walk each `network` cell's `obcm-map.md` §8 node quadtree through a real + reader and collect junction records, keyed by `Node Id` (`obcm-map.md` §8.2's bin-packing means one leaf walk can yield a record more than once, so the collection MUST be idempotent). The set to renumber is the **serialized** one, not the one a graph builder would produce: the serializer mints further synthetic degree-2 junctions after the builder finishes, and they are @@ -521,12 +521,12 @@ This is the most involved rebuild, and its order matters. 5. **Renumber** the surviving nodes densely from 0, in a deterministic order (`(lat, lon)` ascending is sufficient and content-derived). 6. **Rebuild the edge pool.** `Edge Id` names a record by its chunk and its position within that - chunk (`OBCM_Spec.md` §8.4 — it was a pool byte offset before OBCM v14, and either way it is a + chunk (`obcm-map.md` §8.4 — it was a pool byte offset before OBCM v14, and either way it is a property of *placement*), so every edge record is re-emitted and every `Edge Id` re-derived. Edge polyline bytes MAY be copied from the source record (they are self-contained: absolute anchor plus deltas), but their *placement* is new, and the no-straddle rule must be re-applied at the 512-byte chunk granularity. 7. **Re-check the wire limits** (§4.8) and rebuild the node quadtree over the assembly bbox, with - `OBCM_Spec.md` §8.2 bin-packed 512-byte node chunks. + `obcm-map.md` §8.2 bin-packed 512-byte node chunks. 8. **Copy the profile table** after confirming every cell's is identical. An assembler MUST NOT create an edge between two nodes that no single cell joined. Unification only @@ -543,7 +543,7 @@ z-index and priority, plus a marker color. Each assembly stamps one light and on - before any output write, compare the resolved ids against the cells' canonical table. This check also applies when a local schema omits the style assignment and the skins supply explicit ids; - write both style tables with the schema's ids in the schema's order and each skin's values in the - other seven bytes of each 8-byte record (`OBCM_Spec.md` §2); + other seven bytes of each 8-byte record (`obcm-map.md` §2); - write both marker colors into the header. A paired restyle changes only the two small tables and marker fields. Styles may change drawing order freely. An @@ -564,7 +564,7 @@ uses — and MUST cover: the gate that catches a mis-relocated index or a bad offset base, because a wrong `node_bbox` produces geometry in the wrong place *and* an anchor that no longer fits, and a wrong chunk base produces a stream that never meets its `0xFF` sentinel. -3. **Offset-table invariants** of `OBCM_Spec.md` §5.1 for every chunk: monotone offsets, in-region +3. **Offset-table invariants** of `obcm-map.md` §5.1 for every chunk: monotone offsets, in-region end, and span ≤ `Chunk Size`. 4. **Nav integrity.** Every neighbour entry's `Neighbor Id` resolves to a record in the same file; `Degree ≤ 24`; every `Edge Id` decodes to a record whose first and last vertices equal the two @@ -574,35 +574,35 @@ uses — and MUST cover: surface a selection whose largest component covers an implausibly small share of the graph, because that is what a broken seam looks like; it MUST NOT silently repair it. 6. **The file answers for itself.** Two properties of the file alone: its header's `Offset Scale` - MUST cover its length (`OBCM_Spec.md` §1.1), and its length MUST equal what the pre-download + MUST cover its length (`obcm-map.md` §1.1), and its length MUST equal what the pre-download projection said it would be (§4.1). An assembler MUST refuse rather than emit a file past the interior that scale addresses, and MUST apply that refusal **at plan time**, because every byte of an assembly is computable before any of it is written. 7. **Digests.** SHA-256 of the assembled file, and of nothing inside it. In particular the **terrain region has no digest of its own**: it is a run of bytes inside the map, and the map has one identity. Step 8's read-back is what proves the raster is the bytes the catalog served. -8. **The terrain region** — the raster, spliced into `OBCM_Spec.md` §1.3. Every input is checked +8. **The terrain region** — the raster, spliced into `obcm-map.md` §1.3. Every input is checked **before its bytes are copied**, because a bad cell must not reach the map even to be caught on the way out: each downloaded object parses as an OBCT container through the real reader - (`OBCT_Spec.md` §4.5 — magic, version, flags, the posting/cell pairing, the rectangle against the + (`obct-terrain.md` §4.5 — magic, version, flags, the posting/cell pairing, the rectangle against the world grid, and every directory entry against the file's own length, which is what rejects a truncated download and an out-of-bounds offset); its header's `Posting Log2` and `Cell Log2` equal - the catalog's terrain block ([`OBCC_Spec.md` §13.1](OBCC_Spec.md)); it is the `1 × 1` container at + the catalog's terrain block ([`obcc-catalog.md` §13.1](obcc-catalog.md)); it is the `1 × 1` container at exactly its own id that §13.1 requires of a *published cell*; and its SHA-256 equals the one the pinned terrain index published. The written region is then read back through the same reader — - through `OBCM_Spec.md` §1.3's window, once the raster lives inside the map — and + through `obcm-map.md` §1.3's window, once the raster lives inside the map — and every present cell's block MUST equal the block of the object it came from, with every square the assembly did not receive at directory `0`. A failure at any step MUST abort the whole assembly. A partially written map is not a degraded map; it is an unmountable one, which is the correct outcome. The atomicity is the flat store's commit -(`FLAT_Store_Format.md` §5). +(`flat-store-format.md` §5). --- ## 6. Catalog, schema, and skins -The catalog contract is [`OBCC_Spec.md`](OBCC_Spec.md); `schema_version 3` there carries cells, +The catalog contract is [`obcc-catalog.md`](obcc-catalog.md); `schema_version 3` there carries cells, bands, schemas, skins, and region cell-sets. This section states only the parts that are OBCA's to define. @@ -622,7 +622,7 @@ routing profile table, `min_component_edges`, and the **band table** (§1.2). Al into chunk bytes, so it is the identity of a cell store. Critically, the schema also fixes the **style-id assignment**: `obc-pack` numbers feature types -`1`-based in config document order (`OBCM_Spec.md` §2), and those ids are referenced by every +`1`-based in config document order (`obcm-map.md` §2), and those ids are referenced by every feature header in every chunk. A schema digest therefore has one **canonical style assignment** — one id per feature type, in one order. The two skins may change the other seven bytes of each record and their marker colors, and nothing else (§4.7). @@ -635,7 +635,7 @@ therefore skins. Custom schemas remain a local-bake affair for the desktop app. An OBCM version bump or a schema digest change invalidates **every** cell, because assembly copies chunk bytes between files and that is only meaningful under one schema. The bakery's guard MUST refuse to publish a catalog that mixes OBCM versions or schema digests across cells, exactly as -`OBCC_Spec.md` §10 already refuses a mixed-version artifact catalog, and an assembler MUST refuse a +`obcc-catalog.md` §10 already refuses a mixed-version artifact catalog, and an assembler MUST refuse a mixed input set (§4.1). --- @@ -719,7 +719,7 @@ cell. **No feature was decoded**, because every anchor is relative to a leaf who bit-identical in the cell file and in the assembly. Note the layout is breadth-first only down to the cell depth; below it, each cell's subtree is a -contiguous block. That is legal: `OBCM_Spec.md` §4's reader contract requires only that a branch's +contiguous block. That is legal: `obcm-map.md` §4's reader contract requires only that a branch's four children be contiguous, in NW/NE/SW/SE order, and at a **higher** index than the branch, all of which a per-cell block satisfies. An assembler MUST NOT be required to re-interleave cell subtrees into global breadth-first order, and a consumer MUST NOT assume global breadth-first @@ -737,10 +737,10 @@ degree-2 junction and the road is continuous. A junction 3.9 m away on either si ## 8. Where this lives - The map byte format every cell and assembly is an instance of: - [`OBCM_Spec.md`](OBCM_Spec.md); its code authority + [`obcm-map.md`](obcm-map.md); its code authority [`firmware/obc-formats/src/obcm.rs`](../firmware/obc-formats/src/obcm.rs). - The catalog that publishes cells, bands, schemas, skins, and region cell-sets: - [`OBCC_Spec.md`](OBCC_Spec.md). + [`obcc-catalog.md`](obcc-catalog.md). - The drawing quadtree and feature encoding: [`host/obc-draw`](../host/obc-draw). - The navigation graph and its encoding: [`host/obc-network`](../host/obc-network). - Shared grid and byte framing: [`host/obc-map-core`](../host/obc-map-core). diff --git a/specs/OBCC_Spec.md b/specs/obcc-catalog.md similarity index 99% rename from specs/OBCC_Spec.md rename to specs/obcc-catalog.md index b114a11540..07fdcc887a 100644 --- a/specs/OBCC_Spec.md +++ b/specs/obcc-catalog.md @@ -2,7 +2,7 @@ OBCC (OpenBikeComputer Catalog) is the JSON contract between the central cell bakery and map-building consumers. The website and desktop app read the same -catalog, select the same [OBCA](OBCA_Spec.md) cells, and feed them to the same +catalog, select the same [OBCA](obca-assembly.md) cells, and feed them to the same assembler. The device does not read OBCC. The current envelope has `"schema_version": 3`. Consumers MUST reject any other @@ -421,7 +421,7 @@ artifact entries; known-empty cells contribute zero. After digest verification, artifact bytes, selected known-empty identities, and the selected skin are passed to the OBCA assembler. Island pruning happens at assembly, seams unify only exact serialized coordinates, and the resulting mounted map follows -[OBCA_Spec.md](OBCA_Spec.md) §4. +[obca-assembly.md](obca-assembly.md) §4. The website and desktop app MUST use the same selection arithmetic, verified cell bytes, skin, and assembler. Host-specific file saving MUST NOT alter the @@ -436,7 +436,7 @@ incomplete transfer and MUST NOT leave anything selectable as a map. ## 13. Terrain artifacts -Terrain is [OBCT](OBCT_Spec.md) raster on the same OBCA grid, published as a +Terrain is [OBCT](obct-terrain.md) raster on the same OBCA grid, published as a **second artifact class with its own revision track**. It is not a band, not a section, and not covered by §10. diff --git a/specs/OBCM_Spec.md b/specs/obcm-map.md similarity index 99% rename from specs/OBCM_Spec.md rename to specs/obcm-map.md index e6eedfeb72..9fc3ac0afe 100644 --- a/specs/OBCM_Spec.md +++ b/specs/obcm-map.md @@ -167,7 +167,7 @@ and two bakes of the same input agree on them. ### 1.3 The terrain region `Terrain Offset` and `Terrain Length` are a scaled pointer to a region at the file tail holding one -[OBCT](OBCT_Spec.md) container, byte-for-byte the bytes `obc-dem` bakes and the assembler splices. +[OBCT](obct-terrain.md) container, byte-for-byte the bytes `obc-dem` bakes and the assembler splices. Terrain sits last precisely so that splicing it moves no other offset. **Terrain is part of the map, and partial updates are not a supported operation**: a terrain re-bake @@ -180,7 +180,7 @@ because the header occupies byte `0`, so no region can begin there. **A reader hands the region over; it does not parse it.** A reader forms a **window** — a byte source whose offset `0` is the region's first byte and whose length is `Terrain Length × U` — and -gives that to the terrain consumer, which reads it exactly as [`OBCT_Spec.md`](OBCT_Spec.md) +gives that to the terrain consumer, which reads it exactly as [`obct-terrain.md`](obct-terrain.md) describes reading a terrain file. The container carries its own magic, version, header and offset directory, and every offset inside it is relative to its own first byte, which is what makes a window sufficient and a copy unnecessary. @@ -192,7 +192,7 @@ Two consequences: header bounds its content: a reader MUST NOT derive the payload length from the region length, and a consumer MUST NOT read past what the container's own structure addresses. - **A terrain region that will not parse is not a broken map.** Elevation is an enhancement - (`OBCC_Spec.md` §13): a reader whose OBCT parse fails MUST fall back to no elevation, MUST still + (`obcc-catalog.md` §13): a reader whose OBCT parse fails MUST fall back to no elevation, MUST still mount, render and route, and MUST NOT present the map as faulty. A **writer** gets no such clemency and MUST verify the region it splices. @@ -891,7 +891,7 @@ Rules: this field. A self-loop (`a == b`, e.g. a lollipop loop) appears **once** in its node's record, carrying its forward direction's ascent. - **Seam determinism.** A producer that cuts one edge into pieces at a cell border - (`OBCA_Spec.md` §3) integrates each piece over the **same global elevation + (`obca-assembly.md` §3) integrates each piece over the **same global elevation lattice**, so two neighbouring cells' stubs are each the integral of their own geometry over one surface and the pieces' ascents sum to the uncut edge's. - Degree `0` is valid to decode but the packer never emits it (every junction @@ -1119,7 +1119,7 @@ one ≤ 512-byte read. `Profile Count` consecutive **56-byte** records at `Profile Table Offset`, one per bike type. A producer writes **exactly four**, in the bike-type order of -[`OBCR_Spec.md`](OBCR_Spec.md) §1.2: Road = 0, Gravel = 1, MTB = 2, Touring = 3. The count, +[`obcr-route.md`](obcr-route.md) §1.2: Road = 0, Gravel = 1, MTB = 2, Touring = 3. The count, the order and the meaning are fixed; only the weights are tunable. The device picks a record by bike-type value; A\* weights each edge by it. The table is **always present**, even for an empty graph. The reader rejects a `Profile Count` of `0` or `> 8`, and routes a bike type past diff --git a/specs/OBCR_Spec.md b/specs/obcr-route.md similarity index 98% rename from specs/OBCR_Spec.md rename to specs/obcr-route.md index b0a9e018fb..531309f3ef 100644 --- a/specs/OBCR_Spec.md +++ b/specs/obcr-route.md @@ -1,11 +1,11 @@ # OBCR File Format Specification (v5) OBCR (OpenStreetMap Binary Chunked Route) is the binary **route** format, the sibling of -the [`OBCM`](OBCM_Spec.md) map format. A route is one ordered polyline with per-point +the [`OBCM`](obcm-map.md) map format. A route is one ordered polyline with per-point elevation, precomputed ride statistics, an optional table of **waypoints** pinned along the route, and an optional accepted-visit descriptor. The device and the simulator produce it from an uploaded GPX file; the companion app produces it from imported GPX/TCX before a -BLE upload (see [`FLAT_Store_Protocol.md`](FLAT_Store_Protocol.md)). +BLE upload (see [`flat-store-protocol.md`](flat-store-protocol.md)). This document is the normative byte contract. `firmware/obc-formats/src/obcr.rs` is its code authority for versions, fixed lengths, magic, and sentinels; @@ -107,7 +107,7 @@ Assistant candidates and built trip days use their own retention rules. ### 1.2 Bike type and estimated time Every route has one of four fixed bike types. The value is also the index of the map's routing -profile for that type ([`OBCM_Spec.md`](OBCM_Spec.md) §8.6). +profile for that type ([`obcm-map.md`](obcm-map.md) §8.6). | Value | Type | Flat speed `v` (km/h) | Climb cost `k` (0.1 s per metre of ascent) | | :-- | :-- | :-- | :-- | @@ -222,7 +222,7 @@ accepted-visit producer supplies it when it has the original route's exact sourc **Category** reuses the map's browsable POI categories — `1` water · `2` campsite · `3` accommodation · `4` resupply · `5` pharmacy · -`6` bike shop ([`OBCM_Spec.md`](OBCM_Spec.md) §7.4) — and `0` (generic) is first-class. +`6` bike shop ([`obcm-map.md`](obcm-map.md) §7.4) — and `0` (generic) is first-class. Producers map their source symbols onto these ids; §4.1 is the canonical table for GPX. **`Distance Along`** is defined by nearest-point placement: the cumulative route distance at diff --git a/specs/OBCT_Spec.md b/specs/obct-terrain.md similarity index 96% rename from specs/OBCT_Spec.md rename to specs/obct-terrain.md index eb887239c8..6da3bfd81d 100644 --- a/specs/OBCT_Spec.md +++ b/specs/obct-terrain.md @@ -3,12 +3,12 @@ Version 3 adds the geographic surface index in section 8. Sections 1–7 describe the native v1 raster, which remains the base height plane of v3. -OBCT is the terrain artifact: a raster of ground heights on the [OBCA](OBCA_Spec.md) cell grid. It +OBCT is the terrain artifact: a raster of ground heights on the [OBCA](obca-assembly.md) cell grid. It defines the sample lattice (§1), the 512-byte tile and the terrain cell (§2, §3), the container (§4) and the sampling rules (§5). Terrain is a separate artifact class with its own revision track. An assembly's raster is embedded -**verbatim** in the map file's terrain region ([`OBCM_Spec.md` §1.3](OBCM_Spec.md)), which the map +**verbatim** in the map file's terrain region ([`obcm-map.md` §1.3](obcm-map.md)), which the map reader hands over as an opaque window rather than parsing. This document is normative. The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be @@ -19,9 +19,9 @@ arithmetic is [`firmware/obc-formats/src/obct.rs`](../firmware/obc-formats/src/o and consumers import those facts directly rather than transcribing them. The reference reader, sampler and tile cache are [`firmware/obc-elevation`](../firmware/obc-elevation). -Related contracts: [`OBCA_Spec.md`](OBCA_Spec.md) §1 defines the grid this raster sits on; -[`OBCC_Spec.md`](OBCC_Spec.md) §13 publishes terrain cells as a catalog artifact class with its own -revision track; [`OBCM_Spec.md`](OBCM_Spec.md) is the map that carries an assembly's raster, and +Related contracts: [`obca-assembly.md`](obca-assembly.md) §1 defines the grid this raster sits on; +[`obcc-catalog.md`](obcc-catalog.md) §13 publishes terrain cells as a catalog artifact class with its own +revision track; [`obcm-map.md`](obcm-map.md) is the map that carries an assembly's raster, and whose §8 nav graph stores the ascent integrated *from* these samples. --- @@ -31,7 +31,7 @@ whose §8 nav graph stores the ascent integrated *from* these samples. ### 1.1 Origin and posting All coordinates are integer **microdegrees** (µdeg, 1e-6 degrees), the unit of every OBCM -coordinate. The lattice is anchored on the OBCA grid origin ([`OBCA_Spec.md` §1.1](OBCA_Spec.md)): +coordinate. The lattice is anchored on the OBCA grid origin ([`obca-assembly.md` §1.1](obca-assembly.md)): ``` GRID_ORIGIN = −268435456 µdeg (= −2^28, on BOTH axes) @@ -54,7 +54,7 @@ narrowing towards the poles. This is not corrected anywhere and MUST NOT be: the the lattice being derivable from the coordinate by a shift. The lattice does not wrap, and it spans the world box rather than the geographic domain — the -antimeridian and pole rules of [`OBCA_Spec.md` §1.4](OBCA_Spec.md) apply unchanged. +antimeridian and pole rules of [`obca-assembly.md` §1.4](obca-assembly.md) apply unchanged. ### 1.2 Values @@ -84,7 +84,7 @@ values are | Terrain cell | `2^19` µdeg | 58 × 40 km — 1024 × 1024 samples, 2 MiB raw | They are the bakery's choice, published in the catalog. Retuning them is a terrain re-bake, -**not** a version bump of this format (the [`OBCA_Spec.md` §1.5](OBCA_Spec.md) idiom). +**not** a version bump of this format (the [`obca-assembly.md` §1.5](obca-assembly.md) idiom). A reader MUST accept any pairing this document permits (§4.5), not only the v1 one. @@ -113,7 +113,7 @@ One tile spans `16·P` µdeg on each axis — at the v1 posting, `2^13` µdeg ### 3.1 A cell is an OBCA grid square A **terrain cell** of side `S = 2^cell_log2` µdeg is the OBCA cell of that size -([`OBCA_Spec.md` §1.1](OBCA_Spec.md)): the half-open square +([`obca-assembly.md` §1.1](obca-assembly.md)): the half-open square ``` cell(S, ci, cj) = [ GRID_ORIGIN + ci·S , GRID_ORIGIN + (ci+1)·S ) in latitude @@ -210,7 +210,7 @@ There is no separate cell format. | 24 | Reserved | 8 | `uint8[8]` | MUST be zero | The header carries **no bounding box**: the cell rectangle *is* the bounding box (see -[`OBCC_Spec.md` §8](OBCC_Spec.md), and §13.1 for a terrain cell's own entry). +[`obcc-catalog.md` §8](obcc-catalog.md), and §13.1 for a terrain cell's own entry). `Flags` is this format's extension point: a future per-tile packed encoding (§3.2) sets a bit here. A v1 reader MUST refuse a file with any bit set. @@ -264,7 +264,7 @@ recorded ride log already uses `.obct` ([`obc-formats/src/track.rs`](../firmware/obc-formats/src/track.rs)). The magic stays `OBCT`. An assembly's raster is not a file on the card at all: it is spliced into the map file's terrain -region ([`OBCM_Spec.md` §1.3](OBCM_Spec.md)). A **published cell** is an object of its own, named +region ([`obcm-map.md` §1.3](obcm-map.md)). A **published cell** is an object of its own, named by the catalog. --- @@ -580,5 +580,5 @@ A producer SHOULD report the largest lift of a run, the node it is at, and how m reader can check it; `obc-dem bake` prints it. A change of reference archive changes the baked samples, so it is a terrain revision bump and hence -a navigation re-bake (`OBCC_Spec.md` §13.4). The reference DEM keeps its own attribution, which +a navigation re-bake (`obcc-catalog.md` §13.4). The reference DEM keeps its own attribution, which MUST travel with every container derived from it. diff --git a/specs/OBCU_Spec.md b/specs/obcu-update.md similarity index 98% rename from specs/OBCU_Spec.md rename to specs/obcu-update.md index 85ff7c7c13..967c527269 100644 --- a/specs/OBCU_Spec.md +++ b/specs/obcu-update.md @@ -14,7 +14,7 @@ the `obc-boot` bootloader links: carve** through which the armer hands the bootloader the sEMMC soft-peripheral image it boots the card with. -It shares the conventions of the [`OBCM`](OBCM_Spec.md) map and [`OBCR`](OBCR_Spec.md) route +It shares the conventions of the [`OBCM`](obcm-map.md) map and [`OBCR`](obcr-route.md) route formats: **little-endian** integers throughout, an explicit magic, version and CRC frame, and **no runtime discovery** — every field is at a fixed or self-describing offset. @@ -33,7 +33,7 @@ value `crc32("123456789") == 0xCBF43926`). ``` The container is one **update-package object** (kind `7`) in the flat store -([`FLAT_Store_Format.md`](FLAT_Store_Format.md) §3.1). A client uploads it; the card carries no +([`flat-store-format.md`](flat-store-format.md) §3.1). A client uploads it; the card carries no filesystem and no sideload path. It is produced by the `obc-mkimage wrap` / `obc-mkimage sign` host tool and consumed by the app-side armer, which validates the header, the full image CRC, **and the signature** before arming. @@ -88,7 +88,7 @@ can install. `obc-mkimage wrap` refuses a larger image. The **whole container** `64 + Image Len + Sig Len` bytes, so a transfer that gates on a length gates at the **container** ceiling `MAX_CONTAINER_LEN` = `MAX_IMAGE_LEN + 64 + 64` = 2,023,552: a raw image at the cap must not be refused for its own framing. A container is delivered as a `PUT` of kind `7` -([`FLAT_Store_Protocol.md`](FLAT_Store_Protocol.md) §3.6, §4). Bytes past +([`flat-store-protocol.md`](flat-store-protocol.md) §3.6, §4). Bytes past `64 + Image Len + Sig Len` in the delivered file are ignored (§2.3). ### 1.2 Header Version stays 1 — the flash-once bootloader guarantee (normative) @@ -296,7 +296,7 @@ Each **Extent** is 8 bytes: | +4 | Blocks | 4 | `uint32` | Number of 512-byte blocks in the run | **`MAX_EXTENTS` = 8.** A stored object holds at most eight extent ranges -([`FLAT_Store_Format.md`](FLAT_Store_Format.md) §5.3) and one range is one contiguous block run, so +([`flat-store-format.md`](flat-store-format.md) §5.3) and one range is one contiguous block run, so eight is the whole of what an arm can ever resolve. The armer errors out past this rather than truncating the chain, and the decoder rejects an Extent Count above `MAX_EXTENTS`. @@ -317,7 +317,7 @@ slot. This is normative for both the armer and the bootloader; the skip arithmet `obc-dfu`'s install engine. **The rollback snapshot is unsigned.** The armer's copy of the running image, written from the app -slot into the **rollback reserve** ([`FLAT_Store_Format.md`](FLAT_Store_Format.md) §3.1, kind `8`) +slot into the **rollback reserve** ([`flat-store-format.md`](flat-store-format.md) §3.1, kind `8`) before an install, is a **v1/unsigned** container (`Sig Scheme` = `0`). The device cannot reconstruct the original release signature from slot bytes alone, and nothing needs one: the snapshot never passes through the armer's scan (§1.4), and the bootloader's rollback path validates diff --git a/specs/Ride_Archive_Metadata.md b/specs/ride-archive-metadata.md similarity index 98% rename from specs/Ride_Archive_Metadata.md rename to specs/ride-archive-metadata.md index d4ff3604db..7ad5c3f3cf 100644 --- a/specs/Ride_Archive_Metadata.md +++ b/specs/ride-archive-metadata.md @@ -95,7 +95,7 @@ closed. It does not prevent unrelated Ride payload mutations. No automatic expir The trip progress records follow the checkpoint, or the rows when there is no checkpoint. The record count is the remaining payload length divided by 96, at most `16`. Records are in write -order, each with a unique nonzero trip key. `obc-ble-interface-spec.md` §7.7 has the rules. A +order, each with a unique nonzero trip key. `ble-interface.md` §7.7 has the rules. A record never holds a route: route removal and replacement ignore it. | Record offset | Bytes | Value | diff --git a/specs/Ride_Archive_Contract.md b/specs/ride-archive-receipts.md similarity index 95% rename from specs/Ride_Archive_Contract.md rename to specs/ride-archive-receipts.md index ab42fb7fc4..550589ad1a 100644 --- a/specs/Ride_Archive_Contract.md +++ b/specs/ride-archive-receipts.md @@ -1,8 +1,8 @@ # Ride archive and persistence receipts This contract defines the iOS archive boundary and the device archive-proof boundary. -[ARCHIVE_RIDE](FLAT_Store_Protocol.md#312-archive_ride) carries proof into -[card archive metadata](Ride_Archive_Metadata.md). iOS delivers and retries these receipts; the board uses validated proof for its synced indicator. +[ARCHIVE_RIDE](flat-store-protocol.md#312-archive_ride) carries proof into +[card archive metadata](ride-archive-metadata.md). iOS delivers and retries these receipts; the board uses validated proof for its synced indicator. ## Source identity @@ -25,7 +25,7 @@ link. It checks the served revision, length and CRC against the requested source the download. The canonical archive is decoded from the returned payload; an earlier catalog's display fields do not replace the downloaded fields. -[GET](FLAT_Store_Protocol.md#35-get) completion is transport delivery. It is not client storage +[GET](flat-store-protocol.md#35-get) completion is transport delivery. It is not client storage proof. ## Atomic local archive @@ -103,7 +103,7 @@ local saves, never device proof. The device synced indicator uses only validated ## Device archive proof -The wire rules are [`ARCHIVE_RIDE`](FLAT_Store_Protocol.md#312-archive_ride); the stored rows and -the synced indicator are [card archive metadata](Ride_Archive_Metadata.md). The device accepts proof +The wire rules are [`ARCHIVE_RIDE`](flat-store-protocol.md#312-archive_ride); the stored rows and +the synced indicator are [card archive metadata](ride-archive-metadata.md). The device accepts proof only for the exact current finalized Ride source on the mounted card, and a source that has since changed or vanished is terminal for that receipt: the device never recreates the ride or its proof. diff --git a/specs/vectors/README.md b/specs/vectors/README.md index 433e6ad58b..a1eef2b3cd 100644 --- a/specs/vectors/README.md +++ b/specs/vectors/README.md @@ -1,9 +1,9 @@ # Shared wire-protocol test vectors (S0) Binary fixtures pinning the byte layouts of -[`obc-ble-interface-spec.md`](../obc-ble-interface-spec.md), the OBCR route format -([`OBCR_Spec.md`](../OBCR_Spec.md)), the 20-byte ride-sample codec and complete ride-v6 -object, and the OBCT terrain raster ([`OBCT_Spec.md`](../OBCT_Spec.md)), consumed by +[`ble-interface.md`](../ble-interface.md), the OBCR route format +([`obcr-route.md`](../obcr-route.md)), the 20-byte ride-sample codec and complete ride-v6 +object, and the OBCT terrain raster ([`obct-terrain.md`](../obct-terrain.md)), consumed by three implementations: - **Firmware**: `cargo test -p obc-vectors` (workspace `firmware/`) verifies every @@ -23,18 +23,18 @@ A drift on any side fails that side's tests — the files are the contract. |---|---|---| | `route-waypoints.obcr` | OBCR v5 | "Vector Loop", 9-point track at 48°N, 2 waypoints (`Brunnen` @ 0 m with ele 238, `Drinking Water` → category 1, 13 m left of travel; `Pass Summit` mid-route without ele, an unmapped `Viewpoint` → generic, on-route) | | `route-plain.obcr` | OBCR v5 | the same track, no waypoints — must ride identically | -| `route-visit.obcr` | OBCR v5, visit descriptor ([`OBCR_Spec.md`](../OBCR_Spec.md) §6) | a valid 80-byte accepted-visit envelope | +| `route-visit.obcr` | OBCR v5, visit descriptor ([`obcr-route.md`](../obcr-route.md) §6) | a valid 80-byte accepted-visit envelope | | `route-visit-waypoint-overlap.obcr`, `route-visit-index-overlap.obcr` | OBCR v5 §1.1 envelope | each shares the descriptor's last four reserved-zero bytes with another section; both codecs MUST reject them | -| `eta.csv` | [`OBCR_Spec.md`](../OBCR_Spec.md) §1.2 estimate | one row per case: bike type, distance, ascent and the expected whole seconds, for every bike type. `obc-formats` and the `OBCKit` Swift tests read it | +| `eta.csv` | [`obcr-route.md`](../obcr-route.md) §1.2 estimate | one row per case: bike type, distance, ascent and the expected whole seconds, for every bike type. `obc-formats` and the `OBCKit` Swift tests read it | | `track-log.obct` | sample-codec vector (five complete 20-byte records, no header) | shaped for sensor/signed-coordinate coverage only; it is not accepted as a ride or recovery input | | `track-export.gpx` | GPX 1.1, `obc_route::track_to_gpx` | the export of finished `ride-v6.bin` as "Schauinsland & back" — the name's `&` pins XML escaping. Not spec-derived: the exporter's serialization *is* the contract, so this file is its output, and its value is cross-implementation | | `ride-v6.bin` | ride object v6 (spec §7.2) | "Sensor Ride": three exact 20-byte recorded samples (including segment flags and mixed sensor presence), followed by the fixed 154-byte summary footer with 95 m of descent and 756 kJ. The ride started on day index 1 of 3 of the `trip-v3.bin` trip ("Alpen Traverse", key `0x0123_4567_89AB_CDEF`), on a Gravel bike, with max HR 185, FTP 250 and the demo flag set | | `status-command-result.bin` | `status` msg 3 §4.3 | the answer to an accepted `installFw`: `cmd` 3, `ok`, `detail` 0. Pins the four-byte layout and the position of `detail` | | `command-set-clock.bin` | `setClock` §4.4 cmd 5 | `utc` 1783598400 (2026-07-09T12:00:00Z) · `offset_min` 120 | -| `update-container-v1.bin` | OBCU container ([`OBCU_Spec.md`](../OBCU_Spec.md) §1), **unsigned/v1** | a full `UPDATE.BIN` / `fwImage` payload (§7.6, id 0): 64-byte header (`fw_version` `1.2.0+abc1234`, `image_len` 128) + a 128-byte raw image. Decoded by `obc-dfu` (`cargo test -p obc-dfu --test vectors`) and the iOS `OBCUHeader`. It is the shape of a fielded container and of the device-written rollback snapshot, and pairing it with the v2 file below pins the offset-compatibility guarantee across implementations | +| `update-container-v1.bin` | OBCU container ([`obcu-update.md`](../obcu-update.md) §1), **unsigned/v1** | a full `UPDATE.BIN` / `fwImage` payload (§7.6, id 0): 64-byte header (`fw_version` `1.2.0+abc1234`, `image_len` 128) + a 128-byte raw image. Decoded by `obc-dfu` (`cargo test -p obc-dfu --test vectors`) and the iOS `OBCUHeader`. It is the shape of a fielded container and of the device-written rollback snapshot, and pairing it with the v2 file below pins the offset-compatibility guarantee across implementations | | `update-container-v2.bin` | OBCU container (§1), **Ed25519-signed/v2** | the *same* header table and the *same* 128-byte image as v1 — `header_version` still `1` (§1.2) — with `sig_scheme`/`sig_len` in v1's reserved bytes `48..52` and a 64-byte signature trailer after the image. Signed with the committed **test** key (`firmware/obc-dfu/keys/test/`); signing is deterministic, so this is a stable file. A decoder must read every v1 field from it byte-identically | | `trip-v3.bin` | trip object v3 (spec §7.7) | "Alpen Traverse", key `0x0123_4567_89AB_CDEF`, start date 20360 (Monday 2025-09-29), 3 days: route 7 ending on the line at 82,000 m, route 8 leaving the line at 73,600 m, and full-width route `0x1_0000_0063` joining at 400 m with `leave_m` `0xFFFF_FFFF`; 64-byte header + 16 bytes/day | -| `terrain-shard.obcd` | OBCT container ([`OBCT_Spec.md`](../OBCT_Spec.md) §4) | a 2 × 2 cell rectangle at ≈ 46.97°N / 7.98°E over a **plane** (`100 + 3·di + 5·dj` m), with the far cell **absent** (the `0` directory sentinel) and one `NODATA` sample. Posting is the v1 `2^9`; the cell is `2^14`, not the v1 `2^19`, because both are header data. A plane is an *oracle*: bilinear interpolation over one has a closed form, so a second implementation checks itself against arithmetic, and the differing coefficients (3 vs 5) catch a transposed lat/lon | +| `terrain-shard.obcd` | OBCT container ([`obct-terrain.md`](../obct-terrain.md) §4) | a 2 × 2 cell rectangle at ≈ 46.97°N / 7.98°E over a **plane** (`100 + 3·di + 5·dj` m), with the far cell **absent** (the `0` directory sentinel) and one `NODATA` sample. Posting is the v1 `2^9`; the cell is `2^14`, not the v1 `2^19`, because both are header data. A plane is an *oracle*: bilinear interpolation over one has a closed form, so a second implementation checks itself against arithmetic, and the differing coefficients (3 vs 5) catch a transposed lat/lon | | `route-answer.json` | route answer ([`route-api.md`](../route-api.md)) | two routes before (`routes`) and after (`answer`) encoding. The primary route has six points, one unknown height, two legs and four edge channels: two `surfaces` runs, three `pushing` runs, three `closures` runs and three `sac_scale` runs. The corridor route has five points, one leg and a `via` point. The `planner-router` encoder test and the decoder tests of the web planner and `OBCKit` read it | | `signed-routes.json` | route catalog records ([`route-catalog.md`](../route-catalog.md)) and signed-route search queries | 13 records around one start at 47.87°N 8.15°E: hiking and foot routes graded T1 to T5, an ungraded loop, an S2 mountain-bike loop, a bicycle loop, a long route with two stages whose start is 30 km away, a route across a cell edge, and a loop that reaches a cell outside the offline download. A 14th record, a loop that turns back at its own start, lies outside every query and serves only its plan. `grid` lists the cells that have a file; the loader answers null for every other cell. Each query has the cells that the search loads and the expected matches in order. The distance model is a flat projection at the start latitude on a sphere of R = 6371 km. The cell box is the start ± the radius on each axis, and each box edge is at least 0.05 tiles from a tile edge. A route matches when its distance is not more than the radius. `distance_m` is rounded to the metre. Each plan has the plan points and the turnarounds as plan indices (index 0 when a loop turns back at its start). The web planner search test reads it; the phone port reads the same file | | `trailview-features.json`, `trailview-features.bin` | TrailView feature tile ([`trailview-features.md`](../trailview-features.md)) | one tile decoded and its bytes: a line into the buffer, a line with two parts, solved heights and two heights that are not solved, an area of two polygons with a hole, a point outside the tile square, a non-ASCII tag value and a feature without tags. The encoder test of the bake (`host/obc-trailview-features/src/tiles.rs`) and the engine decoder test (`builder/web/src/lib/trailview/stream/features.test.ts`) read both | diff --git a/specs/vectors/eta.csv b/specs/vectors/eta.csv index d4aa6dc48e..7f1a31ffc2 100644 --- a/specs/vectors/eta.csv +++ b/specs/vectors/eta.csv @@ -1,4 +1,4 @@ -# OBCR_Spec.md §1.2 estimate: seconds = floor((36*distance_m + ascent_m*k*v) / (10*v)) +# obcr-route.md §1.2 estimate: seconds = floor((36*distance_m + ascent_m*k*v) / (10*v)) bike,distance_m,ascent_m,seconds 0,0,0,0 0,1,0,0 diff --git a/specs/vectors/manifest.json b/specs/vectors/manifest.json index 15823c634d..b369072fbf 100644 --- a/specs/vectors/manifest.json +++ b/specs/vectors/manifest.json @@ -1,5 +1,5 @@ { - "spec": "obc-ble-interface-spec.md", + "spec": "ble-interface.md", "protocol_version": 4, "crc32_check": { "input": "123456789", @@ -76,7 +76,7 @@ "valid": false }, "terrain-shard.obcd": { - "format": "OBCT v1 terrain shard (OBCT_Spec.md 4) — 32-byte header + row-major uint32 cell directory + cell blocks of 16x16 int16 tiles", + "format": "OBCT v1 terrain shard (obct-terrain.md 4) — 32-byte header + row-major uint32 cell directory + cell blocks of 16x16 int16 tiles", "byte_len": 6192, "version": 1, "posting_log2": 9, diff --git a/specs/wikimedia-content.md b/specs/wikimedia-content.md index b0521077ef..fa23f75524 100644 --- a/specs/wikimedia-content.md +++ b/specs/wikimedia-content.md @@ -2,7 +2,7 @@ This contract defines the landmark and peak content: the compact facts that the prepared source [`wiki`](#the-wiki-source) holds, and how the compilers read them. The compiled device formats in -[obc-data](obc-data.md) and [OBCM](OBCM_Spec.md) are separate outputs. +[obc-data](obc-data.md) and [OBCM](obcm-map.md) are separate outputs. ## Policy owners diff --git a/tools/fixture_catalog.py b/tools/fixture_catalog.py index 2db649b334..4978837240 100644 --- a/tools/fixture_catalog.py +++ b/tools/fixture_catalog.py @@ -50,7 +50,7 @@ def content_type(path: str) -> str: def pin(objects: dict[str, bytes], path: str, body: bytes) -> dict: """Publish `body` at `path` with its digest before the extension, and return the reference a - document carries for it (`OBCC_Spec.md` §9).""" + document carries for it (`obcc-catalog.md` §9).""" digest = hashlib.sha256(body).hexdigest() stem, _, extension = path.rpartition(".") diff --git a/tools/justfile b/tools/justfile index fdb2d7db4a..abe5d9aabe 100644 --- a/tools/justfile +++ b/tools/justfile @@ -418,7 +418,7 @@ bake *args: _run "${client[@]}" list "$prefix" fi ;; terrain) - # The terrain artifact class, on its own revision track (OBCC_Spec.md §13). Same + # The terrain artifact class, on its own revision track (obcc-catalog.md §13). Same # tree as `bake`, deliberately a separate command: it needs DEM tiles and no OSM # extract, and it must never be a step of a schema re-bake. shift diff --git a/tools/testing/suites.toml b/tools/testing/suites.toml index 5218917dfd..90a3b4d627 100644 --- a/tools/testing/suites.toml +++ b/tools/testing/suites.toml @@ -60,7 +60,7 @@ triggers = ["specs/vectors/**", "builder/presets/**", "builder/wasm/tests/fixtur [[package]] name = "obc-usb" -triggers = ["specs/FLAT_Store_Protocol.md", "specs/vectors/flat-store-v4/**"] +triggers = ["specs/flat-store-protocol.md", "specs/vectors/flat-store-v4/**"] [[package]] name = "obc-fixtures" From cb498697443e35b39ba92ada195c087564677cd9 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 29 Oct 2026 08:07:15 +0000 Subject: [PATCH 02/20] docs(specs): give obcr-route.md the common spec header The header names the version and refusal rule, the writers, the readers, the code authority, the test vectors and the related specs. A short Conventions section holds the file-wide rules. Repeated statements, UI display notes and rationale beyond one sentence are gone. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DTHu3LHSGY5CY9j2XmscLi --- specs/obcr-route.md | 65 +++++++++++++++++++-------------------------- 1 file changed, 28 insertions(+), 37 deletions(-) diff --git a/specs/obcr-route.md b/specs/obcr-route.md index 531309f3ef..08ea78facb 100644 --- a/specs/obcr-route.md +++ b/specs/obcr-route.md @@ -1,22 +1,23 @@ -# OBCR File Format Specification (v5) +# OBCR route format -OBCR (OpenStreetMap Binary Chunked Route) is the binary **route** format, the sibling of -the [`OBCM`](obcm-map.md) map format. A route is one ordered polyline with per-point -elevation, precomputed ride statistics, an optional table of **waypoints** pinned along -the route, and an optional accepted-visit descriptor. The device and the simulator produce -it from an uploaded GPX file; the companion app produces it from imported GPX/TCX before a -BLE upload (see [`flat-store-protocol.md`](flat-store-protocol.md)). +A route: one ordered polyline with per-point elevation and surface, ride statistics, optional +waypoints pinned along it, and an optional accepted-visit descriptor. -This document is the normative byte contract. `firmware/obc-formats/src/obcr.rs` is its -code authority for versions, fixed lengths, magic, and sentinels; -`firmware/obc-formats/src/io.rs` owns the byte-source/sink traits and the checked -little-endian primitives used by both producer and reader. +- Version: 5 (`0x05`). A reader refuses every other version. Re-import older routes from their + source files. +- Writers: `firmware/obc-route` (`convert.rs`, GPX import on the device and the simulator); the + companion app (GPX and TCX import before a [protocol v4](flat-store-protocol.md) upload). +- Readers: `firmware/obc-route` (`reader.rs`), the companion app. +- Code authority: `firmware/obc-formats/src/obcr.rs` (version, fixed lengths, magic, + sentinels); `firmware/obc-formats/src/io.rs` (byte source and sink, little-endian primitives). +- Test vectors: [`vectors/`](vectors/README.md) (`route-*.obcr`, [`eta.csv`](vectors/eta.csv)). +- Related: [OBCM](obcm-map.md). -**Versions.** v5 is the only accepted version. Re-import older routes from their source files. +## Conventions -All multi-byte integers are **little-endian**. Coordinates are **microdegrees** -(1e-6 degrees). Distances and elevations are whole **meters**. Every section is reached -through an explicit offset and every count is stored. +- Multi-byte integers are little-endian. +- Coordinates are microdegrees (1e-6 degrees). Distances and elevations are whole metres. +- Every section is reached through an explicit offset and every count is stored. ## File layout @@ -28,11 +29,8 @@ through an explicit offset and every count is stored. [Visit descriptor] (optional: 80 bytes) ``` -Every section is reached by an **explicit offset** (`Index Offset`, `Data Offset`, -`Waypoint Offset`, per-chunk `Byte Offset`), so the physical order is not load-bearing: -the reader accepts any arrangement. The canonical writer emits the index and the waypoint -table **last**, because their sizes and positions are not known until the chunks have -streamed out (§5). +A reader accepts any physical order of the sections. The canonical writer emits the index and +the waypoint table last (§5). --- @@ -63,8 +61,7 @@ streamed out (§5). | 64 | Name | 48 | `char[48]` | UTF-8 route name, null-padded | With the canonical layout, `Data Offset == 160` (chunks follow the header) and -`Index Offset == Data Offset + total chunk-data bytes`. The UI derives km/m from these -meters fields (`distance_km = round(total_distance / 1000)`). +`Index Offset == Data Offset + total chunk-data bytes`. Readers validate the complete fixed header, including the optional-section envelopes. @@ -83,7 +80,7 @@ Readers validate the complete fixed header, including the optional-section envel An absent map key is all zero. A present key names the installed map used for surface attribution; its object ID and revision are nonzero. A file path or a default ID does not establish map identity. Facts from a different map revision are stale; facts without a key -are unbound. Historical surfaces can still be displayed. +are unbound. A descriptor must fit inside the object and must not overlap the index or the waypoint table. Unsupported versions and nonzero absent-section fields are errors. @@ -188,9 +185,8 @@ for each record: Consecutive chunks MUST share their boundary vertex: chunk `k`'s last decoded point equals chunk `k+1`'s anchor. `Point Count` in the header counts that vertex once. -> **Densification:** the converter inserts intermediate points on any decimated segment -> whose Δlon or Δlat would exceed the `int16` range (±32767 µdeg ≈ 3.6 km), so readers need -> no wide-delta path. +A producer inserts intermediate points on any segment whose Δlon or Δlat exceeds the `int16` +range (±32767 µdeg, about 3.6 km). A reader has no wide-delta path. --- @@ -309,15 +305,11 @@ durable checkpoint CRC/length are separate. ## Reference implementation -`firmware/obc-formats` (`no_std`): `obcr.rs` (normative version, sizes, magic, and -sentinels) and `io.rs` ([`ByteSource`](#bytesource)/`ByteSink` + endian primitives). -`firmware/obc-route` (`no_std`): `reader.rs` (`RouteReader`, `RouteSummary`, `ChunkMeta`, -`Waypoint` + `for_each_waypoint`), `convert.rs` (GPX → OBCR), `gpx.rs` (streaming `` -+ `` scans), and `symbol.rs` (§4.1's table). Format-contract tests build synthetic -`.obcr` bytes by hand, mirroring this layout (`obc-route/tests/format.rs` + -`tests/waypoints.rs`); shared phone↔firmware fixtures live in `specs/vectors/`. -`route-visit.obcr` has a valid descriptor envelope. Both codecs MUST reject -`route-visit-waypoint-overlap.obcr` and `route-visit-index-overlap.obcr`; each shares the +`firmware/obc-route`: `reader.rs` (`RouteReader`, `RouteSummary`, `ChunkMeta`, `Waypoint`), +`convert.rs` (GPX to OBCR), `gpx.rs` (`` and `` scans) and `symbol.rs` (§4.1). +`obc-route/tests/format.rs` and `tests/waypoints.rs` build `.obcr` bytes by hand from this +layout. `route-visit.obcr` has a valid descriptor envelope. Both codecs MUST reject +`route-visit-waypoint-overlap.obcr` and `route-visit-index-overlap.obcr`: each shares the descriptor's last four reserved-zero bytes with another section. ### ByteSource @@ -329,5 +321,4 @@ pub trait ByteSource { } ``` -The shared borrow lets the host implementation copy from a `&[u8]` and the device implementation -read a held flat-store object. `RouteReader` holds `&dyn ByteSource`, so it stays monomorphic. +`RouteReader` holds `&dyn ByteSource`. From 65ad9c20fbcf3eee32f223ce8a80ecbc8f65ba45 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 29 Oct 2026 08:09:38 +0000 Subject: [PATCH 03/20] docs(specs): common header and trim for the TrailView specs Both specs open with the common header. The vectors paragraph moves into it, the format-reader rule is stated once, and OSM narrative and stamp rationale are cut to one sentence or removed. Long lines are wrapped at 100 columns. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DTHu3LHSGY5CY9j2XmscLi --- specs/trailview-features.md | 279 ++++++++++++++++++------------------ specs/trailview-tiles.md | 46 +++--- 2 files changed, 167 insertions(+), 158 deletions(-) diff --git a/specs/trailview-features.md b/specs/trailview-features.md index 0949e2c790..2d93aebb78 100644 --- a/specs/trailview-features.md +++ b/specs/trailview-features.md @@ -1,11 +1,19 @@ # TrailView feature tiles -The layer `planner/trailview-features` holds the OSM features that TrailView draws. Its step -reads the OSM of `planner/osm` and the master terrain of the grid of `planner/terrain` in the same -release, and the water polygons of the source `water-polygons`. -The kind table -[`data/trailview/kinds.json`](../data/trailview/kinds.json) selects the features and the tags -that the tiles keep. A new kind is one entry in that table. The format does not change. +The archive of the layer `planner/trailview-features`: the OSM features that TrailView draws, with +their heights. The kind table [`data/trailview/kinds.json`](../data/trailview/kinds.json) selects +the features and the tags that the tiles keep. + +- Version: `features_format` 2. A reader reads only the format that it knows. +- Writers: the bake `host/obc-trailview-features`. Its step reads the OSM of `planner/osm` and the + master terrain of the grid of `planner/terrain` in the same release, and the water polygons of + the source `water-polygons`. +- Readers: the TrailView engine (`builder/web/src/lib/trailview/stream/features.ts`), + `tools/planner_verify.py`. +- Test vectors: [`vectors/trailview-features.json`](vectors/trailview-features.json) and + [`vectors/trailview-features.bin`](vectors/trailview-features.bin): one tile in its decoded form + and its bytes. The encoder of the bake and the two decoders of the readers read both. +- Related: [obc-data.md](obc-data.md), [TrailView tiles](trailview-tiles.md). ## Archive @@ -28,9 +36,8 @@ lower. A tile that meets `bounds` and that the archive does not hold has no feat A tile holds each feature that comes within the buffer of the tile square, cut to the square grown by the buffer. In tile units, the buffer is `buffer_m` on the ground at the tile edge -nearer the pole, so it is at least `buffer_m` at every point of the tile. The widest stamp, the -cut and fill cone of a road, reaches 25 m past the road edge. 40 m holds it for a road up to -30 m wide, so neighbouring terrain nodes stamp the same apron. +nearer the pole, so it is at least `buffer_m` at every point of the tile. 40 m holds the cut +and fill cone of a road up to 30 m wide, which reaches 25 m past the road edge. The cut makes no vertex. A piece of a line keeps the segments that meet the buffered square whole, and one more vertex at each end, so that both tiles know the bend at each end of such a segment. @@ -39,16 +46,15 @@ the side directly. Such a join lies outside the square. Thus inside the buffered tiles that hold a feature hold the same vertices and edges, and a stamp near a tile edge gets the same answer in both tiles. -The buffer is for the stamps. A tile owns the points from 0 to less than 16384 on each axis: a +A tile owns the points from 0 to less than 16384 on each axis: a reader that draws or places features uses only the part of a feature in that square, so a point on a shared edge or in the buffer of two tiles is drawn once. ### Generalization -Zoom 13 keeps each vertex, and the vertices that the road solver adds (see Heights). Below zoom 13, the bake simplifies each line and ring with -Douglas-Peucker to `extent / 2048` tile units, a quarter texel of a 512-texel terrain node. A -vertex that two features share stays, and so do the ends of a line. Thus road junctions stay -joined, and a river keeps its end on the shore of its lake. Then each edge of an area that +Zoom 13 keeps each vertex, and the vertices that the road solver adds (see Heights). Below zoom +13, the bake simplifies each line and ring with Douglas-Peucker to `extent / 2048` tile units. A +vertex that two features share stays, and so do the ends of a line. Then each edge of an area that crosses another edge of the same feature gets back the vertex at which Douglas-Peucker would split it next, until no two edges cross. Thus a ring that does not cross itself in OSM does not cross itself after the simplification, and an island stays inside its lake. The bake simplifies a @@ -76,12 +82,12 @@ part = varint (vertex_count × 2 + outer), vertex × vertex_count vertex = zigzag dx, zigzag dy, then varint z_code when has_z is 1 ``` -`kind`, `key` and `value` are indices from 0 into the strings of the tile. The tags of a feature -are sorted by key. `geometry` is 0 for a point, 1 for a line and 2 for an area. The features of a tile +`kind`, `key` and `value` are indices from 0 into the strings of the tile. The tags of a feature are +sorted by key. `geometry` is 0 for a point, 1 for a line and 2 for an area. The features of a tile come in a fixed order: first the kinds that no solver sets, in the order of the OSM file, the points -and the lines before the areas; then the buildings, the lifts, the rivers and lakes, the roads, paths -and tracks, the bridge piers, the catenary masts, the platforms and the sea. So the bytes of a tile do not -depend on how much of the file lies outside the region. +and the lines before the areas; then the buildings, the lifts, the rivers and lakes, the roads, +paths and tracks, the bridge piers, the catenary masts, the platforms and the sea. So the bytes of a +tile do not depend on how much of the file lies outside the region. ### Coordinates @@ -106,13 +112,14 @@ first vertex of a feature is the difference from (0, 0). The solvers of the bake set the heights of a feature. A feature with no height has `has_z` 0. -Each vertex of a `main-road`, `road`, `path`, `railway`, `tunnel` or `gallery` has the height of the road -surface or of the rails, except a `via_ferrata`, which has none. The class table -[`data/trailview/roads.json`](../data/trailview/roads.json) gives the rules per class: the steepest grade -and a smoothing length. A track is a way with a `railway` of `rail`, `narrow_gauge`, `light_rail` or -`funicular` and no `highway`. The class of a track is its `railway`, or `rack` when it has a `rack` tag -other than `no`; the class of another way is its `highway`. The table lists the tracks first, so at a level crossing the -road meets the track. It is part of the options of the step. The road solver: +Each vertex of a `main-road`, `road`, `path`, `railway`, `tunnel` or `gallery` has the height of the +road surface or of the rails, except a `via_ferrata`, which has none. The class table +[`data/trailview/roads.json`](../data/trailview/roads.json) gives the rules per class: the steepest +grade and a smoothing length. A track is a way with a `railway` of `rail`, `narrow_gauge`, +`light_rail` or `funicular` and no `highway`. The class of a track is its `railway`, or `rack` when +it has a `rack` tag other than `no`; the class of another way is its `highway`. The table lists the +tracks first, so at a level crossing the road meets the track. It is part of the options of the +step. The road solver: - samples the master terrain every 2 m along each way, - fits a smooth height line to the samples, with weights that let a spike or a gully of the terrain @@ -123,59 +130,62 @@ road meets the track. It is part of the options of the step. The road solver: surface of the through road of a junction (1.5 m where paths meet), but not past the middle of a way between two junctions. The through road is the first class of the table at the junction, and of two ways of that class, the pair that runs most nearly straight on, -- puts each vertex of a track on the ground at the height of the nearest track on the ground that runs - the same way within 6 m beside it, so that the tracks of a double line lie at one height. A track - follows only a track after it in the bake's order. A vertex keeps its own height where the terrain - under the two tracks differs by more than 0.5 m, there or within 100 m along its way: a ramp or a - flyover, -- ignores the terrain under a bridge and in a tunnel: these run between their ends. A tunnel runs in a - straight line between the two ends of its chain of tunnel ways that meet end to end, so a side tunnel - or a shaft that joins it follows it and does not lift it. A tunnel chain with an end that meets no way - runs level from its other end, or on the grade of the road that runs on into it there. A bridge never - sags under the straight line between the two ends of its chain of bridge ways, +- puts each vertex of a track on the ground at the height of the nearest track on the ground that + runs the same way within 6 m beside it, so that the tracks of a double line lie at one height. A + track follows only a track after it in the bake's order. A vertex keeps its own height where the + terrain under the two tracks differs by more than 0.5 m, there or within 100 m along its way: a + ramp or a flyover, +- ignores the terrain under a bridge and in a tunnel: these run between their ends. A tunnel runs in + a straight line between the two ends of its chain of tunnel ways that meet end to end, so a side + tunnel or a shaft that joins it follows it and does not lift it. A tunnel chain with an end that + meets no way runs level from its other end, or on the grade of the road that runs on into it + there. A bridge never sags under the straight line between the two ends of its chain of bridge + ways, - raises a bridge that is not a culvert where it crosses a waterway, or a road, a path or a track on - the ground that it does not meet: its road lies at least 1.5 m (water), 4.5 m (road), 2.5 m (path) or - 6 m (track) over it, plus a deck of 1 m, or 0.4 m for a path. The way below may sink to give the room. A bridge whose end - lies on the road that it crosses, within that road's half-width, meets it there and does not rise, -- lets a bridge rise from where it lands at a height that it cannot move no more steeply than 1:1: an - end that meets no other way stands on the terrain, and an end that leaves a junction as a side lies on - the through road. A crossing that asks more lies where the bridge lands and does not raise it, -- sinks a tunnel where a road, a path or a track crosses over it without meeting it: the tunnel's road lies at - least its bore and 1 m under that road's middle and both its edges. A road on the ground over a tunnel - does not move, -- passes one way over the other where two bridges cross that no bridge joins or that both tag a - different `layer`, and where two ways on the ground with a different `layer` cross, without meeting: the one of - the higher `layer`, at an equal layer of the first class of the table, and of one class the later one - in the OSM data, lies over the other by the clearance and the deck above. Both ways may move to give - the room. A way on the ground of a negative `layer` without a `tunnel` tag, such as a corridor of a - station, runs under something that the solver leaves out: no way on the ground passes over or under + the ground that it does not meet: its road lies at least 1.5 m (water), 4.5 m (road), 2.5 m (path) + or 6 m (track) over it, plus a deck of 1 m, or 0.4 m for a path. The way below may sink to give + the room. A bridge whose end lies on the road that it crosses, within that road's half-width, + meets it there and does not rise, +- lets a bridge rise from where it lands at a height that it cannot move no more steeply than 1:1: + an end that meets no other way stands on the terrain, and an end that leaves a junction as a side + lies on the through road. A crossing that asks more lies where the bridge lands and does not raise it, +- sinks a tunnel where a road, a path or a track crosses over it without meeting it: the tunnel's + road lies at least its bore and 1 m under that road's middle and both its edges. A road on the + ground over a tunnel does not move, +- passes one way over the other where two bridges cross that no bridge joins or that both tag a + different `layer`, and where two ways on the ground with a different `layer` cross, without + meeting: the one of the higher `layer`, at an equal layer of the first class of the table, and of + one class the later one in the OSM data, lies over the other by the clearance and the deck above. + Both ways may move to give the room. A way on the ground of a negative `layer` without a `tunnel` + tag, such as a corridor of a station, runs under something that the solver leaves out: no way on + the ground passes over or under it, - puts a vertex that a road shares with a river or a stream, and the crossing of a way tagged `ford` with a waterway, 0.2 m below the water. -The road solver works by area: an area is a tile of zoom 10. The solve of an area holds each way within -500 m of the area, along the ways from the nearest vertex of OSM of a way in a straight line. A bridge or a tunnel -has no length here, so the solve holds it whole, and the ways that it crosses join at their distance from -it in a straight line. Each vertex of OSM in an area, and each vertex that the solver adds after it up to -the next vertex of OSM, takes its height from the solve of that area. A network of bridges and tunnels -that share a point joins the solve whole, also where it runs past the region, with each way on the -ground that meets one of its ends there. In the order of the OSM data that the rules above use, each -way past the region comes after each way of the region. The bake reads OSM to 20 km and one zoom-13 -tile past the region. A network with an end past that which meets no way has no heights: the input cuts -it, so it has no ends to run between. A bridge is a culvert, and a -tunnel passes under a road, by the solve of the area of its first vertex. A solve of an area does not depend on the -other solves, so the heights do not depend on how many solves run at once. - -The tiles keep the vertices of OSM and the vertex at the end of each approach of a side road, and add -a vertex between two of them where the solved height leaves the straight line between them by more -than 20 cm. A bridge shorter than 8 m whose deck lies -no more than 1 m above the terrain is a culvert: the tiles keep it without its `bridge` tag, as a road. -A tunnel whose terrain lies less than its bore (5 m, 2.6 m for a path, 6.5 m for a track) over it -everywhere runs under something that the terrain leaves out, such as a building: the tiles keep it -without its `tunnel` tag, as a way of its class. A `building_passage` and a tunnel under a road, a path or -a track stay tunnels. The tiles leave out a part of -a tunnel that rises more than 5 m over the terrain anywhere farther than 20 m from its ends: the solver -could not put it under the terrain. +The road solver works by area: an area is a tile of zoom 10. The solve of an area holds each way +within 500 m of the area, along the ways from the nearest vertex of OSM of a way in a straight line. +A bridge or a tunnel has no length here, so the solve holds it whole, and the ways that it crosses +join at their distance from it in a straight line. Each vertex of OSM in an area, and each vertex +that the solver adds after it up to the next vertex of OSM, takes its height from the solve of that +area. A network of bridges and tunnels that share a point joins the solve whole, also where it runs +past the region, with each way on the ground that meets one of its ends there. In the order of the +OSM data that the rules above use, each way past the region comes after each way of the region. The +bake reads OSM to 20 km and one zoom-13 tile past the region. A network with an end past that which +meets no way has no heights: the input cuts it, so it has no ends to run between. A bridge is a +culvert, and a tunnel passes under a road, by the solve of the area of its first vertex. A solve of +an area does not depend on the other solves, so the heights do not depend on how many solves run at +once. + +The tiles keep the vertices of OSM and the vertex at the end of each approach of a side road, and +add a vertex between two of them where the solved height leaves the straight line between them by +more than 20 cm. A bridge shorter than 8 m whose deck lies no more than 1 m above the terrain is a +culvert: the tiles keep it without its `bridge` tag, as a road. A tunnel whose terrain lies less +than its bore (5 m, 2.6 m for a path, 6.5 m for a track) over it everywhere runs under something +that the terrain leaves out, such as a building: the tiles keep it without its `tunnel` tag, as a +way of its class. A `building_passage` and a tunnel under a road, a path or a track stay tunnels. +The tiles leave out a part of a tunnel that rises more than 5 m over the terrain anywhere farther +than 20 m from its ends. ### Structures @@ -190,54 +200,55 @@ The bake adds these to the features of the road solver: | tag `stations` | each `aerialway` with an end that has no station of its own | `start`, `end` or `none`: the ends that have one | | point `catenary-mast` | each mast of the contact wire of a `railway` tagged `electrified` `contact_line` or `yes`, but not a funicular | Its height is its track's rail there. Tag `on` `deck` on a bridge | -The masts of a track stand evenly along each of its ways, at most 50 m apart, 3 m beside its middle, or -2.5 m for a narrow gauge (`gauge` below 1200 mm, else `narrow_gauge` and `funicular`). On a bridge a mast -stands against the deck's edge: 2.02 m beside the middle, or 1.62 m for a narrow gauge. A mast stands on -the side away from another track that lies 1 to 8 m beside it; a track with such tracks on both sides -has no masts. The bake leaves out a mast that lies less than 8 m from a mast before it. A track in a -tunnel has no masts. - -Each vertex of a `platform` or a `platform-line` has the height of its top: 0.55 m, or 0.35 m for a narrow -gauge, over the rail of the nearest track within 15 m that is not in a tunnel. A vertex with no track -near takes the median of the other vertices; a platform with no track near has no heights. - -A bridge is the chain of bridge ways whose ends meet two by two. Its piers divide it evenly into spans -of at most 45 m, or 30 m for a path. A pier that falls in water which the bridge crosses for less than -30 m moves 2 m past the nearer bank of it, and one that falls on a road, a path or a tunnel moves 2 m -past its nearer side; one that comes within 6 m of another pier or of an end goes. Water is a waterway -within its half-width and 1 m, a river area or a lake. A road is its line within 1 m and its -half-width: half its `width` (at least 0.4 m), else 6 m for a main road, 3 m for a road or a -`pedestrian` way, 1.5 m for a `cycleway`, 0.6 m for a `via_ferrata` and 1 m for another path. A cable -bridge has no piers. - -Each vertex of an `aerialway` between its ends is a pylon. OSM maps the stations at the ends of the way -and often no pylon between them. A way splits at each vertex where another `aerialway` ends: there is a -mid-station. Where a span is longer than its rig allows, or where a carrier would come nearer the -terrain than its clearance, the bake adds a pylon: in the middle of a long span, else at the point of -the span at least 25 m from both of its ends where the carrier comes nearest. The rigs are those of -`host/obc-trailview-features/src/lifts.rs`. A station at an end that lies inside a mapped building is that building; -of the ways that end at one point, only the first in the OSM data has the station there. The tag -`stations` names the ends that have a station of their own. - -A `building` in an area `landuse=industrial` or `landuse=farmyard` has the tag `landuse` of that area: -of the smallest such area that holds the mean of the vertices of its first outer ring. The tiles do not -hold these areas. - -A `building` has the tag `use`, `pharmacy`, `hospital` or `shop`, when it, a node that lies inside or on its first -outer ring, or an area that holds the mean of that ring's vertices has that use. `amenity` or `healthcare` -`pharmacy` is a pharmacy, `amenity` or `healthcare` `hospital` a hospital, and any `shop` a shop. Of -several uses, the first in that order wins. The tiles do not hold these nodes and areas. +The masts of a track stand evenly along each of its ways, at most 50 m apart, 3 m beside its middle, +or 2.5 m for a narrow gauge (`gauge` below 1200 mm, else `narrow_gauge` and `funicular`). On a +bridge a mast stands against the deck's edge: 2.02 m beside the middle, or 1.62 m for a narrow +gauge. A mast stands on the side away from another track that lies 1 to 8 m beside it; a track with +such tracks on both sides has no masts. The bake leaves out a mast that lies less than 8 m from a +mast before it. A track in a tunnel has no masts. + +Each vertex of a `platform` or a `platform-line` has the height of its top: 0.55 m, or 0.35 m for a +narrow gauge, over the rail of the nearest track within 15 m that is not in a tunnel. A vertex with +no track near takes the median of the other vertices; a platform with no track near has no heights. + +A bridge is the chain of bridge ways whose ends meet two by two. Its piers divide it evenly into +spans of at most 45 m, or 30 m for a path. A pier that falls in water which the bridge crosses for +less than 30 m moves 2 m past the nearer bank of it, and one that falls on a road, a path or a +tunnel moves 2 m past its nearer side; one that comes within 6 m of another pier or of an end goes. +Water is a waterway within its half-width and 1 m, a river area or a lake. A road is its line within +1 m and its half-width: half its `width` (at least 0.4 m), else 6 m for a main road, 3 m for a road +or a `pedestrian` way, 1.5 m for a `cycleway`, 0.6 m for a `via_ferrata` and 1 m for another path. A +cable bridge has no piers. + +Each vertex of an `aerialway` between its ends is a pylon. A way splits at each vertex where another +`aerialway` ends: there is a mid-station. Where a span is longer than its rig allows, or where a +carrier would come nearer the terrain than its clearance, the bake adds a pylon: in the middle of a +long span, else at the point of the span at least 25 m from both of its ends where the carrier comes +nearest. The rigs are those of `host/obc-trailview-features/src/lifts.rs`. A station at an end that +lies inside a mapped building is that building; of the ways that end at one point, only the first in +the OSM data has the station there. The tag `stations` names the ends that have a station of their +own. + +A `building` in an area `landuse=industrial` or `landuse=farmyard` has the tag `landuse` of that +area: of the smallest such area that holds the mean of the vertices of its first outer ring. The +tiles do not hold these areas. + +A `building` has the tag `use`, `pharmacy`, `hospital` or `shop`, when it, a node that lies inside +or on its first outer ring, or an area that holds the mean of that ring's vertices has that use. +`amenity` or `healthcare` `pharmacy` is a pharmacy, `amenity` or `healthcare` `hospital` a hospital, +and any `shop` a shop. Of several uses, the first in that order wins. The tiles do not hold these +nodes and areas. Each vertex of a `river` or `stream` that is not in a tunnel has a water level, never higher than a vertex before it: the vertices run in the direction of the flow. OSM draws a waterway that way; the -bake turns a way whose terrain rises by more than 5 m from end to end. The level is the fit that never -rises (pool-adjacent violators) to the lowest terrain within ±(w/2 + 2 m) across the line, every -5 m, with a fall of at least 0.05 % in a pool, smoothed over ±25 m. The smoothing does not cross a -`waterfall`, a `dam` or a `dam-point`. Waterways that share a vertex are one network: a level never -rises through a junction, and the waterways at a junction have one level there. In a loop of -waterways, the waterways go from the highest start down, and one that flows back into a junction -before it ends below that junction's level. A point in a lake, or within 2 m of its shore, holds the -lake's level. +bake turns a way whose terrain rises by more than 5 m from end to end. The level is the fit that +never rises (pool-adjacent violators) to the lowest terrain within ±(w/2 + 2 m) across the line, +every 5 m, with a fall of at least 0.05 % in a pool, smoothed over ±25 m. The smoothing does not +cross a `waterfall`, a `dam` or a `dam-point`. Waterways that share a vertex are one network: a +level never rises through a junction, and the waterways at a junction have one level there. In a +loop of waterways, the waterways go from the highest start down, and one that flows back into a +junction before it ends below that junction's level. A point in a lake, or within 2 m of its shore, +holds the lake's level. Each vertex of a `lake` has one level. It is the median of the terrain inside the lake when 80 % of that terrain lies within 0.3 m of it; else the 15th percentile of the terrain 2 m inside the shore. @@ -245,29 +256,23 @@ A pond (`water=pond`) takes the lowest terrain 2 m inside its shore, a reservoir or `landuse=reservoir`) the median of the terrain on its outline: its full level. An `ele` tag within 1.5 m of the level wins. Of the other kinds, only the `sea` has a height (see Sea). -The water solver solves each network apart, with each lake whose bounding box comes within 120 m of the -bounding box of one of its waterways, and each other lake alone. A lake near two networks joins them into -one solve. A level depends only on its network and on the lakes near it, so the levels are those of one solve -of all waterways and lakes. +The water solver solves each network apart, with each lake whose bounding box comes within 120 m of +the bounding box of one of its waterways, and each other lake alone. A lake near two networks joins +them into one solve. A level depends only on its network and on the lakes near it, so the levels are +those of one solve of all waterways and lakes. With `has_z` 1, each vertex has a `z_code`. `z_code` 0 means not solved: the height is unknown, never 0 m. Otherwise `z_code − 1` is the zigzag difference in centimetres from the last solved height of the feature, which starts at 0. A height is in metres in the vertical datum of `planner/terrain`. -The shared vectors [`vectors/trailview-features.json`](vectors/trailview-features.json) and -[`vectors/trailview-features.bin`](vectors/trailview-features.bin) are one tile in its decoded -form and its bytes. The Rust encoder of the bake, the Python decoder of `tools/planner_verify.py` and -the TypeScript decoder of the engine read both. - ### Sea -OSM maps the sea only as `natural=coastline` lines. The bake adds the kind `sea`, `min_zoom` 8: each -polygon of the source `water-polygons` (`water-polygons-split-3857.zip` of osmdata.openstreetmap.de) -whose bounding box comes within one zoom-8 tile of the region. It is one area with its islands as -holes, and each of its vertices has the height 0 m, sea level in the vertical datum of -`planner/terrain`. The source splits the sea into pieces that overlap by 100 m at the split lines, -so no gap opens between them. +The bake adds the kind `sea`, `min_zoom` 8: each polygon of the source `water-polygons` +(`water-polygons-split-3857.zip` of osmdata.openstreetmap.de) whose bounding box comes within one +zoom-8 tile of the region. It is one area with its islands as holes, and each of its vertices has +the height 0 m, sea level in the vertical datum of `planner/terrain`. The source splits the sea into +pieces that overlap by 100 m at the split lines, so no gap opens between them. ## Kind table @@ -303,6 +308,6 @@ example, wayside shrines: } ``` -Then add the engine module of the kind, `builder/web/src/lib/trailview/kinds/shrine.ts`, and the model -that it names, `models/shrine.ts`. The engine finds the module by its file name. The tile format, the -bake, the decoder and the other engine files do not change. +Then add the engine module of the kind, `builder/web/src/lib/trailview/kinds/shrine.ts`, and the +model that it names, `models/shrine.ts`. The engine finds the module by its file name. The tile +format, the bake, the decoder and the other engine files do not change. diff --git a/specs/trailview-tiles.md b/specs/trailview-tiles.md index 3f87a532e1..06f394e3f9 100644 --- a/specs/trailview-tiles.md +++ b/specs/trailview-tiles.md @@ -1,9 +1,15 @@ # TrailView tiles -TrailView reads the raster layers of the planner release: the ground and the colour of each -texel, and the ruleset that the ground indexes. The layers are the optional planner layers -`trailview-rules`, `trailview-ground` and `trailview-colour` of [obc-data.md](obc-data.md). The -bakes are `tools/trailview/rules.py`, `tools/trailview/ground.py` and `tools/trailview/colour.py`. +The raster layers of the planner release that TrailView reads: the ground, the colour, the snow +days and the canopy of each texel or cell, and the ruleset that the ground indexes. + +- Version: `trailview_format` 3 (ground, snow days) and 2 (colour); `canopy_format` 1; `format` 3 + of `rules.json`. A reader reads only the format that it knows. +- Writers: `tools/trailview/rules.py`, `tools/trailview/ground.py`, `tools/trailview/colour.py`, + `tools/trailview/snow.py`, `tools/planner_trailview_canopy.py`. +- Readers: the TrailView engine (`builder/web/src/lib/trailview/`), `tools/planner_verify.py`. +- Related: the optional planner layers `trailview-*` of [obc-data.md](obc-data.md), + [the planner snow archive](planner-snow-tiles.md), [TrailView feature tiles](trailview-features.md). ## Archives @@ -33,7 +39,7 @@ The archive metadata is a JSON object: | Key | Value | | --- | --- | -| `trailview_format` | `3` in the ground archive, `2` in a colour archive. A reader reads only the format that it knows | +| `trailview_format` | `3` in the ground archive, `2` in a colour archive | | `archive` | The name of the archive: `ground`, or `q1` to `q4` | | `variant` | Only in a colour archive: the variant of the table in [Colour](#colour) | | `attribution` | Text to show with the layer | @@ -122,9 +128,9 @@ A class matches by its `match`: A `word` legend may have a `qualifier`: `word`, `within` and `class`. Where its word follows the word of the found class within `within` characters, or the value holds its word and no class, the -value names its `class`. A value without a class names nothing. `authority` takes each class that the source names, but a -material source keeps `inland-water`. `refine` takes a class only where `refines` lists the -texel's code for that class. +value names its `class`. A value without a class names nothing. `authority` takes each class that +the source names, but a material source keeps `inland-water`. `refine` takes a class only where +`refines` lists the texel's code for that class. The entry with `rule: satellite-heath` reads `sentinel-2-mosaics`. It makes `material` of each texel whose material it lists in `on`, where the summer quarter, or spring where summer saw @@ -151,17 +157,17 @@ round(255 × refl^(1/2.2)). The NDVI has half the resolution of the texels: a texel with a code of 2 or more takes the mean code m of the texels with a code of 2 or more in its aligned 2 × 2 block, which starts at even texel coordinates. m is rounded half to even and stored as 128 + 4 × floor((m − 128) / 4) + 2. So a stored -code is a multiple of 4 plus 2, and it is below 128 when m is below 128. Codes 0 and 1 stay as they are. This is the last step of the bake, on -each tile of each zoom. +code is a multiple of 4 plus 2, and it is below 128 when m is below 128. Codes 0 and 1 stay as they +are. This is the last step of the bake, on each tile of each zoom. A tile is two images, one after the other: 1. The NDVI tile: a lossless WebP of 256 × 256 pixels, one pixel per aligned 2 × 2 block of texels. Red is the code that the measured texels of the block share, or 0 where no texel is measured. - Green holds a mark of 2 bits for each of the four texels, in reading order from bit 0: north-west, - north-east, south-west, south-east. A mark is 0 or 1, the code of the texel, or 2: the texel - has the code in red. The length of the image is 8 + the RIFF size in bytes 4 to 7, little - endian. + Green holds a mark of 2 bits for each of the four texels, in reading order from bit 0: + north-west, north-east, south-west, south-east. A mark is 0 or 1, the code of the texel, or 2: + the texel has the code in red. The length of the image is 8 + the RIFF size in bytes 4 to 7, + little endian. 2. The colour: an RGB image of 512 × 512 texels in the codec of the variant. In the first and the last quarter, a texel darker than 0.025 in visible reflectance is terrain @@ -243,10 +249,8 @@ with these rules: `trailview-canopy` holds the forest of each cell of zoom 12 at 256 × 256 cells, 26 m at 47° N: the canopy height, the cover, a class of the stems per hectare and the leaf type. It holds no tree -position: the engine places trees that match these values. The layer is the optional planner layer -`trailview-canopy` of [obc-data.md](obc-data.md). Its bake, `tools/planner_trailview_canopy.py`, -reads the zoom 12 tiles out of the blocks of the prepared source [`forest`](obc-data.md#forest), -one window each. +position. Its bake reads the zoom 12 tiles out of the blocks of the prepared source +[`forest`](obc-data.md#forest), one window each. | Field | Value | | --- | --- | @@ -257,13 +261,13 @@ one window each. | Bounds | The region bounds | Cell (column, row) of tile 12/x/y covers that cell of the 256 × 256 grid of the tile. Row 0 is the -north edge and column 0 is the west edge. A tile that the archive does not hold is unknown: no -cell of its window in `forest` is known. The release packs the archive in PMTiles per zoom 11 parent: +north edge and column 0 is the west edge. A tile that the archive does not hold is unknown: no cell +of its window in `forest` is known. The release packs the archive in PMTiles per zoom 11 parent: `maps/tiles/trailview-canopy/--.pmtiles`, with the TileJSON `maps/trailview-canopy.json`. | Metadata | Value | | --- | --- | -| `canopy_format` | `1`. A reader reads only the format that it knows | +| `canopy_format` | `1` | | `archive` | `trailview-canopy` | | `attribution` | Text to show with the layer | From 68b385dbb0cdbb2a49d21da2e609b4bcc61cde00 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 29 Oct 2026 08:10:40 +0000 Subject: [PATCH 04/20] docs(specs): common header and trim for the planner, route and Wikimedia specs Each spec opens with the common header; writers and readers are checked against the code. Reader-behaviour notes, implementation notes, rationale derivations and record wording are cut. Long lines are wrapped at 100 columns. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DTHu3LHSGY5CY9j2XmscLi --- specs/planner-climate-tiles.md | 86 ++++---- specs/planner-offline.md | 141 ++++++------ specs/planner-plan.md | 58 ++--- specs/planner-release.md | 389 ++++++++++++++++----------------- specs/planner-snow-tiles.md | 87 ++++---- specs/route-api.md | 169 +++++++------- specs/route-catalog.md | 198 ++++++++--------- specs/route-package.md | 252 ++++++++++----------- specs/wikimedia-content.md | 291 ++++++++++++------------ 9 files changed, 802 insertions(+), 869 deletions(-) diff --git a/specs/planner-climate-tiles.md b/specs/planner-climate-tiles.md index f8451a09f1..bc403e64e4 100644 --- a/specs/planner-climate-tiles.md +++ b/specs/planner-climate-tiles.md @@ -1,9 +1,13 @@ # Planner climate tiles -The planner climate layer reads one archive per region: `climate.pmtiles`. It -holds rain, temperature and wind for each week of ten years. The bake is -`tools/planner_climate.py`. It reads the daily values of the prepared source -[`climate`](obc-data.md#climate). +The planner climate archive of one region, `climate.pmtiles`: rain, temperature and wind for each +week of ten years. + +- Writers: `tools/planner_climate.py`, from the daily values of the prepared source + [`climate`](obc-data.md#climate). +- Readers: the web planner climate layer (`builder/web/src/lib/planner/layers/climate.ts`), + `tools/planner_verify.py`. +- Related: [planner release](planner-release.md), [planner snow tiles](planner-snow-tiles.md). ## Archive @@ -19,28 +23,25 @@ The tiles are blocks of grid cells. They are not Web Mercator tiles. ## Cells -The grid is the ERA5-Land grid of 0.1° × 0.1° cells. Cell column i, from 0 to -3599, has its centre at longitude −180 + 0.1 i. Cell row j, from 0 to 1800, has -its centre at latitude 90 − 0.1 j. A point is in column ⌊10 (lon + 180) + 0.5⌋ -mod 3600 and row ⌊10 (90 − lat) + 0.5⌋. +The grid is the ERA5-Land grid of 0.1° × 0.1° cells. Cell column i, from 0 to 3599, has its centre +at longitude −180 + 0.1 i. Cell row j, from 0 to 1800, has its centre at latitude 90 − 0.1 j. A +point is in column ⌊10 (lon + 180) + 0.5⌋ mod 3600 and row ⌊10 (90 − lat) + 0.5⌋. | Zoom | Columns × rows | Tile x holds columns | Tile y holds rows | | --- | --- | --- | --- | | 8 | 24 × 16 | 24x to 24x + 23 | 16y to 16y + 15 | | 9 | 12 × 8 | 12x to 12x + 11 | 8y to 8y + 7 | -The zoom 8 tile x, y holds the same cells as the zoom 9 tiles 2x to 2x + 1 and -2y to 2y + 1. +The zoom 8 tile x, y holds the same cells as the zoom 9 tiles 2x to 2x + 1 and 2y to 2y + 1. -The archive holds the cells that touch the bounds. Every other cell, and every -cell without source data (sea), has the missing code in every plane. A tile -without a cell with data is absent. +The archive holds the cells that touch the bounds. Every other cell, and every cell without source +data (sea), has the missing code in every plane. A tile without a cell with data is absent. ## Tile body -The body is a sequence of planes. A plane has one value per cell for each of its -indexes. For each index, from 0 in order, the cells follow in row order: row 0 is -the north edge and column 0 is the west edge. A 2-byte value is little-endian. +The body is a sequence of planes. A plane has one value per cell for each of its indexes. For each +index, from 0 in order, the cells follow in row order: row 0 is the north edge and column 0 is the +west edge. A 2-byte value is little-endian. Zoom 9, the detail tile, has 96 cells and these planes: @@ -75,8 +76,7 @@ The overview body is 183,552 bytes. ## Codes -A value takes the nearest code, with a half rounded up, and is clamped to the -codes of its plane. +A value takes the nearest code, with a half rounded up, and is clamped to the codes of its plane. | Plane | Type | Missing | Codes | Value | | --- | --- | --- | --- | --- | @@ -92,15 +92,13 @@ codes of its plane. ## Time -Year 0 is `first_year`. A day is a local solar day: local time is UTC plus -⌊lon ÷ 15 + 0.5⌋ hours, with the longitude of the cell centre. +Year 0 is `first_year`. A day is a local solar day: local time is UTC plus ⌊lon ÷ 15 + 0.5⌋ hours, +with the longitude of the cell centre. -Week w of a year holds the days 7w to 7w + 6 of the year, from day 0 on -1 January. Week 51 also holds days 364 and 365, so it has 8 days, or 9 days in a -leap year. +Week w of a year holds the days 7w to 7w + 6 of the year, from day 0 on 1 January. Week 51 also +holds days 364 and 365, so it has 8 days, or 9 days in a leap year. -The month of week w is the month of day 7w + 3 of a year that is not a leap -year, in all years. +The month of week w is the month of day 7w + 3 of a year that is not a leap year, in all years. ## Values @@ -125,8 +123,7 @@ Detail planes, for each year and week: A week with a missing hour is missing. -Overview planes, for each week, use the decoded detail values of the years that -are not missing: +Overview planes, for each week, use the decoded detail values of the years that are not missing: | Plane | Value | | --- | --- | @@ -141,20 +138,17 @@ Static planes: | `lapse_tmax`, `lapse_tmin` | Change of the maximum and the minimum temperature with height | | `rose` | Share of the daytime hourly samples of the month, in all years, with wind from the sector | -The temperatures of a cell are at its orography. At height h, a maximum -temperature is T + `lapse_tmax` × (h − `orography`) ÷ 1000, and a minimum -temperature is T + `lapse_tmin` × (h − `orography`) ÷ 1000, with the rate of the -month. +The temperatures of a cell are at its orography. At height h, a maximum temperature is +T + `lapse_tmax` × (h − `orography`) ÷ 1000, and a minimum temperature is +T + `lapse_tmin` × (h − `orography`) ÷ 1000, with the rate of the month. -`lapse_tmax`: the monthly mean of a cell is the mean of the daily maximum -temperatures over the days of the month in all years. A least-squares fit -T = a + b × height + c × lon + d × lat through the 5 × 5 cells around the cell gives -b. When the heights of these cells differ by less than 100 m, b is −6.5 K/km. -`lapse_tmin` is the same for the daily minimum temperatures. +`lapse_tmax`: the monthly mean of a cell is the mean of the daily maximum temperatures over the days +of the month in all years. A least-squares fit T = a + b × height + c × lon + d × lat through the +5 × 5 cells around the cell gives b. When the heights of these cells differ by less than 100 m, b is +−6.5 K/km. `lapse_tmin` is the same for the daily minimum temperatures. -`rose`: the wind comes from the direction atan2(−u, −v), clockwise from north. -Sector s holds the directions from 22.5 s − 11.25° up to, but not including, -22.5 s + 11.25°. +`rose`: the wind comes from the direction atan2(−u, −v), clockwise from north. Sector s holds the +directions from 22.5 s − 11.25° up to, but not including, 22.5 s + 11.25°. ## Metadata @@ -173,11 +167,9 @@ The archive metadata is a JSON object: ## Source -The source is ERA5-Land hourly data, DOI 10.24381/cds.e2161bac, licence CC BY -4.0. The producer of `climate` reads the ECMWF ARCO geo-chunked Zarr stores of -ERA5-Land: `t2m`, `u10`, `v10` and `tp`. `tp` holds the precipitation of each -hour. It uses final data only: a chunk is final in a store that reaches 120 -days past the end of the chunk or, when that is earlier, past the end of the -year that it is read for. A chunk that the store omits has no data. The orography is the ERA5-Land -geopotential file `geo_1279l4_0.1x0.1.grib2_v4_unpack.nc` with the SHA-256 in -`orography_sha256`. +The source is ERA5-Land hourly data, DOI 10.24381/cds.e2161bac, licence CC BY 4.0. The producer of +`climate` reads the ECMWF ARCO geo-chunked Zarr stores of ERA5-Land: `t2m`, `u10`, `v10` and `tp`. +`tp` holds the precipitation of each hour. It uses final data only: a chunk is final in a store that +reaches 120 days past the end of the chunk or, when that is earlier, past the end of the year that +it is read for. A chunk that the store omits has no data. The orography is the ERA5-Land +geopotential file `geo_1279l4_0.1x0.1.grib2_v4_unpack.nc` with the SHA-256 in `orography_sha256`. diff --git a/specs/planner-offline.md b/specs/planner-offline.md index 0de7e52b63..687b230423 100644 --- a/specs/planner-offline.md +++ b/specs/planner-offline.md @@ -1,8 +1,16 @@ # Offline planner bundle -A bundle carries one [planner release](planner-release.md) that the download -service selects from a grid publication. It preserves the exact release manifest -and every runtime file. It omits source mirrors. +A bundle carries one [planner release](planner-release.md) that the download service selects from a +grid publication: the exact release manifest and every runtime file, without source mirrors. This +contract also defines the grid selection, the download service API and the iOS library. + +- Version: bundle manifest `format` `1`; `offline/catalog.json` `format: 3`; selected release + `offline.format: 2`; service catalog `format: 1`. +- Writers: the download service (`tools/planner_downloads.py`); `tools/planner_grid_index.py` + (offline selection metadata). +- Readers: the companion app (`OBCPlanner/OfflineMapStore.swift`). +- Related: [planner release](planner-release.md), [route package](route-package.md), [route + catalog](route-catalog.md). ## Transfer layout @@ -20,57 +28,47 @@ The bundle manifest has these fields: | `release` | Original release manifest `bytes` and `sha256` | | `files` | Every runtime path from the release, with no omissions or extra paths | -Each file entry repeats the release's `bytes` and `sha256`. Its `transport` object -has `bytes`, `sha256`, and `encoding`. Encoding is `identity` or `gzip`. Decoded -bytes match the release entry. Repeated content uses one object. Paths are relative, -contain no `..`, and cannot replace `release.json`. +Each file entry repeats the release's `bytes` and `sha256`. Its `transport` object has `bytes`, +`sha256`, and `encoding`. Encoding is `identity` or `gzip`. Decoded bytes match the release entry. +Repeated content uses one object. Paths are relative, contain no `..`, and cannot replace +`release.json`. ## Grid publication and selection -A grid release has `grid: {format: 2, zoom: 9, map_zoom: 11}`. Selection cells -use Web Mercator XYZ coordinates at zoom 9. Cell IDs are `9-X-Y`. A requested -rectangle selects every intersecting cell, clipped to the release bounds. -The returned coverage contains the complete requested rectangle. - -The publisher creates routing page packs, search databases, and PMTiles packs -once. Map packs group each tile under its ancestor at `min(tile_zoom, 11)`. -Original compressed tile payloads remain unchanged. Terrain is the exception: its -tiles are encoded again with the same pixels, and its zoom 13 tiles have packs of -their own ([terrain tiles](planner-release.md#terrain-tiles)). -A routing pack contains pages with the same cell consumers, up to 16 MiB. -Search retains whole intersecting records and their dependencies. -Each cell also has its [route catalog](route-catalog.md) file. - -`offline/catalog.json` has `format: 3`. It lists cell bounds, logical file -names, map packs, and routing cell descriptors. Its map packs omit the online -places archive and the zoom 13 terrain packs. Each routing descriptor has a manifest path and SHA-256, source -adjacency ranges, and retained geometry bounds. `shared` maps -each map asset path of a selection to its publication file. Files carry the -decoded and transport hashes above. -The service selects these objects and writes only the small selection manifests. -It does not rebuild routing, SQLite databases, or map payloads. - -A selection keeps every glyph range path, `maps/assets/fonts/STACK/RANGE.pbf`. -All range paths of one font stack share `offline/fonts/STACK.pbf`. This file -joins the stack's range files in range order. It keeps only the ranges that -contain a character of a basemap label field or a route network reference in -the release. The label fields are `name`, `name:en`, `name2`, `name3`, their -`pgf:` forms, `ref`, `ref:en`, `shield_text`, and `addr_housenumber`. The -ranges also cover the upper-case form of each text. Text with a character from -U+0600 to U+08FF adds the Arabic presentation forms, U+FB00 to U+FEFF. MapLibre -reads only the glyphs of the requested range from the file. A missing range -file stops the labels of each tile that requests it. - -The selected release has `offline.format: 2`, `id`, `zoom`, `map_zoom`, -`source_routing`, and `cells`. Each cell has `id`, `bounds`, and `files`: the -cell's search databases and route catalog file. -`routing/blocks.json` follows the [routing selection contract](route-package.md#grid-selections). -Map selection takes basemap, overlay and terrain packs, without the zoom 13 -terrain packs. It covers retained road -geometry; terrain includes tile neighbours. `maps/basemap.json`, -`maps/overlays.json` and `maps/terrain.json` are TileJSON for these packs. Their -tile URLs use the host `offline.openbikecomputer.invalid`, which the app serves -from the installed packs. +A grid release has `grid: {format: 2, zoom: 9, map_zoom: 11}`. Selection cells use Web Mercator XYZ +coordinates at zoom 9. Cell IDs are `9-X-Y`. A requested rectangle selects every intersecting cell, +clipped to the release bounds. The returned coverage contains the complete requested rectangle. + +The publisher creates routing page packs, search databases, and PMTiles packs once. Map packs group +each tile under its ancestor at `min(tile_zoom, 11)`. Original compressed tile payloads remain +unchanged. Terrain is the exception: its tiles are encoded again with the same pixels, and its zoom +13 tiles have packs of their own ([terrain tiles](planner-release.md#terrain-tiles)). A routing pack +contains pages with the same cell consumers, up to 16 MiB. Search retains whole intersecting records +and their dependencies. Each cell also has its [route catalog](route-catalog.md) file. + +`offline/catalog.json` has `format: 3`. It lists cell bounds, logical file names, map packs, and +routing cell descriptors. Its map packs omit the online places archive and the zoom 13 terrain +packs. Each routing descriptor has a manifest path and SHA-256, source adjacency ranges, and +retained geometry bounds. `shared` maps each map asset path of a selection to its publication file. +Files carry the decoded and transport hashes above. The service selects these objects and writes +only the selection manifests. It does not rebuild routing, SQLite databases, or map payloads. + +A selection keeps every glyph range path, `maps/assets/fonts/STACK/RANGE.pbf`. All range paths of +one font stack share `offline/fonts/STACK.pbf`. This file joins the stack's range files in range +order. It keeps only the ranges that contain a character of a basemap label field or a route network +reference in the release. The label fields are `name`, `name:en`, `name2`, `name3`, their `pgf:` +forms, `ref`, `ref:en`, `shield_text`, and `addr_housenumber`. The ranges also cover the upper-case +form of each text. Text with a character from U+0600 to U+08FF adds the Arabic presentation forms, +U+FB00 to U+FEFF. A missing range file stops the labels of each tile that requests it. + +The selected release has `offline.format: 2`, `id`, `zoom`, `map_zoom`, `source_routing`, and +`cells`. Each cell has `id`, `bounds`, and `files`: the cell's search databases and route catalog +file. `routing/blocks.json` follows the [routing selection +contract](route-package.md#grid-selections). Map selection takes basemap, overlay and terrain packs, +without the zoom 13 terrain packs. It covers retained road geometry; terrain includes tile +neighbours. `maps/basemap.json`, `maps/overlays.json` and `maps/terrain.json` are TileJSON for these +packs. Their tile URLs use the host `offline.openbikecomputer.invalid`, which the app serves from +the installed packs. ## Download service @@ -86,15 +84,13 @@ the service. Bounds use west, south, east, north. | `GET /bundles/ID/release.json` | Selected release manifest | | `GET /bundles/ID/objects/SHA256` | Transport bytes or HTTP 307 to the immutable object pool | -The ID hashes the publication catalog, rounded bounds, and selection format. -Repeated selections reuse metadata. There is no queued preparation worker or -reservation. Cancellation stops the client request. Error responses contain -`message`. Selections outside source coverage are rejected. +The ID hashes the publication catalog, rounded bounds, and selection format. Repeated selections +reuse metadata. Cancellation stops the client request. Error responses contain `message`. Selections +outside source coverage are rejected. -Objects support HEAD and byte ranges. The service stores the publication origin -with each cached selection. Generated manifests stay in its bounded metadata -cache. Least recently used selections can expire. A missing selection returns -404; the client must request a new selection. The companion bundle omits the +Objects support HEAD and byte ranges. The service stores the publication origin with each cached +selection. Least recently used selections can expire from its bounded metadata cache. A missing +selection returns 404; the client must request a new selection. The companion bundle omits the server language model and device catalog. ## iOS library @@ -108,18 +104,15 @@ server language model and device catalog. | `maps.json` | List of installed maps | | `pending.json` | One resumable download | -Each `maps.json` entry contains `id`, `name`, `region`, `bounds`, and -`installedBytes`. - -The app checks free space before transfer. Its estimate includes filesystem -allocation, missing decoded objects, the four largest temporary compressed -objects, and installation metadata. At most four downloads run at once. Compressed objects are decoded -and removed individually. Verification precedes the atomic library update. -The library is excluded from device backups. Deletion retains shared objects -that another map or the pending download needs. - -Each download prohibits cellular, expensive and constrained networks unless -the user allows them. Maps, routes and search first use a complete installed -release that covers the request. Routing graphs are not combined. -A failed local request falls back to the online service. A valid empty local -search result does not need a network request. +Each `maps.json` entry contains `id`, `name`, `region`, `bounds`, and `installedBytes`. + +The app checks free space before transfer. Its estimate includes filesystem allocation, missing +decoded objects, the four largest temporary compressed objects, and installation metadata. At most +four downloads run at once. Compressed objects are decoded and removed individually. Verification +precedes the atomic library update. The library is excluded from device backups. Deletion retains +shared objects that another map or the pending download needs. + +Each download prohibits cellular, expensive and constrained networks unless the user allows them. +Maps, routes and search first use a complete installed release that covers the request. Routing +graphs are not combined. A failed local request falls back to the online service. A valid empty +local search result does not need a network request. diff --git a/specs/planner-plan.md b/specs/planner-plan.md index c58e90d46e..395e696f36 100644 --- a/specs/planner-plan.md +++ b/specs/planner-plan.md @@ -1,9 +1,13 @@ # Editable planner files -An `.obcplan` file is a UTF-8 JSON object. It contains one editable plan and its -saved versions. It has no browser database ID or database revision, and no routed -line: the planner calculates the line again from the points. Import gives the plan -a new ID and does not replace another plan. +An `.obcplan` file is a UTF-8 JSON object: one editable plan and its saved versions. It has no +browser database ID or database revision, and no routed line: the planner calculates the line again +from the points. Import gives the plan a new ID and does not replace another plan. + +- Version: `version` is `1`. +- Writers: the web planner (`builder/web/src/lib/planner/library.ts`). +- Readers: the web planner; the companion app reads the [plan object](#plan-object) + (`OBCDomain/PlannerPlan.swift`). ## Envelope @@ -15,9 +19,9 @@ a new ID and does not replace another plan. | `trip` | Plan object | | `versions` | Array of version objects, newest first | -A version has a unique non-empty string `id`, a UTC ISO timestamp `at` with milliseconds, an optional -string `name`, a display string `summary`, and a plan object `trip`. Import -calculates each summary from the plan. It rejects duplicate version IDs. +A version has a unique non-empty string `id`, a UTC ISO timestamp `at` with milliseconds, an +optional string `name`, a display string `summary`, and a plan object `trip`. Import calculates each +summary from the plan. It rejects duplicate version IDs. ## Plan object @@ -43,19 +47,17 @@ Required fields are `points`, `routeOrder`, `days`, `budget`, `target`, and `lim | `climbTarget` | Optional finite non-negative number | | `sleep` | Optional `camping`, `hotel`, or `either`: the kind of place the suggested nights look for. Absent means `either`. | -Each point has a unique non-empty string `id`, a string `label`, and a two-number -`coordinate` in longitude/latitude order. -Longitude is from -180 to 180. Latitude is from -90 to 90. +Each point has a unique non-empty string `id`, a string `label`, and a two-number `coordinate` in +longitude/latitude order. Longitude is from -180 to 180. Latitude is from -90 to 90. -The point `kind` is `start`, `finish`, `pass`, `via`, `waypoint`, `detour`, `night`, -or `marker`. Optional fields are boolean `autoLabel`, string `placeKind`, string `placeId` (the search id -of the mapped place that the point stands for, such as `n123`), a -coordinate `anchor`, and `leg` (`routed`, `straight`, `drawn`, or `transfer`). A drawn leg can -have a `drawn` array of coordinates between its endpoints. A drawn coordinate can have a -third finite number, the elevation in metres. A transfer leg is a straight -line that the rider does not ride, such as a train. It adds no ridden distance or time. Optional `turnaround` -is `true` when the point turns the route back. Any point can have an optional string `note`, such as the -description of a route file's waypoint. +The point `kind` is `start`, `finish`, `pass`, `via`, `waypoint`, `detour`, `night`, or `marker`. +Optional fields are boolean `autoLabel`, string `placeKind`, string `placeId` (the search id of the +mapped place that the point stands for, such as `n123`), a coordinate `anchor`, and `leg` (`routed`, +`straight`, `drawn`, or `transfer`). A drawn leg can have a `drawn` array of coordinates between its +endpoints. A drawn coordinate can have a third finite number, the elevation in metres. A transfer +leg is a straight line that the rider does not ride, such as a train. It adds no ridden distance or +time. Optional `turnaround` is `true` when the point turns the route back. Any point can have an +optional string `note`, such as the description of a route file's waypoint. A marker can have a string `legEnd`: the ID of the route point that ends the leg the marker belongs to. In a loop, the start ends the closing leg. The route can pass a marker more than once. A pass at @@ -63,17 +65,15 @@ most 100 m farther from the marker than the nearest pass is a near pass. The pla marker on the near pass whose leg is nearest, in route order, to the leg that ends at `legEnd`. Without `legEnd`, or when no leg ends at it, the planner shows the marker on the nearest pass. -A night has integer `night` from 1 to `days - 1` and ID `night-N`, where `N` is -that number. Night numbers are unique. Markers do not count as route points. -An open plan with two or more route points has one start and one finish. An -incomplete plan has zero points or one endpoint, with optional markers. A loop -has at least two route points, one start, and no finish. +A night has integer `night` from 1 to `days - 1` and ID `night-N`, where `N` is that number. Night +numbers are unique. Markers do not count as route points. An open plan with two or more route points +has one start and one finish. An incomplete plan has zero points or one endpoint, with optional +markers. A loop has at least two route points, one start, and no finish. -The route goes from the start through the points of `routeOrder` to the finish. -A loop goes back to its start. `routeOrder` contains each route point that is not -the start or the finish one time. +The route goes from the start through the points of `routeOrder` to the finish. A loop goes back to +its start. `routeOrder` contains each route point that is not the start or the finish one time. ## Import -Import validates the complete file before saving it. An invalid plan or version -rejects the file. The open plan stays unchanged. +Import validates the complete file before saving it. An invalid plan or version rejects the file. +The open plan stays unchanged. diff --git a/specs/planner-release.md b/specs/planner-release.md index 09346d21cb..10fb768e81 100644 --- a/specs/planner-release.md +++ b/specs/planner-release.md @@ -1,9 +1,18 @@ # Planner release -A release joins map, routing, search and model data. -`release.json` is UTF-8 JSON. The release and its files are immutable. -A release of `obc data update` has the ID of the [named files of the planner](obc-data.md#planner). -The standalone publisher uses the SHA-256 of `release.json`. +A planner release joins map, routing, search and model data. `release.json` is its UTF-8 JSON +manifest. This contract also defines the grid storage, the tile API paths and the catalogue. The +release and its files are immutable. A release of `obc data update` has the ID of the [named files +of the planner](obc-data.md#planner). The standalone publisher uses the SHA-256 of `release.json`. + +- Version: manifest `format` `1`; catalogue `format: 1`; `public/grid.json` `format: 3`. A client + refuses a catalogue `format` that it does not know. +- Writers: `obc data update` (`host/obc-data`), the standalone publisher. +- Readers: the web planner (`builder/web/src/lib/planner/release.ts`), the map builder, the tile + service ([`planner/tiles`](../planner/tiles/README.md)), `obc data serve`, the companion app. +- Related: [offline bundle](planner-offline.md), [route package](route-package.md), [route + catalog](route-catalog.md), [snow tiles](planner-snow-tiles.md), [climate + tiles](planner-climate-tiles.md), [`obc data`](obc-data.md). ## Manifest @@ -17,98 +26,89 @@ The standalone publisher uses the SHA-256 of `release.json`. | `routing_package` | Hash of `routing/manifest.json` or `routing/blocks.json` | | `profiles` | Sorted routing profile IDs | | `attribution` | OSM credit: the `osm-planet` attribution of `data/sources.toml` | -| `landcover_attribution` | Credit of the basemap's land cover: the `daylight-landcover` attribution of `data/sources.toml`. Older releases have none | +| `landcover_attribution` | Credit of the basemap's land cover: the `daylight-landcover` attribution of `data/sources.toml`. Can be absent | | `terrain_attribution` | Elevation source credits: each reference model used, and the `copernicus-glo-30` credit when the bake reads GLO-30 tiles | | `terrain_bounds` | Bounds that include contour neighbour tiles | | `files` | Relative file names, each with `bytes` and `sha256` | -File paths stay inside the release directory. File hashes use lowercase -64-character hex. Maps, search, and routing have the same OSM hash and bounds. -The terrain inputs cover every routing elevation input. Search metadata contains -the OSM hash. Terrain TileJSON contains terrain input hashes. The -[`obc data` release manifest](obc-data.md#release-manifest) records source versions, tool -identities and layer receipts. These are not client files. +File paths stay inside the release directory. File hashes use lowercase 64-character hex. Maps, +search, and routing have the same OSM hash and bounds. The terrain inputs cover every routing +elevation input. Search metadata contains the OSM hash. Terrain TileJSON contains terrain input +hashes. The [`obc data` release manifest](obc-data.md#release-manifest) records source versions, +tool identities and layer receipts. These are not client files. ## Objects and services -The standalone publisher stores release files under `planner/releases/ID/` of R2. It -stores source mirrors under `planner/sources/`, without the local `sources/` path prefix. -A source name starts with its SHA-256. Source mirrors can be shared by releases. A grid -release of `obc data` uses the layout of [Canonical grid storage](#canonical-grid-storage). - -`maps/` contains `basemap.pmtiles`, `places.pmtiles`, `overlays.pmtiles`, -`terrain.pmtiles`, map assets, and their manifest. When the region recipe names a -snow source, it also contains `snow.pmtiles` from the -[snow tile contract](planner-snow-tiles.md). When the recipe has a `climate` field, -it also contains `climate.pmtiles` from the -[climate tile contract](planner-climate-tiles.md). The release `files` hold exactly the -files of the map manifest and the manifest itself. `routing/` contains the three files in the -[route package contract](route-package.md), plus `overlays.sqlite`. The overlay -index stores the routing manifest identity and has the same OSM source. -`search/` contains `pois/REGION.sqlite`, `addresses/REGION.sqlite`, -and `model/`. Local combined packages use `REGION.sqlite`. `routes/` contains `REGION.json` from the -[route catalog contract](route-catalog.md). `device/catalog.json` is a snapshot. Its file references are absolute -URLs to the original immutable cell objects. - -Search packages use schema `6`. Each place stores `website`, `phone`, and -`description` as UTF-8 text. Empty values are empty strings. Each place stores -`country`, the lowercase ISO 3166-1 code of its source record, and `region`, its -state. A Swiss `region` is the German canton name of the opening hours library, -for example `Sankt Gallen`. Opening hours select public holidays by `country` and `region`, and -evaluate times in the metadata `time_zone`. The producer uses -`website` before `contact:website`, and `phone` before `contact:phone`, skipping -empty values. A description uses `description`, then `description:en`, then -`description:de`, then the first nonempty `description:*` key in sorted order. -The producer trims outer whitespace and keeps the description's internal text. -The search query accepts an optional `source` with an OSM identity such as -`n123`, `w123`, or `r123`, or a curated Wikidata identity such as `Q123`. It returns that place without name matching or model -inference. An absent identity returns an empty result. +The standalone publisher stores release files under `planner/releases/ID/` of R2. It stores source +mirrors under `planner/sources/`, without the local `sources/` path prefix. A source name starts +with its SHA-256. Source mirrors can be shared by releases. A grid release of `obc data` uses the +layout of [Canonical grid storage](#canonical-grid-storage). + +`maps/` contains `basemap.pmtiles`, `places.pmtiles`, `overlays.pmtiles`, `terrain.pmtiles`, map +assets, and their manifest. When the region recipe names a snow source, it also contains +`snow.pmtiles` from the [snow tile contract](planner-snow-tiles.md). When the recipe has a `climate` +field, it also contains `climate.pmtiles` from the [climate tile +contract](planner-climate-tiles.md). The release `files` hold exactly the files of the map manifest +and the manifest itself. `routing/` contains the three files in the [route package +contract](route-package.md), plus `overlays.sqlite`. The overlay index stores the routing manifest +identity and has the same OSM source. `search/` contains `pois/REGION.sqlite`, +`addresses/REGION.sqlite`, and `model/`. Local combined packages use `REGION.sqlite`. `routes/` +contains `REGION.json` from the [route catalog contract](route-catalog.md). `device/catalog.json` is +a snapshot. Its file references are absolute URLs to the original immutable cell objects. + +Search packages use schema `6`. Each place stores `website`, `phone`, and `description` as UTF-8 +text. Empty values are empty strings. Each place stores `country`, the lowercase ISO 3166-1 code of +its source record, and `region`, its state. A Swiss `region` is the German canton name of the +opening hours library, for example `Sankt Gallen`. Opening hours select public holidays by `country` +and `region`, and evaluate times in the metadata `time_zone`. The producer uses `website` before +`contact:website`, and `phone` before `contact:phone`, skipping empty values. A description uses +`description`, then `description:en`, then `description:de`, then the first nonempty `description:*` +key in sorted order. The producer trims outer whitespace and keeps the description's internal text. +The search query accepts an optional `source` with an OSM identity such as `n123`, `w123`, or +`r123`, or a curated Wikidata identity such as `Q123`. It returns that place without name matching +or model inference. An absent identity returns an empty result. `place_identities` maps place ids to explicit Wikidata and Wikipedia identities. `place_content` -contains `place_id`, `content` (UTF-8 JSON), and nullable `landmark_id`. The `places` -view joins this content. A landmark reuses matching OSM places, including explicit -Wikidata redirects and validated Wikipedia title redirects. Search results deduplicate their `landmark_id`. A landmark -without a match uses its Wikidata identity and prepared coordinate. Peak articles -join by OSM node id. Distinct summits that share an article stay distinct places. - -Content contains `default_language` and `variants`. Each variant contains `language`, -`text_pages`, and `attribution` (`source_url`, string `revision`, `license_url`). Optional -`photo` contains `online_url`, four complete `credit` strings, those attribution -fields, `file_identity` (`filename`, `page_id`), `page_revision`, and `file_revision` -(`timestamp`, `sha1`). Missing photo identity fields are null. Content excludes -original notice envelopes, device paths, hashes, byte counts, and image payloads. -Grid shards retain the content of their selected places. - -Planner details select English, then the prepared default language. Photo details -make one bounded Commons metadata request when opened. A photo appears only when -its page identity, description revision, file revision, and 500 px thumbnail URL -match the prepared reference. A failed request or mismatch retains the article and -source links. A missing online URL never falls back to an original image. - -The overlay database has SQLite `user_version=2`. `features` stores stable IDs, -layer kinds, cycling and walking minimum zooms, geometry IDs, and attribute IDs. -`attributes` stores distinct JSON properties without the way ID or layer kind. -Route memberships are ordered relation IDs. `routes` stores each relation's JSON -properties once. `geometries` stores the way ID, a point count, and a Postcard `Vec<[i32;2]>`. -Each coordinate pair is longitude and latitude in microdegrees. The first pair -is absolute; each later pair is a difference from the previous pair. Decoding -uses checked addition. Shared geometry retains every source point. - -`places.pmtiles` holds rider places from the POI search database. It has gzip MVT -at zoom 11, extent 4096, and one `pois` layer. Each point has the database's -`kind` and `name`. String properties `lon` and `lat` retain the search coordinates -without tile quantization. The feature ID is `(type << 44) | source_id`, where type -is 1 for nodes, 2 for ways, 3 for relations, and 4 for curated Wikidata items. -Source IDs are positive and below 2^44; a Wikidata item uses the digits after `Q`. Categories are defined in the -[place categories](../planner/search/place-kinds.json). -Archive metadata includes the source `osm_sha256`. Empty tiles are absent, -except that an empty archive contains one empty tile at the southwest bound. - -`overlays.pmtiles` holds the route networks and access restrictions of the -overlay index. It has gzip MVT tiles from zoom 6 to 14, with extent 4096. Its -metadata `routing_package` is the routing manifest identity of the overlay index. -In a grid release, `maps/overlays.json` names the grid routing package, the -SHA-256 of `routing/blocks.json`, which packs the same graph. +contains `place_id`, `content` (UTF-8 JSON), and nullable `landmark_id`. The `places` view joins +this content. A landmark reuses matching OSM places, including explicit Wikidata redirects and +validated Wikipedia title redirects. Search results deduplicate their `landmark_id`. A landmark +without a match uses its Wikidata identity and prepared coordinate. Peak articles join by OSM node +id. Distinct summits that share an article stay distinct places. + +Content contains `default_language` and `variants`. Each variant contains `language`, `text_pages`, +and `attribution` (`source_url`, string `revision`, `license_url`). Optional `photo` contains +`online_url`, four complete `credit` strings, those attribution fields, `file_identity` (`filename`, +`page_id`), `page_revision`, and `file_revision` (`timestamp`, `sha1`). Missing photo identity +fields are null. Content excludes original notice envelopes, device paths, hashes, byte counts, and +image payloads. Grid shards retain the content of their selected places. + +Planner details select English, then the prepared default language. Photo details make one bounded +Commons metadata request when opened. A photo appears only when its page identity, description +revision, file revision, and 500 px thumbnail URL match the prepared reference. A failed request or +mismatch retains the article and source links. A missing online URL never falls back to an original +image. + +The overlay database has SQLite `user_version=2`. `features` stores stable IDs, layer kinds, cycling +and walking minimum zooms, geometry IDs, and attribute IDs. `attributes` stores distinct JSON +properties without the way ID or layer kind. Route memberships are ordered relation IDs. `routes` +stores each relation's JSON properties once. `geometries` stores the way ID, a point count, and a +Postcard `Vec<[i32;2]>`. Each coordinate pair is longitude and latitude in microdegrees. The first +pair is absolute; each later pair is a difference from the previous pair. Decoding uses checked +addition. Shared geometry retains every source point. + +`places.pmtiles` holds rider places from the POI search database. It has gzip MVT at zoom 11, extent +4096, and one `pois` layer. Each point has the database's `kind` and `name`. String properties `lon` +and `lat` retain the search coordinates without tile quantization. The feature ID is +`(type << 44) | source_id`, where type is 1 for nodes, 2 for ways, 3 for relations, and 4 for +curated Wikidata items. Source IDs are positive and below 2^44; a Wikidata item uses the digits +after `Q`. Categories are defined in the [place categories](../planner/search/place-kinds.json). +Archive metadata includes the source `osm_sha256`. Empty tiles are absent, except that an empty +archive contains one empty tile at the southwest bound. + +`overlays.pmtiles` holds the route networks and access restrictions of the overlay index. It has +gzip MVT tiles from zoom 6 to 14, with extent 4096. Its metadata `routing_package` is the routing +manifest identity of the overlay index. In a grid release, `maps/overlays.json` names the grid +routing package, the SHA-256 of `routing/blocks.json`, which packs the same graph. | Layer | Feature ID | Properties | | --- | --- | --- | @@ -116,35 +116,33 @@ SHA-256 of `routing/blocks.json`, which packs the same graph. | `access` | Way ID | `cycling_status`, `walking_status`, `name`, `ref`, `conditional`; `riding`, `walking`, `pushing` and `tags` as JSON text; `cycling_minzoom` and `walking_minzoom`, the overlay index minimum zoom of each restricted mode | | `routes` | Relation ID | The route properties that are not empty: `kind`, `network`, `rank`, `name`, `ref`, `website`, `symbol`, `symbol_text`. The point is the tile origin. | -A feature starts at the lower mode minimum zoom of its overlay index feature, -at most zoom 14. The planner shows access from the `MODE_minzoom` of its travel mode. -A line names only the routes that start at or below the tile zoom. Lines with -equal properties join where exactly two of them meet. The `routes` layer of a -tile holds each route that its lines name. A tile with no features is absent. +A feature starts at the lower mode minimum zoom of its overlay index feature, at most zoom 14. The +planner shows access from the `MODE_minzoom` of its travel mode. A line names only the routes that +start at or below the tile zoom. Lines with equal properties join where exactly two of them meet. +The `routes` layer of a tile holds each route that its lines name. A tile with no features is +absent. -The tile API serves `/releases/ID/basemap.json`, vector tiles at -`/releases/ID/basemap/Z/X/Y.mvt`, `/releases/ID/places.json`, places tiles at -`/releases/ID/places/Z/X/Y.mvt`, `/releases/ID/overlays.json`, overlay tiles at -`/releases/ID/overlays/Z/X/Y.mvt`, Terrarium tiles at +The tile API serves `/releases/ID/basemap.json`, vector tiles at `/releases/ID/basemap/Z/X/Y.mvt`, +`/releases/ID/places.json`, places tiles at `/releases/ID/places/Z/X/Y.mvt`, +`/releases/ID/overlays.json`, overlay tiles at `/releases/ID/overlays/Z/X/Y.mvt`, Terrarium tiles at `/releases/ID/terrain/Z/X/Y.webp`, and, for each data layer of the release, -`/releases/ID/LAYER.json` and its tiles at `/releases/ID/LAYER/Z/X/Y`. It serves -route catalog cells at `/releases/ID/routes/tiles/9-X-Y.json`, and TrailView's ruleset at -`/releases/ID/trailview/rules.json`. Each -TileJSON holds its archive metadata. Data layer tiles that are not MVT or WebP keep their -gzip encoding. An absent tile returns 204. The raw archives -remain downloadable from R2. +`/releases/ID/LAYER.json` and its tiles at `/releases/ID/LAYER/Z/X/Y`. It serves route catalog cells +at `/releases/ID/routes/tiles/9-X-Y.json`, and TrailView's ruleset at +`/releases/ID/trailview/rules.json`. Each TileJSON holds its archive metadata. Data layer tiles that +are not MVT or WebP keep their gzip encoding. An absent tile returns 204. The raw archives remain +downloadable from R2. -Routing, search and downloads APIs have the prefix `/planner-api/releases/ID/`. The final -path component selects `routing`, `search` or `downloads`. `obc data serve` runs the -services of each release that `release.json` of its target names. +Routing, search and downloads APIs have the prefix `/planner-api/releases/ID/`. The final path +component selects `routing`, `search` or `downloads`. `obc data serve` runs the services of each +release that `release.json` of its target names. ## Terrain tiles One terrain pyramid serves the planner and TrailView: format 2 of the layer `planner/terrain`. A -tile is a 512 × 512 lossless WebP in the Terrarium encoding. A pixel is the height at its centre -in metres, R × 256 + G + B / 256 − 32768, rounded to the nearest decimetre: B is the tenths × -25.6, rounded, so B is one of 0, 26, 51, 77, 102, 128, 154, 179, 205 and 230. A pixel with alpha 0 -has no known height. +tile is a 512 × 512 lossless WebP in the Terrarium encoding. A pixel is the height at its centre in +metres, R × 256 + G + B / 256 − 32768, rounded to the nearest decimetre: B is the tenths × 25.6, +rounded, so B is one of 0, 26, 51, 77, 102, 128, 154, 179, 205 and 230. A pixel with alpha 0 has no +known height. | Zoom | Tiles | | --- | --- | @@ -165,114 +163,97 @@ service and an offline area never load zoom 13: `terrain.mbtiles`. An opaque tile can lose its alpha chunk: a decoder then gives alpha 255. The planner reads zooms 0 to 12; TrailView reads zoom 13 too. The heights are the mean pool of the -national terrain models (`terrain`) where they hold the ground, and Copernicus -GLO-30 elsewhere. The two meet in a seam that `host/obc-dem/src/planner.rs` defines. The national -weight is 0 within one coverage cell (about 114 m × 78 m at 47° N) of a cell where the national -model misses a pixel, and rises to 1 over the next 1 km. GLO-30 takes the bias of a national -model there: the median of national − GLO-30 that `index.json` of `terrain` states for the best -model of the tile, or outside the coverage for that of the nearest full cell, faded out over -1 km. The height is continuous, also at a concave edge such as a national border. A height reads -the archive tiles of `terrain_bounds` and one ring of tiles around them, so it does not depend on -the bounds of the bake. The routes read the same heights, without the OSM corrections of -coastlines and summits. +national terrain models (`terrain`) where they hold the ground, and Copernicus GLO-30 elsewhere. The +two meet in a seam that `host/obc-dem/src/planner.rs` defines. The national weight is 0 within one +coverage cell (about 114 m × 78 m at 47° N) of a cell where the national model misses a pixel, and +rises to 1 over the next 1 km. GLO-30 takes the bias of a national model there: the median of +national − GLO-30 that `index.json` of `terrain` states for the best model of the tile, or outside +the coverage for that of the nearest full cell, faded out over 1 km. The height is continuous, also +at a concave edge such as a national border. A height reads the archive tiles of `terrain_bounds` +and one ring of tiles around them, so it does not depend on the bounds of the bake. The routes read +the same heights, without the OSM corrections of coastlines and summits. ## Canonical grid storage -The online services serve grid releases only. A regional release -without `grid` is for local preview. A grid release adds `grid: {format: 2, zoom: 9, map_zoom: 11}`. Its `files` -entries retain logical paths and decoded `bytes` and `sha256`. Each also has -`transport: {bytes, sha256, encoding}`. Encoding is `identity` or `gzip`. -A target of `obc data` stores each distinct transport once at `objects/SHA256`. -The online services and offline installer consume this same pool. -Named metadata stays at `releases/ID/`: `release.json`, `public/`, and -`indexes/GRID_STEP/index.json` for each selected grid. The release manifests of -`obc data` bind each named path to its size and SHA-256. Producer directories and source -archives are not part of these verification indexes. - -Before publication, the verifier checks the packed bytes and named metadata -against their receipts. It composes the indexes with the same source and coverage -checks as the producer. It reads changed decoded map archives, search shards and -routing graphs with their format validators. Unchanged binary payloads retain their -previous verification. The Python group and Node reader dependencies must be -prepared from their locks. A fresh store can restore the named -indexes and client objects without the producer files. Verification does not -install tools, download data or build runtimes. - -The planner release binds service source and locked dependencies in its final index. -The VPS builds services from a commit. - -`public/grid.json` contains `format: 3` and `map_zoom`. A pointer repeats the -transport entry of a file and adds `decoded_bytes`. A file of a grid cell, at -`DIR/Z-X-Y.EXT` under `maps/tiles/` or `routes/tiles/`, has its pointer in the -block index `public/DIR/B-BX-BY.json`: a JSON object that maps `Z-X-Y.EXT` to -the pointer. `B` is the larger of `Z - 4` and 0, and `BX` and `BY` are `X` and -`Y` shifted right by `Z - B` bits, so an index holds at most 16 × 16 cells of a -zoom. Each other asset, TileJSON and device catalog has its pointer at -`public/LOGICAL_PATH.json`. So the count of public keys grows with the area of a -release by one index per block, not by one pointer per cell of each layer. -The tile service resolves a pack through its pointer. A tile without a pack is -absent. A block index holds at most 64 KiB, which the tile service reads. The -tile service and TrailView also read format 2, which has a pointer at -`public/LOGICAL_PATH.json` for each file, for a rollback release. An update of -prod refuses while the tile service does not read the grid format of the release. -No regional map archive is required beside the pool. Grid assets use the tile -service origin. - -The VPS materializes routing, search, and offline selection metadata. Search -uses `search/REGION.grid.json` to list cell files and coverage. Route catalog -cells follow the [route catalog contract](route-catalog.md#files). The -[offline contract](planner-offline.md#grid-publication-and-selection) defines -cell selection and download manifests. +The online services serve grid releases only. A regional release without `grid` is for local +preview. A grid release adds `grid: {format: 2, zoom: 9, map_zoom: 11}`. Its `files` entries retain +logical paths and decoded `bytes` and `sha256`. Each also has +`transport: {bytes, sha256, encoding}`. Encoding is `identity` or `gzip`. A target of `obc data` +stores each distinct transport once at `objects/SHA256`. The online services and offline installer +consume this same pool. Named metadata stays at `releases/ID/`: `release.json`, `public/`, and +`indexes/GRID_STEP/index.json` for each selected grid. The release manifests of `obc data` bind each +named path to its size and SHA-256. Producer directories and source archives are not part of these +verification indexes. + +Before publication, the verifier checks the packed bytes and named metadata against their receipts. +It composes the indexes with the same source and coverage checks as the producer. It reads changed +decoded map archives, search shards and routing graphs with their format validators. Unchanged +binary payloads retain their previous verification. The Python group and Node reader dependencies +must be prepared from their locks. A fresh store can restore the named indexes and client objects +without the producer files. Verification does not install tools, download data or build runtimes. + +The planner release binds service source and locked dependencies in its final index. The VPS builds +services from a commit. + +`public/grid.json` contains `format: 3` and `map_zoom`. A pointer repeats the transport entry of a +file and adds `decoded_bytes`. A file of a grid cell, at `DIR/Z-X-Y.EXT` under `maps/tiles/` or +`routes/tiles/`, has its pointer in the block index `public/DIR/B-BX-BY.json`: a JSON object that +maps `Z-X-Y.EXT` to the pointer. `B` is the larger of `Z - 4` and 0, and `BX` and `BY` are `X` and +`Y` shifted right by `Z - B` bits, so an index holds at most 16 × 16 cells of a zoom. Each other +asset, TileJSON and device catalog has its pointer at `public/LOGICAL_PATH.json`. The tile service +resolves a pack through its pointer. A tile without a pack is absent. A block index holds at most 64 +KiB, which the tile service reads. The tile service and TrailView also read format 2, which has a +pointer at `public/LOGICAL_PATH.json` for each file, for a rollback release. An update of prod +refuses while the tile service does not read the grid format of the release. No regional map archive +is required beside the pool. Grid assets use the tile service origin. + +The VPS materializes routing, search, and offline selection metadata. Search uses +`search/REGION.grid.json` to list cell files and coverage. Route catalog cells follow the [route +catalog contract](route-catalog.md#files). The [offline +contract](planner-offline.md#grid-publication-and-selection) defines cell selection and download +manifests. ## Catalogue `planner/catalog.json` has `format: 1`, `origins` and `active`. `origins` is the `[publication]` -table of `data/planner-runtime.toml`. -`active` contains the release `id`, manifest URL, region, bounds, attribution, -land cover attribution when the release has one, terrain attribution, device catalogue URL, map asset URLs, tile URLs, and routing, -search and downloads API prefixes. `name` is the name of the recipe's region in `data/regions/`. `layers` maps -the name of each data layer of the release to its TileJSON URL. -`trailview_rules` is the URL of TrailView's ruleset when the release has it. -`routes` is the route catalog cell URL template on the tile API origin, +table of `data/planner-runtime.toml`. `active` contains the release `id`, manifest URL, region, +bounds, attribution, land cover attribution when the release has one, terrain attribution, device +catalogue URL, map asset URLs, tile URLs, and routing, search and downloads API prefixes. `name` is +the name of the recipe's region in `data/regions/`. `layers` maps the name of each data layer of the +release to its TileJSON URL. `trailview_rules` is the URL of TrailView's ruleset when the release +has it. `routes` is the route catalog cell URL template on the tile API origin, `/releases/ID/routes/tiles/{cell}.json`, where `{cell}` is a cell ID `9-X-Y`. -The publisher uploads and verifies all release files before the manifest. The -catalogue cache lifetime is 30 seconds. Immutable objects have a one-year -cache lifetime. The web planner and the map builder read the catalogue at page -load. The planner uses `active`, unchanged, as its configuration. The map builder -uses its device catalogue. A client refuses a catalogue `format` that it does not -know. When a release object returns 404 or no answer, a client reads the catalogue +The publisher uploads and verifies all release files before the manifest. The catalogue cache +lifetime is 30 seconds. Immutable objects have a one-year cache lifetime. The web planner and the +map builder read the catalogue at page load. The planner uses `active`, unchanged, as its +configuration. The map builder uses its device catalogue. A client refuses a catalogue `format` that +it does not know. When a release object returns 404 or no answer, a client reads the catalogue again, once, and retries with the new `active` entry. ## Component receipts -`sources.components` maps component names to immutable receipts. Each has -`format: 1`, `key`, `spec`, `files`, and `cost`. The key is SHA-256 of the -canonical JSON `spec`, including its trailing newline. The spec contains -`format`, `name`, `producer`, `inputs`, `options`, `coverage`, and `dependencies`. -Producer identities include effective source code and dependency contracts. -Inputs identify source snapshots and upstream component keys. Coverage uses -`[west,south,east,north]`. A receipt contains only completed, verified files. -Selection includes consumers of changed shared inputs. Terrain and routing -share an elevation identity for source rasters and local elevation code. -Each file entry has `bytes` and `sha256`. Cost fields are `elapsed_seconds`, -`cpu_seconds`, `output_bytes`, and `peak_ram_bytes`; unavailable RAM is null. -`cpu_seconds` counts the whole bake process while the component builds, so it -includes a component that builds at the same time. +`sources.components` maps component names to immutable receipts. Each has `format: 1`, `key`, +`spec`, `files`, and `cost`. The key is SHA-256 of the canonical JSON `spec`, including its trailing +newline. The spec contains `format`, `name`, `producer`, `inputs`, `options`, `coverage`, and +`dependencies`. Producer identities include effective source code and dependency contracts. Inputs +identify source snapshots and upstream component keys. Coverage uses `[west,south,east,north]`. A +receipt contains only completed, verified files. Selection includes consumers of changed shared +inputs. Terrain and routing share an elevation identity for source rasters and local elevation code. +Each file entry has `bytes` and `sha256`. Cost fields are `elapsed_seconds`, `cpu_seconds`, +`output_bytes`, and `peak_ram_bytes`; unavailable RAM is null. `cpu_seconds` counts the whole bake +process while the component builds, so it includes a component that builds at the same time. `sources.grid_components` records the partition and transport receipts. -Search schema 6 metadata `component` is `pois`, `addresses`, or `all`. Metadata -`time_zone` is the IANA time zone of the region recipe. All components and cells -of a release have the same `time_zone`. -Addresses own street records and house records. POIs own the other searchable -places, including locality records. Independent POI IDs are positive integers -below 2^52; independent address street IDs are above 2^52 and below 2^53. -Components share the declared OSM snapshot and regional bounds. Each search -cell retains that snapshot and declares its clipped cell bounds. - -Split search grids use `format: 3`. Their `metadata` describes the complete -region. Each cell has `id`, `bounds`, and `files`. File names are relative to -`search/`: `tiles/pois/CELL.sqlite` and `tiles/addresses/CELL.sqlite`. -Both components are present. Offline catalogue cell files use release-relative -paths. Consumers require one schema and OSM snapshot in all cells, and select cells +Search schema 6 metadata `component` is `pois`, `addresses`, or `all`. Metadata `time_zone` is the +IANA time zone of the region recipe. All components and cells of a release have the same +`time_zone`. Addresses own street records and house records. POIs own the other searchable places, +including locality records. Independent POI IDs are positive integers below 2^52; independent +address street IDs are above 2^52 and below 2^53. Components share the declared OSM snapshot and +regional bounds. Each search cell retains that snapshot and declares its clipped cell bounds. + +Split search grids use `format: 3`. Their `metadata` describes the complete region. Each cell has +`id`, `bounds`, and `files`. File names are relative to `search/`: `tiles/pois/CELL.sqlite` and +`tiles/addresses/CELL.sqlite`. Both components are present. Offline catalogue cell files use +release-relative paths. Consumers require one schema and OSM snapshot in all cells, and select cells by the bounds in each cell's metadata. diff --git a/specs/planner-snow-tiles.md b/specs/planner-snow-tiles.md index c4f8402832..618d7e7eba 100644 --- a/specs/planner-snow-tiles.md +++ b/specs/planner-snow-tiles.md @@ -1,7 +1,12 @@ # Planner snow tiles -The planner snow layer reads one archive per region: `snow.pmtiles`. The bake is -`tools/planner_snow.py`. +The planner snow archive of one region, `snow.pmtiles`: the onset and melt-out of the longest snow +period of each season. + +- Writers: `tools/planner_snow.py`. +- Readers: the web planner snow layer (`builder/web/src/lib/planner/layers/snow.ts`), + `tools/trailview/snow.py`. +- Related: [planner release](planner-release.md), [planner climate tiles](planner-climate-tiles.md). ## Archive @@ -18,13 +23,11 @@ A missing tile is no data. A pixel outside the region bounds is no data. ## Seasons -Season s runs from 1 September of year s to 31 August of year s+1. The day index -is 0 on 1 September. A season has 365 or 366 days. The day step is -⌊day index ÷ 2⌋, from 0 to 182. +Season s runs from 1 September of year s to 31 August of year s+1. The day index is 0 on 1 +September. A season has 365 or 366 days. The day step is ⌊day index ÷ 2⌋, from 0 to 182. -The longest snow period of a season is its longest run of consecutive snow days. -When two runs have the same length, the earliest run counts. Onset is its first -day. Melt-out is its last day. +The longest snow period of a season is its longest run of consecutive snow days. When two runs have +the same length, the earliest run counts. Onset is its first day. Melt-out is its last day. ## Tile body @@ -35,8 +38,8 @@ The body is planar bytes. For each season, from the first season in order: | 0 | 65,536 | Onset, one byte per pixel | | 65,536 | 65,536 | Melt-out, one byte per pixel | -Pixels are in row order. Row 0 is the north edge and column 0 is the west edge. -The body length is `seasons` × 131,072 bytes. +Pixels are in row order. Row 0 is the north edge and column 0 is the west edge. The body length is +`seasons` × 131,072 bytes. ## Byte values @@ -49,9 +52,9 @@ The body length is `seasons` × 131,072 bytes. A sentinel (253, 254, 255) is in both planes of the pixel. -There is snow on day step d of season s when onset_s ≤ d ≤ meltout_s, or when the -pixel has 254. The share of seasons with snow on a date is the count of seasons -with snow divided by the count of seasons that are not 255 at that pixel. +There is snow on day step d of season s when onset_s ≤ d ≤ meltout_s, or when the pixel has 254. The +share of seasons with snow on a date is the count of seasons with snow divided by the count of +seasons that are not 255 at that pixel. ## Metadata @@ -73,52 +76,48 @@ The archive metadata is a JSON object: | `nasa-modis` | MODIS Terra and Aqua daily snow cover (MOD10A1, MYD10A1, collection 6.1) | 500 | | `copernicus-hr-wsi` | HR-WSI Snow Phenology S2 yearly rasters | 20 | -An archive reads `copernicus-hr-wsi` where it has data and `nasa-modis` elsewhere. -A region outside the `extent` of `hr-wsi`, or where HR-WSI has no product, reads `nasa-modis` only. +An archive reads `copernicus-hr-wsi` where it has data and `nasa-modis` elsewhere. A region outside +the `extent` of `hr-wsi`, or where HR-WSI has no product, reads `nasa-modis` only. -The max zoom is the zoom whose pixel size at the middle latitude of the bounds is -nearest to `resolution_m` in log scale. In the Alps, it is 12 for -`copernicus-hr-wsi` and 8 for `nasa-modis`. +The max zoom is the zoom whose pixel size at the middle latitude of the bounds is nearest to +`resolution_m` in log scale. In the Alps, it is 12 for `copernicus-hr-wsi` and 8 for `nasa-modis`. `nasa-modis` snow days: - A day has a clear observation when Terra or Aqua is clear. Terra comes first. - A clear day is a snow day when NDSI ≥ 0.10 or the detector is saturated. -- A day with no clear observation takes the state of the nearest clear day. On a - tie, the earlier clear day counts. +- A day with no clear observation takes the state of the nearest clear day. On a tie, the earlier + clear day counts. - A season with no clear day at a pixel is no data. -- When the source ends before the last season ends, a pixel is no data for that - season when its snow period at the source end can still become the longest. +- When the source ends before the last season ends, a pixel is no data for that season when its snow + period at the source end can still become the longest. -`copernicus-hr-wsi` takes onset (SCO), melt-out (SCM) and duration (SCD) from the -yearly rasters. Its day index also starts at 0 on 1 September. A season without -a raster is no data. +`copernicus-hr-wsi` takes onset (SCO), melt-out (SCM) and duration (SCD) from the yearly rasters. +Its day index also starts at 0 on 1 September. A season without a raster is no data. ## Zoom levels -Both resampling steps use the blend rule. The rule takes weighted input pixels -for each season: +Both resampling steps use the blend rule. The rule takes weighted input pixels for each season: 1. The pixel is 255 when the weight of 255 inputs is more than half of the total. -2. Else the pixel takes the class with the largest weight among dated (0–182), 254 - and 253, in this order on a tie. -3. A dated pixel takes the weighted mean of the dated inputs, for onset and for - melt-out separately, rounded half up. - -At the max zoom, the inputs are the four source pixels around the pixel centre -with bilinear weights. A source pixel outside the source grid is 255. In each -season, a pixel takes the first source in which it is not 255: -`copernicus-hr-wsi`, then `nasa-modis`. - -At a lower zoom, the inputs are the 2 × 2 child pixels, with equal weights. When -`source` is `copernicus-hr-wsi`, each child pixel is smoothed first: it takes the blend of its -3 × 3 neighbourhood in its tile, with equal weights and the median of the dated -inputs instead of the mean. A neighbour outside the tile repeats the edge pixel. The smoothing -applies to every pixel of the tile, also to a pixel that `nasa-modis` fills. +2. Else the pixel takes the class with the largest weight among dated (0–182), 254 and 253, in this + order on a tie. +3. A dated pixel takes the weighted mean of the dated inputs, for onset and for melt-out separately, + rounded half up. + +At the max zoom, the inputs are the four source pixels around the pixel centre with bilinear +weights. A source pixel outside the source grid is 255. In each season, a pixel takes the first +source in which it is not 255: `copernicus-hr-wsi`, then `nasa-modis`. + +At a lower zoom, the inputs are the 2 × 2 child pixels, with equal weights. When `source` is +`copernicus-hr-wsi`, each child pixel is smoothed first: it takes the blend of its 3 × 3 +neighbourhood in its tile, with equal weights and the median of the dated inputs instead of the +mean. A neighbour outside the tile repeats the edge pixel. The smoothing applies to every pixel of +the tile, also to a pixel that `nasa-modis` fills. ## Forest -`nasa-modis`: a MODIS pixel is 255 in all seasons when its mean tree canopy -cover is more than 75 % in Hansen Global Forest Change `treecover2000`. +`nasa-modis`: a MODIS pixel is 255 in all seasons when its mean tree canopy cover is more than 75 % +in Hansen Global Forest Change `treecover2000`. `copernicus-hr-wsi` has no forest mask. diff --git a/specs/route-api.md b/specs/route-api.md index 3f64bf194f..86a7124878 100644 --- a/specs/route-api.md +++ b/specs/route-api.md @@ -1,10 +1,16 @@ # Route API -`POST /v1/route` on the [route service](../planner/service/README.md) calculates -routes. `POST /v1/shape` finds the plan points of a route that follows a line. The -native provider answers both with the same code (`planner_router_call` with the call -`route` or `shape`). `GET /v1/region` describes the routing package. Request and -answer bodies are UTF-8 JSON. +`POST /v1/route` calculates routes. `POST /v1/shape` finds the plan points of a route that follows a +line. `GET /v1/region` describes the routing package. Request and answer bodies are UTF-8 JSON. + +- Writers: the [route service](../planner/service/README.md); the native provider answers `route` + and `shape` with the same code (`planner_router_call` with the call `route` or `shape`). The + answer encoder is `planner/router/src/answer.rs`. +- Readers: the web planner (`builder/web/src/lib/planner/routing.ts`, `route-answer.ts`), the + companion app (`OBCPlanner/PlannerService.swift`, `RouteAnswer.swift`). +- Test vectors: [`vectors/route-answer.json`](vectors/route-answer.json): two routes before and + after encoding. The encoder test and each decoder test read it. +- Related: [route package](route-package.md), [route catalog](route-catalog.md). ## Request @@ -23,13 +29,12 @@ The service rejects unknown fields. ## Answer -The answer is `{"routes": [...]}`. The first route is the primary route. With -`alternatives_only`, the answer holds only the alternative routes, and it can be -empty. `alternatives_only` overrides `alternatives` when both are `true`. When the -primary route fails, an `alternatives_only` request fails with that error, not -with an empty answer. When the deadline or a limit stops the search for -alternatives after the primary route, the answer holds the routes found before -it. Each route has these fields: +The answer is `{"routes": [...]}`. The first route is the primary route. With `alternatives_only`, +the answer holds only the alternative routes, and it can be empty. `alternatives_only` overrides +`alternatives` when both are `true`. When the primary route fails, an `alternatives_only` request +fails with that error, not with an empty answer. When the deadline or a limit stops the search for +alternatives after the primary route, the answer holds the routes found before it. Each route has +these fields: | Field | Value | | --- | --- | @@ -48,26 +53,24 @@ it. Each route has these fields: ### Deltas -`coordinates_udeg` holds two integers for each point: longitude, then latitude. -The first pair is absolute. Each pair after it is the difference from the -previous point. Add the pairs in sequence and divide by 1,000,000 to get degrees. +`coordinates_udeg` holds two integers for each point: longitude, then latitude. The first pair is +absolute. Each pair after it is the difference from the previous point. Add the pairs in sequence +and divide by 1,000,000 to get degrees. -`elevation_dm` and `elapsed_s` hold one integer for each point. Each value is the -difference from the previous value. The value before the first point is zero. -Add the values in sequence. Divide the heights by 10 to get metres. A `null` -height adds nothing: the next height is a difference from the last known height. -The first elapsed value is zero. +`elevation_dm` and `elapsed_s` hold one integer for each point. Each value is the difference from +the previous value. The value before the first point is zero. Add the values in sequence. Divide the +heights by 10 to get metres. A `null` height adds nothing: the next height is a difference from the +last known height. The first elapsed value is zero. -The encoder rounds the absolute value of each point and then calculates the -difference. Thus rounding errors do not add up along the route. +The encoder rounds the absolute value of each point and then calculates the difference, so rounding +errors do not add up. ### Edges -An edge joins point `i` and point `i + 1`. A route with `n` points has `n - 1` -edges. `edges` maps each channel name to a list of runs `[value, edge count]`. -Each run gives one value and the number of consecutive edges with that value. -The run lengths of a channel add up to `n - 1`. A missing channel is `null` on -every edge. A client accepts a channel that it does not know. +An edge joins point `i` and point `i + 1`. A route with `n` points has `n - 1` edges. `edges` maps +each channel name to a list of runs `[value, edge count]`. Each run gives one value and the number +of consecutive edges with that value. The run lengths of a channel add up to `n - 1`. A missing +channel is `null` on every edge. A client accepts a channel that it does not know. | Channel | Value of an edge | | --- | --- | @@ -76,15 +79,12 @@ every edge. A client accepts a channel that it does not know. | `closures` | `null`, or a list of possible closures, each `{"kind", "condition"}` | | `sac_scale` | The OSM `sac_scale` as an integer from `0` (`strolling`) through `1` (`hiking`, T1) to `6` (`difficult_alpine_hiking`, T6), or `null` when the way has none | -The router blocks a mode only for the access value `no`; `dismount` blocks -riding only. It uses an edge that is possibly closed for the mode that the route -uses on it, and reports it. A closure on a node, such as a gate, belongs to the -edges of the road that arrives at the node. A closure applies only to the -direction of travel that its tag names. A closure from an access value makes -the road cost three times its length, and a node cost as much as 300 m of good -road. Thus a route uses the road only where every alternative is more than three -times as long, or where a shaping point is on it. A `seasonal` or `conditional` -closure adds no cost: +The router blocks a mode only for the access value `no`; `dismount` blocks riding only. It uses an +edge that is possibly closed for the mode that the route uses on it, and reports it. A closure on a +node, such as a gate, belongs to the edges of the road that arrives at the node. A closure applies +only to the direction of travel that its tag names. A closure from an access value makes the road +cost three times its length, and a node cost as much as 300 m of good road. A `seasonal` or +`conditional` closure adds no cost: | `kind` | Source | `condition` | | --- | --- | --- | @@ -100,52 +100,47 @@ closure adds no cost: ### Legs and totals -`legs[k].from_index` and `legs[k].to_index` are inclusive point indices. The -first leg starts at index 0. Each leg starts where the previous leg ends. The -last leg ends at index `n - 1`. `surface_m` gives metres for each surface, in -the order above. `seconds` is moving time. Distances and heights are metres. +`legs[k].from_index` and `legs[k].to_index` are inclusive point indices. The first leg starts at +index 0. Each leg starts where the previous leg ends. The last leg ends at index `n - 1`. +`surface_m` gives metres for each surface, in the order above. `seconds` is moving time. Distances +and heights are metres. ### Leg positions -`start` and `end` are opaque strings. Each names the snapped road position of -a leg end: the road and the direction of travel on it. Where a point is not a -turnaround, the `end` of one leg equals the `start` of the next leg. A client -compares positions only for equality and sends them back unchanged. +`start` and `end` are opaque strings. Each names the snapped road position of a leg end: the road +and the direction of travel on it. Where a point is not a turnaround, the `end` of one leg equals +the `start` of the next leg. A client compares positions only for equality and sends them back +unchanged. -A pinned point keeps only the road candidate at that position. Thus a request -for some legs of a trip joins the legs before and after it in the same -direction. The service ignores a position that is not a candidate of the -point, for example a position from a different `package`. +A pinned point keeps only the road candidate at that position. Thus a request for some legs of a +trip joins the legs before and after it in the same direction. The service ignores a position that +is not a candidate of the point, for example a position from a different `package`. -`legs[k].totals` has the fields of the route `totals`, for that leg only. The -sum of the leg totals is the route total. Only `seconds` can differ, by up to -0.5 s for each leg. +`legs[k].totals` has the fields of the route `totals`, for that leg only. The sum of the leg totals +is the route total. Only `seconds` can differ, by up to 0.5 s for each leg. ### Precision -Coordinates are exact: the routing engine stores microdegrees. Heights are -within 0.05 m of the engine value. Elapsed and total seconds are within 0.5 s. -The [vector](vectors/route-answer.json) gives two routes before and after -encoding. The encoder test and each decoder test read it. +Coordinates are exact: the routing engine stores microdegrees. Heights are within 0.05 m of the +engine value. Elapsed and total seconds are within 0.5 s. ## Compression -The service compresses with brotli or gzip, as `Accept-Encoding` permits. It -prefers brotli. +The service compresses with brotli or gzip, as `Accept-Encoding` permits. It prefers brotli. ## Shape -`POST /v1/shape` finds plan points for a line, for example a GPX track. A route -request with these points and the same profile follows the line. +`POST /v1/shape` finds plan points for a line, for example a GPX track. A route request with these +points and the same profile follows the line. | Field | Value | | --- | --- | | `line` | 2 to 2,000 `[longitude, latitude]` pairs in degrees, at most 200 km long | | `profile` | A profile ID from `GET /v1/region` | -The service rejects unknown fields. The body limit is 64 KiB for both requests. -Simplify a track within 10 m and send 6 decimals before the request: then a -200 km track fits. The service also simplifies the line within 10 m. +The service rejects unknown fields. The body limit is 64 KiB for both requests. Simplify a track +within 10 m and send 6 decimals before the request: then a 200 km track fits. The service also +simplifies the line within 10 m. The answer is `{"points": [...], "turnarounds": [...]}`: @@ -154,18 +149,17 @@ The answer is `{"points": [...], "turnarounds": [...]}`: | `points` | 2 to 64 `[longitude, latitude]` pairs in degrees, in route order | | `turnarounds` | Ascending indices in `points` where the line turns back | -Each point is on a road, where the route attaches it. The first point is the -start of the line, and the last point is the end of the line. Send -`turnarounds` unchanged as the `turnarounds` of the route request. +Each point is on a road, where the route attaches it. The first point is the start of the line, and +the last point is the end of the line. Send `turnarounds` unchanged as the `turnarounds` of the +route request. -The route follows the line when its deviation is at most 2 % of the line length. -The deviation is the length of the route farther than 30 m from the line, plus -the length of the line farther than 30 m from the route. The line turns back at -a point when, for 150 m before and after the point, it stays within 30 m of -itself. A hairpin bend is not a turnaround. +The route follows the line when its deviation is at most 2 % of the line length. The deviation is +the length of the route farther than 30 m from the line, plus the length of the line farther than 30 +m from the route. The line turns back at a point when, for 150 m before and after the point, it +stays within 30 m of itself. A hairpin bend is not a turnaround. -One shape request uses one profile and at most 200 route calculations. Each has -the limits of a route request. The deadline is 30 s. +One shape request uses one profile and at most 200 route calculations. Each has the limits of a +route request. The deadline is 30 s. ## Region @@ -180,30 +174,27 @@ the limits of a route request. The deadline is 30 s. | `attribution` | Data credits | | `warnings` | Strings that name known limits of the package data | -A profile ID is an activity, such as `touring`, or an activity, `/` and a variant: -`touring/shorter` or `touring/less-climbing`. Two variants are skill choices. -`mtb/technical` permits every MTB grade; `mtb` permits S2 at most. -`hiking/alpine` permits T4; `hiking` permits T2 at most. Cycling profiles never -use steps. The clients offer the presets of the profiles that the region serves. +A profile ID is an activity, such as `touring`, or an activity, `/` and a variant: `touring/shorter` +or `touring/less-climbing`. Two variants are skill choices. `mtb/technical` permits every MTB grade; +`mtb` permits S2 at most. `hiking/alpine` permits T4; `hiking` permits T2 at most. Cycling profiles +never use steps. The clients offer the presets of the profiles that the region serves. ## Errors An error answer is `{"code", "message"}`. It never contains a substitute route. -The service attaches each point to the nearest road that the profile can use -and that lies in a large connected part of its road graph, within 250 m. When -none is that near, it uses the nearest such road within 1 km. `no_snap` means -that no such road is within 1 km. `no_path` means that no legal route joins two -consecutive points; the service does not try other roads for them. +The service attaches each point to the nearest road that the profile can use and that lies in a +large connected part of its road graph, within 250 m. When none is that near, it uses the nearest +such road within 1 km. `no_snap` means that no such road is within 1 km. `no_path` means that no +legal route joins two consecutive points; the service does not try other roads for them. -`line_too_long` means that a shape line has more than 2,000 points or is longer -than 200 km. `line_not_reproducible` means that the search found no plan of at -most 64 points that follows the line, within its 200 route calculations. +`line_too_long` means that a shape line has more than 2,000 points or is longer than 200 km. +`line_not_reproducible` means that the search found no plan of at most 64 points that follows the +line, within its 200 route calculations. -`busy` means that no worker became free within 1 s. `internal` means that the -service failed, not the request or the data. Send the request again. A body -larger than the limit is an `invalid_request`. A call other than `route` and -`shape` is `not_found`. +`busy` means that no worker became free within 1 s. `internal` means that the service failed, not +the request or the data. Send the request again. A body larger than the limit is an +`invalid_request`. A call other than `route` and `shape` is `not_found`. | Code | Status | | --- | --- | diff --git a/specs/route-catalog.md b/specs/route-catalog.md index 1d6fa4d73d..c873993dd7 100644 --- a/specs/route-catalog.md +++ b/specs/route-catalog.md @@ -1,10 +1,17 @@ # Route catalog -The route catalog lists the signed routes of a region: OSM route relations for -hiking, foot, bicycle and mountain-bike routes. The planner search reads it -online and offline. Each route record also holds the plan that reproduces the -route. The region prepare step bakes the catalog. A relation with a member -outside the extract leaves the catalog. +The route catalog lists the signed routes of a region: OSM route relations for hiking, foot, bicycle +and mountain-bike routes. Each route record also holds the plan that reproduces the route. A +relation with a member outside the extract leaves the catalog. + +- Version: `format` `1`. +- Writers: the region prepare step, `planner/router-build` (`src/catalog/mod.rs`). +- Readers: the planner search, online and offline: the web planner + (`builder/web/src/lib/planner/signed-routes.ts`), the companion app + (`OBCPlanner/SignedRoutes.swift`). +- Test vectors: [`vectors/signed-routes.json`](vectors/signed-routes.json). +- Related: [route API](route-api.md), [planner release](planner-release.md), [offline + bundle](planner-offline.md). ## Files @@ -13,26 +20,24 @@ outside the extract leaves the catalog. | Regional | `routes/REGION.json` | Every record of the region | | Grid | `routes/tiles/9-X-Y.json` | One file for each grid cell | -A file is UTF-8 JSON: `{"format": 1, "routes": [...]}`. The records are in -ascending `id` order. A grid release has no region file. +A file is UTF-8 JSON: `{"format": 1, "routes": [...]}`. The records are in ascending `id` order. A +grid release has no region file. -A grid cell is a Web Mercator tile at zoom 9 whose box overlaps the release -bounds with a positive area. Each grid cell has a file. A cell with no routes -has an empty `routes` list. Equal files share one object in the pool. A record -is in the file of each grid cell in its `cells`, with all of its fields. +A grid cell is a Web Mercator tile at zoom 9 whose box overlaps the release bounds with a positive +area. Each grid cell has a file. A cell with no routes has an empty `routes` list. Equal files share +one object in the pool. A record is in the file of each grid cell in its `cells`, with all of its +fields. -The tile API serves a cell file at `/releases/ID/routes/tiles/9-X-Y.json`. An -offline selection holds the files of its `offline.cells`. A client reads only -the files of covered cells: the grid cells online, and `offline.cells` offline. -A cell outside this set is not downloaded. A covered cell without a file is an -error, not a cell with no routes. In a regional release, the region file covers +The tile API serves a cell file at `/releases/ID/routes/tiles/9-X-Y.json`. An offline selection +holds the files of its `offline.cells`. A client reads only the files of covered cells: the grid +cells online, and `offline.cells` offline. A cell outside this set is not downloaded. A covered cell +without a file is an error, not a cell with no routes. In a regional release, the region file covers every cell. ## Records -A record is a route record or a long route. A field with no value is absent. -Coordinates are integer microdegrees, longitude before latitude. Lengths and -heights are integer metres. +A record is a route record or a long route. A field with no value is absent. Coordinates are integer +microdegrees, longitude before latitude. Lengths and heights are integer metres. | Field | Value | | --- | --- | @@ -57,68 +62,59 @@ heights are integer metres. | `stages` | Long route only: the relation IDs of its stages, in order | | `start_udeg` | Long route only: `[longitude, latitude]` of its start | -Each record has a `name` or a `ref`, or both. The builder cuts a longer -`description` at a word boundary. `symbol` is absent when France is the only -country of the region; the client then draws the `ref`. A region with France and -another country has no catalog yet: the builder has no country lookup. +Each record has a `name` or a `ref`, or both. The builder cuts a longer `description` at a word +boundary. `symbol` is absent when France is the only country of the region; the client then draws +the `ref`. A region with France and another country has no catalog: the builder has no country +lookup. ### Plan -The plan is the start, the shaping points and the finish. The start is vertex 0 -of `line_udeg`. The finish is its last vertex. The shaping points are the -vertices in `via`; vertex 0 and the last vertex are never in `via`. Every route -record has `via`, and it can be empty. `via` has at most 62 entries; a route that -needs more leaves the catalog. - -`turnarounds` has the meaning of `turnarounds` in the -[route API request](route-api.md#request): at each of these shaping points the -plan route turns back on purpose. In the request, the points are the start, the -shaping points and the finish, so the request index of an entry is its position -in `via` plus 1. For a loop, `turnarounds` can hold 0: the plan route turns -back at its start. It is absent when there are none. - -The plan reproduces the route with a profile of each activity that lists the -kind: `hiking/alpine` for `hiking` and `foot`, `mtb/technical` for `mtb`, and -`road`, `gravel` and `touring` for `bicycle`. A client plans a `hiking` or `foot` -route with `hardest` above 1 (T2) with `hiking/alpine`, an `mtb` route with -`hardest` above 2 (S2) with `mtb/technical`, and every other route with the -Balanced profile. Both profiles give the same route when no grade on it is above -the ordinary limit. The main line is the member ways with an -empty, `main`, `forward` or `backward` role, ordered as the Waymarked Trails route -builder orders them, with the forward branch where the route splits by -direction. When that order has more than one piece, the builder joins the -pieces at their nearest ends, from the first piece on, into one chain that uses -each piece once, with each join at most 500 m. When no such chain exists, for -example because a piece ends inside another piece, the member order stays if -each of its gaps is at most 500 m; otherwise the route leaves the catalog. The -router patches each gap. To reproduce the route, the router's route for the plan -with each profile has a deviation of at most 2 % of the main-line length. The -deviation is the length of the routed line that is more than 30 m from the main -line, plus the length of the main line that is more than 30 m from the routed -line. The plan route is the route that the -router gives for the plan with one of these profiles: `touring` for `bicycle`, -else the only one. The plan runs in the member order of the relation, also for a -loop. A route is a loop when its main line is a closed ring, or when it has -`roundtrip=yes` and its ends are within 200 m. A loop plan ends at its start, so -the last vertex equals vertex 0. - -`length_m`, `ascent_m` and `descent_m` are the totals of the route engine for -the plan route. The ascent is the sum of the height rises between consecutive -points of the road geometry. The descent is the sum of the height drops. +The plan is the start, the shaping points and the finish. The start is vertex 0 of `line_udeg`. The +finish is its last vertex. The shaping points are the vertices in `via`; vertex 0 and the last +vertex are never in `via`. Every route record has `via`, and it can be empty. `via` has at most 62 +entries; a route that needs more leaves the catalog. + +`turnarounds` has the meaning of `turnarounds` in the [route API request](route-api.md#request): at +each of these shaping points the plan route turns back on purpose. In the request, the points are +the start, the shaping points and the finish, so the request index of an entry is its position in +`via` plus 1. For a loop, `turnarounds` can hold 0: the plan route turns back at its start. It is +absent when there are none. + +The plan reproduces the route with a profile of each activity that lists the kind: `hiking/alpine` +for `hiking` and `foot`, `mtb/technical` for `mtb`, and `road`, `gravel` and `touring` for +`bicycle`. A client plans a `hiking` or `foot` route with `hardest` above 1 (T2) with +`hiking/alpine`, an `mtb` route with `hardest` above 2 (S2) with `mtb/technical`, and every other +route with the Balanced profile. Both profiles give the same route when no grade on it is above the +ordinary limit. The main line is the member ways with an empty, `main`, `forward` or `backward` +role, ordered as the Waymarked Trails route builder orders them, with the forward branch where the +route splits by direction. When that order has more than one piece, the builder joins the pieces at +their nearest ends, from the first piece on, into one chain that uses each piece once, with each +join at most 500 m. When no such chain exists, for example because a piece ends inside another +piece, the member order stays if each of its gaps is at most 500 m; otherwise the route leaves the +catalog. The router patches each gap. To reproduce the route, the router's route for the plan with +each profile has a deviation of at most 2 % of the main-line length. The deviation is the length of +the routed line that is more than 30 m from the main line, plus the length of the main line that is +more than 30 m from the routed line. The plan route is the route that the router gives for the plan +with one of these profiles: `touring` for `bicycle`, else the only one. The plan runs in the member +order of the relation, also for a loop. A route is a loop when its main line is a closed ring, or +when it has `roundtrip=yes` and its ends are within 200 m. A loop plan ends at its start, so the +last vertex equals vertex 0. + +`length_m`, `ascent_m` and `descent_m` are the totals of the route engine for the plan route. The +ascent is the sum of the height rises between consecutive points of the road geometry. The descent +is the sum of the height drops. ### Line -`line_udeg` holds two integers for each vertex: longitude, then latitude. The -first pair is absolute. Each pair after it is the difference from the previous -vertex. This is the encoding of `coordinates_udeg` in the -[route API](route-api.md#deltas). The vertices are points of the plan route, in -its order. No point of the plan route is more than 50 m from the line. Each -shaping point is a vertex of the line. +`line_udeg` holds two integers for each vertex: longitude, then latitude. The first pair is +absolute. Each pair after it is the difference from the previous vertex. This is the encoding of +`coordinates_udeg` in the [route API](route-api.md#deltas). The vertices are points of the plan +route, in its order. No point of the plan route is more than 50 m from the line. Each shaping point +is a vertex of the line. ### Grades -The grade of a road comes from its way. For `hiking` and `foot`, `sac_scale` -gives the grade: +The grade of a road comes from its way. For `hiking` and `foot`, `sac_scale` gives the grade: | `sac_scale` | Grade | | --- | --- | @@ -129,44 +125,40 @@ gives the grade: | `demanding_alpine_hiking` | T5 | | `difficult_alpine_hiking` | T6 | -For `mtb`, the first digit of `mtb:scale` gives S0 to S5; S6 counts as S5. A -road without a valid value has no explicit grade and counts as T1 or S0. +For `mtb`, the first digit of `mtb:scale` gives S0 to S5; S6 counts as S5. A road without a valid +value has no explicit grade and counts as T1 or S0. -`grades_m[i]` is the length of the plan route with grade T(i + 1), or with grade -S(i) for `mtb`. Each length is rounded, so the sum can differ from `length_m` by -up to 3 m. `hardest` is the largest `i` with an explicit grade on the plan -route. It is absent when the plan route has no explicit grade. +`grades_m[i]` is the length of the plan route with grade T(i + 1), or with grade S(i) for `mtb`. +Each length is rounded, so the sum can differ from `length_m` by up to 3 m. `hardest` is the largest +`i` with an explicit grade on the plan route. It is absent when the plan route has no explicit +grade. ### Long routes -A long route is a relation that holds route relations. Its stages are its child -routes with an empty or `main` role and the same `network`, in member order. A -long route is in the catalog only when each of its stages is in it. A long route -with a child relation that holds route relations leaves the catalog. - -A long route has no `line_udeg`, `via` or `turnarounds`. Its line and its plan -are those of its stages in order. The client loads the stages by their IDs from -the files of the long route's `cells`. The joined plan has the points of the -stage plans in order. A stage finish and the next stage start are one point when -they are the same vertex, and two points otherwise. Each stage turnaround moves -to its position in the joined plan. A client -plans a whole long route only when its joined plan has at most 64 points; -otherwise it offers its stages only. `start_udeg` is the start of its first -stage. `length_m`, `ascent_m`, `descent_m` and `grades_m` are the sums of the -values of its stages. `hardest` is the largest `hardest` of its stages. Its -`cells` are the union of the `cells` of its stages. Thus each file that holds a -stage also holds its long route. - -A stage has `parent` and `stage` only when its long route is in the catalog. -When a route is a stage of two long routes, `parent` is the one with the lower -relation ID. +A long route is a relation that holds route relations. Its stages are its child routes with an empty +or `main` role and the same `network`, in member order. A long route is in the catalog only when +each of its stages is in it. A long route with a child relation that holds route relations leaves +the catalog. + +A long route has no `line_udeg`, `via` or `turnarounds`. Its line and its plan are those of its +stages in order. The client loads the stages by their IDs from the files of the long route's +`cells`. The joined plan has the points of the stage plans in order. A stage finish and the next +stage start are one point when they are the same vertex, and two points otherwise. Each stage +turnaround moves to its position in the joined plan. A client plans a whole long route only when its +joined plan has at most 64 points; otherwise it offers its stages only. `start_udeg` is the start of +its first stage. `length_m`, `ascent_m`, `descent_m` and `grades_m` are the sums of the values of +its stages. `hardest` is the largest `hardest` of its stages. Its `cells` are the union of the +`cells` of its stages. Thus each file that holds a stage also holds its long route. + +A stage has `parent` and `stage` only when its long route is in the catalog. When a route is a stage +of two long routes, `parent` is the one with the lower relation ID. ### Coverage -A segment of the plan route touches each zoom 9 cell whose box intersects the -bounding box of the segment, edges included. `cells` can name a cell outside the -grid; that cell has no file. Offline, a route lies wholly inside the download -when each of its `cells` is the `id` of a cell in `offline.cells`. +A segment of the plan route touches each zoom 9 cell whose box intersects the bounding box of the +segment, edges included. `cells` can name a cell outside the grid; that cell has no file. Offline, a +route lies wholly inside the download when each of its `cells` is the `id` of a cell in +`offline.cells`. ## Examples diff --git a/specs/route-package.md b/specs/route-package.md index 685ad8d7d8..4da1978bd6 100644 --- a/specs/route-package.md +++ b/specs/route-package.md @@ -1,44 +1,52 @@ # Regional routing package +The routing package of one region: a content-addressed manifest and page store that holds the road +graph, its metrics and its geometry, and the grid selections of it. + +- Version: manifest format `8`; index magic `OBCRIDX3`; `routing/blocks.json` `format: 2`. A format + change requires a new manifest format value. +- Writers: [`planner/router-build`](../planner/router-build/README.md) (the `planner/routing` layer + of `obc data`). +- Readers: the [route engine](../planner/router/README.md) (route service, native iOS, WebAssembly). +- Code authority: `planner/router/src/package.rs` (`Manifest`, `Metric`). +- Related: [route API](route-api.md), [planner release](planner-release.md), [offline + bundle](planner-offline.md). + ## Identity and closure -A package directory contains `manifest.json`, `pages.idx`, and `pages.bin`. -A digest is lowercase SHA-256 as 64 hexadecimal characters. Each object name -is the digest of its complete stored bytes. The package ID is the digest of the -exact UTF-8 manifest bytes. Reformatting a manifest changes its ID. +A package directory contains `manifest.json`, `pages.idx`, and `pages.bin`. A digest is lowercase +SHA-256 as 64 hexadecimal characters. Each object name is the digest of its complete stored bytes. +The package ID is the digest of the exact UTF-8 manifest bytes. Reformatting a manifest changes its +ID. -The manifest format is `8`. Its JSON fields are defined by `Manifest` and -`Metric` in `planner/router/src/package.rs`. Bounds are -`[west, south, east, north]` in degrees. Metric IDs equal their profile names. -Source digests identify the input data. Attribution and warnings travel with -the package. +The manifest format is `8`. Its JSON fields are defined by `Manifest` and `Metric` in +`planner/router/src/package.rs`. Bounds are `[west, south, east, north]` in degrees. Metric IDs +equal their profile names. Source digests identify the input data. Attribution and warnings travel +with the package. Bounds define query endpoint coverage. -Every referenced object must be present in a complete download. Missing objects -are errors, not empty graph cells. Objects are immutable. A consumer must not -mix manifests or pages from different packages during a query. +Every referenced object must be present in a complete download. Missing objects are errors, not +empty graph cells. Objects are immutable. A consumer must not mix manifests or pages from different +packages during a query. ## Objects -`pages.bin` contains the stored objects without separators. `pages.idx` starts -with the eight ASCII bytes `OBCRIDX3` and a little-endian `u64` record count. -Each 48-byte record contains a 32-byte SHA-256 digest, a little-endian `u64` -offset in `pages.bin`, and a little-endian `u64` stored length. Records are -sorted by digest bytes. Each digest occurs once. The index has no trailing +`pages.bin` contains the stored objects without separators. `pages.idx` starts with the eight ASCII +bytes `OBCRIDX3` and a little-endian `u64` record count. Each 48-byte record contains a 32-byte +SHA-256 digest, a little-endian `u64` offset in `pages.bin`, and a little-endian `u64` stored +length. Records are sorted by digest bytes. Each digest occurs once. The index has no trailing bytes. Offsets and lengths must lie within `pages.bin`. -A `Table` contains `len`, the number of entries, and `blocks`, the ordered -digests of its index blocks. Each block stores up to 4096 entries. Page tables -store `Vec` digests. Metric `allowed` stores `Vec` snap words. -Metric `costs` stores `Vec`. Manifest `costs` stores -`Vec`. These value tables refer directly to their blocks. -The last block contains the remaining entries. Consumers load bounded blocks. +A `Table` contains `len`, the number of entries, and `blocks`, the ordered digests of its index +blocks. Each block stores up to 4096 entries. Page tables store `Vec` digests. Metric +`allowed` stores `Vec` snap words. Metric `costs` stores `Vec`. Manifest `costs` stores +`Vec`. These value tables refer directly to their blocks. The last block contains +the remaining entries. Consumers load bounded blocks. -Objects are zlib-compressed Postcard 1 values. Serialized field and enum order -is the Rust declaration order in the types below. Format changes require a -new manifest format value. Compression output need not be canonical; the hash -always refers to the actual stored bytes. +Objects are zlib-compressed Postcard 1 values. Serialized field and enum order is the Rust +declaration order in the types below. Compression output need not be canonical; the hash always +refers to the actual stored bytes. | Manifest reference | Decoded type | Ordering | | --- | --- | --- | @@ -50,125 +58,107 @@ always refers to the actual stored bytes. | Manifest `graph` | `base::Topology` direct tables | Directed road ID and ordered turns | | Metric `weights` | `base::Weights` direct columns | Directed road ID and ordered turns | -The optional manifest `closures` is absent when no road has a possible closure. -`sets` is the digest of one object with the distinct closure sets. `roads` is a -column with one set index per directed road; the all-ones value means no -possible closure, and a finite value is below the number of sets. A set entry is -`(modes, {kind, condition})`, as [the route API](route-api.md#edges) lists the -kinds. A set keeps the order of its source: access values before conditions. - -Cells span 10,000 microdegrees on each axis. Cell indices use floor division, -including for negative coordinates. A road appears in all cells crossed by -its geometry segment bounding boxes. The last page can contain fewer entries. -The manifest's spatial keys are one-degree cells. A fine cell's directory key -is `floor(latitude_cell / 100),floor(longitude_cell / 100)`. Fine cell keys are +The optional manifest `closures` is absent when no road has a possible closure. `sets` is the digest +of one object with the distinct closure sets. `roads` is a column with one set index per directed +road; the all-ones value means no possible closure, and a finite value is below the number of sets. +A set entry is `(modes, {kind, condition})`, as [the route API](route-api.md#edges) lists the kinds. +A set keeps the order of its source: access values before conditions. + +Cells span 10,000 microdegrees on each axis. Cell indices use floor division, including for negative +coordinates. A road appears in all cells crossed by its geometry segment bounding boxes. The last +page can contain fewer entries. The manifest's spatial keys are one-degree cells. A fine cell's +directory key is `floor(latitude_cell / 100),floor(longitude_cell / 100)`. Fine cell keys are `latitude_cell,longitude_cell`. Each directory has at most 10,000 cells. -Points store integer microdegree coordinates and `f32` metre heights. -Height `f32::MIN` means unknown. Surface enum order is unknown, paved, compacted, -gravel, dirt, rough. Access bits are bike `1`, foot `2`, and pushing `4`. -A road stores its source way ID and direction relative to that way. +Points store integer microdegree coordinates and `f32` metre heights. Height `f32::MIN` means +unknown. Surface enum order is unknown, paved, compacted, gravel, dirt, rough. Access bits are bike +`1`, foot `2`, and pushing `4`. A road stores its source way ID and direction relative to that way. -Geometry columns store roads with empty shapes, one shape length per road, -latitude deltas, longitude deltas, and elevation bit deltas. Point columns -follow road and shape order. Coordinates accumulate from zero across the page. -Elevation stores each `f32` bit pattern XOR the previous bit pattern, with zero -as the first predecessor. Column lengths must equal the sum of shape lengths. +Geometry columns store roads with empty shapes, one shape length per road, latitude deltas, +longitude deltas, and elevation bit deltas. Point columns follow road and shape order. Coordinates +accumulate from zero across the page. Elevation stores each `f32` bit pattern XOR the previous bit +pattern, with zero as the first predecessor. Column lengths must equal the sum of shape lengths. Reconstruction preserves every coordinate and elevation bit. -The package holds no source OSM objects. Source hashes and attribution identify -the input. Terrain samples belong to the directed road geometry. - -Metric `allowed` holds one snap bit per directed road: road `i` uses bit `i % 64` -of word `i / 64`. Its table length is `ceil(roads / 64)`. A set bit means that -the metric has a finite cost for the road, and that the road lies in the -metric's largest strongly connected component of road states or in one of at -least `connectivity::MINIMUM_COMPONENT` states (`planner/router-build`). Turn rules -and node access shape these components. Queries snap only to roads with a set -bit, so a leg without a path joins points that no legal route connects. - -The manifest contains one directed topology shared by all metrics. A state is the -arrival at one directed road. Its state ID equals its road ID. An edge joins each -pair of roads whose physical endpoints meet, including pairs forbidden by some -or all metrics. Edges are unique and sorted by source road, then destination road. - -`first` is a `Vec` table of length `roads + 1`. It starts at zero, ends at -the edge count, and never decreases. The outgoing edge range of road `i` is -`first[i]..first[i+1]`. `head` stores each destination road. `offsets` is the -narrowest width that holds every position within an outgoing range. A consumer -builds the incoming edges from `first` and `head`, and stores the position of -each incoming edge within its source's outgoing range at this width. - -A column has a `width` and a direct value table. Width is `U8`, `U16`, `U32`, or -`U64`; its blocks store vectors of that exact unsigned type. The all-ones value -means unavailable. Finite values cannot use this sentinel. The writer chooses -the smallest width that holds every finite value without collision. - -Each metric stores two columns. `road_costs` has one exact total cost per road. -`turns` has one exact entry penalty per edge in the shared topology order. -An unavailable road or turn cannot be traversed. A legal transition costs the -turn penalty plus the destination road cost. Both addition and accumulated path -cost use checked `u64` arithmetic. Endpoint departures are the finite incoming +The package holds no source OSM objects. Terrain samples belong to the directed road geometry. + +Metric `allowed` holds one snap bit per directed road: road `i` uses bit `i % 64` of word `i / 64`. +Its table length is `ceil(roads / 64)`. A set bit means that the metric has a finite cost for the +road, and that the road lies in the metric's largest strongly connected component of road states or +in one of at least `connectivity::MINIMUM_COMPONENT` states (`planner/router-build`). Turn rules and +node access shape these components. Queries snap only to roads with a set bit, so a leg without a +path joins points that no legal route connects. + +The manifest contains one directed topology shared by all metrics. A state is the arrival at one +directed road. Its state ID equals its road ID. An edge joins each pair of roads whose physical +endpoints meet, including pairs forbidden by some or all metrics. Edges are unique and sorted by +source road, then destination road. + +`first` is a `Vec` table of length `roads + 1`. It starts at zero, ends at the edge count, and +never decreases. The outgoing edge range of road `i` is `first[i]..first[i+1]`. `head` stores each +destination road. `offsets` is the narrowest width that holds every position within an outgoing +range. A consumer builds the incoming edges from `first` and `head`, and stores the position of each +incoming edge within its source's outgoing range at this width. + +A column has a `width` and a direct value table. Width is `U8`, `U16`, `U32`, or `U64`; its blocks +store vectors of that exact unsigned type. The all-ones value means unavailable. Finite values +cannot use this sentinel. The writer chooses the smallest width that holds every finite value +without collision. + +Each metric stores two columns. `road_costs` has one exact total cost per road. `turns` has one +exact entry penalty per edge in the shared topology order. An unavailable road or turn cannot be +traversed. A legal transition costs the turn penalty plus the destination road cost. Both addition +and accumulated path cost use checked `u64` arithmetic. Endpoint departures are the finite incoming turns of the selected road. No separate endpoint topology is stored. -Metric `costs` has one entry per road. Zero means excluded. Other values index -manifest `costs`, starting at one. A `CostBasis` contains an exact `f64` factor, -`f64` bend coefficient, and ferry flag. The distance cost is the road length in -metres converted to `f64`, multiplied by the factor. Binary Postcard storage -preserves the factors' floating point bits. The shared cost implementation -derives the `RoadCost` curve from these values, the directed road geometry, and -the metric profile. Preparation and queries use the same operation order and -rounding. Curve fractions and cumulative penalties are nondecreasing. Equal -fractions encode a step. The last fraction is `1`, unless there are no penalties. -Total cost is rounded to a positive integer. Rounded interpolated prefix +Metric `costs` has one entry per road. Zero means excluded. Other values index manifest `costs`, +starting at one. A `CostBasis` contains an exact `f64` factor, `f64` bend coefficient, and ferry +flag. The distance cost is the road length in metres converted to `f64`, multiplied by the factor. +Binary Postcard storage preserves the factors' floating point bits. The shared cost implementation +derives the `RoadCost` curve from these values, the directed road geometry, and the metric profile. +Preparation and queries use the same operation order and rounding. Curve fractions and cumulative +penalties are nondecreasing. Equal fractions encode a step. The last fraction is `1`, unless there +are no penalties. Total cost is rounded to a positive integer. Rounded interpolated prefix differences give partial costs. -A stored object and its decoded payload must each fit within 8 MiB. The -manifest must fit within 128 MiB. Consumers verify hashes before decoding. -These bounds protect installation and query memory; they do not set region -coverage or a phone performance target. +A stored object and its decoded payload must each fit within 8 MiB. The manifest must fit within 128 +MiB. Consumers verify hashes before decoding. These bounds do not set region coverage or a phone +performance target. ## Optional search bounds -`landmarks` can be absent. When present, it contains `junctions`, `mapping`, -and `profiles`. `junctions` is a positive integer. `mapping` has one arrival -junction ID per directed road. Each ID is less than `junctions`. `profiles` maps -prepared metric IDs to a positive integer `scale` and `tables`, one through 32 -distance columns. Each column has `junctions` entries. - -These tables store `Vec` deltas. Values accumulate from zero within each -4096-entry block. Mapping values fit `u32`. Distance values fit `u16`; 65535 is -a capped distance, not an unavailable-cost sentinel. - -A distance column `d` of a profile must satisfy `scale * (d(u) - d(v)) <= w(u, v)` -for every legal road-state transition, where `u` and `v` use their mapped -arrival junctions and `w` is the exact transition cost. The builder computes -reverse junction distances with each legal road cost divided by the profile's -`scale` and rounded down. It omits turn penalties from these lower bounds. It -chooses the scale from the farthest junction to the first landmark plus the -farthest junction from it, so that no column of the connected component reaches -the cap; a distance outside the component saturates at 65535. The exact search -still uses all prepared road and turn costs. +`landmarks` can be absent. When present, it contains `junctions`, `mapping`, and `profiles`. +`junctions` is a positive integer. `mapping` has one arrival junction ID per directed road. Each ID +is less than `junctions`. `profiles` maps prepared metric IDs to a positive integer `scale` and +`tables`, one through 32 distance columns. Each column has `junctions` entries. + +These tables store `Vec` deltas. Values accumulate from zero within each 4096-entry block. +Mapping values fit `u32`. Distance values fit `u16`; 65535 is a capped distance, not an +unavailable-cost sentinel. + +A distance column `d` of a profile must satisfy `scale * (d(u) - d(v)) <= w(u, v)` for every legal +road-state transition, where `u` and `v` use their mapped arrival junctions and `w` is the exact +transition cost. The builder computes reverse junction distances with each legal road cost divided +by the profile's `scale` and rounded down. It omits turn penalties from these lower bounds. It +chooses the scale from the farthest junction to the first landmark plus the farthest junction from +it, so that no column of the connected component reaches the cap; a distance outside the component +saturates at 65535. The exact search still uses all prepared road and turn costs. ## Grid selections -`routing/blocks.json` has `format: 2`. `source` is the source manifest SHA-256. -`data` uses the format 8 manifest structure with selected bounds and region. -Each sparse table adds `pages`, an ascending list of source page numbers, -parallel to `blocks`. `len` remains the source column length. An absent page -is unavailable, not an empty page. +`routing/blocks.json` has `format: 2`. `source` is the source manifest SHA-256. `data` uses the +format 8 manifest structure with selected bounds and region. Each sparse table adds `pages`, an +ascending list of source page numbers, parallel to `blocks`. `len` remains the source column length. +An absent page is unavailable, not an empty page. -`roads` contains sorted, disjoint half-open source road ranges. `arcs` is the -number of source outgoing transitions for those roads, before transitions to -absent roads are removed. `snap` maps source spatial keys to page hashes. -`archives` is the sorted unique list of pack IDs. Each pack is +`roads` contains sorted, disjoint half-open source road ranges. `arcs` is the number of source +outgoing transitions for those roads, before transitions to absent roads are removed. `snap` maps +source spatial keys to page hashes. `archives` is the sorted unique list of pack IDs. Each pack is `packs/ID/{pages.idx,pages.bin}` and uses the object encoding above. -A selection retains source geometry, weights, turn penalties, snap bits and -landmark values. Runtime road IDs are compact indices in source road order. -Queries use only transitions whose two roads are present. The snap bits come -from the whole region, so a road near the edge of a selection can be snappable -although the selection has no path to it; such a query returns `no_path`. The -source junction IDs -connect retained roads to the original landmark columns. The selection -manifest hash is its package identity. Different source releases cannot mix. +A selection retains source geometry, weights, turn penalties, snap bits and landmark values. Runtime +road IDs are compact indices in source road order. Queries use only transitions whose two roads are +present. The snap bits come from the whole region, so a road near the edge of a selection can be +snappable although the selection has no path to it; such a query returns `no_path`. The source +junction IDs connect retained roads to the original landmark columns. The selection manifest hash is +its package identity. Different source releases cannot mix. diff --git a/specs/wikimedia-content.md b/specs/wikimedia-content.md index fa23f75524..aaad9f6d90 100644 --- a/specs/wikimedia-content.md +++ b/specs/wikimedia-content.md @@ -1,34 +1,39 @@ # Shared Wikimedia content -This contract defines the landmark and peak content: the compact facts that the prepared source -[`wiki`](#the-wiki-source) holds, and how the compilers read them. The compiled device formats in -[obc-data](obc-data.md) and [OBCM](obcm-map.md) are separate outputs. +The landmark and peak content: the compact facts that the prepared source [`wiki`](#the-wiki-source) +holds, and how the compilers read them. The compiled device formats in [obc-data](obc-data.md) and +[OBCM](obcm-map.md) are separate outputs. + +- Writers: the producer of `wiki` (`host/obc-data-steps/src/wiki.rs`) with the steps of `tools/wiki` + (`facts.py` holds the compact fact format). +- Readers: the landmark and peak compilers (`host/obc-pack/src/landmarks/`), the device cell steps. +- Related: [obc-data `wiki`](obc-data.md#wiki), [OBCM](obcm-map.md), [planner + release](planner-release.md). ## Policy owners -`host/obc-pack/src/landmarks/policy.json` owns the category roots and exclusions. -`policy.rs` owns category precedence and its tests. The sixteen category ids and -the excluded glacier group remain unchanged. Ordinary OSM service-place filters -remain separate. Peak content does not need a landmark category. +`host/obc-pack/src/landmarks/policy.json` owns the category roots and exclusions. `policy.rs` owns +category precedence and its tests. The sixteen category ids and the excluded glacier group remain +unchanged. Ordinary OSM service-place filters remain separate. Peak content does not need a landmark +category. -`specs/content-languages.json` owns supported languages and order: English, -German, French, Spanish. `locale.rs` selects a usable local language through -P131/P37, then P17/P37, then that order. Administrative traversal has depth eight -and at most 64 entities. These dependencies are pinned source inputs. +`specs/content-languages.json` owns supported languages and order: English, German, French, Spanish. +`locale.rs` selects a usable local language through P131/P37, then P17/P37, then that order. +Administrative traversal has depth eight and at most 64 entities. These dependencies are pinned +source inputs. -The existing landmark module owns article, name, photo and credit eligibility. -Both collection compilers use it. Acquisition records facts and API outcomes; -compilation applies these rules. A policy edit recompiles retained facts. Only a -missing fact requires acquisition. There is no separate acquisition rule language. +The landmark module owns article, name, photo and credit eligibility. Both collection compilers use +it. Acquisition records facts and API outcomes; compilation applies these rules. A policy edit +recompiles retained facts. Only a missing fact requires acquisition. There is no separate +acquisition rule language. ## Identity and source pins -A subject id is the resolved Wikidata QID. An exact Wikipedia link without a -Wikidata item uses `wiki--`, as the peak compiler does. The -canonical language follows supported-language order over confirmed language links. -Titles and coordinates are not stable ids. Acquisition retains the exact API -normalization, redirect and language-link proof used to resolve an input link. -It does not infer identity from a name, proximity, filename or camera position. +A subject id is the resolved Wikidata QID. An exact Wikipedia link without a Wikidata item uses +`wiki--`, as the peak compiler does. The canonical language follows +supported-language order over confirmed language links. Titles and coordinates are not stable ids. +Acquisition retains the exact API normalization, redirect and language-link proof used to resolve an +input link. It does not infer identity from a name, proximity, filename or camera position. A pin contains source identity, source revision and SHA-256 of retained bytes: @@ -39,47 +44,44 @@ A pin contains source identity, source revision and SHA-256 of retained bytes: | Commons metadata | Commons page id | description-page revision id | | Image bytes | Commons file identity | file timestamp and upstream SHA-1 | -An image retains its canonical filename and source URL as well as its identity. -Metadata and file revisions are separate: a credit edit must not require another -image download. A successful check time controls freshness, not content identity. -A confirmed missing response retains the exact requested identity, response digest -and check time. It has no invented revision. A refresh may replace that outcome. +An image retains its canonical filename and source URL as well as its identity. Metadata and file +revisions are separate: a credit edit must not require another image download. A successful check +time controls freshness, not content identity. A confirmed missing response retains the exact +requested identity, response digest and check time. It has no invented revision. A refresh may +replace that outcome. -A region, OSM extract, transport implementation or image transform is not part of -a source pin. Release manifests pin the complete source dependency set used for a -compile. The same pin can serve more than one region and collection. +A region, OSM extract, transport implementation or image transform is not part of a source pin. +Release manifests pin the complete source dependency set used for a compile. The same pin can serve +more than one region and collection. ## Retained inputs and Store boundary -The acquired record keeps only fields needed for compilation and provenance: -subject id; supported labels; coordinates; relevant claims and sitelinks; class -and locale dependency pins; exact link proofs; and per-asset outcomes. -Each article retains language, canonical title, page id, revision id, revision -source URL, compact lead HTML, licence and required original notices. -The article API representation must bind that input to its stated revision. -The REST page `with_html` response supplies HTML, page id, current revision and -licence together. Retain its lead and required notices, not its full envelope. -A rendered lead or a later API response must not inherit an earlier revision id. - -Each image candidate retains its source claim or eligible article identity, -Commons identity, metadata/file pins, thumbnail URL, dimensions, relevant subject -metadata and required licence/credit fields. Device conversion uses the standard -500 px thumbnail. The `wiki` source publishes the thumbnail of each photo that the compilers -choose; other thumbnails are not published content. -Full API envelopes, rejected image payloads and request logs are not permanent -shared content. Required proofs and notices must survive compaction. +The acquired record keeps only fields needed for compilation and provenance: subject id; supported +labels; coordinates; relevant claims and sitelinks; class and locale dependency pins; exact link +proofs; and per-asset outcomes. Each article retains language, canonical title, page id, revision +id, revision source URL, compact lead HTML, licence and required original notices. The article API +representation must bind that input to its stated revision. The REST page `with_html` response +supplies HTML, page id, current revision and licence together. Retain its lead and required notices, +not its full envelope. A rendered lead or a later API response must not inherit an earlier revision +id. + +Each image candidate retains its source claim or eligible article identity, Commons identity, +metadata/file pins, thumbnail URL, dimensions, relevant subject metadata and required licence/credit +fields. Device conversion uses the standard 500 px thumbnail. The `wiki` source publishes the +thumbnail of each photo that the compilers choose; other thumbnails are not published content. Full +API envelopes, rejected image payloads and request logs are not permanent shared content. Required +proofs and notices must survive compaction. The producer of `wiki` resolves explicit links to canonical subjects and retains the identity proofs. It acquires the entity, dependency, article and image inputs of the subjects that the -compilers select, and retains each successful response before the next request. A region -compiles from the shards of a `wiki` version, with its OSM associations and the compiler policy, -and makes no Wikimedia request. A fact that the version lacks is no content; the step counts it. +compilers select, and retains each successful response before the next request. A region compiles +from the shards of a `wiki` version, with its OSM associations and the compiler policy, and makes no +Wikimedia request. A fact that the version lacks is no content; the step counts it. -Acquisition returns confirmed present or confirmed missing facts. Transport -failure, incomplete required identity, language-link or dependency pagination, -and server pressure are unresolved work, never confirmed missing content. An -interrupted or failed run of the producer publishes no version. The next run reuses its -successful responses. +Acquisition returns confirmed present or confirmed missing facts. Transport failure, incomplete +required identity, language-link or dependency pagination, and server pressure are unresolved work, +never confirmed missing content. An interrupted or failed run of the producer publishes no version. +The next run reuses its successful responses. ### Compact facts @@ -95,60 +97,57 @@ Each compact JSON object repeats its kind, key, status, identity and revision wh | `category` | One bounded nonrecursive member response, limit and continuation proof | | `file` | File identity and revision, plus a separately retained image asset pin | -A category response has no invented page revision. Its digest pins the bounded -membership observation. `bounded`, `limit`, `limit_reached` and `truncated` -describe its scope. Required metadata pagination must finish; a bounded category -response may retain continuation without following it. +A category response has no invented page revision. Its digest pins the bounded membership +observation. `bounded`, `limit`, `limit_reached` and `truncated` describe its scope. Required +metadata pagination must finish; a bounded category response may retain continuation without +following it. -An image asset has relative `path`, `sha256`, `bytes`, `url` and `input` fields. -`input` is `thumbnail500` or a validated retained `original`. The shards hold the -JSON facts and every asset that they reference. Request envelopes stay in the work directory of -the producer; no version holds them. +An image asset has relative `path`, `sha256`, `bytes`, `url` and `input` fields. `input` is +`thumbnail500` or a validated retained `original`. The shards hold the JSON facts and every asset +that they reference. Request envelopes stay in the work directory of the producer; no version holds +them. -An article pin records the resolved page's actual Wikidata item and alias proof. -A redirect to another subject is a present source fact. Compilation rejects that -article for the requested subject and records an identity omission. Other eligible -languages remain usable. An unresolved API response still blocks acquisition. -A QID subject requires the article's matching Wikidata item. A no-item article -remains eligible for a confirmed synthetic Wikipedia subject. +An article pin records the resolved page's actual Wikidata item and alias proof. A redirect to +another subject is a present source fact. Compilation rejects that article for the requested subject +and records an identity omission. Other eligible languages remain usable. An unresolved API response +still blocks acquisition. A QID subject requires the article's matching Wikidata item. A no-item +article remains eligible for a confirmed synthetic Wikipedia subject. -A thumbnail file fact retains `revision_before` and `revision_after` witnesses. -Both match the canonical filename, file timestamp and SHA-1, required credit -fields, and description revision when present. A retained thumbnail without both -coherent witnesses does not become a shared file pin. +A thumbnail file fact retains `revision_before` and `revision_after` witnesses. Both match the +canonical filename, file timestamp and SHA-1, required credit fields, and description revision when +present. A retained thumbnail without both coherent witnesses does not become a shared file pin. ## Selection and projections -Selection retains a reason for each selected subject and omitted asset. The -selected reason names its landmark category or explicit peak link. Omission -reasons distinguish policy exclusion, no matching category, outside region, -identity failure, absent source content, unsupported text, credit rejection and -image rejection. An API outcome is distinct from a compiler rejection. -A selected article states the chosen language and locale dependency evidence. -A selected photo states its candidate source and identity evidence. - -Landmarks require usable supported article text. Peaks may have article text, -a photo, or both. A peak without content remains an OSM summit. A missing optional -article or photo does not invalidate a map. An unresolved source failure remains -visible in preparation and does not masquerade as optional absence. - -One acquired subject can produce both collections. Landmark OSM links, hours and -routable approaches remain separate from peak node ids and summit coordinates. -Peak records never enter the landmark or service-place query collection. -Regional clipping selects associations without changing the subject identity. - -The device projection contains bounded short text, required attribution and -216 × 240 RGB222 images under the existing OBCM contracts. It works offline. -The planner projection contains selected subject identity, location, category, -article text and provenance, online image references and required credits. -Routing data and downloaded iOS routing maps contain no image payload. -Offline planner detail retains useful text and explains an unavailable photo. -Both projections derive from the same selected inputs and policy. +Selection retains a reason for each selected subject and omitted asset. The selected reason names +its landmark category or explicit peak link. Omission reasons distinguish policy exclusion, no +matching category, outside region, identity failure, absent source content, unsupported text, credit +rejection and image rejection. An API outcome is distinct from a compiler rejection. A selected +article states the chosen language and locale dependency evidence. A selected photo states its +candidate source and identity evidence. + +Landmarks require usable supported article text. Peaks may have article text, a photo, or both. A +peak without content remains an OSM summit. A missing optional article or photo does not invalidate +a map. An unresolved source failure remains visible in preparation and does not masquerade as +optional absence. + +One acquired subject can produce both collections. Landmark OSM links, hours and routable approaches +remain separate from peak node ids and summit coordinates. Peak records never enter the landmark or +service-place query collection. Regional clipping selects associations without changing the subject +identity. + +The device projection contains bounded short text, required attribution and 216 × 240 RGB222 images +under the OBCM contracts. It works offline. The planner projection contains selected subject +identity, location, category, article text and provenance, online image references and required +credits. Routing data and downloaded iOS routing maps contain no image payload. Offline planner +detail retains useful text and explains an unavailable photo. Both projections derive from the same +selected inputs and policy. ## The `wiki` source The prepared source `wiki` ([obc-data](obc-data.md#wiki)) holds the compact facts of each OSM -subject in the selected coverage: an object with a `wikidata` or `wikipedia` tag. Its version has these files: +subject in the selected coverage: an object with a `wikidata` or `wikipedia` tag. Its version has +these files: | Path | Holds | | --- | --- | @@ -156,58 +155,54 @@ subject in the selected coverage: an object with a `wikidata` or `wikipedia` tag | `common.tar.zst` | The classes, and the facts of each subject without a box or whose box touches more than 16 cells. A version that [extends](obc-data.md#wiki) another keeps each earlier common fact whose kind and key no shard of this run holds, until a version of everything replaces it | | `index.json` | `shards`: `path`, `sha256`, `bytes` and `facts` of each shard; `counts`: the subjects that the run read, those in `common`, those without a resolved link, the unresolved work of each kind, and the compiles of the producer that failed (`photo_compile_failures`) | -The version manifest records the coverage, the article languages and `publish_text` of its run. -An earlier fact that a version keeps keeps its languages, so a run with fewer languages keeps the +The version manifest records the coverage, the article languages and `publish_text` of its run. An +earlier fact that a version keeps keeps its languages, so a run with fewer languages keeps the earlier articles; a newer revision is never replaced by an older one. Regional preparation selects -subjects from the regional OSM extract before it asks Commons. An article in an excluded -language is a `missing` fact with `reason: "language_not_prepared"`. This means that the version -excludes it, not that Wikipedia lacks it. It supplies no text or lead-image candidate. Subject -identity still uses the supported-language order; incomplete canonical identity proof remains -unresolved work. - -A shard is a tar under zstd. Its members are `content//.json`, as the shared -content of a step names them. A fact that two shards need is in both, with the same bytes. A -subject has its entity, its links, the locales of the bounded P131 traversal and the articles that -the compilers read. A fact that the version lacks is no content. A member -`content/assets/` is the 500 px thumbnail of a photo that the compilers choose for a -subject; the device cell steps convert it to the device format. - -The producer asks Commons in rounds: the categories of the subjects that the compilers select, then the metadata of their photo candidates (P18, the lead images of their -articles and the members of their categories), then the thumbnails of the files that the -compilers choose. A thumbnail that fails the checks of a compiler makes it choose another file, -so the last round repeats for the subjects with new files until it asks no new file. Without -published text, each article of the version is a fact with its key and the status `missing`: the -compilers of the producer and of the readers see no article, and choose no lead image. Each -round asks only for what the rounds before have not answered, with these limits: one request at -a time, at most four a second, `maxlag` 5, +subjects from the regional OSM extract before it asks Commons. An article in an excluded language is +a `missing` fact with `reason: "language_not_prepared"`. This means that the version excludes it, +not that Wikipedia lacks it. It supplies no text or lead-image candidate. Subject identity still +uses the supported-language order; incomplete canonical identity proof remains unresolved work. + +A shard is a tar under zstd. Its members are `content//.json`, as the shared content +of a step names them. A fact that two shards need is in both, with the same bytes. A subject has its +entity, its links, the locales of the bounded P131 traversal and the articles that the compilers +read. A fact that the version lacks is no content. A member `content/assets/` is the 500 px +thumbnail of a photo that the compilers choose for a subject; the device cell steps convert it to +the device format. + +The producer asks Commons in rounds: the categories of the subjects that the compilers select, then +the metadata of their photo candidates (P18, the lead images of their articles and the members of +their categories), then the thumbnails of the files that the compilers choose. A thumbnail that +fails the checks of a compiler makes it choose another file, so the last round repeats for the +subjects with new files until it asks no new file. Without published text, each article of the +version is a fact with its key and the status `missing`: the compilers of the producer and of the +readers see no article, and choose no lead image. Each round asks only for what the rounds before +have not answered, with these limits: one request at a time, at most four a second, `maxlag` 5, `Retry-After`, three backoffs on 429 and 503, and media at 3 MB/s. When Wikimedia is busy, the -producer waits ten minutes, then twice as long each time up to two hours, and asks again. A -subject whose compile fails alone has no photo in the version: the producer logs it and -`index.json` counts it. When more than half the subjects of a chunk fail, or on an I/O error, the -run stops. +producer waits ten minutes, then twice as long each time up to two hours, and asks again. A subject +whose compile fails alone has no photo in the version: the producer logs it and `index.json` counts +it. When more than half the subjects of a chunk fail, or on an I/O error, the run stops. ## Photo fallback and reachability -Photo order starts with explicit P18, then identity-checked Wikipedia lead images. -A nearby camera position or matching filename is not subject evidence. Coverage -and relevance are measured separately before the old category path is replaced. - -Fallback uses one directly claimed P373 category, one nonrecursive category response -and at most the file bound in `photo_policy.fallback` of `policy.json`. -Continuation beyond that deliberate candidate bound is a complete bounded outcome. -The record states the limit reached. Files require subject evidence and valid credits. - -A target holds the versions of `wiki`. A bake reads the shards of the version that its target -reads, so a warm bake needs no network, and another machine gets the facts and the thumbnails -from the target. A new version refreshes the facts: the content layers and the layers that read -them bake again. Other thumbnails, full originals, rejected candidates and request logs are not -published. The device cell layers publish the final device images. - -`Content.aliases` maps exact Wikidata aliases to canonical subject ids. -`Content.wikipedia_aliases` maps proven `language:title` aliases of accepted article -variants to those ids. Planner search uses these exact identities for OSM association. -Peak node ids and OSM coordinates remain the regional association. -Online photo credits omit device transform notices. A photo reference includes -`file_identity`, `page_revision` and `file_revision`; incomplete provenance gives a -source-page link. Consumers recheck that identity before displaying online photos. -A mutable CDN URL is not a permanent certificate for future bytes. +Photo order starts with explicit P18, then identity-checked Wikipedia lead images. A nearby camera +position or matching filename is not subject evidence. + +Fallback uses one directly claimed P373 category, one nonrecursive category response and at most the +file bound in `photo_policy.fallback` of `policy.json`. Continuation beyond that candidate bound is +a complete bounded outcome. The record states the limit reached. Files require subject evidence and +valid credits. + +A target holds the versions of `wiki`. A bake reads the shards of the version that its target reads, +so a warm bake needs no network, and another machine gets the facts and the thumbnails from the +target. A new version refreshes the facts: the content layers and the layers that read them bake +again. Other thumbnails, full originals, rejected candidates and request logs are not published. The +device cell layers publish the final device images. + +`Content.aliases` maps exact Wikidata aliases to canonical subject ids. `Content.wikipedia_aliases` +maps proven `language:title` aliases of accepted article variants to those ids. Planner search uses +these exact identities for OSM association. Peak node ids and OSM coordinates remain the regional +association. Online photo credits omit device transform notices. A photo reference includes +`file_identity`, `page_revision` and `file_revision`; incomplete provenance gives a source-page +link. Consumers recheck that identity before displaying online photos. A mutable CDN URL is not a +permanent certificate for future bytes. From 78febd730a5d2e790bac659f2c42e3276693f62f Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 29 Oct 2026 08:11:32 +0000 Subject: [PATCH 05/20] docs: common README structure for firmware and the iOS companion The firmware, bootloader, board and companion READMEs follow the common order (setup, build, run or flash, test, reference, pitfalls). The two broken anchor links in firmware/README.md point at live sections. OBCProtocol.md links the BLE and ride-archive specs instead of restating them and keeps only the iOS notes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DTHu3LHSGY5CY9j2XmscLi --- README.md | 3 +- companion-ios/OBCProtocol.md | 41 ++---- companion-ios/README.md | 82 ++++++----- firmware/README.md | 141 +++++++++--------- firmware/obc-boot/README.md | 78 +++++----- firmware/obc-fw-nrf54l/README.md | 240 +++++++++++++++---------------- 6 files changed, 289 insertions(+), 296 deletions(-) diff --git a/README.md b/README.md index d09de9d8dd..2213d4cbac 100644 --- a/README.md +++ b/README.md @@ -138,7 +138,8 @@ This is the current direction, not a release schedule. The hardware build uses the Nordic nRF54LM20 development kit and a Sharp LS021B7DD02 240 × 320 reflective memory LCD. It reads maps and routes from microSD and connects to the companion software -over BLE and USB. The custom PCB and enclosure are still under development. KiCad sources can be found in the repository, CAD sources for the enclosure will be made available later. +over BLE and USB. The custom PCB and enclosure are still under development. KiCad sources are in +[`hardware/`](hardware/README.md); CAD sources for the enclosure follow later. ## Repository layout diff --git a/companion-ios/OBCProtocol.md b/companion-ios/OBCProtocol.md index 00cd6e241e..b089320bd6 100644 --- a/companion-ios/OBCProtocol.md +++ b/companion-ios/OBCProtocol.md @@ -10,16 +10,16 @@ This file records only the choices an iOS contributor needs while reading the im ## Versioning and store identity -Protocol version `4`. `protocolVersion` is exactly one little-endian `u16`. The client issues -`LIST` before its first object operation and takes the exact 128-bit `StoreId` from that response. -Durable phone state is scoped to `(device serial, StoreId)`, so a reset or a card swap cannot -alias an old object id. **A version mismatch is surfaced, never decoded optimistically.** +The client checks `protocolVersion` (`ble-interface.md` §3.3). It issues `LIST` before its first +object operation and takes the exact 128-bit `StoreId` from that response. Durable phone state is +scoped to `(device serial, StoreId)`, so a reset or a card swap cannot alias an old object id. +**A version mismatch is surfaced, never decoded optimistically.** ## Transport -The app discovers DIS (`0x180A`), BAS (`0x180F`) and OBC Control (`3C920000-…`). The DIS Firmware -Revision String is either an installed OBCU version or a bare Git hash; **a hash is never parsed -as a release version**, so it never produces an automatic update offer. +The app discovers DIS, BAS and OBC Control (`ble-interface.md` §3). It follows the §3.1 +firmware-revision dialect: **a bare Git hash is never parsed as a release version**, so it never +produces an automatic update offer. Bulk objects move over one encrypted L2CAP CoC. `L2CAPByteChannel` owns bytes, `BLEChannel` preserves v4 stream-record boundaries, `TransferClient` owns request ids and the @@ -35,39 +35,28 @@ request and stream record at the link. ## Object formats -- Routes are OBCR v5 files. GPX and TCX conversion happens on the phone, and the device stores the - OBCR bytes verbatim. The device never parses XML. +- Routes are OBCR v5 files. The device stores the OBCR bytes verbatim and never parses XML. - Ride bytes are recorded samples plus an OBRF footer. The phone decodes them to `Ride` before archiving or exporting GPX. - Route, ride and trip catalogs are v4 `LIST` entries. Catalog flag bit 3 (`assistantAccepted`) marks an accepted Assistant route; the decoder accepts it and rejects undefined flag bits. -- Trips hold route object ids, not route bytes. Upload the stages first and the trip last. - Deleting a trip does not implicitly delete its routes. +- Trips reference route objects (`ble-interface.md` §7.7). Upload the stages first and the trip + last. Deleting a trip does not implicitly delete its routes. - Firmware images are signed OBCU containers carried as update objects. The wire codecs are under `OBCTransport/Codecs/`; interchange-file parsing is in `OBCFormats`. ## Ride archives -The catalog and download path preserve StoreId, ObjectId, the served Revision, the verified length -and the CRC. The archive publishes the canonical summary, samples, source and local sync state in -one transaction, and **file and directory persistence barriers must succeed before a local archive -receipt is returned**. An in-memory save cannot produce a receipt. - -The coordinator sends ARCHIVE_RIDE after a durable commit, and for a revalidated matching archive -even when no download is needed. **Only the matching ARCHIVE_RIDE response confirms proof**; -STATUS and local save counts cannot. The device uses archive proof for the synced indicator, and -rides stay on the device until the user deletes them. - -A ten-second receipt-response deadline cancels the parked receive. A timeout or a failed receipt -leaves the local archive intact and the device confirmation pending in the sync banner; resume or -reconnect revalidates and retries without downloading saved rides. A changed source is terminal -for that receipt. +`ride-archive-receipts.md` owns the archive commit, the receipt, its deadline and the retry rules. +The coordinator sends ARCHIVE_RIDE; **only the matching ARCHIVE_RIDE response confirms proof**. +Rides stay on the device until the user deletes them. ## Names and imports - **The device name lives in Config.** Renaming is a `Config` object write; there is no rename - command. The UTF-8 name is capped at 48 bytes and truncated only at a Character boundary. + command (`ble-interface.md` §7.3). The app truncates the UTF-8 name to the 48-byte cap only at a + Character boundary. - **The phone accepts GPX and TCX.** Either decodes into `ImportedRoute`, which is encoded as OBCR before upload. `RouteSource` and the import UI expose the same two-format boundary. diff --git a/companion-ios/README.md b/companion-ios/README.md index 5f24a8a79d..31a1aa9ad9 100644 --- a/companion-ios/README.md +++ b/companion-ios/README.md @@ -1,44 +1,24 @@ # OpenBikeComputer iOS companion -An on-ramp. The canonical wire contract is -[`../specs/ble-interface.md`](../specs/ble-interface.md); +The app imports planned routes, pushes them to the device, and syncs recorded rides back. The +canonical wire contract is [`../specs/ble-interface.md`](../specs/ble-interface.md); [`OBCProtocol.md`](OBCProtocol.md) records the iOS-facing mappings. -The app imports planned routes, pushes them to the device, and syncs recorded rides back. +## Setup -## Architecture +Install Xcode 26.x, an iOS simulator runtime, XcodeGen, Node.js 22 or later, and the Rust +toolchain. Personal signing belongs in `project.local.yml`: it is tracked with an empty team id, +and its header says how to keep your edit out of git. -View models depend on capability-sized protocols declared by `OBCTransport`, so the same UI runs -against CoreBluetooth on a device and against the deterministic mock in tests and the simulator. -Use the broad `DeviceTransport` aggregate only for true aggregate consumers and composition-root -wiring. CoreBluetooth stays in `OBCTransport/BLE/`, and mock code stays in `OBCMock` behind -`#if DEBUG`. Tests enforce these boundaries. +## Build -```text -OBCDomain -> OBCTransport -> OBCMock - \-> OBCFormats -OBCUI -> OBCDomain + OBCTransport + OBCPlanner + MapLibre (iOS) -OBCPlanner -> OBCDomain (online client, offline installation and selection) -``` - -`OBCCompanion/` is the composition root and the only target that chooses a concrete transport and planner provider. -`PlannerDataSource` supplies one release for maps, search, routes, and viewport layers. -`obcPlannerSource` passes that provider to map views. The app uses `LocalFirstPlanner`. -`OfflinePlanner` connects installed releases to native routing, search and map files. -Offline maps live in Settings, with a shortcut in the planner's map menu. -`project.yml` is the Xcode project source of truth; **never edit or commit the generated -pbxproj.** - -## Build and test - -Needs Xcode 26.x, an iOS simulator runtime, XcodeGen, Node.js 22 or later, and the Rust toolchain. XcodeGen installs the pinned replay renderer and builds native map styles and search with npm. -App builds link `OBCHost` and compile `planner-service` for the selected Rust iOS target. -Map styles and POI kinds come from the web planner sources. Before package tests without XcodeGen, run -`npm ci --prefix companion-ios/ReplayAssets` from the repository root. -The existing host build script also connects the package to the Rust framework. +Map styles and POI kinds come from the web planner sources. App builds link `OBCHost` and compile +`planner-service` for the selected Rust iOS target. The host build script also connects the +package to the Rust framework. -For a simulator, prepare the framework before XcodeGen resolves the package: +For a simulator, prepare the framework before XcodeGen resolves the package, then use the +`OBCCompanion` scheme: ```sh tools/build-ios-host.sh --sim-only @@ -46,7 +26,16 @@ cd companion-ios xcodegen generate ``` -For package tests, start at the repository root and prepare the macOS slice: +`obc ios-companion` builds Release with real Bluetooth and installs it on the paired iPhone; +`obc ios-device` does the same for the `OBCDevice` shell. + +`project.yml` is the Xcode project source of truth; **never edit or commit the generated +pbxproj.** + +## Test + +Before package tests without XcodeGen, run `npm ci --prefix companion-ios/ReplayAssets` from the +repository root. Then, from the repository root, prepare the macOS slice: ```sh tools/build-ios-host.sh --mac-only @@ -54,11 +43,7 @@ cd companion-ios/Packages/OBCKit swift test --disable-keychain --disable-netrc ``` -CI runs the package tests plus Debug and Release simulator builds. For a simulator, generate the -project and use the `OBCCompanion` scheme. `obc ios-companion` builds Release with real Bluetooth -and installs it on the paired iPhone; `obc ios-device` does the same for the `OBCDevice` shell. -Personal signing belongs in `project.local.yml`: it is tracked with an empty team id, and its -header says how to keep your edit out of git. +CI runs the package tests plus Debug and Release simulator builds. ## Mock and captures @@ -79,3 +64,24 @@ catalogue, such as the one that `obc data serve` prints. Offline downloads use the downloads service of the active release in that catalogue. - On a device, accept the Local Network alert. To answer it again, use Settings › Privacy & Security › Local Network. + +## Architecture + +View models depend on capability-sized protocols declared by `OBCTransport`, so the same UI runs +against CoreBluetooth on a device and against the deterministic mock in tests and the simulator. +Use the broad `DeviceTransport` aggregate only for true aggregate consumers and composition-root +wiring. CoreBluetooth stays in `OBCTransport/BLE/`, and mock code stays in `OBCMock` behind +`#if DEBUG`. Tests enforce these boundaries. + +```text +OBCDomain -> OBCTransport -> OBCMock + \-> OBCFormats +OBCUI -> OBCDomain + OBCTransport + OBCPlanner + MapLibre (iOS) +OBCPlanner -> OBCDomain (online client, offline installation and selection) +``` + +`OBCCompanion/` is the composition root and the only target that chooses a concrete transport and +planner provider. `PlannerDataSource` supplies one release for maps, search, routes, and viewport +layers. `obcPlannerSource` passes that provider to map views. The app uses `LocalFirstPlanner`. +`OfflinePlanner` connects installed releases to native routing, search and map files. Offline maps +live in Settings, with a shortcut in the planner's map menu. diff --git a/firmware/README.md b/firmware/README.md index 81535aa24c..a2ed595974 100644 --- a/firmware/README.md +++ b/firmware/README.md @@ -2,16 +2,16 @@ This directory holds the crates the device image reaches: the shared `no_std` render path, the platform adapters, the board crate and the bootloader. The device application and the desktop -simulator share one rendering path for `.obcm` maps. - -This file is the build, test and dev-loop guide for all the Rust in the repo. For how the system -works, read the docs site: . Per-crate roles are tabulated in the +simulator share one rendering path for `.obcm` maps. This file is the build, test and dev-loop +guide for all the Rust in the repo. For how the system works, read the docs site: +. Per-crate roles are in the [repo README](../README.md#repository-layout). -The workspace is rooted at the repo root (`../Cargo.toml`) and spans `firmware/`, `../host/` and -`../planner/`, `../sim/`, and `../builder/` — one `Cargo.lock`, one `target/`. Only `firmware/` is device-reachable; that is the -rule `firmware/tools/check_dependencies.py` enforces. Dev-dependencies cross the boundary on -purpose, because a dev-dep never enters the `no_std` build, so `cargo test` wants GEOS. +The workspace is rooted at the repo root (`../Cargo.toml`) and spans `firmware/`, `../host/`, +`../planner/`, `../sim/` and `../builder/`: one `Cargo.lock`, one `target/`. Only `firmware/` is +device-reachable; `firmware/tools/check_dependencies.py` enforces that rule. Dev-dependencies +cross the boundary, because a dev-dependency never enters the `no_std` build. So `cargo test` +needs GEOS. Four crates are excluded from the workspace and built from **inside their own directory**, each with its own `Cargo.lock`, `fmt`, `clippy`, `test` and CI job: @@ -23,14 +23,14 @@ with its own `Cargo.lock`, `fmt`, `clippy`, `test` and CI job: | [`obc-desktop`](../builder/desktop/README.md) | the Tauri app: a platform webview | | [`obc-sensor-sim`](obc-sensor-sim/README.md) | the sensor simulator: its own MCU target | -## Prerequisites +## Setup | For… | You need | | :-- | :-- | | Anything Rust | A stable toolchain (`rustup`). | -| The desktop simulator | Just Rust — the GUI is pure eframe/egui. | +| The desktop simulator | Only Rust — the GUI is pure eframe/egui. | | The packer (`obc-pack`) | System **GEOS ≥ 3.14** (`brew install geos`), its only native dependency. | -| The desktop app (`obc-desktop`) | **CMake** and a C++ compiler (it vendors GEOS), plus Node. Linux also wants WebKitGTK — see [its README](../builder/desktop/README.md). | +| The desktop app (`obc-desktop`) | **CMake** and a C++ compiler (it vendors GEOS), plus Node. Linux also needs WebKitGTK — see [its README](../builder/desktop/README.md). | | Compiling the shared crates for the device | `rustup target add thumbv8m.main-none-eabihf`. | ## Build @@ -43,14 +43,53 @@ cargo build --release # → target/release/{obc-sim, obc-pack} cargo build -p obc-app --target thumbv8m.main-none-eabihf ``` -The board crate is built from inside its own directory. Its target comes from -`.cargo/config.toml`, which cargo finds by working directory, so building it through -`--manifest-path` from here silently targets the host and fails: +Build the board crate from inside its own directory. Cargo finds its target in +`.cargo/config.toml` by working directory, so a build through `--manifest-path` from here silently +targets the host and fails: ```sh cd firmware/obc-fw-nrf54l && cargo build --release # see that crate's README to flash ``` +### Web builder core + +Build the browser core and its test-only device variant with `wasm-pack`: + +```sh +# From builder/web. +npm run build:wasm +``` + +The frontend needs this output before `npm run check`, `npm test`, or `npm run build`. +See [the bridge README](../builder/wasm/README.md) for paths and checks. + +## Run + +### Simulator + +`obc-sim` renders a packed `.obcm` map through the exact code the firmware runs. Pack one with +`obc pack `. + +```sh +obc sim # the Grimsel demo +./target/release/obc-sim map.obcm # device look, 240×320, 3× window scale +./target/release/obc-sim map.obcm --size 480x640 --scale 2 +./target/release/obc-sim map.obcm --gpx ride.gpx # replay a GPX as a fake GPS +./target/release/obc-sim map.obcm --png out.png # headless one-frame render +``` + +See the [simulator guide](../sim/desktop/README.md) or `obc-sim --help` for the rest. + +### Web demo (`obc-web-demo`) + +The landing page's live demo is the shared crates compiled to wasm. Trunk drives the build +(`rustup target add wasm32-unknown-unknown` and `cargo install trunk` once): + +```sh +trunk serve --config docs/Trunk.toml # http://127.0.0.1:8080/ +trunk build --release --config docs/Trunk.toml # → docs/dist/, what CI and Pages deploy +``` + ## Test ```sh @@ -81,9 +120,9 @@ regenerates it in the same pull request, with the reason stated. ## Format -`.rustfmt.toml` is committed, so let rustfmt own style. Formatting takes five invocations, and CI -checks all of them: the workspace is a *virtual* manifest, so `--all` is required or it formats -nothing, and `--all` skips the four excluded crates. +`.rustfmt.toml` is committed, so let rustfmt own style. CI checks all five invocations. The +workspace is a *virtual* manifest, so `--all` is required or it formats nothing, and `--all` skips +the four excluded crates. `obc fmt` runs all five. ```sh cargo fmt --all # the workspace @@ -93,50 +132,11 @@ cargo fmt --manifest-path builder/desktop/Cargo.toml cargo fmt --manifest-path firmware/obc-sensor-sim/Cargo.toml ``` -`obc fmt` runs all five. - -## Run the simulator - -`obc-sim` renders a packed `.obcm` map through the exact code the firmware runs. Pack one with -`obc pack `. - -```sh -obc sim # the Grimsel demo -./target/release/obc-sim map.obcm # device look, 240×320, 3× window scale -./target/release/obc-sim map.obcm --size 480x640 --scale 2 -./target/release/obc-sim map.obcm --gpx ride.gpx # replay a GPX as a fake GPS -./target/release/obc-sim map.obcm --png out.png # headless one-frame render -``` - -See the [simulator guide](../sim/desktop/README.md) or `obc-sim --help` for the rest. - -## Run the web demo (`obc-web-demo`) - -The landing page's live demo is the same shared crates compiled to wasm. Trunk drives the build -(`rustup target add wasm32-unknown-unknown` and `cargo install trunk` once): - -```sh -trunk serve --config docs/Trunk.toml # http://127.0.0.1:8080/ -trunk build --release --config docs/Trunk.toml # → docs/dist/, what CI and Pages deploy -``` - -## Build the web builder core - -Build the browser core and its test-only device variant with `wasm-pack`: - -```sh -# From builder/web. -npm run build:wasm -``` - -The frontend needs this output before `npm run check`, `npm test`, or `npm run build`. -See [the bridge README](../builder/wasm/README.md) for paths and checks. - ## Firmware update images (OBCU) A field update is an OBCU container a client uploads as an update-package object. The byte format -is [`obcu-update.md`](../specs/obcu-update.md); the codec and boot decision live in `obc-dfu`, and the -producer is `obc-mkimage`. The pipeline is objcopy, then wrap and sign: +is [`obcu-update.md`](../specs/obcu-update.md); the codec and boot decision live in `obc-dfu`, and +the producer is `obc-mkimage`. The pipeline is objcopy, then wrap and sign: ```sh # From the board crate, whose .cargo/config.toml selects the nRF54L target. @@ -156,18 +156,17 @@ cargo run -p obc-mkimage -- wrap \ cargo run -p obc-mkimage -- inspect UPDATE.BIN ``` -`wrap` refuses an image over `MAX_IMAGE_LEN` and warns if the binary's first word is not a -plausible initial stack pointer, which usually means an ELF or a wrong section order slipped -through. +`wrap` refuses an image over `MAX_IMAGE_LEN`. It warns if the binary's first word is not a +plausible initial stack pointer, which usually means an ELF or a wrong section order. **Signing is not optional on the device.** Without a seed, `wrap` emits an unsigned container and the armer rejects it. `obc-mkimage sign` attaches the trailer to an already-wrapped container, so -an artifact can be built on one machine and signed on the one that holds the key; `keygen` makes a -keypair. Keys, the `OBCU_SIGNING_SEED` secret and the **rotation still owed before the first real -release** are in [`obc-dfu/keys/README.md`](obc-dfu/keys/README.md). +you can build an artifact on one machine and sign it on the one that holds the key; `keygen` makes +a keypair. Keys, the `OBCU_SIGNING_SEED` secret and the **rotation still owed before the first +real release** are in [`obc-dfu/keys/README.md`](obc-dfu/keys/README.md). To install, upload the container and confirm on the device, or trigger the armer over the debug -VCOM link ([board README](obc-fw-nrf54l/README.md#driving-it-from-a-host-debug-uart)). The armer +VCOM link ([board README](obc-fw-nrf54l/README.md#host-input-and-firmware-update)). The armer validates the object, writes the running image into a rollback reserve, arms the boot-state page and resets into `obc-boot` ([its README](obc-boot/README.md) has the LED codes). @@ -197,14 +196,14 @@ the mix-up. `--posting-log2` and `--cell-log2` default to the v1 baked pairing. Both are OBCT header data, so a different pairing is a re-bake, not a format change. -`fetch` is the only thing that touches the network. A bake is a pure function of a tile directory -and a box, and byte-identical output for identical inputs is a contract pinned by a digest test. A -source void becomes `NODATA` and nothing is ever inpainted. +`fetch` is the only command that uses the network. A bake is a pure function of a tile directory +and a box: identical inputs give byte-identical output, and a digest test pins that contract. A +source void becomes `NODATA`; nothing is inpainted. Anything derived from GLO-30 must carry the Copernicus credit; `bake` prints it. Its single copy is the `copernicus-glo-30` entry of [`data/sources.toml`](../data/sources.toml). For a published catalog, use `obc bake terrain`, which drives this crate as a library over the -curated coverage (see the root [README](../README.md#baking-and-publishing-the-catalog)). The two -committed terrain sidecars are regenerated by -[`fixtures/build-map-package.sh terrain`](../fixtures/build-map-package.sh). +curated coverage (usage: the `bake` recipe in [`tools/justfile`](../tools/justfile), or +`just --show bake`). [`fixtures/build-map-package.sh terrain`](../fixtures/build-map-package.sh) +regenerates the two committed terrain sidecars. diff --git a/firmware/obc-boot/README.md b/firmware/obc-boot/README.md index 879647c408..e0e94fe3af 100644 --- a/firmware/obc-boot/README.md +++ b/firmware/obc-boot/README.md @@ -1,41 +1,14 @@ # obc-boot — the nRF54L bootloader -The 32 KB first-stage bootloader for the SD-staged DFU path. It reads the `BOOT_STATE` RRAM page, -decodes it with the shared [`obc-dfu`](../obc-dfu) crate, and runs `obc_dfu::engine`: an `Armed` -page verifies the staged image's CRC, flashes the app slot through RRAMC, verifies the readback, -writes `Trial` and jumps into the new image; a `Trial` still present at a later entry rolls back. -This crate only wires the card transport, RRAMC and GPIO into that engine (`src/semmc.rs`, -`src/install.rs`, `src/led.rs`). The byte formats and the boot decision table are normative in +The 32 KB first-stage bootloader for the SD-staged DFU path. It decodes the `BOOT_STATE` RRAM page +with the shared [`obc-dfu`](../obc-dfu) crate and runs `obc_dfu::engine`. This crate only wires the +card transport, RRAMC and GPIO into that engine (`src/semmc.rs`, `src/install.rs`, `src/led.rs`). +The byte formats and the boot decision table are normative in [`obcu-update.md`](../../specs/obcu-update.md). -The card transport is the sEMMC soft peripheral. The card exists only behind a ~13.6 KB -coprocessor image this crate cannot embed, so the app-side armer stages that image into the -`SEMMC_STAGE` RRAM carve before every arm and `src/semmc.rs` validates it before it boots it on -the FLPR. - -RRAM layout ([`memory.x`](memory.x), which must agree with `../obc-fw-nrf54l/build.rs`): - -| Address | Region | Size | -| --- | --- | ---: | -| `0x0000_0000` | `obc-boot` (this crate) | 32 KB | -| `0x0000_8000` | app slot (`obc-fw-nrf54l`) | 1976 KB | -| `0x001F_6000` | `SEMMC_STAGE` (the staged sEMMC blob) | 20 KB | -| `0x001F_B000` | `BOOT_STATE` page | 4 KB | -| `0x001F_C000` | `SETTINGS` page (the app's) | 4 KB | - -## LED codes (LED0 — the bootloader's entire UI) - -| Pattern | Meaning | -| :-- | :-- | -| one short pulse | proof of life on every entry, then the app boots | -| slow heartbeat | verifying the staged image; nothing is written yet | -| fast heartbeat | flashing the app slot and reading it back | -| 2 blinks, then boot | the arm was abandoned and the old app is intact: the staged image or the staged sEMMC blob failed verification, or the card stayed unreadable past the retry budget | -| 3 blinks, pause, repeat | the card is missing or reads are failing; retrying with backoff. Reinsert the card or power-cycle. A pre-erase `Armed` arm gives up after about a minute; a `Rollback` or a mid-flash error retries forever | -| SOS, forever | the readback never matched. Halted, still `Armed`, so a power cycle retries the install | - -The panel is not dark during an install: the app paints a static "Installing update" card as its -last frame, and `src/com.rs` keeps the memory-in-pixel glass alive with a software COM wave. +The card transport is the sEMMC soft peripheral, a ~13.6 KB coprocessor image this crate cannot +embed. The app-side armer stages it into the `SEMMC_STAGE` RRAM carve before every arm, and +`src/semmc.rs` validates it before it boots it on the FLPR. ## Build @@ -80,12 +53,11 @@ cd ../obc-fw-nrf54l && cargo run --release # app @ 0x8000 No LED blink and no boot means there is no bootloader at `0x0`. One blink and then nothing means there is no app at `0x8000`. -## Bring-up: the install path runs only on glass +## Test -The `Idle` fast path is the only path an ordinary boot takes, and no host test can run this crate -at all. The card, the FLPR, the RRAM writes, the trial boot and the rollback therefore have one -verification: a board. Run it after any change to `src/semmc.rs`, `src/install.rs` or the engine -wiring. +No host test can run this crate, and an ordinary boot takes only the `Idle` fast path. So only a +board verifies the card, the FLPR, the RRAM writes, the trial boot and the rollback. Run this +after any change to `src/semmc.rs`, `src/install.rs` or the engine wiring. 1. Build the app twice, with version strings that differ. Wrap and sign both with `obc-mkimage`. 2. Write the second container to the card as a kind-7 update object with the WebUSB client. @@ -107,7 +79,33 @@ wiring. A fault on this path leaves the `Armed` page behind, so every later boot repeats it. Reflash both images to recover. -## Constraints +## LED codes (LED0 — the bootloader's entire UI) + +| Pattern | Meaning | +| :-- | :-- | +| one short pulse | proof of life on every entry, then the app boots | +| slow heartbeat | verifying the staged image; nothing is written yet | +| fast heartbeat | flashing the app slot and reading it back | +| 2 blinks, then boot | the arm was abandoned and the old app is intact: the staged image or the staged sEMMC blob failed verification, or the card stayed unreadable past the retry budget | +| 3 blinks, pause, repeat | the card is missing or reads are failing; retrying with backoff. Reinsert the card or power-cycle. A pre-erase `Armed` arm gives up after about a minute; a `Rollback` or a mid-flash error retries forever | +| SOS, forever | the readback never matched. Halted, still `Armed`, so a power cycle retries the install | + +The panel is not dark during an install: the app paints a static "Installing update" card as its +last frame, and `src/com.rs` keeps the memory-in-pixel glass alive with a software COM wave. + +## RRAM layout + +[`memory.x`](memory.x) must agree with `../obc-fw-nrf54l/build.rs`. + +| Address | Region | Size | +| --- | --- | ---: | +| `0x0000_0000` | `obc-boot` (this crate) | 32 KB | +| `0x0000_8000` | app slot (`obc-fw-nrf54l`) | 1976 KB | +| `0x001F_6000` | `SEMMC_STAGE` (the staged sEMMC blob) | 20 KB | +| `0x001F_B000` | `BOOT_STATE` page | 4 KB | +| `0x001F_C000` | `SETTINGS` page (the app's) | 4 KB | + +## Pitfalls - The image must fit **32 KB** of flash. `firmware/tools/resource_guard.py boot` gates it in CI, against the default shape. The `rtt` feature must never be load-bearing for the budget. diff --git a/firmware/obc-fw-nrf54l/README.md b/firmware/obc-fw-nrf54l/README.md index 24ad34d03d..c27706aec5 100644 --- a/firmware/obc-fw-nrf54l/README.md +++ b/firmware/obc-fw-nrf54l/README.md @@ -16,79 +16,6 @@ each change; blue dots mean unwritten. Settings persist across power cycles. 3. **VCOM hardware flow control → OFF.** `debug-uart` never asserts RTS. With HWFC on, telemetry works but injected fixes and buttons do not. -## Full pin map - -**Port P2 — MCU/fast domain.** All 11 pins are used: the microSD bus and the panel's source bus, -two of them shared. The card's six pads are fixed by Nordic's sEMMC soft peripheral. - -| Pin | sEMMC | Display | Notes | -|-------|-------|---------|--------------------------------------------------------------| -| P2.00 | D3 | **B0** | shared — `CTRLSEL` per mode; internal pull-up in storage mode | -| P2.01 | CLK | — | card only; parked as an input in display mode | -| P2.02 | D0 | — | card only; parked as an input in display mode | -| P2.03 | D2 | — | card only; parked as an input in display mode | -| P2.04 | D1 | **B1** | shared — `CTRLSEL` per mode; internal pull-up in storage mode | -| P2.05 | CMD | — | card only; parked as an input in display mode | -| P2.06 | — | R0 | source data (even-`x` R) | -| P2.07 | — | BCK | source shift clock | -| P2.08 | — | R1 | source data (odd-`x` R) | -| P2.09 | — | G0 | source data (even-`x` G) | -| P2.10 | — | G1 | source data (odd-`x` G) | - -The packed wire word is `DATA_MASK = 0x751` (`B0`→0, `B1`→4, `R0`→6, `R1`→8, `G0`→9, `G1`→10). -`even`/`odd` is 0-based `x`, so the `*0` lines carry `x = 0, 2, 4, …`; the panel datasheet numbers -columns from 1 and calls that same line the *odd* column. - -**Pad configuration per mode** (`src/semmc.rs`, `configure_storage_pads` / `configure_display_pads`): - -| | the six card pads | the four card-only pads | -|---|---|---| -| **storage** | Output, input Disconnect, E0/E1 drive, `CTRLSEL = VPR`, `GPIOHSPADCTRL.BIAS = 2`; internal pull-up on `D3`/`D1` only | same — all six are the card's | -| **display** | `P2.00`/`P2.04` → Output, S drive, no pull, `CTRLSEL = GPIO`, `GPIOHSPADCTRL.BIAS = 2` | Input, no pull, `CTRLSEL = GPIO`; the external pull-ups hold the bus idle-high and the card stays inert | - -`GPIOHSPADCTRL.BIAS` is port-global, not per-pin, so both modes set the same constant 2 -(`semmc::HS_PAD_BIAS`). Only `D3`/`D1` get an internal pull-up, because this desk breakout carries -its own resistors on the rest. **The production board should fit external 10–100 kΩ pull-ups on -`CMD`/`DAT0–3` (none on `CLK`) and turn all internal pulls off.** - -**Port P1 — PERI domain, ≤8 MHz** (gate/BSP, sensors, COM, VCOM and buttons): - -| Pin | Signal | Notes | -|-------|--------------|-----------------------------------------------------------| -| P1.03 | I²C SCL | shared GPS + altimeter + compass bus (TWIM22) | -| P1.04 | I²C SDA | same bus | -| P1.05 | GPS TX-Ready | optional DDC data-ready IRQ (active-high) | -| P1.06 | unused PWM pad | held low; do not wire | -| P1.07 | unused PWM pad | held low; do not wire | -| P1.08 | BTN2 | BACK | -| P1.09 | BTN1 | DOWN | -| P1.10 | GSP | gate start pulse | -| P1.11 | GCK | gate clock | -| P1.12 | GEN | gate enable | -| P1.13 | INTB | frame envelope | -| P1.14 | BSP | source sub-line start (the lone P1 source line) | -| P1.16 | VCOM TX | UARTE20 → host (`debug-uart` builds only) | -| P1.17 | VCOM RX | UARTE20 ← host (`debug-uart` only; needs HWFC OFF) | -| P1.22 | VCOM (COM) | COM electrode, HighDrive | -| P1.23 | VB | COM electrode | -| P1.24 | VA | COM electrode (inverse phase) | -| P1.25 | LED1 | liveness heartbeat | -| P1.26 | BTN0 | UP | -| P1.27 | backlight | **PROVISIONAL** — PWM20 ch0, 1 kHz; also DK LED2 | - -**Ports P0 and P3:** - -| Pin | Signal | Notes | -|-------|---------|-------| -| P0.05 | BTN3 | SELECT; low-power domain | -| P3.00 | piezo A | **PROVISIONAL** — PWM21 ch0; header P5 | -| P3.01 | piezo B | **PROVISIONAL** — PWM21 ch1; header P5; wire a passive piezo between A and B, not to ground | - -**Schematic-time: the backlight gate needs an external pull-down.** P1.27 idles low, but only -after `PanelBacklight::new` has run (`src/panel_power.rs`). Before that, and through all of -`obc-boot`, the pin is input, no pull, high impedance, and System OFF stops the PWM wherever the -waveform left the line. A floating MOSFET gate is not a defined lamp state. - ## Build and flash Flash [`obc-boot`](../obc-boot/README.md) first: run `obc flash-boot` from the @@ -115,6 +42,45 @@ are refused. Set `OBC_DEMO_RIDE_FILE` to an absolute path for another marked dem | `resource-report` | `.obc_resources` for the resource guard; never flash or package this diagnostic image. | | `flat-store-reset` | Destructive `flat_store_bench` maintenance mode. | +## Host input and firmware update + +Flash `debug-uart` and disable VCOM HWFC. From the repository root: + +```sh +obc flash debug-uart +obc uart # optional: path/to/ride.gpx +obc debug # flash and open the feeder +``` + +The feeder sends buttons and sensor values, and displays telemetry. +`--list` shows serial ports. Disconnect J3 if the test also needs BLE. +On macOS, use `cu.usbmodem*133`, not `*131` or `tty.*`. Use pyserial; +`stty` with `printf` or `cat` loses termios settings across opens. + +For a [signed update package](../README.md#firmware-update-images-obcu) in the store, send +`dfu-install\n` to the live CDC port and read the `D` status lines: + +```sh +uv venv /tmp/dfu-venv && uv pip install --python /tmp/dfu-venv/bin/python pyserial +/tmp/dfu-venv/bin/python -c "import serial; p = serial.Serial('/dev/cu.usbmodem<...>133', 115200, \ + rtscts=False, timeout=90); p.write(b'dfu-install\n'); [print(p.readline()) for _ in range(40)]" +``` + +The device resets into `obc-boot` to install it; see its README for LED codes. +Unsigned packages and triggers during recording are refused. If VCOM returns +nothing and RTT has no `dfu:` line, power-cycle the DK to clear the CDC bridge. + +## Flat store bench + +```sh +cargo run --release --bin flat_store_bench +cargo run --release --bin flat_store_bench --features flat-store-reset +``` + +**Both commands destroy the partition table and every object from raw LBA 0.** +The bench refuses another `StoreId` unless `FORCE_REINIT` overrides it. +See the [bench module](src/bin/flat_store_bench.rs) for phases and output. + ## Display and microSD The M33 renders one 240 × 320 RGB222 plane and sends up to 16 dirty-row spans to @@ -202,6 +168,87 @@ Prototype VID/PID is `1209:0001`. MS OS 2.0 descriptors bind WinUSB on Windows. | Plug is not detected | Both VREGUSB interrupt handlers remain bound. | | Enumeration at 12 Mb/s | Full-speed fallback; 512 B bulk descriptors are invalid. | +## Peak View + +The menu appears when the selected map (lowest-ID Map object) has indexed terrain. +See [simulator controls](../../sim/desktop/README.md). Generation uses the shared +128 KiB scratch arena; Back must release it before navigation, rendering or USB. +`obc-bake bake` writes the surface index. Convert standalone native terrain with +`obc-dem surface native.obcd indexed.obcd`. + +## Full pin map + +**Port P2 — MCU/fast domain.** All 11 pins are used: the microSD bus and the panel's source bus, +two of them shared. The card's six pads are fixed by Nordic's sEMMC soft peripheral. + +| Pin | sEMMC | Display | Notes | +|-------|-------|---------|--------------------------------------------------------------| +| P2.00 | D3 | **B0** | shared — `CTRLSEL` per mode; internal pull-up in storage mode | +| P2.01 | CLK | — | card only; parked as an input in display mode | +| P2.02 | D0 | — | card only; parked as an input in display mode | +| P2.03 | D2 | — | card only; parked as an input in display mode | +| P2.04 | D1 | **B1** | shared — `CTRLSEL` per mode; internal pull-up in storage mode | +| P2.05 | CMD | — | card only; parked as an input in display mode | +| P2.06 | — | R0 | source data (even-`x` R) | +| P2.07 | — | BCK | source shift clock | +| P2.08 | — | R1 | source data (odd-`x` R) | +| P2.09 | — | G0 | source data (even-`x` G) | +| P2.10 | — | G1 | source data (odd-`x` G) | + +The packed wire word is `DATA_MASK = 0x751` (`B0`→0, `B1`→4, `R0`→6, `R1`→8, `G0`→9, `G1`→10). +`even`/`odd` is 0-based `x`, so the `*0` lines carry `x = 0, 2, 4, …`; the panel datasheet numbers +columns from 1 and calls that same line the *odd* column. + +**Pad configuration per mode** (`src/semmc.rs`, `configure_storage_pads` / `configure_display_pads`): + +| | the six card pads | the four card-only pads | +|---|---|---| +| **storage** | Output, input Disconnect, E0/E1 drive, `CTRLSEL = VPR`, `GPIOHSPADCTRL.BIAS = 2`; internal pull-up on `D3`/`D1` only | same — all six are the card's | +| **display** | `P2.00`/`P2.04` → Output, S drive, no pull, `CTRLSEL = GPIO`, `GPIOHSPADCTRL.BIAS = 2` | Input, no pull, `CTRLSEL = GPIO`; the external pull-ups hold the bus idle-high and the card stays inert | + +`GPIOHSPADCTRL.BIAS` is port-global, not per-pin, so both modes set the same constant 2 +(`semmc::HS_PAD_BIAS`). Only `D3`/`D1` get an internal pull-up, because this desk breakout carries +its own resistors on the rest. **The production board should fit external 10–100 kΩ pull-ups on +`CMD`/`DAT0–3` (none on `CLK`) and turn all internal pulls off.** + +**Port P1 — PERI domain, ≤8 MHz** (gate/BSP, sensors, COM, VCOM and buttons): + +| Pin | Signal | Notes | +|-------|--------------|-----------------------------------------------------------| +| P1.03 | I²C SCL | shared GPS + altimeter + compass bus (TWIM22) | +| P1.04 | I²C SDA | same bus | +| P1.05 | GPS TX-Ready | optional DDC data-ready IRQ (active-high) | +| P1.06 | unused PWM pad | held low; do not wire | +| P1.07 | unused PWM pad | held low; do not wire | +| P1.08 | BTN2 | BACK | +| P1.09 | BTN1 | DOWN | +| P1.10 | GSP | gate start pulse | +| P1.11 | GCK | gate clock | +| P1.12 | GEN | gate enable | +| P1.13 | INTB | frame envelope | +| P1.14 | BSP | source sub-line start (the lone P1 source line) | +| P1.16 | VCOM TX | UARTE20 → host (`debug-uart` builds only) | +| P1.17 | VCOM RX | UARTE20 ← host (`debug-uart` only; needs HWFC OFF) | +| P1.22 | VCOM (COM) | COM electrode, HighDrive | +| P1.23 | VB | COM electrode | +| P1.24 | VA | COM electrode (inverse phase) | +| P1.25 | LED1 | liveness heartbeat | +| P1.26 | BTN0 | UP | +| P1.27 | backlight | **PROVISIONAL** — PWM20 ch0, 1 kHz; also DK LED2 | + +**Ports P0 and P3:** + +| Pin | Signal | Notes | +|-------|---------|-------| +| P0.05 | BTN3 | SELECT; low-power domain | +| P3.00 | piezo A | **PROVISIONAL** — PWM21 ch0; header P5 | +| P3.01 | piezo B | **PROVISIONAL** — PWM21 ch1; header P5; wire a passive piezo between A and B, not to ground | + +**Schematic-time: the backlight gate needs an external pull-down.** P1.27 idles low, but only +after `PanelBacklight::new` has run (`src/panel_power.rs`). Before that, and through all of +`obc-boot`, the pin is input, no pull, high impedance, and System OFF stops the PWM wherever the +waveform left the line. A floating MOSFET gate is not a defined lamp state. + ## Board connection and recovery `obc board doctor` reports probes, serial owners and USB enumeration on macOS @@ -226,50 +273,3 @@ resetting. Use the exact installed ELF, features and `DEFMT_LOG` for decoding. | J3 does not enumerate | Check native cable, VID/PID and RTT VBUS lines. J4 serial does not prove native USB works. | | J3 enumerates but connection fails | Close the desktop or browser interface owner, then reconnect. | | Restart and capture boot logs | `obc board run ELF --preverify` verifies, resets and streams RTT. It programs only if the installed image differs. `--preverify` applies to `run` and `download`. | - -## Host input and firmware update - -Flash `debug-uart` and disable VCOM HWFC. From the repository root: - -```sh -obc flash debug-uart -obc uart # optional: path/to/ride.gpx -obc debug # flash and open the feeder -``` - -The feeder sends buttons and sensor values, and displays telemetry. -`--list` shows serial ports. Disconnect J3 if the test also needs BLE. -On macOS, use `cu.usbmodem*133`, not `*131` or `tty.*`. Use pyserial; -`stty` with `printf` or `cat` loses termios settings across opens. - -For a [signed update package](../README.md) in the store, send `dfu-install\n` -to the live CDC port and read the `D` status lines: - -```sh -uv venv /tmp/dfu-venv && uv pip install --python /tmp/dfu-venv/bin/python pyserial -/tmp/dfu-venv/bin/python -c "import serial; p = serial.Serial('/dev/cu.usbmodem<...>133', 115200, \ - rtscts=False, timeout=90); p.write(b'dfu-install\n'); [print(p.readline()) for _ in range(40)]" -``` - -The device resets into `obc-boot` to install it; see its README for LED codes. -Unsigned packages and triggers during recording are refused. If VCOM returns -nothing and RTT has no `dfu:` line, power-cycle the DK to clear the CDC bridge. - -## Flat store bench - -```sh -cargo run --release --bin flat_store_bench -cargo run --release --bin flat_store_bench --features flat-store-reset -``` - -**Both commands destroy the partition table and every object from raw LBA 0.** -The bench refuses another `StoreId` unless `FORCE_REINIT` overrides it. -See the [bench module](src/bin/flat_store_bench.rs) for phases and output. - -## Peak View - -The menu appears when the selected map (lowest-ID Map object) has indexed terrain. -See [simulator controls](../../sim/desktop/README.md). Generation uses the shared -128 KiB scratch arena; Back must release it before navigation, rendering or USB. -`obc-bake bake` writes the surface index. Convert standalone native terrain with -`obc-dem surface native.obcd indexed.obcd`. From 9910be753e0c7b030b8e0354e37a829515d3ed49 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 29 Oct 2026 08:12:23 +0000 Subject: [PATCH 06/20] docs(specs): common header and trim for OBCA and OBCT Both specs open with the common header and a Conventions section. The OBCA alignment proof is cut to the statement, its conditions and its corollaries. Rationale, a version-history aside and a repeated tolerance are cut. Every MUST and every table row stays. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DTHu3LHSGY5CY9j2XmscLi --- specs/obca-assembly.md | 515 +++++++++++++++++++---------------------- specs/obct-terrain.md | 149 ++++++------ 2 files changed, 310 insertions(+), 354 deletions(-) diff --git a/specs/obca-assembly.md b/specs/obca-assembly.md index d43cbb98e8..1b00c088a3 100644 --- a/specs/obca-assembly.md +++ b/specs/obca-assembly.md @@ -1,22 +1,29 @@ -# OBCA — OpenBikeComputer Map Assemblies (v1) - -OBCA is the contract that turns a **catalog of pre-baked grid cells** into the map a rider -carries. It defines the cell grid (§1), the alignment theorem that lets an assembler copy a cell's -chunk bytes verbatim (§2), what a baked **cell** is (§3), and what an **assembly** built from cells -must do (§4). - -OBCA introduces **no new OBCM version and changes no OBCM semantics**. Every cell is an ordinary -[OBCM](obcm-map.md) file that today's reader parses unchanged. What OBCA adds is a set of -*constraints* on how those files are produced. - -This document is normative. The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be -interpreted as in RFC 2119. - -Related contracts: [`obcm-map.md`](obcm-map.md) is the byte format of every cell and -assembly; [`obcc-catalog.md`](obcc-catalog.md) is the catalog manifest that publishes cells and names -the selectable regions; [`obct-terrain.md`](obct-terrain.md) is the terrain raster, a second artifact -class on **this same grid** (§1) with its own revision track, carried **inside** the assembled map -(`obcm-map.md` §1.3). +# OBCA map assemblies + +How a catalog of baked grid cells becomes one map: the cell grid (§1), the alignment theorem that +lets an assembler copy a cell's chunk bytes verbatim (§2), what a baked cell is (§3) and what an +assembly built from cells must do (§4). + +- Version: 1. OBCA adds no OBCM version and changes no OBCM semantics. Every cell and every + assembly is an ordinary [OBCM](obcm-map.md) file that the reader parses unchanged; OBCA only + adds constraints on how a producer writes those files. +- Writers: [`host/obc-bake`](../host/obc-bake) (cuts and publishes cells, curated region list); + `host/obcm-assemble` (assemblies, also used by `builder/wasm`). +- Readers: [`firmware/obc-reader`](../firmware/obc-reader) (the device, and every verify pass of + §4.8). +- Code authority: [`host/obc-map-core`](../host/obc-map-core) (`src/grid.rs`: grid, cell + identity, band cell sizes; shared byte framing). +- Related: [OBCM](obcm-map.md) (byte format of every cell and assembly); + [`obcc-catalog.md`](obcc-catalog.md) (the catalog that publishes cells and names the selectable + regions); [`obct-terrain.md`](obct-terrain.md) (the terrain raster: a second artifact class on + **this same grid** (§1) with its own revision track, carried **inside** the assembled map, + `obcm-map.md` §1.3). + +## Conventions + +- The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are as in RFC 2119. +- All coordinates are integer **microdegrees** (µdeg, 1e-6 degrees), the unit of every OBCM + coordinate. --- @@ -24,9 +31,6 @@ class on **this same grid** (§1) with its own revision track, carried **inside* ### 1.1 Origin, world box, and cell sizes -All coordinates in this document are integer **microdegrees** (µdeg, 1e-6 degrees), the unit of -every OBCM coordinate. - ``` GRID_ORIGIN = −268435456 µdeg (= −2^28, on BOTH axes) WORLD_SIDE = 536870912 µdeg (= 2^29) @@ -42,22 +46,19 @@ cell(S, i, j) = [ GRID_ORIGIN + i·S , GRID_ORIGIN + (i+1)·S ) in latitude ``` with `0 ≤ i, j < WORLD_SIDE / S`. Cells of one size tile the world box exactly and without -overlap; cells of different sizes nest, because every permitted `S` divides `2^28` and therefore -divides `GRID_ORIGIN` and `WORLD_SIDE`. +overlap. Cells of different sizes nest, because every permitted `S` divides `2^28` and so divides +`GRID_ORIGIN` and `WORLD_SIDE`. Three properties of this definition are load-bearing: -- **Square in µdeg, not in metres.** An OBCM quadtree halves latitude and longitude *together* - (`obcm-map.md` §4), so a node can only ever coincide with a cell if the cell is square in the - quadtree's own units. Cells are therefore taller than wide on the ground — at 47°N a `2^18` - cell is ≈ 29 km × 20 km. Producers MUST NOT "correct" this with per-latitude sizes; it would - break §2. -- **One origin for every band and every size.** Two bands whose cells did not nest would make the - assembly bbox satisfy two incompatible alignment conditions. -- **A power-of-two origin, not −90/−180.** `−90 000 000` is not a multiple of any candidate cell - size, so a grid anchored there would have no size at which cell boundaries and quadtree - midpoints agree. `−2^28` is divisible by every permitted `S`, and the world box it spans - (≈ ±268°) contains the whole geographic domain. +- **Square in µdeg, not in metres.** An OBCM quadtree halves latitude and longitude together + (`obcm-map.md` §4), so a node can coincide with a cell only if the cell is square in µdeg. + Producers MUST NOT use per-latitude sizes; they break §2. +- **One origin for every band and every size.** Cells of all bands nest, so the assembly bbox has + one alignment condition. +- **A power-of-two origin, not −90/−180.** `−90 000 000` is not a multiple of any permitted cell + size. `−2^28` is divisible by every permitted `S`, and its world box (≈ ±268°) contains the + whole geographic domain. ### 1.2 Bands @@ -68,9 +69,7 @@ property of the **schema digest** (§6), not of this format; the v1 values are Two rules are normative here: - **Coverage.** An assembly of a selection MUST include, per band, exactly those cells of that - band whose square intersects the selection, and MUST NOT include any other. Because coarse bands - use larger cells, the same rule yields precise coverage at fine bands and whole covering cells at - coarse ones. + band whose square intersects the selection, and MUST NOT include any other. - **Partition.** Every ladder LOD MUST belong to exactly one band, and the nav/POI sections to exactly one band. @@ -82,40 +81,40 @@ The canonical textual id of a cell is // e.g. 18/1204/1052 ``` -with `i` and `j` zero-padded to `max(4, decimal_width(WORLD_SIDE / S − 1))` digits — four for every -size at or above `2^16`, which is every band the v1 table uses, and wider for the smaller sizes -§1.1 still permits. Producers MUST widen rather than truncate. Catalog entries (§6) and object paths -use this id, so a cell's URL is derivable from its **band** and its id plus a base URL — the band, -because two bands may share a cell size ([`obcc-catalog.md` §2](obcc-catalog.md)) — while its -**square is derivable from its id alone**, which is why a catalog cell entry carries no bbox -([`obcc-catalog.md` §8](obcc-catalog.md)). +with `i` and `j` zero-padded to `max(4, decimal_width(WORLD_SIDE / S − 1))` digits: four for every +size at or above `2^16` (every band of the v1 table), wider for the smaller sizes §1.1 permits. +Producers MUST widen rather than truncate. + +Catalog entries (§6) and object paths use this id. A cell's URL derives from its **band**, its id +and a base URL; the band is necessary because two bands may share a cell size +([`obcc-catalog.md` §2](obcc-catalog.md)). A cell's **square derives from its id alone**, so a +catalog cell entry carries no bbox ([`obcc-catalog.md` §8](obcc-catalog.md)). Implementations MAY use the packed convenience key `(log2(S) << 24) | (i << 12) | j`, valid for `S ≥ 2^18`. It is not normative and MUST NOT appear in any published artifact. ### 1.4 Domain edges: poles and the antimeridian -The grid is defined over the world box, which is strictly larger than the geographic domain, and -it **does not wrap**. Three consequences are normative: +The grid covers the world box, which is strictly larger than the geographic domain, and it **does +not wrap**. Three consequences are normative: -- **Cells MAY extend past ±90° / ±180°.** A cell's bbox is exactly its grid square, and producers - MUST NOT clamp it: clamping would destroy the power-of-two span that §2 depends on. Coordinates - outside the geographic domain simply carry no features, so the overhang costs empty leaves and - nothing else. Every field involved is an OBCM `int32` µdeg, which holds the whole world box - (`|2^28| < 2^31`), so nothing overflows. **Consumers MUST NOT assume an OBCM header bbox lies - within ±90 000 000 / ±180 000 000** — the reader only ever does bbox intersection on these - values, which is well-defined outside the domain. +- **Cells MAY extend past ±90° / ±180°.** A cell's bbox is exactly its grid square. Producers MUST + NOT clamp it: clamping destroys the power-of-two span of §2. Coordinates outside the geographic + domain carry no features. Every field involved is an OBCM `int32` µdeg, which holds the whole + world box (`|2^28| < 2^31`). **Consumers MUST NOT assume an OBCM header bbox lies within + ±90 000 000 / ±180 000 000.** The reader only does bbox intersection on these values, which is + well-defined outside the domain. - **No cell straddles the antimeridian**, and the columns either side of ±180° are not adjacent on - the grid. An assembly therefore MUST NOT combine cells from both sides: its bbox would have to - span the globe, and no seam unification (§4.6) can ever join a node at `+179.999…°` to one at - `−179.999…°`. A selection crossing the antimeridian is two assemblies. -- **The poles are not special.** The grid rows nearest ±90° overhang the pole; no routable way - reaches the overhang, so no boundary junction (§3.4) can be produced there. + the grid. An assembly MUST NOT combine cells from both sides: no seam unification (§4.6) joins a + node at `+179.999…°` to one at `−179.999…°`. A selection that crosses the antimeridian is two + assemblies. +- **The poles are not special.** The grid rows nearest ±90° overhang the pole. No routable way + reaches the overhang, so no boundary junction (§3.4) occurs there. ### 1.5 The v1 band table (schema `bikepacking`, revision 1) -These values are **schema data**: a catalog states them (§6), a producer reads them from the -catalog, and retuning them is a re-bake rather than a change to this document. +These values are **schema data**: a catalog states them (§6) and a producer reads them from the +catalog. A change to them is a re-bake, not a change to this document. | Band | Cell size | ≈ at 47°N | Carries | | :-- | :-- | :-- | :-- | @@ -140,32 +139,31 @@ Let `S_MAX` be the largest cell size in the schema's band table. An **assembly b ``` that is **grid-aligned**: `A_lat ≡ A_lon ≡ GRID_ORIGIN (mod S_MAX)`, with `2^n ≥ S_MAX` and -`n ≤ 29`. (The span is a power of two and identical on both axes; the *position* need only be -`S_MAX`-aligned, so the box stays tight to the selection.) +`n ≤ 29`. The span is the same power of two on both axes; the position need only be +`S_MAX`-aligned, so the box stays tight to the selection. > **Theorem.** For a quadtree built over a grid-aligned assembly bbox per > [`obcm-map.md` §4](obcm-map.md), and for every band size `S = 2^s` in the table, the nodes at > depth `d = n − s` are **exactly** the cells of size `S` that tile the assembly bbox — same > minimum corner, same span, to the microdegree. -Two corollaries that implementations lean on: +Two corollaries that implementations use: - `n ≥ s` for every band, so the cell depth is never negative — guaranteed by `2^n ≥ S_MAX`. - The four children are emitted **NW, NE, SW, SE**, and the assembler writes its own upper tree (depths `0 .. d`) breadth-first, so the cell → depth-`d` slot mapping is a pure function of `(i, j)` and needs no search. Below the cell depth the layout is per-cell blocks rather than - global breadth-first order, which is legal — see §7. + global breadth-first order, which is legal (§7). -A cell artifact's own header bbox **is** its cell (§3.1), so the cell file's root node is that cell -and its subtree is the same subdivision the assembly performs at that position — identical node -bboxes at every level, to the microdegree. A feature's anchor is stored relative to its **leaf's** -minimum corner (`obcm-map.md` §5.2) and its deltas are relative to the previous vertex, so every -byte of a chunk decodes to the same absolute geometry in the cell file and in the assembly. §4.3 -states what is therefore copied and what must be rebuilt. +A cell file's header bbox is its cell (§3.1), so its subtree has the same node bboxes as the +assembly at that position. A feature's anchor is relative to its **leaf's** minimum corner +(`obcm-map.md` §5.2) and its deltas to the previous vertex, so every chunk byte decodes to the +same absolute geometry in the cell file and in the assembly. §4.3 states what is copied and what +is rebuilt. -Two cosmetic costs are inherent to cutting at cell boundaries and are accepted: dash phase restarts -at a cell edge, and the packer's `merge_fills` / `merge_lines` unions cannot cross a cell boundary, -so an assembly carries slightly more features than a single-shot bake of the same area. +An assembly differs from a single-shot bake of the same area in two accepted ways: dash phase +restarts at a cell edge, and the packer's `merge_fills` / `merge_lines` unions do not cross a cell +boundary, so the assembly carries more features. --- @@ -175,30 +173,29 @@ so an assembly carries slightly more features than a single-shot bake of the sam A baked cell MUST be a complete, valid OBCM file of the catalog's OBCM version, and: -- its **header bbox MUST be exactly its grid cell** (§1.1), not the content-derived box a normal - pack computes, because §2 needs the box to be the grid square; +- its **header bbox MUST be exactly its grid cell** (§1.1, §2), not the content-derived box a + normal pack computes; - it MUST write the **complete ladder** in its LOD table — one entry per schema LOD, in ladder order, with each entry's `Max Meters/Pixel` taken from the schema (so LOD 0 is `+inf` and the - sequence is strictly decreasing, exactly as `obcm-map.md` §3 requires). LODs outside the cell's - band are written **empty**: `Index Node Count = 0`, `Chunk Count = 0`, and the single-`0`-entry - offset table `obcm-map.md` §5.1 mandates for an empty region. A reader walks an empty LOD and - finds nothing, so a cell of any band stays a legal, openable map; + sequence is strictly decreasing, as `obcm-map.md` §3 requires). LODs outside the cell's band are + written **empty**: `Index Node Count = 0`, `Chunk Count = 0`, and the single-`0`-entry offset + table `obcm-map.md` §5.1 mandates for an empty region. A cell of any band is therefore a valid, + openable map; - the POI section and the nav section MUST be present, per `obcm-map.md` §7/§8, and MUST be **empty** unless the cell's band carries them; - both style tables MUST be the schema digest's **canonical table** (§6.2) — right ids, right count, right order, placeholder values — and both marker colors the schema's placeholder. -Because the full ladder is written, **geometry-band membership is not recorded in the cell's -bytes**. It is a property of the schema digest, read from the catalog (§6). A producer MUST NOT -infer a cell's band from which of its LODs happen to be non-empty: a legitimately empty cell — open -sea — is indistinguishable that way. +The cell's bytes do not record its geometry band. Band membership is a property of the schema +digest, read from the catalog (§6). A producer MUST NOT infer a cell's band from which of its LODs +are non-empty: an empty cell (open sea) is indistinguishable that way. ### 3.2 Determinism > **Same source snapshot + same schema digest + same cell ⇒ byte-identical file.** -This is what lets the catalog content-address cells, lets a re-bake be a no-op, and lets two -independently baked neighbours agree on a seam coordinate (§3.4). Producers MUST ensure that: +Two neighbours baked independently depend on this to agree on a seam coordinate (§3.4). Producers +MUST ensure that: - no wall clock, hostname, path, thread count, or map/set iteration order reaches the bytes; - every list written is ordered by a **content-derived** key; @@ -208,13 +205,13 @@ independently baked neighbours agree on a seam coordinate (§3.4). Producers MUS - the schema digest, not the machine, fixes every threshold — simplification tolerances, cull areas, merge passes, and the island-prune threshold of §3.5. -Read "source snapshot" as the whole **source set**: a cell's bytes are a function of (source -snapshot set, schema digest, crop), where the source set is every co-baked extract whose coverage -intersects the cell — cut once from an ingest of exactly that set (§3.7) — and the crop is whatever -box that ingest was reduced to, which drops edge-crossing relations exactly as an extract's own -boundary does. Both belong to the determinism key, and neither may depend on the order the extracts -were named: a producer MUST key its cut plans by the **sorted** source set, so that permuting the -extracts on a command line cannot change a single output byte. +"Source snapshot" means the whole **source set**. A cell's bytes are a function of (source +snapshot set, schema digest, crop). The source set is every co-baked extract whose coverage +intersects the cell, cut once from an ingest of exactly that set (§3.7). The crop is the box that +ingest was reduced to; it drops edge-crossing relations as an extract's own boundary does. Both +belong to the determinism key, and neither may depend on the order the extracts were named: a +producer MUST key its cut plans by the **sorted** source set, so that a permutation of the +extracts on a command line changes no output byte. A bakery MUST record the source extract identity and snapshot date per cell and MUST NOT publish two different byte sequences under one (cell, schema digest, snapshot) triple. @@ -227,9 +224,8 @@ half-open square (§1.1), and clipping is a geometric operation, so the rules ar - a feature wholly inside the cell is written unchanged; - a feature crossing the boundary is **clipped at the edge line**; the clip vertices lie exactly on the edge (integer µdeg), so the two neighbours' clipped pieces meet with no gap and no overlap; -- a polygon clipped by an edge is closed along that edge. The resulting seam is invisible when the - neighbour is present and is a straight edge at the coverage boundary when it is not — which is - the honest rendering of a coverage hole; +- a polygon clipped by an edge is closed along that edge. The seam is invisible when the neighbour + is present and is a straight edge at the coverage boundary when it is not; - a feature reduced to nothing (zero-length line, zero-area polygon) by the clip is dropped; - the per-LOD sub-pixel area cull (`min_area_px`) is applied to the **clipped** geometry, so a polygon may survive in one cell and be culled in its neighbour. The packer never culls lines. @@ -239,12 +235,9 @@ be written twice and drawn twice. ### 3.4 Cutting the navigation graph: deterministic boundary junctions -Every cell edge is a border, so naturally coincident junctions are not enough and proximity is -worse than nothing. - **Boundary junctions.** For every routable way that crosses or touches the cell boundary, a cell -MUST materialise a junction record at the crossing coordinate. The coordinate is computed as -follows, and the computation MUST be used verbatim by both neighbours: +MUST materialise a junction record at the crossing coordinate. Both neighbours MUST use this +computation verbatim: 1. Take the way's polyline as it exists in the **source snapshot**, in source vertex order. 2. For each segment and each cell-edge line it crosses (a constant latitude or longitude `c`): @@ -258,101 +251,91 @@ follows, and the computation MUST be used verbatim by both neighbours: lon = c ``` in exact `i64` arithmetic with banker's rounding, and symmetrically for a line at latitude - `c`. Both neighbours see the same `P`, `Q`, `c` and therefore produce the same integer pair. + `c`. Both neighbours see the same `P`, `Q`, `c` and so produce the same integer pair. 3. A segment exactly **collinear** with an edge line belongs to the cell on the lower side of that line (the cell for which the line is a `min` edge, per the half-open convention), so it is written exactly once. -4. The boundary junction is materialised in **both** adjacent cells, each carrying its own stub - edge inward. This is deliberate duplication: the pair is what §4.6 unifies. +4. The boundary junction is materialised in **both** adjacent cells, each with its own stub edge + inward. §4.6 unifies the pair. **Only boundary-derived and real OSM junction nodes are load-bearing at a seam.** A cell MUST classify junction-ness from the **source snapshot's** way set, not from the ways that survive -inside the cell, and MUST NOT rely on any *interior* synthetic node coinciding with anything. -Interior synthetic nodes are minted by the packer's own edge splits at a midpoint index of the edge -*as that run sees it* (`nav.rs::split_edge`, plus the serializer's `obcm-map.md` §8.4 chunk-fit and -span splits), so two runs over a different set of ways place them metres apart. +inside the cell, and MUST NOT rely on any *interior* synthetic node coinciding with anything. The +packer mints interior synthetic nodes at a midpoint index of an edge as that run sees it +(`nav.rs::split_edge`, plus the serializer's `obcm-map.md` §8.4 chunk-fit and span splits), so two +runs over a different set of ways place them metres apart. -**Exact-coordinate unification only — an epsilon snap is forbidden.** At a cell seam, genuinely -*different* junctions sit as close as **3.9 m**, so a tolerance large enough to be useful would fuse -distinct nodes and invent turns. Producers and assemblers MUST use exact integer equality and MUST -NOT offer a tolerance knob. +**Exact-coordinate unification only — an epsilon snap is forbidden.** At a cell seam, different +junctions sit as close as **3.9 m**, so a useful tolerance would fuse distinct nodes and invent +turns. Producers and assemblers MUST use exact integer equality and MUST NOT offer a tolerance +knob. **Wire limits survive.** Unifying two junctions unions their adjacency and recomputes nothing, so `obcm-map.md` §8.3's degree cap of 24, its `int16` neighbour deltas, and its `uint16` `Cost M` all -hold. Boundary junctions are ordinary degree-2 nodes. An assembler MUST nevertheless re-check the -cap (§4.8) rather than assume it. +hold. Boundary junctions are ordinary degree-2 nodes. An assembler MUST still re-check the cap +(§4.8). ### 3.5 Island pruning at bake time: strictly interior only -A hard cut severs the road network at a line, so a fragment can hold fewer than the schema's -`min_component_edges` in *each* of two cells while being a good road once assembled. If both -neighbours drop their half, no assembly-time work can recover bytes that were never written. -Therefore: +A cut can leave a road fragment below the schema's `min_component_edges` in *each* of two cells +although the assembled road is above it. Therefore: - A cell bake MUST prune only components that are **strictly interior** to the cell — no node of the component lies on the cell boundary — and MUST NOT prune any component touching the boundary, however small. -- The real pruning pass runs at **assembly** time (§4.6), over the merged graph, where component - sizes are finally true. -- `min_component_edges` is a property of the **schema digest**, never of the skin. Two cells - pruned at different thresholds do not assemble into a graph with consistent semantics. +- The real pruning pass runs at **assembly** time (§4.6), over the merged graph. +- `min_component_edges` is a property of the **schema digest**, never of the skin. ### 3.6 POIs and hours -POIs are points, so cell assignment is unambiguous: a POI belongs to the **one** cell whose -half-open square contains its coordinate. A `network`-band cell MUST write every POI in its square -and no other, and MUST carry its own deduplicated hours pool with `HoursRef` values local to the -cell. Both are rebuilt at assembly (§4.5), so a per-cell pool may be locally optimal and globally -redundant. +A POI belongs to the **one** cell whose half-open square contains its coordinate. A `network`-band +cell MUST write every POI in its square and no other, and MUST carry its own deduplicated hours +pool with `HoursRef` values local to the cell. Both are rebuilt at assembly (§4.5), so a per-cell +pool need not be deduplicated across cells. ### 3.7 Provenance and partial cells (D3) -A cell baked from a regional extract is **not** the cell a covering source would produce, because -each extract lacks the side roads that create the neighbour's junctions. So: +A cell baked from a regional extract is **not** the cell a covering source produces: each extract +lacks the side roads that create the neighbour's junctions. So: - Every cell MUST record its **source extent**: the identifier of each source extract it was baked from, and that extract's snapshot date. - A cell whose sources do **not** fully cover its square is **`partial`**. Coverage is decided against the sources' own coverage geometry (for Geofabrik-style extracts, the region polygon - plus its complete-ways overhang), not against the packed content's bbox — content can be - legitimately empty. + plus its complete-ways overhang), not against the packed content's bbox, because content can be + empty. - A catalog MUST mark a partial cell as such (§6.1), and a consumer MUST NOT present a partial cell as canonical coverage. - A bakery MUST replace a partial cell when a covering source becomes available, and MUST NOT - publish a canonical cell and a partial cell for the same (cell, schema digest) pair. Co-baking - a border cell from every extract that touches it is the sanctioned way to make it canonical - without a planet source. + publish a canonical cell and a partial cell for the same (cell, schema digest) pair. Co-baking a + border cell from every extract that touches it makes it canonical without a planet source. ### 3.8 Known-empty coverage A covering source can prove that a band's canonical payload for a cell is empty. OBCC may -represent those cells as compact **known-empty** row ranges rather than as complete empty OBCM -objects. - -- The assertion is per `(schema digest, band, cell)` and carries the same - source-set identities and snapshot dates as an artifact. -- It is canonical coverage: a partial source MUST NOT produce a known-empty - assertion. Absence from both the artifact list and the known-empty ranges - remains a coverage hole. -- A catalog MUST NOT publish an artifact and a known-empty assertion for the - same `(band, cell)`. The assertion contributes zero bytes and no partial flag. -- A consumer includes selected known-empty identities in coverage, hole, and - assembly-bbox arithmetic, but downloads and grafts no bytes for them. At the - cell depth the assembler emits the same empty leaf the corresponding empty - artifact would have contributed. - -An assembly containing no artifact at all has no cell from which to verify the -schema digest's binary style and routing-profile tables. An assembler MUST -refuse that all-known-empty input rather than borrow an unselected artifact as -an implicit metadata source. +represent those cells as **known-empty** row ranges rather than as complete empty OBCM objects. + +- The assertion is per `(schema digest, band, cell)` and carries the same source-set identities + and snapshot dates as an artifact. +- It is canonical coverage: a partial source MUST NOT produce a known-empty assertion. Absence + from both the artifact list and the known-empty ranges remains a coverage hole. +- A catalog MUST NOT publish an artifact and a known-empty assertion for the same + `(band, cell)`. The assertion contributes zero bytes and no partial flag. +- A consumer includes selected known-empty identities in coverage, hole, and assembly-bbox + arithmetic, but downloads and grafts no bytes for them. At the cell depth the assembler emits + the same empty leaf the corresponding empty artifact would have contributed. + +An assembly that contains no artifact has no cell from which to verify the schema digest's binary +style and routing-profile tables. An assembler MUST refuse that all-known-empty input rather than +borrow an unselected artifact as an implicit metadata source. --- ## 4. The assembly contract -An **assembly** is **one** OBCM file built from catalog cells for one selection and two authored skins, -carrying its terrain raster in `obcm-map.md` §1.3's region when the selection has elevation. This -section defines what an assembler does. +An **assembly** is **one** OBCM file built from catalog cells for one selection and two authored +skins, carrying its terrain raster in `obcm-map.md` §1.3's region when the selection has +elevation. This section defines what an assembler does. ### 4.1 Inputs and preconditions @@ -367,36 +350,32 @@ known-empty identities (§3.8). It MUST refuse to proceed if: - any selected cell is missing, and the caller has not accepted the resulting hole; or - any selected cell is `partial`, and the caller has not accepted the reduced coverage. -Missing cells are legal and produce **empty leaves**; the renderer already paints backdrop there, -so a selection with holes is well-formed by construction. Known-empty cells also produce empty -leaves but are explicit canonical coverage, not holes. What is not legal is a *silent* hole. +Missing cells are legal and produce **empty leaves**. Known-empty cells also produce empty leaves +but are explicit canonical coverage, not holes. A *silent* hole is not legal. -**The assembly's size is known before anything is downloaded.** A cell's `bytes` is published per -cell ([`obcc-catalog.md` §8](obcc-catalog.md)) and, for a named region, per band in the catalog root -(`bytes_by_band`, [`obcc-catalog.md` §6](obcc-catalog.md)), and the remaining overheads do not depend on +**The assembly's size is known before any download.** A cell's `bytes` is published per cell +([`obcc-catalog.md` §8](obcc-catalog.md)) and, for a named region, per band in the catalog root +(`bytes_by_band`, [`obcc-catalog.md` §6](obcc-catalog.md)). The other overheads do not depend on content: the header, the schema's style table, the LOD table, the POI and nav directories, and the per-LOD offset tables, whose sizes follow from the cell count. Therefore: - A consumer **MUST** project the file before the download and **MUST refuse the selection** if the projection exceeds the per-file wall, which is **whichever is smaller**: the interior the file's `Offset Scale` covers ([`obcm-map.md` §1.1](obcm-map.md)), and what the consumer's read seam - addresses. It MUST NOT begin fetching an assembly it cannot legally write **or read**. Both walls - are stated by reference rather than as literals, because a number copied into this section would - go stale against either. + addresses. It MUST NOT begin to fetch an assembly it cannot legally write **or read**. - A producer SHOULD warn above **seven eighths** of that wall, and the refusal MUST name the - coverage as the thing to reduce. One file has no split, so coverage is the only lever. + coverage as the thing to reduce. One file has no split. - An assembler **MUST** fail rather than emit an over-size file, and MUST NOT "solve" one by splitting the nav graph, dropping POIs, or degrading coverage silently. -- §4.8's verify re-checks the file's actual size, so the projection is bounded on both ends: - refused before the fetch, and re-asserted before the write. +- §4.8's verify re-checks the file's actual size before the write. The terrain raster is projected the same way. Its size is `32 + 4 · rows · cols + present · T² · 512` — a header, a directory over the assembly rectangle, and -one fixed-size block per square that has a published object — and every term is known from the +one fixed-size block per square that has a published object. Every term is known from the catalog: the rectangle from the assembly bbox and the terrain block's `cell_log2`, the block length from the `posting_log2`/`cell_log2` pairing (`obct-terrain.md` §3.2), and which squares are present from the pinned terrain index. A consumer MUST show the raster's bytes as their own figure rather -than folding them into the map's ([`obcc-catalog.md` §13.3](obcc-catalog.md)). +than fold them into the map's ([`obcc-catalog.md` §13.3](obcc-catalog.md)). ### 4.2 Choosing the assembly bbox @@ -409,13 +388,12 @@ The assembler computes the minimal grid-aligned box (§2.1) containing every sel `A_lon + 2^n ≥ B.max_lon`. 4. The assembly bbox is `[A_lat, A_lat + 2^n) × [A_lon, A_lon + 2^n)`, written into the header. -The box is square, so a selection that is much wider than tall (or vice versa) is padded with -empty leaves; that costs one `uint32` per empty node and nothing else. The assembler MUST NOT -shrink the box to the content afterwards — that would destroy the alignment the whole scheme rests -on. +The box is square, so a selection much wider than tall (or the opposite) is padded with empty +leaves, at one `uint32` per empty node. The assembler MUST NOT shrink the box to the content +afterwards: that destroys the alignment of §2. -The box MAY extend past the far edge of the world box, which is treated exactly like §1.4's domain -overhang: the cells out there do not exist, so their leaves are empty. It MUST NOT extend past the +The box MAY extend past the far edge of the world box, which is treated like §1.4's domain +overhang: the cells there do not exist, so their leaves are empty. It MUST NOT extend past the `int32` µdeg range, which `n ≤ 29` already guarantees. ### 4.3 Copied verbatim vs rebuilt @@ -432,17 +410,17 @@ overhang: the cells out there do not exist, so their leaves are empty. It MUST N | **POI section + hours pool** | Rebuilt (§4.5). | | **Nav section** (directory, node tree, node chunks, edge pool) | Rebuilt (§4.6). | | **Profile table** (`obcm-map.md` §8.6) | Copied from the cells after checking every cell agrees; it is schema data. | -| **Terrain cell blocks** ([`obct-terrain.md`](obct-terrain.md) §3.2) | **Verbatim**, into a fresh directory over the assembly rectangle. Terrain assembly is *placement*, not grafting: the lattice is global and half-open, so two neighbouring cells already agree about every sample and there is nothing to relocate, re-index or unify. | +| **Terrain cell blocks** ([`obct-terrain.md`](obct-terrain.md) §3.2) | **Verbatim**, placed into a fresh directory over the assembly rectangle. Nothing is relocated, re-indexed or unified: the lattice is global and half-open, so neighbouring cells already agree about every sample. | An assembler MUST NOT decode a geometry chunk in the normal path. Decoding is for verification (§4.8). The raster's cell rectangle **is** the assembly bbox: a terrain cell no larger than the assembly square tiles it exactly, because the assembly corner is congruent to `GRID_ORIGIN` modulo `S_MAX` -and therefore modulo the terrain cell too. An assembler MUST refuse a terrain `cell_log2` larger -than the assembly's `span_log2` rather than overhang the box or grow it (§4.2 forbids growing it). -Squares the selection covers but the catalog does not publish an object for — canonically void -ocean ([`obcc-catalog.md` §13.6](obcc-catalog.md)), or ground outside the dataset — are directory `0`. +and so modulo the terrain cell too. An assembler MUST refuse a terrain `cell_log2` larger than the +assembly's `span_log2` rather than overhang the box or grow it (§4.2 forbids growing it). Squares +the selection covers but for which the catalog publishes no object — canonically void ocean +([`obcc-catalog.md` §13.6](obcc-catalog.md)), or ground outside the dataset — are directory `0`. ### 4.4 Grafting geometry, per LOD @@ -457,10 +435,9 @@ For each ladder LOD `L`, with band size `S = 2^s` and cell depth `d = n − s`: 3. Append each present cell's relocated subtree, then its offset-table entries, then its chunk bytes. 4. `Chunk Size` for the LOD is the schema's value; the assembler MUST verify every copied offset - pair still spans at most that (`obcm-map.md` §5.1) — a cell that violated it would poison the - assembly. + pair still spans at most that (`obcm-map.md` §5.1). -A cell whose LOD `L` region is empty contributes an empty leaf, exactly like an absent cell. +A cell whose LOD `L` region is empty contributes an empty leaf, like an absent cell. ### 4.5 Merging POIs @@ -490,47 +467,43 @@ content precedence and bounded source-copy checks. Landmark queries remain separ ### 4.6 Merging the navigation graph -This is the most involved rebuild, and its order matters. +The steps run in this order. -1. **Read the serialized node set.** Walk each `network` cell's `obcm-map.md` §8 node quadtree through a real - reader and collect junction records, keyed by `Node Id` (`obcm-map.md` §8.2's bin-packing - means one leaf walk can yield a record more than once, so the collection MUST be idempotent). - The set to renumber is the **serialized** one, not the one a graph builder would produce: the - serializer mints further synthetic degree-2 junctions after the builder finishes, and they are - in the bytes. +1. **Read the serialized node set.** Walk each `network` cell's `obcm-map.md` §8 node quadtree + through a real reader and collect junction records, keyed by `Node Id` (`obcm-map.md` §8.2's + bin-packing means one leaf walk can yield a record more than once, so the collection MUST be + idempotent). Renumber the **serialized** set, not the set a graph builder produces: the + serializer mints more synthetic degree-2 junctions after the builder finishes, and they are in + the bytes. 2. **Unify seam nodes, and only seam nodes.** Two records unify iff their coordinates are **exactly** equal *and* the coordinate lies on a boundary line of the `network` band's grid (a latitude or longitude congruent to `GRID_ORIGIN` modulo that band's cell size). Unification - unions their adjacency. Restricting to boundary lines is not an optimisation: whole-map - coordinate keying would also fuse the *interior* coordinate collisions that exist in a single - file — vertically stacked bridge/tunnel junctions — inventing a turn between a bridge and the - road beneath it. + unions their adjacency. Without the boundary-line condition, the interior coordinate collisions + of a single file (stacked bridge/tunnel junctions) would fuse and invent a turn between a bridge + and the road beneath it. 3. **Deduplicate adjacency** keyed on `(unified endpoint pair, Cost M, Way Kind, edge polyline)`. - The distinction that matters at a unified boundary junction: the two stubs meeting there run in - *opposite* directions and are different edges, so both MUST survive; only an edge two cells both - wrote in full — which the half-open ownership rules of §3.3 and §3.4(3) should already prevent — - collapses to one. + At a unified boundary junction the two stubs run in *opposite* directions and are different + edges, so both MUST survive. Only an edge that two cells both wrote in full collapses to one; + the half-open ownership rules of §3.3 and §3.4(3) should already prevent that. 4. **Prune islands** over the merged graph with the schema's `min_component_edges`, keeping the largest component plus every component at or above the threshold — the pass §3.5 deferred from - bake time. This is the only place where the threshold means what it says: an island in the - *map*, not in a *cell*. "Largest" is by node count, then by edge count, and — because two - components can tie on both while only one of them can be kept — then by the component holding - the **lowest-numbered node** of the collection order §4.6.1 read the cells in. The tie-break MUST + bake time. "Largest" is by node count, then by edge count, then by the component that holds the + **lowest-numbered node** of the collection order §4.6.1 read the cells in. The tie-break MUST be a property of the graph rather than of the search that found the components, or two assemblers of the same cells disagree about which islet reached the map. 5. **Renumber** the surviving nodes densely from 0, in a deterministic order (`(lat, lon)` ascending is sufficient and content-derived). 6. **Rebuild the edge pool.** `Edge Id` names a record by its chunk and its position within that - chunk (`obcm-map.md` §8.4 — it was a pool byte offset before OBCM v14, and either way it is a - property of *placement*), so every edge record is re-emitted and every `Edge Id` re-derived. Edge polyline bytes MAY be copied from the - source record (they are self-contained: absolute anchor plus deltas), but their *placement* is - new, and the no-straddle rule must be re-applied at the 512-byte chunk granularity. + chunk (`obcm-map.md` §8.4), so it depends on placement: every edge record is re-emitted and + every `Edge Id` re-derived. Edge polyline bytes MAY be copied from the source record (they are + self-contained: absolute anchor plus deltas), but their *placement* is new, and the no-straddle + rule must be re-applied at the 512-byte chunk granularity. 7. **Re-check the wire limits** (§4.8) and rebuild the node quadtree over the assembly bbox, with `obcm-map.md` §8.2 bin-packed 512-byte node chunks. 8. **Copy the profile table** after confirming every cell's is identical. An assembler MUST NOT create an edge between two nodes that no single cell joined. Unification only -ever joins *through* a coincident junction, so a merged route that steps between two nodes sharing +joins *through* a coincident junction, so a merged route that steps between two nodes that share no source cell is a bug. ### 4.7 Stamping the skins @@ -546,24 +519,22 @@ z-index and priority, plus a marker color. Each assembly stamps one light and on other seven bytes of each 8-byte record (`obcm-map.md` §2); - write both marker colors into the header. -A paired restyle changes only the two small tables and marker fields. Styles may change drawing order freely. An -assembler MUST reject a skin that does not cover every id in the schema's table, and MUST reject one -that names a feature type the schema does not have: silently defaulting a missing style would ship a -map with an invisible layer. +A paired restyle changes only the two style tables and the marker fields. Styles may change drawing +order freely. An assembler MUST reject a skin that does not cover every id in the schema's table, +and MUST reject one that names a feature type the schema does not have. ### 4.8 Verify obligations -An assembly is self-made and outside the catalog's guarantees, so it MUST be verified before it is -written to a device. The verify runs through the **real reader** — the same crate the firmware -uses — and MUST cover: +An assembly is outside the catalog's guarantees, so it MUST be verified before it is written to a +device. The verify runs through the **real reader** — the same crate the firmware uses — and MUST +cover: -1. **Parse.** Header (magic, version, bbox), both style tables, LOD table, POI directory, nav directory - and profile table all parse and validate. +1. **Parse.** Header (magic, version, bbox), both style tables, LOD table, POI directory, nav + directory and profile table all parse and validate. 2. **Every chunk, every feature.** Walk each non-empty LOD's quadtree and decode every feature of - every chunk. Any malformed, truncated, or capacity-exceeded outcome fails the assembly. This is - the gate that catches a mis-relocated index or a bad offset base, because a wrong `node_bbox` - produces geometry in the wrong place *and* an anchor that no longer fits, and a wrong chunk - base produces a stream that never meets its `0xFF` sentinel. + every chunk. Any malformed, truncated, or capacity-exceeded outcome fails the assembly. This + catches a mis-relocated index (wrong `node_bbox`, so an anchor no longer fits) and a bad offset + base (a stream that never meets its `0xFF` sentinel). 3. **Offset-table invariants** of `obcm-map.md` §5.1 for every chunk: monotone offsets, in-region end, and span ≤ `Chunk Size`. 4. **Nav integrity.** Every neighbour entry's `Neighbor Id` resolves to a record in the same file; @@ -571,48 +542,45 @@ uses — and MUST cover: endpoints' coordinates; both directions of an edge agree on `Edge Id`, `Cost M`, and `Way Kind`; every `int16` neighbour delta reconstructs the neighbour's stored coordinate. 5. **Nav reachability, as a report.** Emit the merged component histogram. An assembler SHOULD - surface a selection whose largest component covers an implausibly small share of the graph, - because that is what a broken seam looks like; it MUST NOT silently repair it. + surface a selection whose largest component covers an implausibly small share of the graph (the + sign of a broken seam); it MUST NOT silently repair it. 6. **The file answers for itself.** Two properties of the file alone: its header's `Offset Scale` - MUST cover its length (`obcm-map.md` §1.1), and its length MUST equal what the pre-download - projection said it would be (§4.1). An assembler MUST refuse rather than emit a file past the - interior that scale addresses, and MUST apply that refusal **at plan time**, because every byte - of an assembly is computable before any of it is written. -7. **Digests.** SHA-256 of the assembled file, and of nothing inside it. In particular the - **terrain region has no digest of its own**: it is a run of bytes inside the map, and the map has - one identity. Step 8's read-back is what proves the raster is the bytes the catalog served. + MUST cover its length (`obcm-map.md` §1.1), and its length MUST equal the pre-download + projection (§4.1). An assembler MUST refuse rather than emit a file past the interior that scale + addresses, and MUST apply that refusal **at plan time**, because every byte of an assembly is + computable before any of it is written. +7. **Digests.** SHA-256 of the assembled file, and of nothing inside it. The **terrain region has + no digest of its own**: the map has one identity. Step 8's read-back proves the raster is the + bytes the catalog served. 8. **The terrain region** — the raster, spliced into `obcm-map.md` §1.3. Every input is checked - **before its bytes are copied**, because a bad cell must not reach the map even to be caught on - the way out: each downloaded object parses as an OBCT container through the real reader - (`obct-terrain.md` §4.5 — magic, version, flags, the posting/cell pairing, the rectangle against the - world grid, and every directory entry against the file's own length, which is what rejects a - truncated download and an out-of-bounds offset); its header's `Posting Log2` and `Cell Log2` equal - the catalog's terrain block ([`obcc-catalog.md` §13.1](obcc-catalog.md)); it is the `1 × 1` container at - exactly its own id that §13.1 requires of a *published cell*; and its SHA-256 equals the one the - pinned terrain index published. The written region is then read back through the same reader — - through `obcm-map.md` §1.3's window, once the raster lives inside the map — and - every present cell's block MUST equal the block of the object it came from, with every square the - assembly did not receive at directory `0`. - -A failure at any step MUST abort the whole assembly. A partially written map is not a degraded map; -it is an unmountable one, which is the correct outcome. The atomicity is the flat store's commit -(`flat-store-format.md` §5). + **before its bytes are copied**: each downloaded object parses as an OBCT container through the + real reader (`obct-terrain.md` §4.5 — magic, version, flags, the posting/cell pairing, the + rectangle against the world grid, and every directory entry against the file's own length); its + header's `Posting Log2` and `Cell Log2` equal the catalog's terrain block + ([`obcc-catalog.md` §13.1](obcc-catalog.md)); it is the `1 × 1` container at exactly its own id + that §13.1 requires of a *published cell*; and its SHA-256 equals the one the pinned terrain + index published. The written region is then read back through the same reader — through + `obcm-map.md` §1.3's window, once the raster lives inside the map — and every present cell's + block MUST equal the block of the object it came from, with every square the assembly did not + receive at directory `0`. + +A failure at any step MUST abort the whole assembly. A partially written map is unmountable, which +is the correct outcome. The atomicity is the flat store's commit (`flat-store-format.md` §5). --- ## 6. Catalog, schema, and skins -The catalog contract is [`obcc-catalog.md`](obcc-catalog.md); `schema_version 3` there carries cells, -bands, schemas, skins, and region cell-sets. This section states only the parts that are OBCA's to -define. +The catalog contract is [`obcc-catalog.md`](obcc-catalog.md); `schema_version 3` there carries +cells, bands, schemas, skins, and region cell-sets. This section states only the parts that OBCA +defines. ### 6.1 What the catalog must say about a cell For each published cell: its id (§1.3), its band, its schema digest, its OBCM version read from its own header, its size, its SHA-256, its URL, its source extents and snapshot dates, and whether it is **`partial`** (§3.7). A consumer must be able to price an assembly — cell count and total -bytes, **per band** — from the manifest alone, before fetching anything, which is what makes §4.1's -projection arithmetic rather than estimation. +bytes, **per band** — from the manifest alone, before it fetches anything (§4.1). ### 6.2 Schema owns ids; skin owns values @@ -621,30 +589,28 @@ per level), simplification tolerances, cull thresholds, the merge passes, `Chunk routing profile table, `min_component_edges`, and the **band table** (§1.2). All of it is baked into chunk bytes, so it is the identity of a cell store. -Critically, the schema also fixes the **style-id assignment**: `obc-pack` numbers feature types -`1`-based in config document order (`obcm-map.md` §2), and those ids are referenced by every -feature header in every chunk. A schema digest therefore has one **canonical style assignment** — -one id per feature type, in one order. The two skins may change the other seven bytes of each record -and their marker colors, and nothing else (§4.7). +The schema also fixes the **style-id assignment**: `obc-pack` numbers feature types `1`-based in +config document order (`obcm-map.md` §2), and every feature header in every chunk references those +ids. A schema digest therefore has one **canonical style assignment** — one id per feature type, in +one order. The two skins may change the other seven bytes of each record and their marker colors, +and nothing else (§4.7). The hosted catalog has **exactly one** schema: the 14-LOD bikepacking ladder. Hosted "presets" are -therefore skins. Custom schemas remain a local-bake affair for the desktop app. +therefore skins. Custom schemas are local bakes in the desktop app. ### 6.3 Lockstep and the bake guard An OBCM version bump or a schema digest change invalidates **every** cell, because assembly copies -chunk bytes between files and that is only meaningful under one schema. The bakery's guard MUST -refuse to publish a catalog that mixes OBCM versions or schema digests across cells, exactly as -`obcc-catalog.md` §10 already refuses a mixed-version artifact catalog, and an assembler MUST refuse a -mixed input set (§4.1). +chunk bytes between files, which is valid only under one schema. The bakery's guard MUST refuse to +publish a catalog that mixes OBCM versions or schema digests across cells, as `obcc-catalog.md` §10 +refuses a mixed-version artifact catalog, and an assembler MUST refuse a mixed input set (§4.1). --- ## 7. Worked example A two-cell assembly at a toy scale: one band, `S_MAX = S_fine = 2^18 = 262144`, and a two-LOD -ladder both of whose levels sit in that band. The two cells are neighbours in longitude, on the -Rhine above Basel: +ladder both of whose levels sit in that band. The two cells are neighbours in longitude: ``` cell A = 18/1204/1052 : lat [47185920, 47448064) lon [7340032, 7602176) @@ -677,7 +643,7 @@ node 4 SE = lat [47185920, 47448064) lon [7602176, 7864320) → cell B Nodes 3 and 4 *are* cells A and B, to the microdegree. -**Grafting LOD 1.** Say cell A's LOD 1 index is five nodes — a root branch plus its four children, +**Grafting LOD 1.** Cell A's LOD 1 index is five nodes — a root branch plus its four children, two of which hold chunks — ``` @@ -715,10 +681,9 @@ branch delta is `block_base − 1` for a block starting at `block_base` (A's nod The offset table is A's entries verbatim followed by B's shifted by A's total: `[0, 1200, 2000, 3500]` for three chunks. The 3 500 bytes of chunk payload are one `memcpy` per -cell. **No feature was decoded**, because every anchor is relative to a leaf whose bbox is -bit-identical in the cell file and in the assembly. +cell. No feature is decoded. -Note the layout is breadth-first only down to the cell depth; below it, each cell's subtree is a +The layout is breadth-first only down to the cell depth; below it, each cell's subtree is a contiguous block. That is legal: `obcm-map.md` §4's reader contract requires only that a branch's four children be contiguous, in NW/NE/SW/SE order, and at a **higher** index than the branch, all of which a per-cell block satisfies. An assembler MUST NOT be required to re-interleave cell @@ -726,27 +691,21 @@ subtrees into global breadth-first order, and a consumer MUST NOT assume global order. **The seam.** A road crosses the shared edge at `lon = 7602176`. Both cells cut it there and both -materialise a junction at the same integer pair (§3.4) — cell A ends its stub there, cell B begins -its stub there. At assembly the two records unify by exact coordinate, which they are eligible for -because the coordinate lies on a fine-band boundary line; their adjacency unions into one ordinary -degree-2 junction and the road is continuous. A junction 3.9 m away on either side is a -*different* junction and stays separate — which is exactly why there is no tolerance knob. +materialise a junction at the same integer pair (§3.4): cell A ends its stub there, cell B begins +its stub there. At assembly the two records unify by exact coordinate, because the coordinate lies +on a fine-band boundary line. Their adjacency unions into one ordinary degree-2 junction and the +road is continuous. --- ## 8. Where this lives +Writers, readers and the grid code are in the header. Also: + - The map byte format every cell and assembly is an instance of: [`obcm-map.md`](obcm-map.md); its code authority [`firmware/obc-formats/src/obcm.rs`](../firmware/obc-formats/src/obcm.rs). -- The catalog that publishes cells, bands, schemas, skins, and region cell-sets: - [`obcc-catalog.md`](obcc-catalog.md). - The drawing quadtree and feature encoding: [`host/obc-draw`](../host/obc-draw). - The navigation graph and its encoding: [`host/obc-network`](../host/obc-network). -- Shared grid and byte framing: [`host/obc-map-core`](../host/obc-map-core). -- The reader every verify pass (§4.8) and the device itself run: - [`firmware/obc-reader`](../firmware/obc-reader). -- The bakery that cuts and publishes cells, and the curated region list that names the - selections: [`host/obc-bake`](../host/obc-bake). - The conceptual tour, with diagrams: the docs site's [data formats](../docs/content/software/formats.md) page. diff --git a/specs/obct-terrain.md b/specs/obct-terrain.md index 6da3bfd81d..cfee26409b 100644 --- a/specs/obct-terrain.md +++ b/specs/obct-terrain.md @@ -1,28 +1,31 @@ -# OBCT — OpenBikeComputer Terrain Tiles (v1 and v3) - -Version 3 adds the geographic surface index in section 8. Sections 1–7 describe the native -v1 raster, which remains the base height plane of v3. - -OBCT is the terrain artifact: a raster of ground heights on the [OBCA](obca-assembly.md) cell grid. It -defines the sample lattice (§1), the 512-byte tile and the terrain cell (§2, §3), the container -(§4) and the sampling rules (§5). - -Terrain is a separate artifact class with its own revision track. An assembly's raster is embedded -**verbatim** in the map file's terrain region ([`obcm-map.md` §1.3](obcm-map.md)), which the map -reader hands over as an opaque window rather than parsing. - -This document is normative. The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be -interpreted as in RFC 2119. - -Its code authority for magic, version, field offsets, fixed lengths, sentinels and the layout -arithmetic is [`firmware/obc-formats/src/obct.rs`](../firmware/obc-formats/src/obct.rs); producers -and consumers import those facts directly rather than transcribing them. The reference reader, -sampler and tile cache are [`firmware/obc-elevation`](../firmware/obc-elevation). - -Related contracts: [`obca-assembly.md`](obca-assembly.md) §1 defines the grid this raster sits on; -[`obcc-catalog.md`](obcc-catalog.md) §13 publishes terrain cells as a catalog artifact class with its own -revision track; [`obcm-map.md`](obcm-map.md) is the map that carries an assembly's raster, and -whose §8 nav graph stores the ascent integrated *from* these samples. +# OBCT terrain tiles + +A raster of ground heights on the [OBCA](obca-assembly.md) cell grid: the sample lattice (§1), the +512-byte tile and the terrain cell (§2, §3), the container (§4), the sampling rules (§5), the +geographic surface index (§8) and the crest-lift producer rule (§9). + +- Version: `0x01` and `3`. Both are current. §1–§5 define the native v1 raster, which is also the + base height plane of v3; §8 defines what v3 adds and overrides. A v1 reader refuses every + version other than `0x01` (§4.5). Every reader rejects version 2 (§8.1). +- Writers: `host/obc-dem` (`obc-dem bake`, cell blocks); `host/obc-bake` (terrain stage, published + cells); `host/obcm-assemble` (an assembly's raster, `obca-assembly.md` §4.3). +- Readers: [`firmware/obc-elevation`](../firmware/obc-elevation) (reference reader, sampler and + tile cache); `host/obcm-assemble` (verify, `obca-assembly.md` §4.8). +- Code authority: [`firmware/obc-formats/src/obct.rs`](../firmware/obc-formats/src/obct.rs) + (magic, version, field offsets, fixed lengths, sentinels, layout arithmetic); `obct_surface.rs` + for §8. Producers and consumers import these facts and do not transcribe them. +- Test vectors: `vectors/terrain-shard.obcd` (§5.6). +- Related: [`obca-assembly.md`](obca-assembly.md) §1 defines the grid; + [`obcc-catalog.md`](obcc-catalog.md) §13 publishes terrain cells as a catalog artifact class with + its own revision track; [`obcm-map.md`](obcm-map.md) carries an assembly's raster **verbatim** + in its terrain region ([`obcm-map.md` §1.3](obcm-map.md)), which the map reader hands over as an + opaque window and does not parse. The OBCM §8 nav graph stores the ascent integrated *from* + these samples. + +## Conventions + +- The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are as in RFC 2119. +- All multi-byte integers are **little-endian**. --- @@ -31,7 +34,8 @@ whose §8 nav graph stores the ascent integrated *from* these samples. ### 1.1 Origin and posting All coordinates are integer **microdegrees** (µdeg, 1e-6 degrees), the unit of every OBCM -coordinate. The lattice is anchored on the OBCA grid origin ([`obca-assembly.md` §1.1](obca-assembly.md)): +coordinate. The lattice is anchored on the OBCA grid origin +([`obca-assembly.md` §1.1](obca-assembly.md)): ``` GRID_ORIGIN = −268435456 µdeg (= −2^28, on BOTH axes) @@ -47,26 +51,23 @@ lon(j) = GRID_ORIGIN + j·P ``` with `0 ≤ i, j < WORLD_SIDE / P`. The same expression applies on **both** axes: the lattice is -square in µdeg, not in metres. +square in µdeg, not in metres. A producer or consumer MUST NOT correct this for latitude: a shift +derives the lattice index from the coordinate. -A `2^9` posting is therefore ≈ 57 m in latitude everywhere and ≈ 39 m in longitude at 47°N, -narrowing towards the poles. This is not corrected anywhere and MUST NOT be: the contract rests on -the lattice being derivable from the coordinate by a shift. - -The lattice does not wrap, and it spans the world box rather than the geographic domain — the +The lattice does not wrap, and it spans the world box rather than the geographic domain. The antimeridian and pole rules of [`obca-assembly.md` §1.4](obca-assembly.md) apply unchanged. ### 1.2 Values -A sample is a signed 16-bit little-endian integer: +A sample is a signed 16-bit integer: | Value | Meaning | | :-- | :-- | | `-32767 … 32767` | height in **whole metres**, orthometric (EGM2008) | | `-32768` (`i16::MIN`) | **`NODATA`** — no height is known at this sample | -Heights are orthometric — height above the geoid — **not** ellipsoidal. A producer MUST NOT write -`-32768` as a real height. +Heights are above the geoid, **not** ellipsoidal. A producer MUST NOT write `-32768` as a real +height. A sample is a **point sample** of the source at the lattice node, with one exception. At a crest node the sample MAY instead be the maximum reference ground height inside the node's half-posting @@ -83,8 +84,8 @@ values are | Posting `P` | `2^9` µdeg | 57 × 39 m | | Terrain cell | `2^19` µdeg | 58 × 40 km — 1024 × 1024 samples, 2 MiB raw | -They are the bakery's choice, published in the catalog. Retuning them is a terrain re-bake, -**not** a version bump of this format (the [`obca-assembly.md` §1.5](obca-assembly.md) idiom). +The bakery chooses them and publishes them in the catalog. A change to them is a terrain re-bake, +**not** a version bump of this format (as in [`obca-assembly.md` §1.5](obca-assembly.md)). A reader MUST accept any pairing this document permits (§4.5), not only the v1 one. @@ -101,8 +102,8 @@ tile bytes[ (row · 16 + col) · 2 ] … +2 row, col ∈ 0..16, little-endi Row-major, and **rows advance latitude**: `row` steps north by one posting, `col` steps east by one posting. The 32 bytes of one row are 16 consecutive longitudes at one latitude, and the tile's first -sample is its **minimum** corner in both axes. A north-up source scanline, such as a GeoTIFF, is -flipped by the baker on the way in; a consumer never flips anything. +sample is its **minimum** corner in both axes. The baker flips a north-up source scanline (such as +a GeoTIFF) on input; a consumer never flips anything. One tile spans `16·P` µdeg on each axis — at the v1 posting, `2^13` µdeg ≈ 910 m of latitude. @@ -158,7 +159,7 @@ reserved for a future per-tile encoding. ### 3.3 Addressing a sample Given a lattice sample `(i, j)` inside a present cell whose block starts at absolute offset -`base`, its byte offset is a chain of shifts and masks — no division, no table, no search: +`base`, its byte offset is a chain of shifts and masks: ``` li = i − ci·S/P lj = j − cj·S/P (sample within the cell) @@ -182,8 +183,6 @@ offset = base + (ti·T + tj)·512 + (r·16 + c)·2 ... ``` -All multi-byte integers are **little-endian**. - The same layout is both published artifacts: - a **terrain cell** — what a bakery publishes and a catalog names — is a container whose cell @@ -212,8 +211,8 @@ There is no separate cell format. The header carries **no bounding box**: the cell rectangle *is* the bounding box (see [`obcc-catalog.md` §8](obcc-catalog.md), and §13.1 for a terrain cell's own entry). -`Flags` is this format's extension point: a future per-tile packed encoding (§3.2) sets a bit here. -A v1 reader MUST refuse a file with any bit set. +`Flags` is this format's extension point (§3.2, §8.1). A v1 reader MUST refuse a file with any bit +set. A reader follows the `Directory Offset` field rather than the fixed v1 value. @@ -263,9 +262,9 @@ A terrain artifact's file extension is **`.obcd`** (8.3: `.OBD`), *not* `.obct`: recorded ride log already uses `.obct` ([`obc-formats/src/track.rs`](../firmware/obc-formats/src/track.rs)). The magic stays `OBCT`. -An assembly's raster is not a file on the card at all: it is spliced into the map file's terrain -region ([`obcm-map.md` §1.3](obcm-map.md)). A **published cell** is an object of its own, named -by the catalog. +An assembly's raster is not a file on the card: it is spliced into the map file's terrain region +([`obcm-map.md` §1.3](obcm-map.md)). A **published cell** is an object of its own, named by the +catalog. --- @@ -273,7 +272,7 @@ by the catalog. This section is normative and exhaustive. **Two independent implementations MUST produce bit-identical results for the same `(lat, lon)` against the same file.** Every step below is -integer arithmetic for that reason. +integer arithmetic. ### 5.1 The algorithm @@ -319,8 +318,8 @@ permitted posting) and MUST NOT be evaluated in floating point. h = num ≥ 0 ? (num + P²/2) / P² : −((−num + P²/2) / P²) (truncating division) ``` -Since no corner is `NODATA` at this point, `num / P²` is a weighted mean of values in -`−32767 … 32767`, so the result always fits `int16`. +No corner is `NODATA` here, so `num / P²` is a weighted mean of values in `−32767 … 32767` and +the result always fits `int16`. ### 5.3 Cell seams and coverage edges @@ -339,10 +338,10 @@ Step 3 applies to **absence only**. A failed read — of a directory entry, of a absence, and an implementation MUST NOT let one fall into the clamp: it makes the whole sample `None` per §5.1. -Clamping is the coverage-edge rule. It makes the surface flatten over the last half posting at the -outer boundary of coverage instead of jumping to `None`. Step 3 fires for a hole *inside* the -rectangle too: terrain plateaus for at most one posting as it approaches a hole, and then step 3 of -§5.1 makes the hole itself `None`. +Clamping is the coverage-edge rule: the surface is flat over the last half posting at the outer +boundary of coverage and does not jump to `None`. Step 3 also applies at a hole *inside* the +rectangle: terrain is flat for at most one posting next to the hole, and §5.1 step 3 makes the +hole itself `None`. ### 5.4 `NODATA` propagation @@ -360,7 +359,7 @@ elevation. across cell and tile seams, which is the point of §5.3 step 2. - **Exactness on a plane.** If the sampled region's heights are an affine function of the lattice indices, the interpolated value equals that function evaluated at the query point, rounded per - §5.2. This is what makes a synthetic plane an *oracle* for a second implementation. + §5.2. A synthetic plane is therefore a test *oracle* for a second implementation. ### 5.6 Worked example @@ -404,11 +403,11 @@ specified below. The header remains 32 bytes. `Version` MUST be `3`, flag bit 0 MUST be set, and bytes `24..32` MUST be zero. Flag bit 1 indicates the cross-cell maximum index in section 8.3. Other flag bits MUST be zero, so `Flags` is `1` or `3`. Producers MUST set both bits (`Flags = 3`); readers also -accept `Flags = 1` without that index. The directory retains one little-endian `uint32` offset per geographic cell. -Zero means absent. Every present cell MUST start at a multiple of 512 bytes relative to the -container. The producer MUST pad the header and directory to that boundary with zero bytes. -When flag bit 1 is set, the cross-cell index starts at that boundary and precedes all cell blocks. -The complete container MUST fit within `uint32` byte addressing. +accept `Flags = 1` without that index. The directory retains one little-endian `uint32` offset per +geographic cell. Zero means absent. Every present cell MUST start at a multiple of 512 bytes +relative to the container. The producer MUST pad the header and directory to that boundary with +zero bytes. When flag bit 1 is set, the cross-cell index starts at that boundary and precedes all +cell blocks. The complete container MUST fit within `uint32` byte addressing. A cell contains a complete height pyramid. Level `i` has posting `posting_log2 + i`. The last level has 16 posts along each axis, so the level count is @@ -488,9 +487,8 @@ children inside the rectangle remain `32767` and propagate to the root. The last 1×1. The producer MUST pad the index end to 512 bytes with zeros. No cell block may overlap the index or its padding. A reader MUST reject a truncated index. -One index serves all surface levels. Native cell bounds include the high-edge vertices, and -every coarser level selects a subset of those vertices. Thus the native maximum also bounds -all coarser bilinear surfaces. An 8×8 cell rectangle needs 170 index bytes plus 342 padding +One index serves all surface levels: native cell bounds include the high-edge vertices, and every +coarser level selects a subset of them. An 8×8 cell rectangle needs 170 index bytes plus 342 padding bytes. ### 8.4 Assembly @@ -504,15 +502,14 @@ bytes inside the OBCT prefix and cell blocks are zero padding. ## 9. Crest lifts (producer rule) -A native posting of `2^9` µdeg cannot hold a rock tower. A **lift** is the correction: at a node -where a finer reference DEM says the ground stands far above the native bilinear surface, and the -terrain there is convex, the producer raises the sample to the reference ground. This is the -section 1.2 exception, and it is a producer rule, not a container feature. The lifted value is in -the sample itself, so section 8's pyramid, maxima, error codes and cross-cell index need no rule of +A **lift** raises a native sample to the reference ground at a node where a finer reference DEM +says the ground stands far above the native bilinear surface and the terrain is convex. This is the +section 1.2 exception. It is a producer rule, not a container feature: the lifted value is in the +sample itself, so section 8's pyramid, maxima, error codes and cross-cell index need no rule of their own. -Copernicus stays the base raster. A national DTM supplies lifts only: it sits below a surface model -over every forest and town, so it MUST NOT become the base. +Copernicus stays the base raster. A national DTM supplies lifts only and MUST NOT become the base: +it sits below a surface model over every forest and town. ### 9.1 The rule @@ -549,8 +546,8 @@ lattice, so that is 64 probes per node at the v1 posting. A probe that is a **maximum over an area** must have its gap measured against the **highest point of the surface over that same area**, not against the surface under one point of it. Each archive pixel is the maximum of the source pixels inside its square and may have come from anywhere in that -square, so the gap the bakery credits it is `pixel − max(surface over the pixel's square)`. Measured -this way, the gap can only read low, which loses a correction rather than inventing ground. +square, so the gap the bakery credits it is `pixel − max(surface over the pixel's square)`. The gap +then can only read low: it loses a correction and never invents ground. This document therefore does **not** promise that two producers agree byte for byte on a lifted cell. It promises the identity of section 9.2: where there is no coverage, there is no difference. @@ -568,15 +565,15 @@ cell with coverage has the same byte length as one without. ### 9.3 What a producer MUST NOT do -A lift MUST NOT be negative: section 9 raises crests, and a reference that sits below our surface in -a hollow is not a reason to edit the lattice there. A producer MUST NOT use a lift to carry any -other correction; a systematic disagreement with the source is a re-bake of the source, not a lift. +A lift MUST NOT be negative: section 9 only raises crests. A producer MUST NOT use a lift to carry +any other correction; a systematic disagreement with the source is a re-bake of the source, not a +lift. There is **no fixed ceiling** on a lift, and a producer MUST NOT impose one. The gap and `node_max` bound a lift, and nothing else may. A producer SHOULD report the largest lift of a run, the node it is at, and how many lifts exceed -200 m, so that an operator sees a broken reference. Nothing in a container records this and no +200 m, so that an operator can see a broken reference. Nothing in a container records this and no reader can check it; `obc-dem bake` prints it. A change of reference archive changes the baked samples, so it is a terrain revision bump and hence From 52884d45ebe8c8533da421124ff3fdd6b227701f Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 29 Oct 2026 08:12:57 +0000 Subject: [PATCH 07/20] docs(specs): common header and trim for obc-data.md The spec opens with the common header and a Conventions section. Rationale and consequence clauses, Commands rows that repeat their sections, and text that restates planner-offline.md and wikimedia-content.md are cut or linked. Headings and the generated Error codes and JSON schemas sections are byte-identical, so the api.rs spec test still holds. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DTHu3LHSGY5CY9j2XmscLi --- specs/obc-data.md | 511 +++++++++++++++++++++------------------------- 1 file changed, 234 insertions(+), 277 deletions(-) diff --git a/specs/obc-data.md b/specs/obc-data.md index d4d4257200..5d84a21da8 100644 --- a/specs/obc-data.md +++ b/specs/obc-data.md @@ -1,12 +1,24 @@ -# obc data +# obc data registry and releases -`obc data` reads the data registry: the external sources that a bake uses, the regions, and -the targets. It fetches sources into the store, bakes layers and updates the release that a -target serves. The crate is `host/obc-data`. -All files under `data/` are TOML. A file with an unknown key is refused. +The data registry under `data/` (sources, regions, targets, archive datasets, machines), the +store, the archive, the layers that a bake makes, the releases that a target serves, and the +`obc data` commands with their errors and JSON output. -The landmark and peak content reads the prepared source `wiki`, which -[Shared Wikimedia content](wikimedia-content.md#the-wiki-source) defines. +- Version: store `layout` 3; release manifest format 1. Each section gives its refusal rule. +- Writers: people (`data/*.toml`); `host/obc-data` (store, archive, targets, releases); the + `obc data` binary of `host/obc-data-steps` (the steps of the products and the producers of the + prepared sources). +- Readers: `host/obc-data`, `tools/data_registry.py`, `tools/check_data_sources.py`, `obc-bake` + (`data/regions/`), the planner services and the device. +- Code authority: `host/obc-data`. Its Rust types generate [Error codes](#error-codes) and + [JSON schemas](#json-schemas), and a test compares them with this file. +- Related: [Shared Wikimedia content](wikimedia-content.md#the-wiki-source) (the prepared source + `wiki`), [planner release](planner-release.md), [offline planner bundle](planner-offline.md), + [TrailView tiles](trailview-tiles.md), [OBCC catalog](obcc-catalog.md). + +## Conventions + +- All files under `data/` are TOML. A file with an unknown key is refused. ## Files @@ -30,7 +42,7 @@ One `[[source]]` table per source. | `version` | string | yes | How upstream names a version: `date`, `release`, `commit` or `digest` | | `start` | string | no | The version that a step list reads while no serving release reads the source. It has the form of `version` | | `refresh` | integer or string | no, `"manual"` | The policy, which the settings of a target can replace: `1` through `65535` whole days, or `"manual"` | -| `keep` | string | yes | Where the copy that a bake reads stays, so that no outage stops a bake: `archive`, the [archive](#archive) holds the whole dataset; `store`, the store keeps what each fetch took; `prepared`, we keep what we make: the [archive](#prepared-sources) keeps a prepared source and a source that only a prepare reads, and R2 keeps our own asset. `tools/check_data_sources.py` refuses a source without it | +| `keep` | string | yes | Where the copy that a bake reads stays: `archive`, the [archive](#archive) holds the whole dataset; `store`, the store keeps what each fetch took; `prepared`, we keep what we make: the [archive](#prepared-sources) keeps a prepared source and a source that only a prepare reads, and R2 keeps our own asset. `tools/check_data_sources.py` refuses a source without it | | `redistribute` | boolean | yes | The licence lets us give the upstream bytes to others | | `credential` | table | no | `env`: the environment variables a fetch needs; or `file`: the file that holds them. `~/` is the home directory. `register`: the `https://` page where a person gets it, and `hint`: what to copy there, in one line. See [Credentials of sources](#credentials-of-sources) | | `publish_text` | boolean | no, `true` | Only for a `prepared` source of article text: `false` keeps the text out of its versions | @@ -62,8 +74,8 @@ Rules: - An id is listed once. - Each kind of fetch but `installed` and `prepared` has a `url`, and the `url` starts with `https://`. - A `prepared` source has `version = "date"`, `redistribute = true` and `keep = "prepared"`. -- Data changes only on purpose: only the OSM sources, so that roads stay current, and the prepared - sources have a `refresh` in days. A prepared source changes when a person prepares a version. +- Only the OSM sources and the prepared sources have a `refresh` in days. A prepared source + changes when a person prepares a version. - `refresh` in days needs `version = "date"`, because only a date version has an age. - A credential has `env` or `file`, not both. A `file` is `~/.cdsapirc`, `~/.config/openbikecomputer/cdse-s3.env` or `~/.config/openbikecomputer/clms-key.json`. @@ -156,7 +168,7 @@ The bakes read this directory: | `fixtures/build-map-package.sh` | The `box` regions of the fixtures | `tools/data_registry.py box ID [--lat-first]` prints the box of a `box` region and refuses -every other kind, so Python and shell never resolve a union or a Geofabrik area. +every other kind. ### `data/planner.toml` @@ -201,8 +213,7 @@ age is separate from data age. Each fetch gives its params, serving version, state, reason, age and upstream observation. The observation gives the result, the latest probe time and the last successful probe time and -version. A failed probe keeps the last success. A capture result has no probe time: the capture -policy is not evidence of a network check. The source row gives missing credentials, but a +version. A failed probe keeps the last success. A capture result has no probe time. The source row gives missing credentials, but a credential blocks only a new fetch. ### Source overview @@ -246,12 +257,12 @@ with the columns SOURCE, LICENCE, RULE, COVERS, STATUS, WHERE, SIZE and DATE, an the source for each target. `--layer [LAYER]` gives each name of `used_by`, or LAYER only, with the `data` sources and assets -whose `used_by` has it, but no source that only a prepare reads. It gives no verdict: whether a -bake still downloads depends on the region, which `--region` tells. +whose `used_by` has it, but no source that only a prepare reads. It does not tell what a bake +still downloads; `--region` does. `--region ID|PATH[,PATH…]` and `--box W,S,E,N` name a region as `update` does, with each optional -part on. They need the products of `obc data`: `obc-data-plumbing` refuses. A Geofabrik area that the cached index lacks refuses. The output lists the step lists of -the products for the region, as the [overview](#overview) does, but with no fetch: the sources are +part on. They need the products of `obc data`: `obc-data-plumbing` refuses. A Geofabrik area +that the cached index lacks refuses. The output lists the step lists of the products for the region, as the [overview](#overview) does, but with no fetch: the sources are the fetches that the step lists read, and the sources that only a prepare reads of the prepared sources among them, by their `extent`, the bounds of their `areas` in the index, and for a national model (`dtm-…`, `canopy-`) of a region that names countries, its country ``. A fetch @@ -278,8 +289,8 @@ of its coverage. A version covers the region when one of its scopes is `null`, or each box of the region is inside one box of a scope that names no countries or each country of the region. `for_region` also counts the fetches, the held fetches and their known bytes, the known bytes to download and the fetches -of unknown size. Each layer has the most of the needs of the sources that its steps read. The output sums the known bytes to -download, and names the sources of unknown size or need and the sources to prepare. +of unknown size. Each layer has the most of the needs of the sources that its steps read. The +output sums the known bytes to download, and names the sources of unknown size or need and the sources to prepare. ## Store @@ -352,9 +363,8 @@ it runs in, and the store: - A snapshot record is reached when an output of a root release read its source and version. The newest record of each source, by the latest `retrieved` of its files, is also reached: a plan - step list reads it when nothing else names a version (see [Versions](#versions)), and a source whose - upstream gives only its newest file cannot give it again. So is the newest version of each request record (`requests/`), by the - latest `retrieved` of its files, such as the extract of each Geofabrik area, and each version + step list reads it when nothing else names a version (see [Versions](#versions)). So is the + newest version of each request record (`requests/`), by the latest `retrieved` of its files, such as the extract of each Geofabrik area, and each version that `obc data source refresh` took (`refreshed/`). - An object is reached when a reached snapshot record or a reached layer has it, or when its SHA-256 is a file of an output of a root release, or is in `fixtures/catalog.toml` or a JSON or @@ -373,12 +383,12 @@ reached snapshot record, with the reasons: `served by TARGET, …`, `newest of t `newest of a request` and `refreshed`. Then one entry for the reached layers of each step (`inputs kept`). Then one entry for each kind of root that names objects that no reached record or layer has: `release` or `fixture`. Then one entry for the unpacked copies that stay: `in the archive` or -`archive not read`. The size of -an entry is the size of its files, and the plan gives the size of `partial/`. The collection -takes the store lock alone, or refuses to start while a run or a fetch holds it. A run holds it -until it finishes; a run of `serve` does not hold it. A collection deletes each snapshot record, each object and each unpacked copy that is not -reached, and empties `partial/`. It deletes the `.complete` of a copy first, so a stopped -collection leaves no damaged copy that is marked whole. Receipts and upstream checks stay. A clean that cannot read a target +`archive not read`. The size of an entry is the size of its files, and the plan gives the size +of `partial/`. The collection takes the store lock alone, or refuses to start while a run or a +fetch holds it. A run holds it until it finishes; a run of `serve` does not hold it. A collection +deletes each snapshot record, each object and each unpacked copy that is not reached, and empties +`partial/`. It deletes the `.complete` of a copy first, so a stopped collection leaves no damaged +copy that is marked whole. Receipts and upstream checks stay. A clean that cannot read a target deletes nothing. ## Fetch @@ -404,8 +414,8 @@ Which version a fetch gets: A download stops after four failed tries in a row, and waits 2, 4 and 8 seconds between them. A try that adds bytes to a resumed part resets the count; a try that starts the file again does -not. A try receives its body for at most 15 minutes, so a -stalled transfer becomes a retry. It retries a connection error, a cut-off or stalled body, a +not. A try receives its body for at most 15 minutes; a stalled transfer is then a retry. It +retries a connection error, a cut-off or stalled body, a refused resume and HTTP 408, 429 and 5xx. It keeps the bytes it has in `partial/`. The next try, or the next fetch, asks for the rest with `Range` and `If-Range`, with the strong `ETag` or else the `Last-Modified` of the first response. A `206` answer with another validator, or a @@ -482,8 +492,7 @@ producer of [`wiki`](#wiki) reads them. `era5-land` has none: only the producer never upstream. Without that file in the archive, the fetch fails and names the fix. The capture reads the `layer` of the one GeoPackage or FileGDB of the zip, 5000 features to a page, in the order of their feature ids. It reads that file from its copy in `unpacked//` of - the [store](#store), because a read in a deflated member inflates the member from its start. - While the copy has no `.complete`, one capture at a time makes it: it checks the SHA-256 of + the [store](#store). While the copy has no `.complete`, one capture at a time makes it: it checks the SHA-256 of the zip against its manifest, refuses while the disk of the store cannot take the members that the copy lacks and keep 4 GiB free, unpacks them, flushes them to disk and then writes `.complete`. A stopped unpack resumes with the members in place. Each later box reads the @@ -494,8 +503,7 @@ producer of [`wiki`](#wiki) reads them. `era5-land` has none: only the producer stand for; a resumed run that finds others deletes the pages. - A `sentinel-2-mosaics` file is `/.tif`: the window over `bbox`, one pixel wider on each side, of a band of a quarterly mosaic tile of the year, as the STAC search lists it. -- The version is the day of the fetch, because the service answers with current data. Another - day comes only from the store. A request that a started version of a `manual` source lacks joins +- The version is the day of the fetch. Another day comes only from the store. A request that a started version of a `manual` source lacks joins that version, so requests over several days have one version. - A source with a `credential` that is not on this machine fails before the program runs, and the error names the variables or the file. A capture that the [archive](#archive) serves needs @@ -504,10 +512,9 @@ producer of [`wiki`](#wiki) reads them. `era5-land` has none: only the producer ### Archive The archive keeps whole datasets and the versions of the prepared sources on drives of their -own, so that no outage stops a bake. `OBC_DATA_ARCHIVE` names its roots, separated by `:`. -Without the variable, or with an empty value, there is no archive. Each root is a complete -archive. A dataset is wholly in one root, and so are the versions of a prepared source, so a failed -drive loses only its own. A prepared version has no other copy. Only `obc data archive` writes +own. `OBC_DATA_ARCHIVE` names its roots, separated by `:`. Without the variable, or with an empty +value, there is no archive. Each root is a complete archive. A dataset is wholly in one root, and +so are the versions of a prepared source. A prepared version has no other copy. Only `obc data archive` writes `mirror/`, `manifest/` and `.partial/`, and only `obc data source prepare`, `obc data source copy` and `obc data unlock --source` write `prepared/`. @@ -597,21 +604,19 @@ The version of a dataset: removes leaves the manifest. - A file of the manifest that is in place with its `bytes` stays without a download when upstream gives its size, its `ETag` and its `Last-Modified` day, as far as upstream gives them. Only a - bucket listing gives the `ETag`: the `ETag` of a `HEAD` can differ from one request to the next - and from that of a `GET`. + bucket listing gives the `ETag`: the `ETag` of a `HEAD` is not stable. - A file on disk with the `Content-Length` and the `Last-Modified` day of upstream as its size and modification day is read in place once. - Each other file downloads into `.partial//` with the retries and the resume of [Fetch](#fetch), four files at a time, and goes into `mirror/` after its check. An `ETag` of 32 hex digits in a bucket listing, which S3 and GCS give for a file of one part, is its MD5. -- A kept file has no other source, so a get never gives it other bytes than the `sha256` that the - manifest records in `files` or `missing`. A new kept file is read in place for its SHA-256. To +- A get never gives a kept file other bytes than the `sha256` that the manifest records in + `files` or `missing`. A new kept file is read in place for its SHA-256. To replace a kept file, a person puts the new file in place and removes its entry from the manifest. - A file that the get cannot take keeps its entry of the manifest before while that file is in place; else it is in `missing`. The get fails when a file failed. -- The get writes the manifest each minute and at the end, so a stopped get keeps the files that - it has. The next get resumes. Only one get or verify of a dataset runs at a time. +- The get writes the manifest each minute and at the end. The next get resumes. Only one get or verify of a dataset runs at a time. #### Verify @@ -626,8 +631,7 @@ place. The verify fails when a file differs or is not in place. `obc data archive status` gives each root with its free space, and each dataset of `data/archive/`: its sources, coverage, root, state, version, the day of the last get, the files that the last get listed and their known bytes, the verified files and their bytes, and the -`bytes_estimate`. It also names each source with -`keep = "archive"` that no dataset has. It reads only the manifests and the sizes of the files. +`bytes_estimate`. It also names each source with `keep = "archive"` that no dataset has. It reads only the manifests and the sizes of the files. | State | When | | --- | --- | @@ -654,16 +658,15 @@ that the last get listed and their known bytes, the verified files and their byt An `http`, `glo30` or `github` fetch looks for the file of each URL that the store does not have, before it downloads. The day of a file stands for `Last-Modified`: its `modified`, or else the version of its dataset when that is a day. For a `date` source without a version, the latest day of -its files is the version. Only a URL that the archive has no day for gets a `HEAD` request upstream, -so a fetch offline needs a day in the archive for each of its URLs. The archive file serves when it is the -version: its SHA-256 is that of the record of the version or of the `digest` version, or else its -day is on or before the day of a date version. Otherwise the fetch writes why and downloads the -file. So a GLO-30 tile, which the fetch reads at the `start` `2022-05-09`, serves only when its day -is `2022-05-09` or earlier. `obc data source refresh` asks upstream, not the archive. +its files is the version. Only a URL that the archive has no day for gets a `HEAD` request +upstream, so a fetch offline needs a day in the archive for each of its URLs. The archive file +serves when it is the version: its SHA-256 is that of the record of the version or of the `digest` +version, or else its day is on or before the day of a date version. Otherwise the fetch writes +why and downloads the file. `obc data source refresh` asks upstream, not the archive. The `corine-land-cover` capture looks for the manifest with its `dataset` and `file`. When the -archive has the zip, the capture checks its SHA-256 against the manifest, and `tools/clms.py --archive -ZIP` takes the file from that zip as from an order. An `archive-features` ground capture reads only +archive has the zip, the capture checks its SHA-256 against the manifest, and +`tools/clms.py --archive ZIP` takes the file from that zip as from an order. An `archive-features` ground capture reads only the archive, see [Fetch](#fetch). ### Credentials of sources @@ -682,8 +685,8 @@ values, never a value. | `file = "~/.config/openbikecomputer/clms-key.json"` | that file | None: a person saves the JSON service key whole, as the `hint` says, and the command refuses it | Each `obc data` command but `obc data ui`, and `obc bake publish --target r2`, loads -`credentials.env` when it starts, and its steps get the variables. A variable that the environment already sets wins. `overridden` in the output -names such variables: the new value serves only after they are removed from the environment. +`credentials.env` when it starts, and its steps get the variables. A variable that the +environment already sets wins. `overridden` in the output names such variables: the new value serves only after they are removed from the environment. Each acquisition check has 15 seconds. The store keeps its answer or failure for one hour. Equal normalized requests share a check. A changed acquisition URL or check kind invalidates @@ -779,9 +782,7 @@ Code is not in the key. A change of code with the same version reuses the files change that gives other output changes the `revision` of its layer; a change of the readers changes its `format`. -A step input enters a key with its digest, not with its key. A rebuild that gives the same -files gives the same digest, so the keys of the steps that read it do not change, and the -engine reuses them. A snapshot version enters a key the same way, by the digest of its files. +A step input enters a key with its digest, not with its key. A snapshot version enters a key the same way, by the digest of its files. The recipe of a step is the SHA-256 of the same object without digests and with `client`: each input as `{"kind", "name", "files"}` for a step and `{"kind", "name", "version", "params", @@ -808,17 +809,17 @@ A step starts in the repository root, in a process group of its own. On Linux it when the engine ends; the programs that it started can stay. Its standard output and standard error go to the standard error of the engine. Exit status 0 is success. When a Rust function fails, its child process writes the error as the last line of standard error and exits -with 1. The objects are read-only. While a step -runs, `output` and `metrics` are in `partial/layer-/`; the engine removes that directory -when the step ends, also when it fails. `TMPDIR` of a step is the empty directory `tmp` there, beside -`output`; a step makes no other file with that name. A program puts no socket in `TMPDIR`: its path -can be longer than the 108 bytes of a socket path. A failed step writes no receipt. A declared directory may be empty when the step creates it. -A declared output that does not exist is an error. +with 1. The objects are read-only. While a step runs, `output` and `metrics` are in +`partial/layer-/`; the engine removes that directory when the step ends, also when it fails. +`TMPDIR` of a step is the empty directory `tmp` there, beside `output`; a step makes no other file +with that name. A program puts no socket in `TMPDIR`: its path can be longer than the 108 bytes of +a socket path. A failed step writes no receipt. A declared directory may be empty when the step +creates it. A declared output that does not exist is an error. ### Offline -A step reads the data inputs of its request. A step does not use the network; fetchers are the only network users. A step must not write to its -inputs: their paths, and the links of a view, are objects of the store, and a step that runs as +A step reads the data inputs of its request. A step does not use the network; fetchers are the +only network users. A step must not write to its inputs: their paths, and the links of a view, are objects of the store, and a step that runs as root can write to a read-only object. The engine does not enforce this. A step that needs a package or a tool finds it installed, or reads it as a snapshot. @@ -849,7 +850,7 @@ ends. | `built` | `YYYY-MM-DDTHH:MM:SSZ` | | `wall_ms` | The time from start to end, in milliseconds | | `cpu_ms` | User and system time in milliseconds of the process of the step and of the children that it waited for, such as Osmium (`wait4`). `null` when the system does not report it | -| `peak_rss_bytes` | The peak resident set of the process of the step, or of the largest child that it waited for. It is not the sum of processes that run at the same time. Linux starts the peak of a forked process at the resident set of the engine, which runs no step itself. `null` when the system does not report it | +| `peak_rss_bytes` | The peak resident set of the process of the step, or of the largest child that it waited for. It is not the sum of processes that run at the same time. `null` when the system does not report it | | `own_process` | Whether the step ran in a process of its own. Without it (`false` when absent), `cpu_ms` and `peak_rss_bytes` can be those of the engine, and no estimate reads the peak | | `bytes_in` | The size of the input files | | `bytes_out` | The size of `files` | @@ -868,7 +869,7 @@ looks at the steps in dependency order: - The plan does not list the other steps: the store has their layers. A group is one change. Two builds are in the same group when one reads the layer of the other. -Thus a run of one group never needs a build of another group. Two groups can need the same +Two groups can need the same fetch. An `id` names a group only in the plan that it comes from. | Key | Value | @@ -882,19 +883,15 @@ fetch. An `id` names a group only in the plan that it comes from. The estimate of a build is `wall_ms`, `bytes_out`, `client_bytes` (the size of the files that a client reads) and `peak_rss_bytes` of the newest receipt, by `built`, of the same step with the same options. Without one, it is the `cost` of the output of the step in the serving release with the -same options, else the newest receipt of the step, else that `cost` with other options, or `null`. Its `peak_rss_bytes` is `null` unless it comes from a -receipt with `own_process`: the cost in a release does not tell whose peak it is. +same options, else the newest receipt of the step, else that `cost` with other options, or +`null`. Its `peak_rss_bytes` is `null` unless it comes from a receipt with `own_process`. An estimate from a receipt or a cost with other options than the step, such as another region, is a guess (`guess`). It scales `wall_ms`, `bytes_out` and `client_bytes` by the input size of the step against theirs: the input bytes (`bytes_in`) when the store has every input of the step and the receipt read some, else the area of the option `bounds` (`[west, south, east, north]` in -degrees, on a sphere) when both have it, else not at all. Its `peak_rss_bytes` does not scale: it -is the peak of the largest process, and a step that works by chunks keeps it at any size. So it is -a lower bound. On a first bake of a region the store seldom has every input of a step, so a guess -mostly scales by area. +degrees, on a sphere) when both have it, else not at all. Its `peak_rss_bytes` does not scale. -The `bytes` of a fetch is the size of its files in the -record of the version. When that record does not list them all, it is the size in the record of +The `bytes` of a fetch is the size of its files in the record of the version. When that record does not list them all, it is the size in the record of another version that lists them all, the last in byte order. Without `files`, the files are those that a fetch with the same `params` gave, or else every file. Otherwise `bytes` is `null`. A run fetches a version with the same `params` once, with the files of every group that needs it. @@ -903,10 +900,10 @@ fetches a version with the same `params` once, with the files of every group tha The `device/catalog` step consumes selected layer paths and their receipt metadata. Map and terrain layers list their empty cells beside the cells: a map layer in `cells//empty.json`, a terrain -layer in `metadata/`, with the decoded reference credits. The `cells/` -and `terrain/` payloads and `metadata/` are client files: a target holds the metadata that the -catalog and the map cells read, so another machine bakes them without the leaf. Every file of the catalog layer is a client file: the -pinned satellites in `objects/`, which it names `/objects/`, and the root `catalog.json` +layer in `metadata/`, with the decoded reference credits. The `cells/` and `terrain/` payloads +and `metadata/` are client files: a target holds the metadata that the catalog and the map cells +read, so another machine bakes them without the leaf. Every file of the catalog layer is a client +file: the pinned satellites in `objects/`, which it names `/objects/`, and the root `catalog.json` with the schema, terrain, licence and region files, which are the named files of the device. The option `credits` holds the OSM source entry and the credits of GLO-30 and of the landmark articles, from `data/sources.toml`, so that a credit edit changes the key. @@ -924,8 +921,7 @@ pick can reuse prior verification. An update and a rollback own one run each. It starts after the arguments are checked, and finishes after the switch, and for an update after the collection that follows it. -`update --dry-run` creates none. -A read-only command creates none. A failure after the run starts returns its id in `error.run`. +`update --dry-run` and a read-only command create none. A failure after the run starts returns its id in `error.run`. A run fetches the fetches of a plan, one after another, with the fetchers of [Fetch](#fetch). Then it builds the builds of the plan. A layer that a planned step reads, and that the plan does @@ -937,8 +933,7 @@ the serving release is unchanged: the step does not bake, also when the store la new release keeps the files, the digest and the cost of that output by reference, with the versions and the client of the step now. This needs the store to have each file of that output that no client of the serving release reads. `--only` bakes the steps of the named layers also -when they are unchanged. So a leaf whose input files are the same, such as the OSM of a leaf -after a refresh of the extract, does not bake on any machine. +when they are unchanged. - A step starts when the layers that it reads are built. - At most one step per core runs at a time. @@ -947,15 +942,12 @@ after a refresh of the extract, does not bake on any machine. end of the run starts first: its estimated `wall_ms` and that of the longest chain of the steps that read it. A step without an estimate counts as the longest. - The memory that the steps that run at the same time reserve is not more than three quarters of - the physical memory of the machine. The rest is for the engine, the page cache and other - programs. A step reserves its estimated peak (`peak_rss_bytes` of the estimate). + the physical memory of the machine. A step reserves its estimated peak (`peak_rss_bytes` of the estimate). A step with a guess, or without a peak, reserves from its measured runs too, the larger of the two: the receipts of the step, also with other options or of another leaf of its tiled step, with `own_process`, a peak and input bytes, in the store or of this run. They give their smallest peak, or, when that is more, its input bytes (`bytes_in`) times the peak per input byte - of the run that read the most. The peak per input byte of a run that read little is mostly the - fixed memory of a process. Another step of the layer does not count: a copy and a parse of one - extract differ greatly in memory. A step without a peak and without a measured run reserves + of the run that read the most. Another step of the layer does not count. A step without a peak and without a measured run reserves 512 MiB and 16 times its input bytes. A step that reserves more than the memory runs alone. - A step with worker processes reserves its largest process, not their sum. It gets `OBC_DATA_MEMORY_BYTES`: the part of the budget that the steps that run do not reserve when it @@ -965,8 +957,7 @@ after a refresh of the extract, does not bake on any machine. - A step that does not fit in the free memory holds back the steps after it, so that it starts first. - After a fetch fails, no step starts. After a step fails, no other step starts, and the steps - that run finish. The layers that the run built stay in the store, so the next update reuses - them: it continues after the failed step. + that run finish. The layers that the run built stay in the store; the next update reuses them. The id of a run is its start time in UTC, `YYYY-MM-DD-HHMMSS`, with `-2`, `-3` and so on when another run has that id. The folder `runs//` and a status `starting` reserve the id. The @@ -979,10 +970,11 @@ process never started. [JSON schemas](#json-schemas)): its arguments after `obc data`, its process (while it starts, the process that reserved it, then the process that it starts), its state (`starting`, `running`, `ok`, `failed` or `stopped`), each phase that it entered with its state and why it - waits (`waiting`, see [Router ready](#router-ready)), the progress of each layer that it bakes (when its first step started, steps done of the total, and - for a tiled layer the leaves done, the leaves that bake now and the leaves whose steps are all - unchanged), its fetches and steps (each `unchanged` or not, with when it started and ended, in milliseconds after the start of the run), the - releases that it switched the target to, and its error with the code, the fix and the steps + waits (`waiting`, see [Router ready](#router-ready)), the progress of each layer that it bakes + (when its first step started, steps done of the total, and for a tiled layer the leaves done, + the leaves that bake now and the leaves whose steps are all unchanged), its fetches and steps + (each `unchanged` or not, with when it started and ended, in milliseconds after the start of + the run), the releases that it switched the target to, and its error with the code, the fix and the steps whose layers the store keeps. The run replaces it with one rename, at most each second while events come, at each phase and at the start of each fetch. A reader ignores keys that it does not know. @@ -994,8 +986,7 @@ process never started. `obc data runs RUN --stop` writes it, and sends SIGTERM to the process group of a `running` run, once. When a run stops, the process group of each step that runs gets SIGTERM: the step ends with the programs that it started, and fails. A detached run whose process finds it at its - start does nothing more. A run that - stops frees the lock of its target, keeps what it baked, and ends as `stopped`; so does a + start does nothing more. A run that stops frees the lock of its target, keeps what it baked, and ends as `stopped`; so does a detached `serve`. `obc data update` runs detached. It makes the review in its own process, refuses as the review @@ -1021,7 +1012,7 @@ A step list gets the version of each fetch that it reads from one function version of any fetch of a source with a `start` is the newest serving version of the source: one version names all its fetches, such as the GLO-30 tiles of every region. A version of the store serves a fetch without `NAME=VALUE`s only when no record of a fetch with `NAME=VALUE`s - names it, so a source that lost its `NAME=VALUE`s never reads an old box. + names it. 3. Otherwise it reads the newest version upstream: the product names the fetch, and the update fetches it. A source whose URL needs a `NAME=VALUE`, such as the GLO-30 tiles, has no one newest version: a fetch of it without that value fails with `failed`, and the fix names the code of the @@ -1030,12 +1021,10 @@ A step list gets the version of each fetch that it reads from one function A `date` or a `release` version is newer by its parts: a run of digits by its number, any other run as text, so `v1.9` comes before `v1.11`; two versions with equal numbers, such as `v1.0` and `v1.00`, are in the order of their text. A `commit` or a `digest` has no order: rule 2 gives -its `start`, so that a change of `data/sources.toml` moves it; else the version that -`obc data source refresh` took last on this machine, when the target serves a version that the -targets served then, or none: a refresh never moves a target that another machine moved since; -else the serving version; else the one -version in the store, when the store has exactly one. So an update after an update that changed -nothing reads the same versions. A `prepared` source has the rule of +its `start`; else the version that `obc data source refresh` took last on this machine, when the +target serves a version that the targets served then, or none: a refresh never moves a target +that another machine moved since; else the serving version; else the one version in the store, +when the store has exactly one. A `prepared` source has the rule of [Prepared sources](#prepared-sources). Before it lists the steps, an update finds the fetches that the step lists read, with only small @@ -1093,8 +1082,7 @@ outline of the region touches. The outline of a `box` region is its box; the out Both products resolve the same source areas. A Geofabrik definition names its areas directly. For a Box, the cached index selects candidates by polygon coverage: of the areas whose polygons -meet the Box, each area that the others cover drops out, largest first. So neighbouring areas -cover a sliver of the Box before their parent does. Full `.poly` files must then cover the +meet the Box, each area that the others cover drops out, largest first. Full `.poly` files must then cover the complete Box. A gap tries the parent extract before selecting bulk inputs. An uncovered Box gives an actionable error. Country metadata does not remove required geography. @@ -1105,8 +1093,7 @@ when a capture holds older extract or polygon bytes. Each product sorts and dedu A multi-area input uses prepared `osmium merge --with-history`, then `time-filter` at the latest object timestamp present in the union. For an overlapping object, its highest supplied version -wins. Unique supplied objects remain. An extract cannot reveal a deletion absent from its bytes; -this is a union of available snapshots, not a snapshot at one common date. Conflicting payloads +wins. Unique supplied objects remain. Conflicting payloads at the same object type, id and version are malformed OSM inputs. The merge, the map crop, the cut and the terrain of the planner run the Osmium of `OBC_OSMIUM`, or @@ -1123,11 +1110,10 @@ A leaf reads the archive tiles of its terrain cells: those of the windows that t `obct-terrain.md` §9 reads, which have a halo of two postings. A model whose `extent` meets one of those tiles in the version of `terrain` that the target reads (see [Prepared sources](#prepared-sources)): the step list reads its manifest, and the leaf reads the blocks `18//.tif` of the -manifest that hold those tiles and its `index.json`. So a machine with an empty store plans a -region from the manifest alone. A leaf where the version has no block of its tiles has no -`device/reference` layer. Where a tile whose block the version lacks meets the outline of the region and is in the -`geofabrik-poly` polygon of one of the -`areas` of a `dtm-*` model, a note on `device/terrain` names the model and the command that adds +manifest that hold those tiles and its `index.json`. A leaf where the version has no block of its +tiles has no `device/reference` layer. Where a tile whose block the version lacks meets the +outline of the region and is in the `geofabrik-poly` polygon of one of the `areas` of a `dtm-*` +model, a note on `device/terrain` names the model and the command that adds the region; the version that adds it is a new input of the leaves that read it. | Layer | Reads | Options | Files | @@ -1153,15 +1139,15 @@ A single-area definition keeps the unsuffixed content layer names. Other definit content layer per collection and source area. Article inputs keep separate file views; canonical content merge rules deduplicate overlaps without losing unique articles. -`device/source/`, `device/region-osm`, `device/osm`, `device/reference/-` and all content -layers are private intermediates. Their complete records stay in release provenance. +`device/source/`, `device/region-osm`, `device/osm`, `device/reference/-` and all +content layers are private intermediates. Their complete records stay in release provenance. #### `planner` The planner reads the `countries` of the region, or else the countries of its source areas in the Geofabrik index, sorted; a region without countries is blocked. It reads the `time_zone` of the -region, or else `UTC`. It uses the shared source selection above. Its bounds enclose the requested Box or union of selected outlines. -Each snapshot, such as `copernicus-glo-30`, the extract, the `.poly` and the GLO-30 tile list, is +region, or else `UTC`. It uses the shared source selection above. Its bounds enclose the +requested Box or union of selected outlines. Each snapshot, such as `copernicus-glo-30`, the extract, the `.poly` and the GLO-30 tile list, is at its version (see [Versions](#versions)), as for `device`. The other options come from [`data/planner.toml`](#dataplannertoml). A GLO-30 input reads the tile of each 1° square that its box touches and that `copernicus-glo-30-tiles` names; a box at sea @@ -1201,22 +1187,20 @@ reads their maximum: the blocks `mean/18//.tif` that hold the arch Each map producer has a `planner//grid` step, where `` is `basemap`, `places`, `terrain`, `overlays`, `climate`, `snow`, `trailview-features`, `trailview-ground` or -`trailview-canopy`. It reads only the archive of its -producer. +`trailview-canopy`. It reads only the archive of its producer. `planner/trailview-colour//grid` reads one archive of the colour step, as the kind -`trailview-colour-`. -`planner/trailview-snow//grid` reads one archive of the snow day step, as the kind -`trailview-snow-`. -It converts terrain MBTiles with the pinned Python PMTiles library. Each tile goes into the -archive of its zoom 11 parent, or its own tile when its zoom is less than 11. The tile bytes -stay the same, except for terrain: it encodes each terrain tile again with the same pixels, and -puts zoom 13 in archives of its own ([terrain tiles](planner-release.md#terrain-tiles)). Each +`trailview-colour-`. `planner/trailview-snow//grid` reads one archive of the snow +day step, as the kind `trailview-snow-`. It converts terrain MBTiles with the pinned Python +PMTiles library. It groups the tiles as +[grid publication](planner-offline.md#grid-publication-and-selection) says: each tile goes into +the archive of its zoom 11 parent, or its own tile when its zoom is less than 11, and zoom 13 +terrain has archives of its own ([terrain tiles](planner-release.md#terrain-tiles)). Each archive is packed as `maps/tiles//--.pmtiles`, and its TileJSON as `maps/.json`, with the transport rules of [the offline contract](planner-offline.md). The step writes the packed bytes in `objects/`, and selects `objects` and `index.json` for clients. Its `index.json` has `format: 1`, `kind`, `map_zoom: 11`, `files`, -`source` and `metadata`. `files` maps logical names to source and transport hashes, sizes and encoding. -`source` names the converted archive bytes, or the raw MBTiles when it has no tiles. `metadata` +`source` and `metadata`. `files` maps logical names to source and transport hashes, sizes and +encoding. `source` names the converted archive bytes, or the raw MBTiles when it has no tiles. `metadata` is the TileJSON. Empty terrain has TileJSON and no tile archives. The parts of the planner are `map`, `routing`, `search`, `climate`, `snow`, @@ -1225,11 +1209,8 @@ and `trailview-canopy`; the device maps are the part `device`. A planner step is part that is on makes it or reads it: `map` makes the basemap, places, assets and terrain grids, `routing` the routing and overlays grids, `search` the model and the two search grids; the other parts make their own steps. The fonts, the index and the catalog read only the grids that stay. -So `trailview-features` alone bakes the OSM, the terrain and the features, and `map` without `search` still bakes the POIs that -the places read. `trailview-ground` makes the material, lithology and region steps, and needs `trailview-rules`. -`trailview-snow` needs `snow`. -All producers are intermediate layers. Grid steps select their `objects/` and their +`trailview-snow` needs `snow`. All producers are intermediate layers. Grid steps select their `objects/` and their `index.json`. | Layer | Reads | Files | @@ -1278,10 +1259,11 @@ The pointer names routing, search and downloads under `dir` target. [`obc data deploy`](#deploy) puts the follower of `prod` on its server and the tile Worker on Cloudflare. -Downloads jobs return a ready quote with an absolute pinned `source` URL. The client reads the -bundle, release and objects through that source, so a stable entry switch cannot mix releases. -The private selection key binds the object pool as well as the catalog and bounds. A quote -stays valid while its bundle exists; a missing bundle requires a new quote. Job creation completes synchronously; there is no mutable polling path. +The [download service](planner-offline.md#download-service) returns a ready quote with an +absolute pinned `source` URL. The client reads the bundle, release and objects through that +source. The private selection key binds the object pool as well as the catalog and bounds. A +quote stays valid while its bundle exists; a missing bundle requires a new quote. Job creation +completes synchronously; there is no polling path. A grid cell is a zoom 9 Web Mercator tile that the bounds of the region overlap, clipped to the bounds, with the id `9--`. The JSON objects that `planner/routing` writes have their keys in @@ -1339,8 +1321,7 @@ One table per target. The name of the table is the name of the target, in lowerc | `dir` | string | `dir` only, optional: an absolute path, or a path that starts with `~/` for the home directory. Without it, the folder with the name of the target beside the store | The file has `prod`, the public bucket, and `local`, a folder on each machine. A `dir` target and -the store do not overlap, also through a link, so a clean of the store never deletes a file of a -target. +the store do not overlap, also through a link. `obc data` reads an R2 target through the S3 API when it has the credentials, because the public origin can serve a cached pointer. Else, and always in [serve](#serve), it reads through the public @@ -1350,10 +1331,9 @@ URL, with each part of a key percent-encoded, so a `#` in a key under `sources/` a small document through the S3 API, signed with AWS Signature Version 4, and the files of a release with rclone. `OBC_R2_BUCKET` must be the `bucket` of the target. A command that writes an R2 target without these variables refuses (`blocked`); so do an update and a rollback without -rclone, which lists and checks the files of a release. A tagged read and a -review need the credentials too: the review id holds the tag of `release.json`, which is the ETag -of the S3 API, and the public origin can give another form of it, such as the weak ETag of a -compressed answer. +rclone, which lists and checks the files of a release. A tagged read and a review need the +credentials too: the review id holds the tag of `release.json`, the ETag of the S3 API, and the +public origin can give another form of it. ### Target layout @@ -1365,21 +1345,20 @@ A key is a path with `/` between its parts. No part is empty, `.` or `..`. | `lock.json` | The [lock](#lock): the run that changes the target, or free | | `planner/next.json` | Optional: a release whose planner services [serve](#serve) loads before a switch to it, see [Router ready](#router-ready) | | `releases/.json` | The [manifest](#release-manifest) of a release. Immutable | -| `releases//` | A file that a client finds by name. `` is the [digest](#keys) of the files of the folder, listed by ``. So other files make another folder. Immutable | +| `releases//` | A file that a client finds by name. `` is the [digest](#keys) of the files of the folder, listed by ``. Immutable | | `objects/` | A client file of an output. Immutable | | `cell-catalog/catalog.json`, `planner/catalog.json` | The pointers: a copy of the object that the serving release names for the key | Only client files of outputs go to a target: an output that no client reads can derive from a source that we may not give to others, and a bucket is public. No target holds a version of a -[prepared source](#prepared-sources). A step that reads such an output -of a kept layer reads it from the store of the machine. +[prepared source](#prepared-sources). A step that reads such an output of a kept layer reads it +from the store of the machine. ### Release manifest `releases/.json` is compact JSON with the keys of each object in byte order. The id is the SHA-256 of these bytes. The manifest has format 1. A change that a follower of the format before -cannot read raises the format. A follower reads its own format and the one before, so a deploy of -the services keeps the serving release. It says its format at `/planner-api/version`, and the review +cannot read raises the format. A follower reads its own format and the one before. It says its format at `/planner-api/version`, and the review of an update of prod names a follower that cannot read the release. | Key | Meaning | @@ -1440,10 +1419,7 @@ beside the others, and gives it to no client. Before it writes `release.json`, a switch to another release needs each pointer to hold nothing, the bytes of the serving release, or the bytes of the release that it switches to. A switch to the -serving release writes its pointers again, and sets `since` of the serving release: a pointer -that drifted served other bytes until then. So each pointer holds the bytes of the serving release -or of the rollback, and a rollback after a stopped switch needs nothing of the release that it -leaves. Drift is a pointer whose bytes are not those of the serving release. Nothing deletes a +serving release writes its pointers again, and sets `since` of the serving release. Drift is a pointer whose bytes are not those of the serving release. Nothing deletes a pointer key: clients of older publishers read these keys. A `dir` target must be on a local file system. A conditional write holds `/.lock` while it @@ -1459,7 +1435,7 @@ that is not 2xx is an error. `release.json`, `planner/next.json` and `lock.json` before the switch, each key of the release must be on the target with its size, else nothing switches. -Older publishers wrote other keys of the `prod` bucket, such as `cell-catalog/objects/`, +The `prod` bucket also holds keys of older publishers, such as `cell-catalog/objects/`, `planner/releases/` and `reference/v1/`. An update and a rollback never write or delete them. ### Lock @@ -1490,19 +1466,20 @@ it. - The server can apply a write whose answer does not come, so a later write on the old tag fails. After a write that fails or gets no answer, the run reads `lock.json`: when it is what the run wrote, the write is done; when it names this run on this machine, the run writes again on its - tag. Else another run took the lock: the run stops and changes nothing more. The write at the start of the phase `switch` checks this immediately before the - switch. After the switch, a lost lock stops only the collection. + tag. Else another run took the lock: the run stops and changes nothing more. The write at the + start of the phase `switch` checks this immediately before the switch. After the switch, a lost + lock stops only the collection. - At its end, the run frees the lock with a conditional write of `{"state": "free"}`: also when - it fails or panics. When another run took the lock meanwhile, the run reports that. SIGINT and SIGTERM stop the run before its next fetch, step or phase, and - end the steps that run, so it frees the lock too, and a run that a signal stopped before the switch does not switch. A second + it fails or panics. When another run took the lock meanwhile, the run reports that. SIGINT and + SIGTERM stop the run before its next fetch, step or phase, and end the steps that run, so it + frees the lock too. A run that a signal stopped before the switch does not switch. A second signal ends the process at once. - A process that ends without that, such as on SIGKILL, a crash or an out-of-memory kill, keeps - the lock. On its machine, on Linux, the next run of the same store takes the lock over at once, with a - conditional write, and writes that to its log: the store shows that the process of the run - ended (see [Runs](#runs)), and no process of its session runs, such as a program of a - step. On another machine, `obc data unlock TARGET --from MACHINE` frees the - lock of a run of MACHINE whose heartbeat is 10 minutes old or older (`STALE` in `lock.rs`): ten - heartbeats that did not come. It asks first. When that run continues, its next write fails, + the lock. On its machine, on Linux, the next run of the same store takes the lock over at once, + with a conditional write, and writes that to its log: the store shows that the process of the + run ended (see [Runs](#runs)), and no process of its session runs, such as a program of a step. + On another machine, `obc data unlock TARGET --from MACHINE` frees the lock of a run of MACHINE + whose heartbeat is 10 minutes old or older (`STALE` in `lock.rs`). It asks first. When that run continues, its next write fails, and it does not switch. - The heartbeat is the time of the clock of the holder. A `dir` target is on a local file system, so the holder and `unlock` read the clock of one machine. On an R2 target, `unlock` measures the @@ -1576,10 +1553,9 @@ its bakes and the checks of the products read: - A step that bakes and reads a file of a fetch that the serving release lists must find the same SHA-256 in the record of the store: before the review, and again after the fetches of the update, before any bake. Else nothing changes (`verify_failed`). -- A fetch that the serving release read, of a source whose upstream gives only its newest version (a - capture), refuses the update (`blocked`) before the fetch, also while the update lists the steps, when the store lacks - it and a capture cannot give it: this machine cannot get that version again. A capture gives the - version of today, and a version of a manual source that the store started (see [Fetch](#fetch)). +- A fetch that the serving release read, of a source whose upstream gives only its newest version + (a capture), refuses the update (`blocked`) before the fetch, also while the update lists the + steps, when the store lacks it and a capture cannot give it. A capture gives the version of today, and a version of a manual source that the store started (see [Fetch](#fetch)). The fix names the machine that made the serving release. A refresh that moves the source to a new version is the other way. - When the store and the serving release give a fetched file the same name, size and SHA-256, the @@ -1595,8 +1571,8 @@ its bakes and the checks of the products read: The review of an update says what it does before it changes a target. `--dry-run` writes the review and creates no run. `--plan` makes the review in a detached run, as [Runs](#runs) says: the run logs each fetch with its progress and each product that it plans, writes the review to -`runs//review.json`, and bakes nothing. It takes no lock: it writes nothing to the target. `runs RUN` gives that review. To -list the steps, an update fetches what the step lists need, which can be bulk data. It fetches into +`runs//review.json`, and bakes nothing. It takes no lock: it writes nothing to the target. +`runs RUN` gives that review. To list the steps, an update fetches what the step lists need, which can be bulk data. It fetches into the store only, and it never writes to the target. - `layers`: each layer that bakes or goes, with its cause (`stale` with the cause of @@ -1613,11 +1589,9 @@ the store only, and it never writes to the target. reference, what this machine fetches upstream and downloads from the target, the steps that it builds, and the steps whose output the store has from an earlier build, each with `built`. A reused step does not build; its client files go to the target. -- `cost`: each bake has the estimate of the newest receipt in the store of the step with the same - options, else of the `cost` of the step in the serving release with the same options, else of - the newest receipt of the step, else of that `cost`. `upload_bytes` is the size of the files of the bakes and of the reused steps that a client reads. - `unknown` lists each bake without an estimate, with a guess or without a peak, and each fetch - whose size is not known. +- `cost`: the estimate of each bake, as [Plan](#plan) says. `upload_bytes` is the size of the + files of the bakes and of the reused steps that a client reads. `unknown` lists each bake + without an estimate, with a guess or without a peak, and each fetch whose size is not known. - `machine`: the memory of the machine and the free bytes of the disk of the store after the listing. A bake whose estimated peak, also a guess, is more than the memory, or fetches, downloads and bakes that leave less than 4 GiB free on the disk, refuse the update. @@ -1654,8 +1628,8 @@ second rollback undoes the first. A target without a rollback refuses. The targe key of the rollback release, else nothing changes. On an R2 target, the follower must also serve the planner release of the rollback: the rollback asks its `ready` once, as [Router ready](#router-ready) says, and any answer but 200 refuses (`blocked`) with its reason. The -follower keeps that planner release running, so it waits for nothing. Then it switches as [Publish and switch](#publish-and-switch) says. It holds the [lock](#lock) -as an update does. A rollback bakes, uploads and deletes nothing: the next update collects. +follower keeps that planner release running. Then it switches as +[Publish and switch](#publish-and-switch) says. It holds the [lock](#lock) as an update does. A rollback bakes, uploads and deletes nothing: the next update collects. ### Collection @@ -1691,8 +1665,8 @@ and the archive is the only place that keeps it: no target holds a version. A ba as it reads any other source. A prepared source is a row of `data/sources.toml` with `fetch.kind = "prepared"`. The `obc data` -binary links a producer for each name: the producer of `` makes the source ``. Only a source whose licence lets us give its bytes to others is -prepared. +binary links a producer for each name: the producer of `` makes the source ``. Only a +source whose licence lets us give its bytes to others is prepared. The name of a version is the UTC date when its first run starts, `YYYY-MM-DD`. A later version of the same day adds `.2`, `.3` and so on: `2026-10-28`, `2026-10-28.2`, `2026-10-28.10`. Versions @@ -1701,8 +1675,8 @@ are in the order of the date, then of the number; a plain date is number 1. The home of a prepared source is the folder `prepared//` of the first root of `OBC_DATA_ARCHIVE` that has it. Each version of the source is in its home, because a version can keep the files of an earlier one (`reused`). When a later root also has files in -`prepared//`, each command that reads or writes the source warns: only the home serves. A test -root does this on purpose. The home holds: +`prepared//`, each command that reads or writes the source warns: only the home serves. The +home holds: | Path | Holds | | --- | --- | @@ -1771,7 +1745,7 @@ is compact JSON with the keys of each object in byte order: exist, such as a drive that is not mounted: that root can be the home. A source without a home gets one in the first root with 4 GiB free. To put it in another root, a person makes the empty folder `prepared//` there first. A run that made the home and then refuses or fails - removes the home again while it holds only the lock, so a refusal never fixes the root. While + removes the home again while it holds only the lock. While a target of `data/targets.toml` lists a version in its legacy folder `sources//` that the home lacks, with its manifest SHA-256 and that manifest, or cannot be read, the prepare refuses: [copy](#copy) it first. @@ -1779,17 +1753,19 @@ is compact JSON with the keys of each object in byte order: `versions.json` and has no manifest in the home. A run keeps its work in `prepared///` of the [store](#store) of its machine. The same command resumes the version: the producer does not make a file of the journal again, unless the home - lacks it. A resume with another archive root, region or language selection refuses. When the manifest of a stopped version is - in the home and names the files of its journal, its run stopped after the manifest: the run + lacks it. A resume with another archive root, region or language selection refuses. When the + manifest of a stopped version is in the home and names the files of its journal, its run stopped after the manifest: the run only adds the version to `versions.json`, and the review says `listing`. Another stopped version that is not newer than the newest complete version in the home, or whose manifest is in the home, is not resumed: the run starts a new version, deletes the work of the old one, and the review names it in `abandoned`. So a run never writes a file of a complete version. -3. The review names the root of the home, the version, the inputs with the size of their files in the store or on - this machine (`unknown` lists an input whose size neither knows), the files of the version at - most (`files`, `null` when the producer cannot tell before the run), the files - that earlier runs moved into the home, the cost from the newest version in the home, only when its scope and languages match, the bytes that go into the store, with the disk that the producer says its programs take, and into the archive (unknown without such a version), the check of this machine as in the - [review](#review) of an update, which also refuses when the source or an input has a +3. The review names the root of the home, the version, the inputs with the size of their files in + the store or on this machine (`unknown` lists an input whose size neither knows), the files of + the version at most (`files`, `null` when the producer cannot tell before the run), the files + that earlier runs moved into the home, the cost from the newest version in the home, only when + its scope and languages match, the bytes that go into the store, with the disk that the + producer says its programs take, and into the archive (unknown without such a version), the + check of this machine as in the [review](#review) of an update, which also refuses when the source or an input has a `credential` that is not on the machine, or when the root lacks the free space for the bytes of the version and 4 GiB more, and for each target of `data/targets.toml` the layers of its serving release that read an older version. `--dry-run` writes it. Its id is the @@ -1804,19 +1780,19 @@ is compact JSON with the keys of each object in byte order: have no manifest: no run resumes them. Before each 100 deletes, it checks that it still holds the lock. A version that `versions.json` lists and that has no manifest refuses the run: its manifest is lost, and the fix restores it from the backup. -6. The producer writes each file. `terrain`, `forest` and `climate` run the programs of - eight tiles at a time. The run moves each file into the home at once, records its SHA-256 in the journal - and deletes it from the work. No reader reads a version before its manifest, so the move - replaces a file that a stopped run left with other bytes. The run shows each tile that a program makes as a step `/` - of the layer ``, with its time, CPU, peak memory and the bytes of its files, so the page - shows its progress as it shows the steps of an update. +6. The producer writes each file. `terrain`, `forest` and `climate` run the programs of eight + tiles at a time. The run moves each file into the home at once, records its SHA-256 in the + journal and deletes it from the work. No reader reads a version before its manifest, so the + move replaces a file that a stopped run left with other bytes. The run shows each tile that a + program makes as a step `/` of the layer ``, with its time, CPU, peak memory and + the bytes of its files. 7. The run checks that it still holds the lock, and that the home has each file of the journal with its size. It writes the manifest only when the file is absent: a manifest that an earlier run of the version wrote stays when it names the same files. Then a conditional write adds the version to `versions.json`, with the time of the manifest, and the work of the version goes. -`--region` and `--box` make a version of that region only. `obc data runs RUN --stop` stops a run, and the same command resumes -it. `obc data unlock --source ID --from MACHINE` frees a lock that a stopped run left. +`--region` and `--box` make a version of that region only. `obc data runs RUN --stop` stops a run, +and the same command resumes it. `obc data unlock --source ID --from MACHINE` frees a lock that a stopped run left. A test of a producer uses a root of its own: name it first in `OBC_DATA_ARCHIVE`, and make the empty folder `prepared//` in it. Then the prepare and each update with that variable use the @@ -1824,22 +1800,20 @@ versions of the test root, and the other roots still serve their datasets. #### Read -The archive is the reference: each target reads the versions that the home lists. Before an update or the -[overview](#overview) lists the steps, each prepared source (an update) or each that the steps read -(the overview) gets a version in the -`format` of the producer of this `obc data`: the serving version of a `manual` source, else the -newest version of the home. So a new version makes each layer that reads an older one stale, with the cause -`source`, in each target. When no root has the home, such as on a machine without the +The archive is the reference: each target reads the versions that the home lists. Before an +update or the [overview](#overview) lists the steps, each prepared source (an update) or each +that the steps read (the overview) gets a version in the `format` of the producer of this +`obc data`: the serving version of a `manual` source, else the newest version of the home. So a +new version makes each layer that reads an older one stale, with the cause `source`, in each +target. When no root has the home, such as on a machine without the archive, a target keeps the version that it serves, and a bake from the store needs no archive. When the home lists no version in that format and the target serves none, the listing blocks the layers that read the source (`blocked`), and the fix names `OBC_DATA_ARCHIVE` and the command that -prepares one. A product can bake a layer -without the source instead, as `device/terrain` does without a national terrain model: then a note +prepares one. A product can bake a layer without the source instead, as `device/terrain` does without a national terrain model: then a note of the [review](#review) names the source and the command. -A test root can have other files under the name of a version. So the manifest is the identity of a -version: a step reads -`/`, and the record in the store, the `sources` entry of a release +The manifest is the identity of a version: a step reads `/`, and the +record in the store, the `sources` entry of a release and the fetch name it. Text output shows it as ` ()`; with `--json`, a command gives the whole version. @@ -1898,8 +1872,7 @@ format. polygon of a leaf of the region meets; for a version of everything, each block with a tile of a model. A run keeps the version of each polygon that the first run of the version read. A version makes each block of its region from each model with a tile in it, also from a model of - a country that the region does not name. So the pixels of a block do not depend on the region, - and a version that makes a block of the version before again makes it from all its models. + a country that the region does not name. So the pixels of a block do not depend on the region. Each other block of the version before stays in `reused`; the manifest names the scopes of the versions before in `extends`. The review counts the two pools of the blocks that the boxes of the region meet, and `index.json`; for a region with a `geofabrik` leaf, `files` is `null`. The review @@ -1909,17 +1882,14 @@ format. - For each block, `host/obc-dem/reference/ingest.py fetch --block - -= …` pools the rasters of each model of each tile of the block onto the lattice in one pass, and folds the models of the tile worst first by `PRIORITY`: a model keeps - each pixel where it has a height. So the best model with a height holds each pixel. Before it + each pixel where it has a height. Before it uses a model whose row has a `summit`, the program pools the tiles near that summit and refuses the model when the highest height within about 1 km of it is more than 10 m from the surveyed height, and names a check that finds no height. `--shared`, one directory for the blocks of a - prepare, keeps the passed checks, so one prepare checks a model once. Each lattice pixel that no + prepare, keeps the passed checks. Each lattice pixel that no centre of a model reaches and that lies between four of its centres with a height takes their bilinear height. A model whose step is longer than the east side of a lattice pixel in the tile - asks for the tile with two of its steps of margin. So a model coarser than the lattice fills its - footprint. A finer model whose step is longer than half the east side can leave a pixel without - a centre next to a void, and the fill gives that pixel the bilinear height. On a tile of `es` - (5 m) and one of `ch` (2 m), the fill writes no pixel. A raster of one row or one column fills + asks for the tile with two of its steps of margin. A raster of one row or one column fills nothing. The program writes `18//.tif` and `mean/18//.tif`, then `-.tiles`, a JSON object of each tile `/` of the block with a height: `sources`, its sources best first, and `samples`, the bias samples of its mean: the decimetres of @@ -1943,11 +1913,8 @@ format. `copernicus-glo-30-tiles` names; a land square that the fetch does not give is an error. A sample in a square that the list does not name is at sea and does not count, and the run logs how many. A model with fewer than 1024 samples with GLO-30 has no bias, and a later version - measures it again. The number is chosen: the samples of four full tiles, so that a test version - of one tile does not fix the bias of a model. A model keeps the bias of the version before, so a - region added later does not change it. A version that extends no version, such as the first - version of a format, measures every model over all its tiles. The heights of a tile are thus the - same in each bake of one version of `terrain`. + measures it again. A model keeps the bias of the version before. A version that extends no + version, such as the first version of a format, measures every model over all its tiles. - The plan of `planner/reference` reads `index.json` (a fetch of a prepared version with the param `file` gives that file too). It refuses a version that states no bias of the best model of a tile that it reads, and names the fix: a prepare of a box of about four tiles in the middle of @@ -1988,8 +1955,7 @@ same format. its `extent`. - The blocks of a region are the blocks with a tile of a model that a box or a `geofabrik-poly` polygon of a leaf of the region meets; for a version of everything, the blocks with a tile of a - national model. A run - keeps the version of each polygon that the first run of the version read, as for + national model. A run keeps the version of each polygon that the first run of the version read, as for [`terrain`](#terrain). A version has a file for each block of its region, also for a block without a pixel, and makes each zoom 12 tile of the block from each model that has the tile, also from a model of a country that the region does not name. So the pixels of a block do not depend @@ -2023,10 +1989,8 @@ same format. order. - `global` reads `canopy-global`, Canopy Height Maps v2, through HTTP range requests: a COG per zoom 10 tile, whose pixels of zoom 17 lie on the cells, 32 × 32 to a cell. Its heights are whole metres, - where 0 is also no data. It reads low and its tree tops are artefacts, so a cell takes 0.982 times - its height plus 3.73 m, and the stems that fill 41.7 % of its cover with crowns of that height. The - factors are fitted against `canopy-ch` on the canopy cells of eight tiles of the Jura, the - Mittelland and the Alps. + where 0 is also no data. A cell takes 0.982 times its height plus 3.73 m, and the stems that fill + 41.7 % of its cover with crowns of that height. - `ch` reads `canopy-ch` through HTTP range requests, in strips of whole rows of its internal tiles, so each internal tile downloads once: at each 0.5 m pixel, the newest acquisition that has it. @@ -2058,9 +2022,8 @@ same format. - The years are the ten years from `climate.first_year` of [`data/planner.toml`](#dataplannertoml). For each block of its region, a version holds `orography//.gz` and `//.gz` of each year, also for a block without land. A file that the version before holds stays in - `reused`, and the run does not make it again: final data of a year does not change. So a new - region or a new year adds files, and a new format makes each file again. The manifest names the - scopes of the version before in `extends`. + `reused`, and the run does not make it again: final data of a year does not change. The manifest + names the scopes of the version before in `extends`. - For each block, `python -m tools.planner_climate prepare --block - --years ,…` writes the files that the version lacks: with `--orography DIR` also the orography, from the orography file that the first program downloads into `DIR` of the run. It reads the chunks of @@ -2090,8 +2053,8 @@ same format. The orography file has one plane: the geopotential of the cell in m²/s², float64, from the orography file of the climate archive. With these types, `planner/climate` computes the same values as from the hourly data, so its archive has the same bytes except `inputs`. -- The run moves the files into the home in batches of 64. The manifest names `era5-land` at the date of the - version, with the [digest](#keys) of the SHA-256 of the files that it made. +- The run moves the files into the home in batches of 64. The manifest names `era5-land` at the + date of the version, with the [digest](#keys) of the SHA-256 of the files that it made. #### `wiki` @@ -2113,21 +2076,19 @@ in the shards of [Shared Wikimedia content](wikimedia-content.md#the-wiki-source archives serially, with Range resume and the reviewed size and ETag, then prepares the source. A changed remote identity refuses. The archive becomes visible only after the download ends. Credentials stay in the local environment and private token cache. No review or manifest holds - them. The manifest names - `wikidata`, `wikipedia` and `osm-planet` or `geofabrik-extracts` at the days of their dumps, with the - [digest](#keys) of the SHA-256 of the files. + them. The manifest names `wikidata`, `wikipedia` and + `osm-planet` or `geofabrik-extracts` at the days of their dumps, with the [digest](#keys) of the + SHA-256 of the files. - `--languages en,de` selects article languages from `specs/content-languages.json`. Without the flag, all supported languages are selected. Unknown or empty codes refuse; duplicates are removed and codes have supported order. The review, its id and the manifest keep the - selection. Only selected languages require HTML and language-link dumps. An excluded article - is a `missing` fact with `reason: "language_not_prepared"`; it supplies no lead-image candidate. - The identity order for a page without a Wikidata item stays the supported-language order. - Without the dumps that prove its canonical identity, its link stays unresolved. + selection. Only selected languages require HTML and language-link dumps. + [The `wiki` source](wikimedia-content.md#the-wiki-source) gives the facts of an excluded + language. - Start saves the reviewed input plan before it downloads. A resume uses this plan, including the URLs, sizes and object identities. The review ID covers these decisions, but not whether the download has since completed. After acquisition, the run pins each dump with its path, size, - modification time and SHA-256, - and `publish_text`. A resume reads exactly those dumps, and refuses when one is gone or changed, + modification time and SHA-256, and `publish_text`. A resume reads exactly those dumps, and refuses when one is gone or changed, or `publish_text` changed; the fix deletes the work of the version, so the next run starts a new one. A dump of the pinned size with another modification time is the pinned dump when its SHA-256 is. `tools.wiki` gets each SHA-256 and does not hash a dump again. A row without @@ -2138,8 +2099,8 @@ in the shards of [Shared Wikimedia content](wikimedia-content.md#the-wiki-source the version. A step whose output the run has does not run again, and `tools.wiki` does not repeat an import that finished with the same input. - The selection is the rule of the compilers: each subject of a landmark category, and each peak - with a resolved link of a node that the map takes as a summit. Only their articles are read, and only the photos that the compilers - choose for them come from Commons, as [Shared Wikimedia content](wikimedia-content.md#the-wiki-source) + with a resolved link of a node that the map takes as a summit. Only their articles are read, and + only the photos that the compilers choose for them come from Commons, as [Shared Wikimedia content](wikimedia-content.md#the-wiki-source) says. - A box keeps the subjects whose box meets it. A region whose leaves are all Geofabrik regions reads the extract of each of their areas. An object in two extracts is the object of its @@ -2152,16 +2113,15 @@ in the shards of [Shared Wikimedia content](wikimedia-content.md#the-wiki-source run, unless the run lacks the fact, has an older revision of it, or has it only as an article of a language that the run did not prepare. Then the fact keeps its earlier bytes and its asset, also its languages. A shard writes its facts in the order of kind and key, and an earlier shard - whose members do not change stays in `reused`. So `common.tar.zst` keeps earlier facts until a - version of everything replaces them. A version of everything extends no version. + whose members do not change stays in `reused`. A version of everything extends no version. - An extension whose row has another `publish_text` than the version before refuses before its - first step: the kept articles would not match the manifest. The fix is the `publish_text` of the - version before, or a version of everything. Without published text, no shard has article text. + first step. The fix is the `publish_text` of the + version before, or a version of everything. - The run moves the shards into the home in batches of 64, then `index.json`. A resume names the shards in the home to `shards`, which does not write them again. -- With `publish_text = false`, each article in the shards is a fact with its key and the status - `missing`, without text, and no lead image is a photo. The review says "text published" or - "text kept local". +- With `publish_text = false`, each article is a `missing` fact, as + [the `wiki` source](wikimedia-content.md#the-wiki-source) says. The review says "text published" + or "text kept local". ### Serve @@ -2215,14 +2175,14 @@ then writes a warning: an update can delete the files of that release. The services of a planner release read its files under `routing/`, `search/` and `offline/` in `/serve///`. The document `releases//release.json` that the -manifest names lists them. Each file comes from an object of the store of serve, which downloads an object -that it lacks from `objects/` of the target and checks its SHA-256: a link to an `identity` +manifest names lists them. Each file comes from an object of the store of serve, which downloads +an object that it lacks from `objects/` of the target and checks its SHA-256: a link to an `identity` object, or the decoded bytes of a `gzip` object under a temporary name. No write opens an existing file. Each file must have the SHA-256 and the size that the planner release gives. When serve stops, it ends each download at its next read and waits for it within the 10 seconds of its grace; a wait before a retry, a connection or a hash does not see the stop. -Serve removes `/serve//` when it starts and when it stops; `obc data clean` does not -touch it. The objects are in a store of serve alone, `/follower//`. After the +Serve removes `/serve//` when it starts and when it stops; `obc data clean` does +not touch it. The objects are in a store of serve alone, `/follower//`. After the releases that the two files name change, and after downloads, serve deletes there each object and each unfinished download that the services of none of them read, once no download runs. A download for a planner release that no release names any more stops at its next object. For a @@ -2259,8 +2219,8 @@ commands in a row: the login, the configuration of `ssh` and the host key. | Action | Checks | | --- | --- | | Publish: an update of `prod` and `r2 legacy` | The `OBC_R2_*` variables; rclone on PATH | -| `deploy services` | `OBC_VPS_SSH`, which does not start with `-`; this machine logs in as root without a prompt, with a host key that it knows; the one-time setup on the VPS: the user `obc-data`, `/opt/obc-data`, Rust and uv of that user, a C compiler, Python and Node of the `[target]` of `data/planner-runtime.toml`, and the enabled unit. The Caddy route of the API is a check that the deploy does not wait for. `tools/vps/` and `data/planner-runtime.toml` have no change that the commit does not hold, because the deploy runs the script of the checkout | -| `deploy tiles` | The two variables; the token may list the Workers of the account; `planner/tiles` has no change that the commit does not hold, because `wrangler deploy` reads the files of the checkout | +| `deploy services` | `OBC_VPS_SSH`, which does not start with `-`; this machine logs in as root without a prompt, with a host key that it knows; the one-time setup on the VPS: the user `obc-data`, `/opt/obc-data`, Rust and uv of that user, a C compiler, Python and Node of the `[target]` of `data/planner-runtime.toml`, and the enabled unit. The Caddy route of the API is a check that the deploy does not wait for. `tools/vps/` and `data/planner-runtime.toml` have no change that the commit does not hold | +| `deploy tiles` | The two variables; the token may list the Workers of the account; `planner/tiles` has no change that the commit does not hold | | Both | The commit of the checkout is on `origin/develop` after a fetch. An older commit of develop serves to go back | A refused login names its causes: the VPS does not know the key of this machine, the host does @@ -2277,8 +2237,7 @@ changed goes only with `ssh-keygen -R`. Then it asks, or needs `--yes` without a appends the key under the name and to the file that `ssh` reads for the host: its `HostKeyAlias` and its first `UserKnownHostsFile`. It is the one change of a check. -`deploy services|tiles --dry-run` checks the credential, the commit and the files, -and writes the commit, what the deploy changes, its commands and the arguments that start it. A +`deploy services|tiles --dry-run` checks the credential, the commit and the files, and writes the commit, what the deploy changes, its commands and the arguments that start it. A deploy starts only with `--expect COMMIT`, the commit of the checkout now, and asks first, or needs `--yes` without a terminal. It is a detached [run](#runs), and the output of each command goes to its log. One run at a time deploys each part. The commands get only `PATH`, `HOME`, the user, the @@ -2291,13 +2250,12 @@ services|tiles` reads the values on standard input, as `obc data` gives the machine that runs it and, for each target, the release that it serves, the machine that made it and when, its rollback, for a `dir` target how [serve](#serve) gives it to each app, the run that holds its lock or why `lock.json` cannot be read, the region and the parts -that are on, the state of -each layer against the steps of now, the blocked steps, the notes of the layers as in the -[review](#review), the pointers that drift, an error that -stopped the judgement, and the command to run next. While a run holds the lock of a `dir` target, +that are on, the state of each layer against the steps of now, the blocked steps, the notes of the +layers as in the [review](#review), the pointers that drift, an error that stopped the judgement, +and the command to run next. While a run holds the lock of a `dir` target, the next step is to wait for that run, unless its process ended on this machine: then the next -update takes the lock over. It fetches only small metadata, and checks upstream with the -checks of the last hour. A target that serves nothing has the next command +update takes the lock over. It fetches only small metadata, and checks upstream with the checks +of the last hour. A target that serves nothing has the next command `obc data update TARGET --region ID`. For an R2 target, the next command is the review (`--dry-run`): its update needs the id that the owner approved. @@ -2309,20 +2267,20 @@ checks of the last hour. A target that serves nothing has the next command | `obc data update TARGET [--region ID\|PATH[,PATH…] \| --box W,S,E,N] [--add LAYER]… [--remove LAYER]… [--refresh SOURCE=DAYS\|manual]… [--only LAYER,…] [--dry-run \| --plan \| --expect ID] [--accept-format-change] [--yes] [--json]` | The [update](#update) of a target. The flags change its settings. `--dry-run` writes the [review](#review) and changes nothing. `--plan` makes the review in a run and changes nothing; with `--json`, it writes the run and a `null` review id. Else it starts a detached run, as [Runs](#runs) says, and writes its log until it ends: the steps that it baked or reused, the release that the target serves, its rollback, the number of kept outputs and what the collection did. With `--json`, it writes the run and the review id | | `obc data rollback TARGET [--yes] [--json]` | The [rollback](#rollback) of a target. Writes the release that the target serves and its rollback | | `obc data router TARGET [--json]` | For an R2 target, asks the follower once, as [Router ready](#router-ready) says, for the planner release of the serving release and of the rollback. Writes each answer, and the release that `planner/next.json` names. Changes nothing. Refuses a `dir` target | -| `obc data unlock (TARGET \| --source ID) --from MACHINE [--yes] [--json]` | Frees the [lock](#lock) of a target, or with `--source` the lock of a [prepared source](#prepared-sources) in the archive, that a run of MACHINE holds with a heartbeat 10 minutes old or older. Asks first. Writes the holder that it removed | +| `obc data unlock (TARGET \| --source ID) --from MACHINE [--yes] [--json]` | Frees the [lock](#lock) of a target, or with `--source` the lock of a [prepared source](#prepared-sources), that a run of MACHINE holds with a heartbeat 10 minutes old or older. Asks first. Writes the holder that it removed | | `obc data sources [--check-now] [--json]` | Every source with licence, and for each target that serves a release its policy, its versions and its [state](#state-of-a-source); the versions in the local store; for a prepared source the root of its home, its versions there and the run that holds its lock; and how this machine holds it, as the [source overview](#source-overview) says. Rows are in kind order: data, then assets, then tools. An upstream check of the last hour serves, except with `--check-now` | | `obc data sources --layer [LAYER] [--json]` | For each layer, or LAYER, the sources that it reads with their status, as the [source overview](#source-overview) says | | `obc data sources (--region ID\|PATH[,PATH…] \| --box W,S,E,N) [--json]` | The sources that a bake of the region reads, what each layer and each source still needs, and what a bake still downloads or prepares, as the [source overview](#source-overview) says. Fetches nothing | | `obc data source refresh ID [--json]` | Fetches upstream's newest version of a `digest` source without `start` or params, and writes `refreshed/.json` of the store with the versions that the targets serve: the next update of a target that serves one of them, or none, reads it, as [Versions](#versions) says. Refuses another source. The fetch is a detached [run](#runs): a terminal follows it, and `--json` writes `{run}` and ends | | `obc data source prepare ID [--region ID\|PATH[,PATH…] \| --box W,S,E,N] [--languages L,…] [--dry-run \| --expect ID] [--json]` | [Prepares](#prepare) a version of a prepared source in the archive. `--dry-run` writes the review. Else it starts a detached run and writes its log until it ends; with `--json`, it writes the run and the review id | -| `obc data source copy ID --from TARGET [--dry-run] [--json]` | [Copies](#copy) the versions of a prepared source from the legacy folder `sources//` of a target into the archive. `--dry-run` writes the plan. Else it starts a detached run and writes its log until it ends; with `--json`, it writes `{run}` | +| `obc data source copy ID --from TARGET [--dry-run] [--json]` | [Copies](#copy) the versions of a prepared source from a target into the archive. `--dry-run` writes the plan. Else it starts a detached run and writes its log until it ends; with `--json`, it writes `{run}` | | `obc data source credential ID [--json]` | Reads the values of the credential of a source on standard input and writes them, as [Credentials of sources](#credentials-of-sources) says. Writes the file and the names of the values | | `obc data fetch SOURCE[@VERSION] [NAME=VALUE…] [--json]` | Fetches the version, or else the newest file upstream. Writes the store path of each file | -| `obc data archive status [--json]` | The [status](#status) of the archive: each root with its free space, and each dataset with its root, version, state and files. Changes nothing | -| `obc data archive get DATASET [--root PATH] [--json]` | [Gets](#get) the files of the dataset that its root does not hold, and writes its manifest. Writes the status of the dataset | -| `obc data archive verify DATASET [--json]` | [Verifies](#verify) each file of the manifest of the dataset. Writes the status of the dataset | +| `obc data archive status [--json]` | The [status](#status) of the archive. Changes nothing | +| `obc data archive get DATASET [--root PATH] [--json]` | [Gets](#get) the dataset. Writes the status of the dataset | +| `obc data archive verify DATASET [--json]` | [Verifies](#verify) the dataset. Writes the status of the dataset | | `obc data areas [QUERY] [--json]` | The regions of `data/regions/` and the Geofabrik areas of the cached index whose id or name holds `QUERY`. Fetches nothing | -| `obc data clean [--apply [--yes]] [--json]` | The plan of [Clean](#clean): the snapshot records and the objects that nothing reaches, what stays and why, and the size of `partial/`. `--apply` asks, then cleans. With `--json` and `--apply`, the plan goes to standard error, and the output is what it did | +| `obc data clean [--apply [--yes]] [--json]` | The plan of [Clean](#clean). `--apply` asks, then cleans. With `--json` and `--apply`, the plan goes to standard error, and the output is what it did | | `obc data runs [--json]` | Every run in the store, newest first: id, arguments, state, time, and the size of its fetches and of the layers that it built | | `obc data runs RUN [--json]` | One run: its status, with the users of each step and its time in the last run that built it, the last 50 lines of its log with its path, and the review of a run of `update --plan` | | `obc data runs RUN --follow [--json]` | The log of the run as it comes, until it ends; with `--json`, its status each time it changes. Ctrl-C ends only the following. The exit status is the one of the run | @@ -2382,15 +2340,14 @@ A request that does not, and a command that the page does not run, get HTTP 403 The R2 client in `host/obc-data` reaches the bucket for `obc bake publish --target r2`, `obc bake clean-r2`, `obc fixtures publish` and the firmware publish workflow. rclone moves the -bytes. -`obc data r2` is plumbing for scripts: those commands call it, and a person deletes a key with +bytes. `obc data r2` is plumbing for scripts: those commands call it, and a person deletes a key with `obc data r2 delete`. It does not change the state of a release. Once the bucket has the `release.json` of `obc data`, `obc bake publish --target r2` refuses to run, and `obc data r2 put` and `delete` refuse `release.json`, `lock.json`, `planner/next.json`, a pointer key, a key under `objects/` or `releases/`. `obc data r2 delete` refuses a key under `sources//` of a prepared source while the [archive](#prepared-sources) lacks a version that -`sources//versions.json` of the bucket lists, with its manifest SHA-256 and that manifest. `obc bake clean-r2` refuses to run when the pointer -of its prefix has `release`, or is not JSON. `obc data` never writes under `reference/v1/`. +`sources//versions.json` of the bucket lists, with its manifest SHA-256 and that manifest. +`obc bake clean-r2` refuses to run when the pointer of its prefix has `release`, or is not JSON. `obc data` never writes under `reference/v1/`. ### Credentials @@ -2420,7 +2377,7 @@ key and never sends it. Rules: - The `r2` commands are plumbing below the rule of [Errors](#errors). Only `delete` and - `legacy --delete` ask, because a delete cannot be undone. `put` and `get` never ask. + `legacy --delete` ask: a delete cannot be undone. `put` and `get` never ask. - A key or a prefix is never empty, has no empty, `.` or `..` part, and has no control character. The bucket root is never a target. - Verify passes when the object has the size of the file, and the same MD5 when both sides @@ -2435,13 +2392,13 @@ Rules: - `legacy` lists each key under `cell-catalog/` and `planner/`, or under the one `--prefix`, that `obc data` does not write. It refuses while the target serves no release. It refuses `reference/v1` while `reference/v1/index.json` is missing or does not parse, and while the - `index.json` of the newest version of `terrain` in the archive lacks a key of its `sources`. The digest is the - SHA-256 of one line ` \n` per key, sorted by key. With `--delete`, it lists again, + `index.json` of the newest version of `terrain` in the archive lacks a key of its `sources`. The + digest is the SHA-256 of one line ` \n` per key, sorted by key. With `--delete`, it lists again, refuses when the digest is not `--expect`, asks, and then deletes in a detached [run](#runs): as `delete --legacy`, with the counts, the bytes and the digest as its plan. The run lists again and refuses another digest too. One run at a time deletes the keys of a target. With no key, the run deletes nothing. With `--json`, it writes `{run}`; else it follows the log of the run. -- `delete` cannot be undone. It prints the plan: the bucket, and each object with its size and +- `delete` prints the plan: the bucket, and each object with its size and upload time. With `--json`, the plan goes to standard error, and the output is the objects that it deleted. Then it asks as [Errors](#errors) says. - `delete` appends one line per object to `removed.jsonl` at the bucket root before it deletes: From 5c46b2d7e05436bf3690c7e8557be34683da3b29 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 29 Oct 2026 08:12:57 +0000 Subject: [PATCH 08/20] docs: common README structure for host, tools, fixtures and data The READMEs use the standard section names and order. Design rationale, explanation that a spec already holds and filler are cut or linked. Provenance, licence, date and digest text in the fixture READMEs is unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DTHu3LHSGY5CY9j2XmscLi --- data/trailview/README.md | 4 +- fixtures/README.md | 15 ++--- fixtures/sources/peak-view/README.md | 14 ++--- fixtures/sources/ride-assistant/CAPTURE.md | 27 ++++---- .../ride-assistant/PLACES-VALIDATION.md | 8 +-- fixtures/sources/ride-assistant/README.md | 53 ++++++++-------- fixtures/sources/route-import/README.md | 4 +- host/obc-bake/assets/README.md | 27 ++++---- host/obc-data/README.md | 41 ++++++------ host/obc-dem/reference/README.md | 62 +++++++++---------- host/obc-pack/src/landmarks/README.md | 12 ++-- host/obc-search-bake/README.md | 8 ++- host/obc-trailview-features/README.md | 4 +- specs/vectors/README.md | 20 +++--- tools/verification/README.md | 4 +- tools/vps/README.md | 19 +++--- 16 files changed, 162 insertions(+), 160 deletions(-) diff --git a/data/trailview/README.md b/data/trailview/README.md index 1eb2abcb19..7d1e95398a 100644 --- a/data/trailview/README.md +++ b/data/trailview/README.md @@ -14,13 +14,13 @@ The rules that the TrailView ground layers index and that the engine reads. The | `ground.json` | The ground sources of the material and the lithology, in the order they apply: [specs/trailview-tiles.md](../../specs/trailview-tiles.md#ground-sources) | | `roads.json` | The rules per class of road and track, from the most important down: the grade and smoothing of the road solver, the stamp profile of the engine | -## Check +## Test ```sh uv run --locked --group trailview python -m tools.trailview.rules ``` -## What bites +## Pitfalls - Codes in `materials.json` are indexes in the ground tiles. Append only. - A region id is a region and a place: one id per region, and one per country or subdivision diff --git a/fixtures/README.md b/fixtures/README.md index 2f5906314c..2cf0b8ac68 100644 --- a/fixtures/README.md +++ b/fixtures/README.md @@ -6,7 +6,11 @@ crate's `assets/` directory. The catalog is tracked. Large generated and captured bytes are not. -## Daily use +## Setup + +Python 3.11 or newer. No Python package dependencies. + +## Run ```sh obc fixtures list # scenarios, cache state, download size @@ -27,8 +31,6 @@ extraction are atomic, and archives, manifests and every extracted file are veri `tracked_sources` mappings make CI fail if a small authored source changes without its external package being rebuilt. -Python 3.11 or newer. No Python package dependencies. - ## Add or replace data 1. Make a clean staging directory holding only the package's final layout. @@ -55,9 +57,8 @@ terrain before its map, or use the `all` target. | Country-scale raw landmark captures | The map-baker's local source cache only | **Do not register or publish a country capture as a development fixture**, in any profile. The -1 GiB archive limit is a backstop, not -a target: use the smallest input that covers the scenario. Generated design-review screenshots -belong in a pull request, not in a runtime asset folder. +1 GiB archive limit is a backstop, not a target: use the smallest input that covers the scenario. +Generated design-review screenshots belong in a pull request, not in a runtime asset folder. ## Package provenance @@ -112,7 +113,7 @@ The verifier rebuilds discovery and compares two offline compilations. `fixtures.openbikecomputer.com` is a read-only custom domain for the EU-jurisdiction R2 bucket `obc-dev-fixture`. Developers and CI need no cloud credentials. Upload credentials are -maintainer-only and are deliberately not shared with the production map publisher. +maintainer-only and are not shared with the production map publisher. The publisher reads `OBC_FIXTURE_R2_BUCKET`, `OBC_FIXTURE_R2_ENDPOINT`, `OBC_FIXTURE_R2_ACCESS_KEY_ID` and `OBC_FIXTURE_R2_SECRET_ACCESS_KEY` from diff --git a/fixtures/sources/peak-view/README.md b/fixtures/sources/peak-view/README.md index 0fc2e55073..003584744c 100644 --- a/fixtures/sources/peak-view/README.md +++ b/fixtures/sources/peak-view/README.md @@ -8,13 +8,13 @@ Observer locations and the peak catalogue are in [locations.json](locations.json coordinates are reconstructed from rounded bearings and distances; production map summit records keep exact OSM coordinates. -## Obtain +## Setup -`obc fixtures sync sim-peak-view`. The package is also in the `sim` profile, and the simulator +`obc fixtures sync sim-peak-view`. The package is also in the `sim` profile; the simulator resolves it through the standard fixture cache. `OBC_PEAK_TERRAIN_DIR` overrides it for a local bake. **The simulator never downloads data.** -## Generate +## Build Install NumPy and Pillow, then run from the repository root: @@ -41,7 +41,7 @@ coverage radii of 28, 53 and 103 km. The generator puts the finest available sou coordinate onto one 2^9-microdegree lattice; coarser outer sources extend coverage and interpolation adds no detail. `obc-dem surface` then creates the lower levels and the bounds. Geographic cells span 2^16 microdegrees. Runtime level changes happen at 25 and 50 km and all rays -end at 100 km, so the extra coverage is what lets a nearby observer position work. +end at 100 km; the extra coverage lets a nearby observer position work. `--source-levels PATH` reuses the three source grids from an earlier bake. That directory must hold `{site}-9.obcd`, `{site}-10.obcd` and `{site}-11.obcd` per site, and this path reads and @@ -54,7 +54,7 @@ generator, and the location and Rust catalogues. Lighting is fixed relative to geography, 25 degrees above the horizon and to the north-west. It describes terrain form only: **no snow, no rock texture, no cast shadows and no real sunlight.** -Source resolution and level changes limit detail. It is not a photographic reconstruction. +Source resolution and level changes limit detail. Named summits use measured terrain at their catalogue distance and within half a degree of their bearing, with a quarter-degree tolerance on visibility. **Terrain is never raised to meet a @@ -68,8 +68,8 @@ completed image and touches no terrain storage. Terrain starts 2 m from the observer. Curvature uses an effective Earth radius with refraction coefficient 0.13. -A GPS displacement above 20 m starts a replacement job; smaller changes keep the current panorama, -to limit GPS jitter. At a moved observer, ground elevation plus two metres is the viewing height. +A GPS displacement above 20 m starts a replacement job; smaller changes keep the current panorama. +At a moved observer, ground elevation plus two metres is the viewing height. Missing distant terrain and clipped coverage show a partial-coverage notice and bearing marks. Missing observer terrain or a failed read is an unavailable state. diff --git a/fixtures/sources/ride-assistant/CAPTURE.md b/fixtures/sources/ride-assistant/CAPTURE.md index 4efd88a9a0..8f9d405fb2 100644 --- a/fixtures/sources/ride-assistant/CAPTURE.md +++ b/fixtures/sources/ride-assistant/CAPTURE.md @@ -4,8 +4,8 @@ A one-shot acquisition command that writes country-scale raw sources to a local directory. **Do not publish that directory to the dev fixture bucket or the production maps bucket**, and do not register it as a fixture; see [the fixture package workflow](../../README.md#add-or-replace-data) for what does belong there. -The compiled landmark content travels in ordinary maps, so no simulator or device ever needs the -raw archive. +The compiled landmark content travels in ordinary maps; no simulator or device needs the raw +archive. ## Acquire @@ -28,8 +28,8 @@ python3 tools/landmark_capture.py \ ``` Candidates are the QIDs the extract's own objects carry in a `wikidata` tag. Nothing is matched by -name or by coordinate, so discovery is offline and every landmark has a map object with an -approach. A tag that is not a QID is listed in `rejected` and does not stop the run. +name or by coordinate: discovery is offline, and every landmark has a map object with an approach. +A tag that is not a QID is listed in `rejected` and does not stop the run. The command uses two workers and at most ten requests per second in total. It sends `maxlag=5` and a descriptive User-Agent, and it waits for the `Retry-After` of a `429` or a `503` before it asks @@ -37,19 +37,18 @@ again. It captures the entities fifty per request, their P279 class closure, the articles at exact revisions, and the P18 and lead-image candidates with their Commons metadata. A batch whose response is larger than the compiler reads is asked for again in halves. Responses land in `entities/`, `classes/`, `locales/`, `articles/` and `images/`, with -`manifest.json` recording every URL, byte count, SHA-256 and outcome. The response -limit is 32 MiB. +`manifest.json` recording every URL, byte count, SHA-256 and outcome. The response limit is +32 MiB. -`obc-bake landmark-content` makes every text, image and attribution decision afterwards, offline. Site -eligibility comes from the exact polygon and the category policy, not from P17 country claims or a -curated list, and the acquisition checks its category file digest against the compiler's embedded -policy digest so the two cannot disagree. +`obc-bake landmark-content` makes every text, image and attribution decision afterwards, offline. +Site eligibility comes from the exact polygon and the category policy, not from P17 country claims +or a curated list. The acquisition checks its category file digest against the compiler's embedded +policy digest. **Resume with the same arguments and the same output directory.** A completed request is reused only after its URL, byte count and digest check out. Failed requests stay failed. A changed candidate list, boundary, category policy, language set or locale policy needs a **new** output -directory. -Never run two capture processes against one output directory. +directory. Never run two capture processes against one output directory. Manifest schema 2 records the request identity of every Commons category page. You can resume a schema 1 capture in the same output directory to add missing continuation pages and write schema @@ -57,8 +56,8 @@ schema 1 capture in the same output directory to add missing continuation pages category records because they cannot prove that every page was consumed. An incomplete acquisition exits with status 2 and keeps the bytes it captured. Unresolved source -requests or class closure make country coverage incomplete: do not report their -absence as zero landmarks or zero images. A `wikidata` tag that names no item is a fact about the +requests or class closure make country coverage incomplete: do not report their absence as zero +landmarks or zero images. A `wikidata` tag that names no item is a fact about the extract, not a failure: `unresolved_identities` counts those and coverage stays complete. Keep a separate copy of the built `obc-bake` when another build can replace the target binary; the diff --git a/fixtures/sources/ride-assistant/PLACES-VALIDATION.md b/fixtures/sources/ride-assistant/PLACES-VALIDATION.md index 7615028a90..c2ab004fa8 100644 --- a/fixtures/sources/ride-assistant/PLACES-VALIDATION.md +++ b/fixtures/sources/ride-assistant/PLACES-VALIDATION.md @@ -10,13 +10,13 @@ python3 fixtures/verify-assistant-places.py It reads the pinned `assistant-osm/switzerland.osm.pbf` from the fixture cache and crops it to `8.17,46.55,8.38,46.74`, keeping complete source ways at the crop edge. The test uses the repository routing preset and the production ingest, OBCM serializer, map reader and place -continuation query, so it checks place identity and access rather than rendering: it packs real -service records and the real navigation graph with an empty render layer and no DEM. +continuation query. It checks place identity and access, not rendering: it packs real service +records and the real navigation graph with an empty render layer and no DEM. A place gets an approach only through its source node's membership in a routable way. **No proximity join is used**, so a place with a road nearby but no such membership keeps an unavailable approach. The shared query returns bounded pages in stable order and keeps the original eligibility time -across continuation. No coverage map can prove that an installed map has no internal holes, so -results report `coverage_complete: false`, including for an empty result. +across continuation. Results report `coverage_complete: false`, including for an empty result: no +coverage map can prove that an installed map has no internal holes. diff --git a/fixtures/sources/ride-assistant/README.md b/fixtures/sources/ride-assistant/README.md index 3f99d86808..accabc90ee 100644 --- a/fixtures/sources/ride-assistant/README.md +++ b/fixtures/sources/ride-assistant/README.md @@ -22,7 +22,7 @@ source needs a new immutable package. No acquisition command runs during map gen `fixtures/verify-assistant-places.py` runs the `fixtures.assistant-places` suite against the pinned Swiss PBF; see [place query validation](PLACES-VALIDATION.md). -## Build the maps offline +## Build Build the shipping tools once while Cargo dependencies are available, then work from cached inputs: @@ -62,6 +62,31 @@ pre-extract would change the data in the boundary cells. These map-selection bou from the wider acquisition and replay bounds in `regions.geojson`. The catalog region ID is `europe/switzerland`; the assembled package is `sim-assistant-meiringen`. +## Run + +```sh +tools/obc sim assistant-dunlough-access +tools/obc sim assistant-monaco-dense +tools/obc sim assistant-out-and-back +tools/obc sim assistant-forward-rejoin +tools/obc sim assistant-loop-crossing +``` + +Once the packages and Cargo dependencies are cached, these need no live source API. + +For a persistent card, start with a path that does not exist: + +```sh +tools/obc sim assistant-dunlough-access -- --create-card .artifacts/west-cork.obc +cargo run --release -p obc-sim -- --card .artifacts/west-cork.obc --physical \ + --clock 2026-09-14T10:00 --utc-offset-min 60 +``` + +The second command reopens saved routes and recordings without importing the fixture again. +**The card is user state**: fixture sync does not replace it, so create a new card to test a new +package. See the [simulator README](../../../sim/desktop/README.md) for imports and recording +recovery. + ## Sources and licences Each `sources[]` entry in the JSON records below pins `path`, `url`, `retrieved_at`, `bytes` and @@ -112,29 +137,3 @@ The Dunlough paths carry `sac_scale=mountain_hiking` and mud or ground surfaces unknown. Hours are absent on the selected landmark objects. Dunlough Castle is Wikidata `Q5315471` through OSM way `300189816`, with real article and photo Sources; a truthful unavailable Visit is a valid result there. - -## Run the scenarios - -```sh -tools/obc sim assistant-dunlough-access -tools/obc sim assistant-monaco-dense -tools/obc sim assistant-out-and-back -tools/obc sim assistant-forward-rejoin -tools/obc sim assistant-loop-crossing -``` - -These use ordinary map loading and authored GPS replay. Once the packages and Cargo dependencies -are cached they need no live source API. - -For a persistent card, start with a path that does not exist: - -```sh -tools/obc sim assistant-dunlough-access -- --create-card .artifacts/west-cork.obc -cargo run --release -p obc-sim -- --card .artifacts/west-cork.obc --physical \ - --clock 2026-09-14T10:00 --utc-offset-min 60 -``` - -The second command reopens saved routes and recordings without importing the fixture again. -**The card is user state**: fixture sync does not replace it, so create a new card to test a new -package. See the [simulator README](../../../sim/desktop/README.md) for imports and recording -recovery. diff --git a/fixtures/sources/route-import/README.md b/fixtures/sources/route-import/README.md index 68937c4519..fc158936ec 100644 --- a/fixtures/sources/route-import/README.md +++ b/fixtures/sources/route-import/README.md @@ -12,5 +12,5 @@ test holds the two byte-identical. | --- | --- | --- | --- | | `komoot-schwarzwald.gpx` | komoot export, `creator="https://www.komoot.de"`, "Schwarzwald Tour · Tag 2": 150 track points and five ``. The symbols are Garmin names komoot copies through — `Flag, Blue` three times, `Restaurant`, and `Fishing Hot Spot Facility`, which no category fits. Who exported it is not recorded anywhere in the tree; it arrived as the iOS companion's import fixture. | ODbL-1.0 for the track, which is komoot's routing output over OSM ways. **The authored part — the chosen route and the five waypoint names — is unconfirmed**, pending the owner. | `companion-ios/Packages/OBCKit/Sources/OBCMock/Fixtures/sample-import.gpx` | -SwiftPM processes `Sources/OBCMock/Fixtures` as the `OBCMock` target's bundled resources, and a -resource must live inside its own target directory. The iOS copy therefore stays where it is. +Do not move the iOS copy: SwiftPM bundles `Sources/OBCMock/Fixtures` as resources of the `OBCMock` +target, and a resource must live inside its own target directory. diff --git a/host/obc-bake/assets/README.md b/host/obc-bake/assets/README.md index 045223a4ad..af54763163 100644 --- a/host/obc-bake/assets/README.md +++ b/host/obc-bake/assets/README.md @@ -5,10 +5,9 @@ Teningen camera through the production map renderer. The file is never published itself. `teningen-preview.obcd` is the OBCT terrain sidecar for the same square. `obc-bake` does not use -it yet; it exists so the preview geometry has real elevation available when terrain reaches the -shared render path. +it yet. -## Regenerate +## Build `fixtures/build-map-package.sh` is the only supported way, as it is for every built fixture here. The map is packed from a Geofabrik `europe/germany/baden-wuerttemberg/freiburg-regbez` extract @@ -30,13 +29,11 @@ fixtures/build-map-package.sh terrain ``` It is baked at the real v1 posting (`2^9` µdeg) with a `2^16` cell, so the 1 × 2 cell rectangle is -65 576 B rather than a 2 MiB v1 cell that is mostly outside the crop. `obct-terrain.md` §1.3 makes -both header data and §4.5 requires a reader to accept the pairing. +65 576 B. `obct-terrain.md` §1.3 makes both header data and §4.5 requires a reader to accept the +pairing. -There is no terrain in the map, deliberately: the Rhine plain has nothing to show in a 1.2 km -frame, and a preview whose subject is the skin should not double as a contour test. There is no -golden preview PNG either — the tests assert that the skins render distinctly and -deterministically, and that residential fills cover the frame. +The map has no terrain, and there is no golden preview PNG: the tests assert that the skins render +distinctly and deterministically, and that residential fills cover the frame. ## When to refresh it @@ -44,11 +41,10 @@ deterministically, and that residential fills cover the frame. checks the assignment before it starts a region bake and fails with this path rather than publishing stale previews. -One case deliberately owes no refresh: feature types *appended* to the schema take the next free -ids and leave every id in this file meaning what it meant, so `check_source` in -`host/obc-bake/src/previews.rs` requires the fixture's table to be a leading run of the schema's -assignment rather than all of it. A schema that stops covering an id this file carries still -fails. +Feature types *appended* to the schema owe no refresh: they take the next free ids, so +`check_source` in `host/obc-bake/src/previews.rs` requires the fixture's table to be a leading run +of the schema's assignment rather than all of it. A schema that stops covering an id this file +carries still fails. ## The published camera @@ -56,8 +52,7 @@ The 240 × 240 image is centred at `7.814,48.130` at 5 m/px. The live skin edito camera, allows pan and zoom, and treats the crop above as its dense coverage. The crop is wider than the frame, and the packer's relation-complete selection also pulls in every member of a land-cover multipolygon reached from inside it, which keeps residential, forest and farmland fills -whole. That is already enough for the interactive preview to select every rung of the ladder, so -no second browser fixture is needed. +whole. OSM complete-way retention can push the OBCM header beyond the requested bbox. **Those overhanging coordinates are not a licence to pan into sparse space**: at wide scales the camera diff --git a/host/obc-data/README.md b/host/obc-data/README.md index be3467860b..c78833fd71 100644 --- a/host/obc-data/README.md +++ b/host/obc-data/README.md @@ -10,19 +10,11 @@ The [data contract](../../specs/obc-data.md) defines bytes and admission checks. Prepare Rust, Python with `uv`, GEOS and Osmium before an update, and a Java 21 JDK and Maven for the planner layers; `obc data` does not install them. `obc data setup` shows what this machine can do and the tools and credentials that it has. -A changed producer does not rebuild its layer. Raise the `revision` of the layer in the change -that changes its output; a reader that needs new data raises its `format`. - -Each step runs in a process of its own. A Rust step shows as `obc-data obc_data_step FUNCTION` -in `ps`. Add the function of a new Rust step to `host/obc-data-steps/src/functions.rs`: a -function that is not there fails its step. A test binary that runs a Rust step needs a test -`obc_data_step` that calls `obc_data::engine::child` with its functions. Without that test, the -step fails because it did not write its output. `data/targets.toml` names the targets: `local` is a folder beside the store, and `prod` the public bucket. An update or a rollback of `prod` needs the `OBC_R2_*` variables and rclone. The -settings of a target (region, parts, refresh policies) are in the release that it -serves, so they need no commit. +settings of a target (region, parts, refresh policies) are in the release that it serves; they +need no commit. | Configuration or credential | Purpose | | --- | --- | @@ -51,7 +43,7 @@ serves, so they need no commit. Set a source credential on the Machine screen, or pipe it to `obc data source credential ID`. Put other secrets, such as the R2 key, in `credentials.env`, never in `tools/obc.local`. -## Daily commands +## Run | Command | Action | | --- | --- | @@ -88,12 +80,6 @@ or stops frees the lock. A crash or an out-of-memory kill keeps it; the next run machine takes it over. From another machine, `obc data unlock TARGET --from MACHINE` frees it after 10 minutes without a heartbeat. -`cargo test -p obc-data --test r2_live -- --ignored` writes the bucket of the `OBC_R2_*` variables -in a throwaway folder `test/