Skip to content

SOURCES.md

Latest commit

 

History

History
105 lines (84 loc) · 7.19 KB

SOURCES.md

File metadata and controls

105 lines (84 loc) · 7.19 KB
Raw

Test data provenance

Everything under testdata/spp/ is vendored from the reference implementation this project is validated against. Nothing here was written by hand; using real mission files is the whole point — a hand-rolled XTCE fixture would only test the assumptions of whoever wrote it.

Upstream
Project lasp/space_packet_parser
Commit 6de220ff25e75d0a8b6258086f9d99ce9eae820b (2026-08-07)
Version 6.1.2 — the commit's own string, which the 6.1.2 release also reports; see below
Path tests/test_data/
Licence BSD 3-Clause, © 2023 University of Colorado — see testdata/spp/LICENSE.txt

The BSD 3-Clause licence permits redistribution provided the copyright notice and licence text are retained, which testdata/spp/LICENSE.txt does.

Files

Definition + packet-stream pairs (used by the differential tests)

Definition Packet stream Packets Exercises
jpss/jpss1_geolocation_xtce_v1.xml jpss/J01_G011_LZ_2021-04-09T00-00-00Z_V01.DAT1 7200 container inheritance, ComparisonList, IEEE-754 floats
jpss/contrived_inheritance_structure.xml (same stream) 7200 BooleanExpression, ANDedConditions, Condition
ctim/ctim_xtce_v1.xml ctim/ccsds_2021_155_14_39_51 1499 1.6 MB definition, twosComplement, fixed-size UTF-8 strings
suda/suda_combined_science_definition.xml suda/sciData_2022_130_17_41_53.spl 13 enumerations, dynamically sized binary (DynamicValue + LinearAdjustment); stream needs skip_header_bytes=4
idex/idex_combined_science_definition.xml idex/sciData_2023_052_14_45_05 78 polymorphic containers, dynamically sized binary
jpss/jpss1_geolocation_xtce_v1.xml ctim/ccsds_2021_155_14_39_51 1499 rejection case: a definition pointed at a stream it does not describe, so both implementations must refuse every packet. Without it, the "we agree this cannot be decoded" half of the contract goes untested — the other five cases have no unrecognised packets at all

Definitions only (parser coverage)

File Exercises
test_xtce.xml the xtce: prefixed namespace form
test_xtce_default_namespace.xml the default-namespace form
test_xtce_no_namespace.xml the no-namespace form
test_xtce_4byte.xml minimal single-container definition
udp_packet.xml BinaryDataEncoding with DynamicValue sizing
SpaceSystem.xsd the XTCE 1.2 schema, for reference
jpss/ccsdspy_jpss1_geolocation.csv upstream cross-check table, unused here

Written for this project

Ten files here are not vendored. All exist for the same reason: an implementation path that the real mission files never reach, and so would never be compared against anything.

File Why
numeric_edges.xml the mission files contain one 32-bit float, no 16-bit float, and no numeric field spanning nine bytes
calibrators.xml no mission file has a calibrator at all — grepping all five returns nothing
boolean_criteria.xml the one mission file with a <BooleanExpression> has a single conjunction of equalities: no disjunction, no nesting, and no operator but ==
byte_order.xml + byte_order_stream.bin no mission file sets byteOrder — so there is no recorded telemetry with a little-endian field, and no bytes to put in front of the reference
arrays.xml no mission file has an <ArrayParameterType>, and neither does the reference: it raises NotImplementedError and calls the feature roadmap
aggregates.xml the same for <AggregateParameterType>, plus the two nested in each other, which is where the naming can go wrong
mil_1750a.xml + mil_1750a_stream.bin no mission file uses MIL-STD-1750A — but the reference implements it, so this one is a golden case like the six real ones
context_calibrators.xml + context_calibrators_stream.bin no mission file has a <ContextCalibratorList> — the reference implements one, so this is a golden case too, and the only stream whose bytes are shaped so that the criteria actually fire. One of its contexts is selected by an enumeration's label, which makes it the only place a criterion comparing a string is checked against Python rather than against the interpreter alone
commands.xml no mission file has a <CommandMetaData> section, and the reference has no command support of any kind — not even a NotImplementedError: the string does not appear in its source, and a definition carrying one loads with the command half silently ignored

byte_order_stream.bin, mil_1750a_stream.bin and context_calibrators_stream.bin are the only packet streams here that were not flown. It is written by tools/gen_byte_order_stream.py from a fixed seed, so regenerating it produces the same file and the golden taken over it stays meaningful. All three are golden cases like the six real ones, which makes little-endian, MIL-STD-1750A and context calibration the only features whose interpreted paths are checked against space_packet_parser directly. Little-endian found two divergences the first time it ran; the other two matched. See SUPPORTED.md.

Neither is a substitute for a real sample; they are a supplement to one. calibrators.xml in particular is built around a difference that only shows up in the last bit: the reference raises an integral raw value to its power exactly and rounds once, and a float raw by repeated squaring, which rounds at every step. Two of its parameters carry identical polynomial terms over the two encodings so that a generator collapsing them into one routine is caught.

Golden files

testdata/golden/*.json are generated by tools/gen_goldens.py running the Python reference, and are committed so that CI needs no Python interpreter. Each file holds:

  • full raw and engineering values for the first 64 packets;
  • a SHA-256 digest over the canonical encoding of every packet in the stream, so a mismatch in the tail cannot hide behind the truncated detail section;
  • the reference implementation's own load and parse timings, as the baseline for xtce bench.

Regenerate with:

$ python3.12 -m venv .venv
$ .venv/bin/pip install \
    'space_packet_parser @ git+https://github.com/lasp/space_packet_parser@6de220ff25e75d0a8b6258086f9d99ce9eae820b'
$ .venv/bin/python tools/gen_goldens.py

Install from the commit, not from PyPI. pip install space_packet_parser==6.1.2 succeeds — 6.1.2 is a real release — but it is not this commit. The commit is ahead of the tag: it reads a <UnitSet> holding several <Unit> elements as a tuple of units, where the released 6.1.2 raises NotImplementedError on the second one. Both call themselves 6.1.2, so pip show cannot tell them apart, and a golden regenerated against the release would differ with nothing to say why. No definition under testdata/ uses compound units today, so the two agree on everything here — that is luck, and the next definition added is under no obligation to preserve it.

Values are encoded losslessly: integers as JSON numbers, floats as {"f": "<float.hex()>"} so the IEEE-754 bit pattern round-trips exactly, and bytes as {"b": "<hex>"}.