Skip to content

Wrap wirelog's typed row API so float columns work in a Session - #188

Merged
justinjoy merged 1 commit into
mainfrom
feat/typed-float-session
Sep 6, 2026
Merged

justinjoy merged 1 commit into
mainfrom
feat/typed-float-session

Conversation

@justinjoy

@justinjoy justinjoy commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Wraps wirelog 0.60.0's typed row C API so a float column is usable from the advanced Session. No version bump and no CHANGELOG entry — this is the feature only; releasing it is a separate decision.

Why five entry points, not just insert

1.0.6 brought float support, but only through a program's source text. The untyped entry points carry int64 lanes, and the engine refuses them where a float is involved — verified against the C source and the live engine:

  • wirelog_session_insert / wirelog_session_remove refuse the float-bearing relation
  • wirelog_session_set_delta_cb and the untyped snapshot refuse whenever the program carries a float anywhere, including in a compound slot

So insert() into such a relation, and step() / snapshot() on such a program, do not work at all. Wrapping insert alone would have shipped a write with no way to read it back or drive the session.

with Program.from_string(SRC) as prog, Session(prog) as s:
    s.insert_typed("sample", [(1, 2.5), (2, 3.5)])
    s.step_typed()            # [("seen", (1, 2.5), 1), ("seen", (2, 3.5), 1)]
    s.remove_typed("sample", [(1, 2.5)])
    s.step_typed()            # [("seen", (1, 2.5), -1)]

Column types come from the relation's declaration, so callers pass plain Python rows and never build lane arrays.

API

  • Session.insert_typed / remove_typed / snapshot_typed / step_typed / set_typed_delta_callback
  • TypedRowError(ExecError) — carries the TypedErrorCode, offending row index and logical column, and wirelog's own diagnostic. Existing except ExecError handlers keep working.
  • TypedErrorCode exported, so err.typed_code is comparable without importing a module docs/api-stability.md declares private.
  • pyrewire._core.lanes — the value ↔ lane codec, shared with the typed delta trampoline (which cannot import session without a cycle).

Value policy. str is refused: a STRING column carries an intern id and the advanced Session has no forward-intern entry point, so coercing would write a wrong id (int("5") is a valid id for a different symbol). Anything integer-like or float-like is accepted, so NumPy scalars work as they do through insert_batch. Integer-likes go through operator.index and stay exact above 2⁵³. Where PyreWire can tell the float conversion lost the value it refuses rather than storing it wrong — Decimal(2**63 - 1) names an integer no binary64 can hold. NaN and the infinities are rejected by the engine, and a batch is validated whole before any of it is applied.

Left unwrapped: wirelog_session_make_compound_typed and the wirelog_extension_* family. Neither is needed for a float column. A float inside a compound term does stay unreachable as a consequence, since the untyped make_compound refuses a FLOAT argument.

Note for whoever cuts the next release

This adds two names to pyrewire.__all__ (TypedRowError, TypedErrorCode), taking it from 47 to 49. Every 1.0.x release held it at 47, and docs/api-stability.md names new APIs and exception subclasses as minor-release surface — so the release that ships this should be a minor bump, not a patch. docs/api-stability.md is updated here to list both names, which its contract test requires; nothing else version-related is touched.

Compatibility

The typed symbols first exist in wirelog 0.60.0 while the documented floor is 0.52.0, so registration is guarded by has_typed_row_api(), mirroring the existing wirelog_program_get_relation_ir precedent. An unguarded attribute access raises AttributeError at import time and makes the package unimportable on an older engine — that regression was caught during review. The typed methods raise WirelogVersionError there instead. The engine pin is unchanged.

Validation

wirelog 0.60.0 wirelog 0.54.0 (pre-typed)
full suite 652 passed, 10 skipped 629 passed, 33 skipped

mypy --strict clean over all 28 source files; black, isort, flake8 clean. The ABI was checked against wirelog-types.h field-for-field, and the typed structs satisfy every check in the engine's own session_typed_rows validator.

The single failure on both engines, test_installed_wheel_is_recognized_as_typed_by_mypy, is environmental and reproduces on unmodified main: the test builds a venv with system_site_packages=True, which from inside another venv resolves to the base interpreter where mypy is not installed.

https://claude.ai/code/session_01MqHwvCimcvrg1osEY5MH7o

return
rel = relation.decode() if relation else ""
state.queue.append(("typed_delta", rel, decode_typed_row(row[0]), int(diff)))
except BaseException as exc: # never propagate to C
Comment thread tests/test_typed_rows.py Fixed
Comment thread tests/test_typed_rows.py Fixed
Comment thread tests/test_typed_rows.py Fixed
@justinjoy
justinjoy force-pushed the feat/typed-float-session branch from 15bcae1 to 67d7249 Compare September 5, 2026 11:24
Comment thread tests/test_typed_rows.py

def test_a_raising_float_conversion_is_reported_as_a_type_error():
class Exploding:
def __float__(self):
Comment thread tests/test_typed_rows.py Fixed
@justinjoy
justinjoy force-pushed the feat/typed-float-session branch 2 times, most recently from c4f39f2 to f7d4a9a Compare September 5, 2026 12:14
@justinjoy justinjoy changed the title Wrap wirelog's typed row API so float columns work in a Session Wrap wirelog's typed row API so float columns work in a Session (1.1.0) Sep 5, 2026
wirelog 0.60.0 added float support, but a float could reach a program
only through its source text. The untyped session entry points carry
int64 lanes, and the engine refuses them outright where a float column
is declared: wirelog_session_insert refuses the float-bearing relation,
while wirelog_session_set_delta_cb and the untyped snapshot refuse
whenever the program declares a float column anywhere. So insert(),
step() and snapshot() do not work on such a program, and wrapping insert
alone would have shipped a write with no way to read it back.

Wrapped: insert_typed, remove_typed, snapshot_typed, step_typed and
set_typed_delta_callback. Column types come from the relation's
declaration, so callers pass plain Python rows and never build lane
arrays. Left unwrapped: wirelog_session_make_compound_typed and the
wirelog_extension_* family. Neither is needed for a float column, though
a float inside a compound term does stay unreachable as a result, since
the untyped make_compound refuses a FLOAT argument.

TypedRowError subclasses ExecError so existing handlers keep working
while gaining the row index, logical column and wirelog's own bounded
diagnostic. The message buffer is caller-owned, which is why the struct
field is POINTER(c_char) and not c_char_p - the latter surfaces as an
immutable bytes and the engine's message would land where nothing can
read it. TypedErrorCode is exported alongside it, since typed_code is
public and naming it otherwise means importing from a module
docs/api-stability.md declares private.

The lane conversion lives in _core.lanes rather than on Session because
the typed delta trampoline needs it too and cannot import the session
module without a cycle. A FLOAT lane is host-order IEEE-754 bits, so
every float crosses through an explicit reinterpret; UINT32/UINT64
decode unsigned and BOOL decodes to bool, where the untyped path returns
everything signed. encode_lane accepts anything integer-like or
float-like, so a NumPy scalar works as it does through insert_batch, and
refuses a value that would not survive the conversion rather than
storing it wrong - Decimal(2**63 - 1) names an integer no binary64 can
hold. str is the deliberate exclusion: a STRING column carries an intern
id and the advanced Session has no forward-intern entry point, so
coercing would write a wrong id.

insert_typed and remove_typed take TypedRow, not the EasySession Row
alias, and the typed readers return LaneValue. Row admits str because
EasySession.insert auto-interns it; declaring it here would let mypy
wave through the one input that always raises and reject the NumPy
scalars that work. The row count and the descriptor array size both come
from one materialized list, so a Sequence whose __len__ changes between
them cannot make wirelog read past the allocation.

_delta_cb can now hold either callback kind, so set_delta_callback(),
step() and close() dispatch on the armed kind. Reusing the other kind's
handle hands ctypes a mismatched function pointer, which it rejects only
after the stored callable has been overwritten. Clearing through the
matching entry point is insurance rather than a fix: 0.60.0's
set_delta_cb nulls typed_delta_cb unconditionally and gates its
FLOAT refusal on a non-NULL callback, so either clear disarms both today
- but nothing in the public C contract promises that stays true.

Registration is guarded by has_typed_row_api(), mirroring
wirelog_program_get_relation_ir in _parser. These symbols first ship in
0.60.0 while the loader floor still admits 0.52.0, and an unguarded
attribute access would raise AttributeError at import time and make the
package unimportable against an older engine. The typed methods raise
WirelogVersionError there instead.

Validated against both engines built from source, and mypy --strict
clean over all 28 source files.

Claude-Session: https://claude.ai/code/session_01MqHwvCimcvrg1osEY5MH7o
@justinjoy
justinjoy force-pushed the feat/typed-float-session branch from f7d4a9a to bcffe5d Compare September 6, 2026 07:01
@justinjoy

Copy link
Copy Markdown
Contributor Author

Response to the github-code-quality findings

Six inline findings across three passes. Two are addressed in 0c0ac5f; four are deliberate and I'm leaving them, with reasons below so a reviewer doesn't have to re-derive them.

Addressed

Module imported with both import and import from (reported twice — tests/test_typed_rows.py at the import as site and again at the import from site). Genuine; both spellings of pyrewire._core.callbacks sat inside one function. Unified on callbacks_mod._REGISTRY, which resolves both reports.

Unused local variables _rel / _diff (reported twice, same statement). Underscore-prefixed and not flagged by flake8, but the finding is a fair prompt: naming values only to discard them is weaker than checking them. The signed-zero test now asserts the whole event instead of the value alone:

[(relation, (key, value), diff)] = events
assert (relation, key, diff) == ("seen", 1, 1)
assert value == 0.0
assert math.copysign(1.0, value) > 0

That is a stronger test, not just a quieter one.

Not addressed — deliberate

except BaseException in _typed_delta_trampoline (src/pyrewire/_core/callbacks.py:143).

This is a hard requirement of the module, stated in its own docstring:

No exceptions into C. Python exceptions raised inside a callback must not propagate across the FFI boundary. The trampolines try/except BaseException and stash the error on the registry slot. CallbackHandle.drain() then re-raises after wirelog has returned control to Python.

wirelog invokes these from its own worker threads. An exception that escapes here crosses into C and takes the process down; the catch is what converts it into a Python-side re-raise at a safe point. The two pre-existing trampolines do exactly the same at lines 92 and 115 — only line 143 is flagged because only it is in this diff, so the finding reads as a new deviation when it is in fact the established pattern being followed.

It is also load-bearing rather than defensive: mutating this handler to swallow instead of stash is caught by the suite.

Non-standard exception raised in special method (Exploding.__float__, tests/test_typed_rows.py:739-741).

The suggestion is that the method "does not need to be implemented". Implementing it to raise is the fixture — the test exists to prove that a __float__ which raises is reported as a TypeError rather than leaking an arbitrary exception out of encode_lane:

def test_a_raising_float_conversion_is_reported_as_a_type_error():
    class Exploding:
        def __float__(self):
            raise RuntimeError("boom")

    with pytest.raises(TypeError, match="__float__ raised"):
        encode_lane(Exploding(), ColumnType.FLOAT)

Removing the method removes the test. The same shape covers the __eq__ case a few tests below.

Verification after the change

653 passed / 10 skipped on wirelog 0.60.0, 630 passed / 33 skipped on 0.54.0 (the older engine that lacks the typed symbols); black, isort, flake8 clean. The single failure on both engines, test_installed_wheel_is_recognized_as_typed_by_mypy, is environmental and reproduces on unmodified main.

https://claude.ai/code/session_01MqHwvCimcvrg1osEY5MH7o

@justinjoy
justinjoy force-pushed the feat/typed-float-session branch from bcffe5d to 0c0ac5f Compare September 6, 2026 08:18
@justinjoy justinjoy changed the title Wrap wirelog's typed row API so float columns work in a Session (1.1.0) Wrap wirelog's typed row API so float columns work in a Session Sep 6, 2026
@justinjoy
justinjoy merged commit 4aba4ff into main Sep 6, 2026
17 checks passed
@justinjoy
justinjoy deleted the feat/typed-float-session branch September 6, 2026 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant