Skip to content

fix: reject subscriber settings that delete data or never consume - #227

Merged
lesnik512 merged 1 commit into
mainfrom
fix/strict-subscriber-validation
Oct 17, 2026
Merged

lesnik512 merged 1 commit into
mainfrom
fix/strict-subscriber-validation

Conversation

@lesnik512

Copy link
Copy Markdown
Member

Closes #215. For 0.15.0: subscriber settings that would delete data or never consume now raise when the subscriber is registered.

What raises now

All checks run in OutboxSubscriberConfig._validate, so every construction path goes through them: @broker.subscriber, @router.subscriber and OutboxRoute.

  • max_deliveries below 1 raises ValueError. Each row has deliveries_count >= 1 by the time it's checked, so 0 put every row over the cap and deleted the queue (or sent it to the DLQ) without running the handler. None still means no cap.
  • NaN or infinite min_fetch_interval, max_fetch_interval or lease_ttl_seconds raises ValueError. NaN slipped past every <= check, and an infinite TTL broke make_interval in the fetch SQL.
  • A bool passed as a count (max_workers, fetch_batch_size, terminal_flush_batch_size, max_deliveries) raises TypeError. fetch_batch_size=True used to run as 1.
  • Queue names that are empty, longer than 255 characters, not a str, or listed twice in one subscriber now raise. The messages and limits match the publish-side checks, so a subscriber can't listen on a name nothing can publish to.
  • include_router(prefix=...) raises ValueError on the broker and on OutboxRouter. The prefix used to be dropped silently, so a router ported from Kafka or Redis never consumed. The FastAPI router's HTTP prefix is unaffected, because it never reaches the broker.

Default retry strategy after validation

The registrator used to fill in ExponentialRetry before validation ran. As a result, ack_policy=REJECT_ON_ERROR warned "retry_strategy is ignored" even when no strategy had been passed, and the "max_deliveries without retry_strategy" branch could never be reached. The config now validates the value as passed and fills in the default afterwards. _default_retry_strategy moves to retry.py so the config can call it.

  • REJECT_ON_ERROR with no strategy is silent. With an explicit active strategy it still warns.
  • The max_deliveries warning now fires only for NoRetry(). With no strategy, the default retries, so the cap can be reached.
  • Three tests in test_fake.py that suppressed the false warning no longer need to. Their suppressions are removed.

Release note (0.15.0, breaking)

Subscriber registration now rejects settings that previously passed and then deleted data or consumed nothing: max_deliveries of 0 or less (use None for no cap), NaN or infinite min_fetch_interval / max_fetch_interval / lease_ttl_seconds, a bool for an integer setting, queue names that are empty, longer than 255 characters or repeated in one subscriber, and include_router(prefix=...). ack_policy=REJECT_ON_ERROR without a retry_strategy no longer warns.

Tests

Each check has a test, and every test failed before the change except the guard that the default strategy is still applied. Docs updated: docs/usage/subscriber.md (option table and validation paragraph) and docs/usage/router.md (include_router prefix).

Verification

  • 678 tests pass in docker at 100% coverage, and the full suite also passes on SQLAlchemy 2.0.0.
  • One run under heavy machine load failed test_publish_with_activate_in_delays_delivery on timing. It passed on the next two runs, and this change doesn't touch that path.
  • just bench-check passes, ruff is clean, ty shows the same 6 diagnostics as main, and the docs build.
  • No hot-path change: all the checks run once, at registration.

@github-actions

Copy link
Copy Markdown

Benchmark gate

✅ gate passed

scenario msg/s delete/msg WALrec/msg WALB/msg fpi upd del dead_tup
consumer/w1/b10 1620 1.000 6.68 910 242 5000 5000 10000
consumer/w1/b100 2031 1.000 7.10 1144 243 5000 5000 10000
consumer/w2/b10 1994 1.000 6.72 924 242 5000 5000 10000
consumer/w2/b100 2947 1.000 7.10 1144 243 5000 5000 10000
consumer/w4/b10 2113 1.000 6.72 929 242 5000 5000 10000
consumer/w4/b100 3329 1.000 7.09 1143 243 5000 5000 10000
consumer/w1/b100/tfbs100 11364 0.010 6.07 1117 243 5000 5000 10000
consumer/w1/b100/interval1-10 2061 1.000 7.10 1144 243 5000 5000 10000
producer/w1/b100 6415 0.000 3.04 585 1 0 0 0
producer/w1/b100/ppt1 1228 0.000 4.10 670 23 0 0 0

Gated (fails the build): delete_calls + tuple counters (upd/del/ins) + the producer's insert_calls, exact; select_calls and other_calls within +2; wal_records within a 10% band; fetch_calls within 2x. msg/s, WAL bytes and total calls are informational (timing/FPI noise).

@lesnik512
lesnik512 merged commit 8112f73 into main Oct 17, 2026
13 checks passed
@lesnik512
lesnik512 deleted the fix/strict-subscriber-validation branch October 17, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Subscriber validation accepts settings that delete data or never consume

1 participant