[lykakis]

Summary

• Introduce backend-neutral pending, selected-router, routed-work and coordinator contracts with typed ownership identities.
• Implement bounded, non-durable memory storage and activate the standalone outbound pipeline with fail-fast backend profile validation.
• Connect HTTP/SMPP admission, multipart ownership, single-destination routing, provider completion, required DLR handoff, retry policies and graceful shutdown.
• Preserve embedding extension points and legacy entry points; document the supported lifecycle and guarantees.
• Integrate asynchronous, batched provider DLR persistence: coordinated completion retains source ownership until persistence commits and required handoffs finish.
• Bound completion monitoring and use per-worker FIFO preparation rather than per-message contention polling; expose unexpected standalone processor termination through readiness.

Verification

• Local full reactor verification passed 415 core unit tests, 54 integration tests and 53 app tests, with packaging and zero Checkstyle violations.
• The performance harness verification passed 157 tests.
• Additional regressions cover source retention until async persistence commit, persistence failure propagation, rejection handoff timing, completion-monitor retention, worker FIFO/concurrency limits and shutdown.
• Backend, native E2E, CodeQL/code quality and DCO checks passed on the AWS-tested application revision 4271407fdec71e82b189b33c99ab0bc927f35469.

AWS performance evidence

Both runs below tested application revision 4271407fdec71e82b189b33c99ab0bc927f35469, JVM mode, DLR enabled, offered 350 messages/s, unlimited worker TPS, 120-second warm-up, 600-second measurement and 120-second maximum drain. Sendium ran on t4g.small, the harness on c7g.large, and PostgreSQL 17.11 on db.t4g.small.

Protocol	Measurement messages accepted / provider-completed / receipts forwarded	Provider completions during measurement	Provider-completion p99	Drain	Evidence
HTTP	210,000 / 210,000 / 210,000	349.952/s	107.711 ms	1.984 s	Run 41662595485
SMPP	210,000 / 210,000 / 210,000	349.977/s	108.863 ms	1.342 s	Run 41795543177

Both observations had zero generator misses, rejections, admission failures/timeouts, duplicate receipts or unfinished accepted work after drain. Minute-boundary provider backlog remained small and bounded, and no late forward-progress stall appeared. Evidence validation and cleanup passed.

The existing provider-batching SMPP reference at 350/s is additional context, not a new main-branch matched control: it tested a different application branch and harness revision. These single AWS observations demonstrate the tested workload, not maximum capacity or an indefinite-duration guarantee. HTTP receipt-forwarding latency includes its callback path and must not be compared as an identical boundary to SMPP receipt return.

Architecture: before and after

The standalone outbound path changes from passing messages between runtime queues to coordinating explicitly owned stages, retaining accepted work until provider processing and required handoffs finish. The diagrams compare main 8a67c23 with this feature and show shipped defaults.

Before: queue-driven processing

HTTP submissions entered the legacy router queue; SMPP submissions first passed through protocol-specific ingress buffering and multipart handling. The router applied routing rules and enqueued messages for destination workers. Workers submitted to providers, while callbacks and worker policies handled outcomes and retries.

flowchart TB
    http["HTTP"] --> routerQueue["Router queue"]
    smpp["SMPP"] --> ingressBatch["SMPP buffer (8)<br/>100 events / 100 ms"]
    ingressBatch --> routerQueue
    routerQueue --> routing["Routing"]
    routing --> workerQueue["Worker queue"]
    workerQueue --> worker["Worker"]
    worker --> provider["Provider"]
    provider -. "Outcomes / DLRs" .-> tracking["DLR subsystem"]

Loading

The queue provider already abstracted queue access. What was missing was a shared, backend-neutral outbound lifecycle record connecting admission, route assignment, in-flight provider processing, and final cleanup. Responsibility was distributed across queue membership, worker state, and callbacks.

After: coordinator-owned stages

The standalone application assembles an OutboundCoordinator and three stage stores. Accepted sources remain retained while later stages execute. References such as (1) point to the notes below; numbers in the diagrams show shipped defaults. Store capacities and per-cycle take limits are separate bounds.

flowchart TB
    ingress["HTTP / SMPP"]
    cycle["Cycle (7)<br/>100 ms idle backoff"] -.-> selected
    ingress --> pending["Pending (1)<br/>10,000 sources max"]
    pending -. "Admitted" .-> ack["Client ACK"]
    pending -->|"100/cycle max"| selected["Selected (2)<br/>1,000 unassigned items max"]
    selected -->|"100/cycle max"| routing["Routing"]
    routing --> routed["Routed (3)<br/>10,000 selections max"]
    routed -->|"100/worker/cycle max"| dispatch["Dispatch (4)<br/>1 preparation slot/worker"]
    dispatch --> provider["Provider"]
    provider -. "Retry" .-> dispatch
    provider --> handoff["Handoff (5)"]
    handoff --> cleanup["Cleanup (6)"]

Loading

Only Pending, Selected and Routed are outbound stores. Dispatch, handoff and cleanup are processing phases. The coordinator governs ownership and transitions across them; arrows show progression, not deletion of earlier records.

Separate DLR batches - already present on main

Provider-event persistence and client receipt delivery are separate batches:

flowchart TB
    events["Provider events"] --> processor["DLR batch (9)<br/>100 events / 500 ms<br/>10,000 outstanding max"]
    processor --> postgres["PostgreSQL"]
    postgres -. "Accept/reject commit" .-> handoff["Handoff (5)"]
    postgres -->|"Final outcome"| delivery["Pending DLR"]
    delivery --> http["HTTP return (10)<br/>500 claims / 1 s schedule<br/>10 callbacks/host max"]
    delivery --> smpp["SMPP return (11)"]

Loading

Diagram notes

• (1) Pending — PendingMessageStore: up to 10,000 accepted sources, including held multipart parts and in-flight sources. Admit before HTTP 202 / SMPP success. When full, new admissions fail: HTTP 503 or SMPP STATUS_THROTTLED; they are not acknowledged as accepted. Existing work keeps processing, and cleanup frees space for later admissions. Moving a message to the next stage does not free its pending slot. An aggregate retains every original source ID.
• (2) Selected — SelectedRouterStore: up to 1,000 queued or taken unassigned items. Select up to 100 ready items/cycle, limited by free slots, and route up to 100 takes/cycle through existing routing rules. When full, additional ready work stays in Pending until a routing slot is free. Recording a destination frees that slot but keeps bookkeeping until cleanup. A routing miss or recoverable lookup failure returns the item for another routing attempt; a terminal routing drop discards it. An aggregate is one selected item; held parts wait for reassembly/publication before selection.
• (3) Routed — RoutedWorkStore: up to 10,000 retained selections across all destinations, using the same configured capacity as pending. Records the chosen destination and queues assigned work for that worker. “Unfinished” means still waiting to be sent, awaiting provider outcomes, running retries, or waiting for required handoff/cleanup. Taking an item for dispatch does not remove its ownership record. If destination recording cannot succeed, retain the original assignment intent and retry it without repeating routing lookup; work is not silently dropped or sent without a record.
• (4) Dispatch: take up to 100 items per eligible worker/cycle into per-worker FIFO execution. Preparation concurrency is threadCount (default 1), not the take batch size. Busy or paused workers leave work waiting; the TPS limit controls how quickly attempts can send. The shared worker TPS cap defaults to 0 = unlimited. Async responses do not hold preparation slots, but their messages still occupy pending capacity. Provider outcomes drive completion or the configured retry policy; multipart requests and replacement attempts stay under the original source ownership.
• (5) Handoff: after provider processing, finish required tracking/persistence, including its commit and any required DLR enqueue. If this fails, keep the message owned and retry the handoff, not an already accepted provider request. Acceptance/rejection commit notification alone does not bypass other handoff conditions. This stage does not wait for final handset delivery or for the client to acknowledge a later delivery receipt.
• (6) Cleanup: only after all provider parts/replacement attempts are terminal and required handoffs succeed: complete routed work, release pending sources, remove selected bookkeeping, then forget the routed record. This frees Pending and Routed capacity. If cleanup partly fails, remember successful steps and retry the remaining cleanup without resending provider requests or repeating a successful handoff.
• (7) Cycle: one standalone processing thread drives selection, routing and provider takes. All three 100-item limits share sendium.message.selection-batch-size, applied separately. Do not wait for batches to fill. Continue immediately after productive routing/dispatch; back off 100 ms when idle, paused or unsuccessful.
• (8) SMPP buffer: gather up to 100 ingress events or wait up to 100 ms. Main uses this protocol-specific path before routing. In the coordinated path it remains for multipart/reassembly processing; ordinary single-part SMPP admission goes directly to pending storage. This is separate from outbound selection batching.
• (9) DLR persistence batch — ProviderDlrProcessor: up to 10,000 outstanding events, including processing/completion. Collect up to 100 events, waiting up to 500 ms from the oldest event, then persist. Acceptance, rejection and final-receipt events share this processor. 100 events is not 100 messages; one message can produce multiple events. Collection delay is not an end-to-end commit-latency guarantee.
• (10) HTTP receipt return: claim up to 500 due receipts per invocation, scheduled every 1 second, with no overlapping batches and at most 10 callbacks in flight per host. A slow batch skips later checks; 500 is not a guaranteed callbacks-per-second rate.
• (11) SMPP receipt return: send deliver_sm and complete the delivery attempt after the required successful deliver_sm_resp acknowledgements. This path does not use the HTTP batch-500 limit.

Settings behind the diagram

Setting / limit	Default	Controls
sendium.message.pending.capacity	10,000 sources	Total retained accepted sources; also supplies routed-store capacity
sendium.message.router-queue.capacity	1,000 items	Queued plus taken, unassigned routing items
sendium.message.selection-batch-size	100 items	Per-cycle selection, routing-take and per-worker provider-take budgets
Worker threadCount / tps	1 / 0	Concurrent preparation threads per worker / shared TPS cap; 0 means unlimited
sendium.dlr.batch.size	100 events	Provider-event persistence batch
sendium.dlr.batch.max-delay	500 ms	Maximum collection waiting from the oldest event before proceeding with the batch
sendium.dlr.batch.queue-capacity	10,000 events	Outstanding provider-persistence events, including processing/completion
HTTP DLR return constants	500 / 1 s / 10 per host	Due-receipt claim limit / schedule / callback concurrency

None of the per-cycle batch values is a messages-per-second cap. The SMPP request window is another independent limit; our performance preset uses a 1,000-request window.

Important boundaries

• DLR persistence is separate from outbound storage. Current main already has asynchronous, batched PostgreSQL DLR persistence and receipt forwarding. This PR integrates with that subsystem; it does not introduce it.
• Outbound completion does not wait for final handset delivery or downstream receipt acknowledgement. It waits for the provider processing outcome and required handoff, including its persistence commit.
• Only memory/memory/memory outbound storage is implemented. The contracts create a replacement boundary for later backends; they do not make accepted outbound work durable today. Restart can still lose unfinished outbound work.
• Shutdown drains active provider executions and required handoffs before closing workers/stores; it does not persist the remaining queued backlog.
• The new standalone execution path currently supports one SMPP-client destination per selection and rejects copied routes. Legacy entry points remain available for existing library callers.

Current guarantees

• The supported outbound profile is memory/memory/memory; accepted and in-flight outbound work remains non-durable across process restart.
• Pending sources stay owned through provider processing and required persistence/handoff completion; queue insertion alone is not terminal success.
• The new lifecycle records one destination per selection and explicitly rejects copied routes; legacy entry points remain available.

Refs #338
Related epic: #337

[lykakis]

This parameter is required by CDI: @observes StartupEvent registers the startup callback and @priority establishes its ordering. The event payload is intentionally unused. Retaining the parameter; an explanatory source comment is prepared locally.

[lykakis]

The @observes StartupEvent parameter registers early startup logging and profile resolution in CDI. Removing it would remove the observer registration even though the payload is unused. Retaining it; an explanatory source comment is prepared locally.

[lykakis]

The @observes ShutdownEvent parameter registers provider draining before worker shutdown. Its payload is intentionally unused, but the parameter and annotation are required for CDI lifecycle invocation. Retaining it; an explanatory source comment is prepared locally.

[lykakis]

The @observes StartupEvent parameter registers pipeline activation after worker startup. Its payload is intentionally unused, but removing the parameter would remove the CDI observer registration. Retaining it; an explanatory source comment is prepared locally.

[lykakis]

Completed defensively snapshots sources through copySources(), which calls Set.copyOf(). The sources.clear() line cited by the alert intentionally mutates the caller collection in a regression test. Subsequent assertions prove Completed retains both source IDs and completed.sources().clear() throws UnsupportedOperationException. OutboundWorkTest passes; no additional copy is needed.