Skip to content

feat: add rust workspace with queue crate - #1684

Merged
ferhatelmas merged 2 commits into
masterfrom
ferhat/rust
Nov 10, 2026
Merged

ferhatelmas merged 2 commits into
masterfrom
ferhat/rust

Conversation

@ferhatelmas

Copy link
Copy Markdown
Member

What kind of change does this PR introduce?

feat

What is the current behavior?

Events are sent to and processed by the existing ts pg-boss queue.

What is the new behavior?

Add a standalone Rust queue transport with an AWS SQS implementation.
It supports sending and receiving messages, visibility timeouts, acknowledgement and structured batch error reporting.

Also add ElasticMQ based contract tests and a CI workflow.

Additional context

Transport is not yet connected to producers or workers so existing event processing continues to use pg-boss

@ferhatelmas
ferhatelmas requested a review from a team as a code owner November 9, 2026 15:00
Copilot AI balanced review requested due to automatic review settings November 9, 2026 15:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟡 rust/crates/queue/src/sqs.rs — On real AWS SQS, callers extending a stale receipt may get a generic Error::Sqs instead of the documented Error::StaleReceipt. The mapping at sqs.rs:204-205 compares e.code() against two bare strings, "MessageNotInflight" and "ReceiptHandleIsInvalid". Those strings match what ElasticMQ returns. AWS SQS can report the same condition under other codes. Fix: match on the typed service-error variants, and also map the AWS stale-receipt forms. That means AWS.SimpleQueueService.MessageNotInflight and InvalidParameterValue when its message says the message is not available for a visibility change. [also at: rust/crates/queue/src/sqs.rs:205 - On real AWS SQS, calling extend with a receipt for a deleted or expired message may return a generic Error::Sqs, not the documented StaleReceipt.]

    Why this was flagged

    This applies to deployments against real AWS SQS, not ElasticMQ, which is the only endpoint CI tests. Here is the trigger. A worker calls extend (sqs.rs:485) with a receipt whose visibility already lapsed, or whose message was deleted or redelivered. AWS SQS then commonly answers InvalidParameterValue ("Value ... for parameter ReceiptHandle is invalid. Reason: Message does not exist or is not available for visibility timeout change."). For query-compatible errors it can also send AWS.SimpleQueueService.MessageNotInflight from the x-amzn-query-error header. Neither matches the string patterns at sqs.rs:495, so the error falls to sqs(e). That contradicts the trait doc at lib.rs:201, which promises Error::StaleReceipt. Callers that branch on StaleReceipt to drop a lost claim will see an opaque transport error and may retry or crash. The contract test stale_receipt (contract.rs:770) runs only against ElasticMQ, so it does not catch this. This is new code, so the base branch has no such behaviour.

    Verification: The trigger is extend running against real AWS SQS with a receipt whose message was deleted or delivered again. In rust/crates/queue/src/sqs.rs:204-205, only "MessageNotInflight" and "ReceiptHandleIsInvalid" become StaleReceipt, so AWS's InvalidParameterValue becomes the generic Error::Sqs. The only test, contract.rs:166-181, runs only against ElasticMQ.

Comment thread rust/crates/queue/src/sqs.rs Resolved
Comment thread rust/crates/queue/src/sqs.rs Resolved
Comment thread rust/crates/queue/src/sqs.rs Resolved
Comment thread rust/crates/queue/src/lib.rs Resolved
@coveralls

coveralls commented Nov 9, 2026 •

Copy link
Copy Markdown

Coverage Report for CI Build 42819578232

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage remained the same at 87.484%

Details

  • Coverage remained the same as the base build.
  • Patch coverage: No coverable lines changed in this PR.
  • No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 15902
Covered Lines: 14354
Line Coverage: 90.27%
Relevant Branches: 10080
Covered Branches: 8376
Branch Coverage: 83.1%
Branches in Coverage %: Yes
Coverage Strength: 1300.99 hits per line

💛 - Coveralls

@ferhatelmas

Copy link
Copy Markdown
Member Author

@claude review always

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed the updated push and found no new bugs. Since this is a new 2.6k-line crate with its own CI workflow, and CODEOWNERS covers all paths, someone on the storage team should still take a look.

What was reviewed:

  • Three of my earlier points are fixed. delete now reports a rejected receipt (such as ReceiptHandleIsInvalid) as Failed. send now splits chunks by total body bytes as well as by count. DelaySeconds is now always sent, so an explicit 0 overrides the queue's default delay.
  • The extend thread about the 12-hour cap counted from receive is still open. The code and docs for that path haven't changed.
  • I checked how outcomes are mapped back to inputs. Ids out of range, duplicated or from another chunk are rejected, and that chunk is reset to Unknown.
  • I checked the new workflow. It has read-only contents permission, runs only on pull_request and push, pins its actions to commit SHAs, and points ElasticMQ at localhost only.
Extended reasoning...

The change adds a standalone Rust workspace with an SQS-backed queue crate. It also adds an ElasticMQ compose service, a CI workflow, and a .dockerignore entry. The crate isn't wired into any producers or workers yet. It touches no auth, crypto or data-exposure code, and the workflow has read-only permissions on non-privileged triggers. I deferred instead of approving because the change is large, .github/CODEOWNERS assigns every path to a team, and one of my earlier review threads is still open.

Still open from earlier reviews (1):

  • Unresolved: 1 minor or pre-existing.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed the latest push and found no new bugs. The four points I raised earlier look addressed in the current code. Since this is a new 2.7k-line crate in a path covered by CODEOWNERS, a human review is still worthwhile.

What was reviewed:

  • send now splits batches by 10 entries and 1 MiB, rejects any single body over 1 MiB, and always sets DelaySeconds, so an explicit zero delay overrides the queue default.
  • delete now reports rejected receipts as Failed instead of Succeeded. extend maps only not-in-flight or unavailable messages to StaleReceipt. Other provider errors come back as VisibilityChange with code and message, and the 12-hour window limit is now documented.
  • record_outcome checks entry ids for range and duplicates. The CI workflow uses only pull_request with read-only contents permission, and its actions are pinned to commit SHAs.
Extended reasoning...

The change adds a standalone Rust workspace with an SQS transport crate (about 330 lines of implementation and about 680 lines of tests), an ElasticMQ compose service and a CI workflow. Nothing calls the crate yet. It touches no auth, crypto or data-exposure surface. The workflow runs on pull_request with contents: read and SHA-pinned actions. The latest commit plausibly addresses all four earlier inline findings, but the repo-wide CODEOWNERS entry and the size of the new crate rule out auto-approval.

Signed-off-by: Ferhat Elmas <elmas.ferhat@gmail.com>
Signed-off-by: Ferhat Elmas <elmas.ferhat@gmail.com>
@ferhatelmas
ferhatelmas enabled auto-merge (squash) November 10, 2026 12:53
@ferhatelmas
ferhatelmas merged commit f59d7ee into master Nov 10, 2026
34 checks passed
@ferhatelmas
ferhatelmas deleted the ferhat/rust branch November 10, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants