Repository navigation
feat: add rust workspace with queue crate - #1684
Conversation
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🟡
rust/crates/queue/src/sqs.rs— On real AWS SQS, callers extending a stale receipt may get a genericError::Sqsinstead of the documentedError::StaleReceipt. The mapping at sqs.rs:204-205 comparese.code()against two bare strings, "MessageNotInflight" and "ReceiptHandleIsInvalid". Those strings match what ElasticMQ returns. AWS SQS can report the same condition under other codes. Fix: match on the typed service-error variants, and also map the AWS stale-receipt forms. That meansAWS.SimpleQueueService.MessageNotInflightandInvalidParameterValuewhen its message says the message is not available for a visibility change. [also at: rust/crates/queue/src/sqs.rs:205 - On real AWS SQS, calling extend with a receipt for a deleted or expired message may return a generic Error::Sqs, not the documented StaleReceipt.]Why this was flagged
This applies to deployments against real AWS SQS, not ElasticMQ, which is the only endpoint CI tests. Here is the trigger. A worker calls
extend(sqs.rs:485) with a receipt whose visibility already lapsed, or whose message was deleted or redelivered. AWS SQS then commonly answersInvalidParameterValue("Value ... for parameter ReceiptHandle is invalid. Reason: Message does not exist or is not available for visibility timeout change."). For query-compatible errors it can also sendAWS.SimpleQueueService.MessageNotInflightfrom the x-amzn-query-error header. Neither matches the string patterns at sqs.rs:495, so the error falls tosqs(e). That contradicts the trait doc at lib.rs:201, which promisesError::StaleReceipt. Callers that branch onStaleReceiptto drop a lost claim will see an opaque transport error and may retry or crash. The contract teststale_receipt(contract.rs:770) runs only against ElasticMQ, so it does not catch this. This is new code, so the base branch has no such behaviour.Verification: The trigger is
extendrunning against real AWS SQS with a receipt whose message was deleted or delivered again. In rust/crates/queue/src/sqs.rs:204-205, only "MessageNotInflight" and "ReceiptHandleIsInvalid" becomeStaleReceipt, so AWS'sInvalidParameterValuebecomes the genericError::Sqs. The only test, contract.rs:166-181, runs only against ElasticMQ.
Coverage Report for CI Build 42819578232Warning Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes. Coverage remained the same at 87.484%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats💛 - Coveralls |
40e5bba to
59318f1
Compare
|
@claude review always |
There was a problem hiding this comment.
I re-reviewed the updated push and found no new bugs. Since this is a new 2.6k-line crate with its own CI workflow, and CODEOWNERS covers all paths, someone on the storage team should still take a look.
What was reviewed:
- Three of my earlier points are fixed.
deletenow reports a rejected receipt (such asReceiptHandleIsInvalid) asFailed.sendnow splits chunks by total body bytes as well as by count.DelaySecondsis now always sent, so an explicit 0 overrides the queue's default delay. - The
extendthread about the 12-hour cap counted from receive is still open. The code and docs for that path haven't changed. - I checked how outcomes are mapped back to inputs. Ids out of range, duplicated or from another chunk are rejected, and that chunk is reset to
Unknown. - I checked the new workflow. It has read-only
contentspermission, runs only onpull_requestandpush, pins its actions to commit SHAs, and points ElasticMQ at localhost only.
Extended reasoning...
The change adds a standalone Rust workspace with an SQS-backed queue crate. It also adds an ElasticMQ compose service, a CI workflow, and a .dockerignore entry. The crate isn't wired into any producers or workers yet. It touches no auth, crypto or data-exposure code, and the workflow has read-only permissions on non-privileged triggers. I deferred instead of approving because the change is large, .github/CODEOWNERS assigns every path to a team, and one of my earlier review threads is still open.
Still open from earlier reviews (1):
- Unresolved: 1 minor or pre-existing.
There was a problem hiding this comment.
I re-reviewed the latest push and found no new bugs. The four points I raised earlier look addressed in the current code. Since this is a new 2.7k-line crate in a path covered by CODEOWNERS, a human review is still worthwhile.
What was reviewed:
sendnow splits batches by 10 entries and 1 MiB, rejects any single body over 1 MiB, and always sets DelaySeconds, so an explicit zero delay overrides the queue default.deletenow reports rejected receipts asFailedinstead ofSucceeded.extendmaps only not-in-flight or unavailable messages toStaleReceipt. Other provider errors come back asVisibilityChangewith code and message, and the 12-hour window limit is now documented.record_outcomechecks entry ids for range and duplicates. The CI workflow uses onlypull_requestwith read-onlycontentspermission, and its actions are pinned to commit SHAs.
Extended reasoning...
The change adds a standalone Rust workspace with an SQS transport crate (about 330 lines of implementation and about 680 lines of tests), an ElasticMQ compose service and a CI workflow. Nothing calls the crate yet. It touches no auth, crypto or data-exposure surface. The workflow runs on pull_request with contents: read and SHA-pinned actions. The latest commit plausibly addresses all four earlier inline findings, but the repo-wide CODEOWNERS entry and the size of the new crate rule out auto-approval.
Signed-off-by: Ferhat Elmas <elmas.ferhat@gmail.com>
Signed-off-by: Ferhat Elmas <elmas.ferhat@gmail.com>
365383b to
185c846
Compare
What kind of change does this PR introduce?
feat
What is the current behavior?
Events are sent to and processed by the existing ts pg-boss queue.
What is the new behavior?
Add a standalone Rust queue transport with an AWS SQS implementation.
It supports sending and receiving messages, visibility timeouts, acknowledgement and structured batch error reporting.
Also add ElasticMQ based contract tests and a CI workflow.
Additional context
Transport is not yet connected to producers or workers so existing event processing continues to use pg-boss