Repository navigation
fix(queue): serialize maps as objects for json content type - #1093
Open
PedroRosalba wants to merge 1 commit into
Open
PedroRosalba wants to merge 1 commit into
PedroRosalba wants to merge 1 commit into
Conversation
Queue::send and send_batch serialize the message with
serde_wasm_bindgen::to_value, which writes Rust maps (HashMap, BTreeMap,
serde_json::Value::Object) as JS Map and byte buffers as Uint8Array. The
default content type is json, and the runtime JSON-encodes the body, so a
Map arrives as {} and a Uint8Array as an object keyed by index: consumers
silently lose every map-typed field.
Serialize the body with serialize_maps_as_objects and
serialize_bytes_as_arrays when the content type is json; v8 and text sends
are unchanged. Add a test that sends a struct with a HashMap and a
serde_json::Value field through send and send_batch and asserts the
round-trip.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Queue::send/send_batchserialize the message withserde_wasm_bindgen::to_value, whose default serializer writes Rust maps (HashMap,BTreeMap,serde_json::Value::Object) as a JSMapand byte buffers as aUint8Array. The default content type isjson, and the runtime JSON-encodes the body before storing it, so aMaparrives as{}and aUint8Arrayas an object keyed by index. Consumers silently lose every map-typed field, and no consumer-side change can recover them because the data is already gone on the wire.This is the producer-side half of #1013: the reporter's payload holds
serde_json::Valueobjects, which are exactly the values that get flattened to{}. #1048 (body_json,content_type) addresses the consumer side and is complementary to this change; both are needed for that issue.Fix
When the content type is
json, serialize withserialize_maps_as_objects(true)andserialize_bytes_as_arrays(true). Thev8andtextpaths are unchanged.json_compatible()was deliberately not used becauseserialize_missing_as_nullwould change the wire format of everyOption::Nonefield.Observable changes (json path only): map fields now arrive with their data instead of
{};serde_bytesfields arrive as arrays instead of index-keyed objects; a map with non-string keys now returns an error at send time instead of silently sending{}.Tests
Adds a
MapQueueBodymessage with aHashMapand aserde_json::Valuefield, routes forsendandsend_batch, and a consumer branch that records what arrived. The new specs fail onmain(2 failures) and pass with this change; full suite 147 passed / 4 skipped.Tested on macOS (Apple Silicon) only.
Refs #1013