Skip to content

Add UTS test specs for LiveObjects path-based API#473

Open
paddybyers wants to merge 22 commits into
mainfrom
uts-liveobjects
Open

Add UTS test specs for LiveObjects path-based API#473
paddybyers wants to merge 22 commits into
mainfrom
uts-liveobjects

Conversation

@paddybyers

@paddybyers paddybyers commented May 13, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds portable UTS test specs for the LiveObjects path-based API (~330 tests across 21 files)
  • Covers PathObject, Instance, batch operations, LiveCounter/LiveMap CRDT internals, ObjectsPool sync state machine, RealtimeObject lifecycle, value types, subscriptions, and integration scenarios
  • Specs are language-independent pseudocode at uts/objects/
  • Corresponding ably-js translations: Add UTS tests for LiveObjects path-based API ably-js#2219

Test plan

  • ably-js translations of these specs pass (290 tests, 1 pending for unimplemented LiveMap#clear())

🤖 Generated with Claude Code

@github-actions github-actions Bot temporarily deployed to staging/pull/473 May 13, 2026 12:52 Inactive
@paddybyers paddybyers requested a review from ttypic May 13, 2026 13:48
@github-actions github-actions Bot temporarily deployed to staging/pull/473 May 14, 2026 07:18 Inactive
@paddybyers paddybyers force-pushed the uts-integration-proxy branch from f355624 to 2e12074 Compare May 14, 2026 20:17
Base automatically changed from uts-integration-proxy to main May 14, 2026 20:18
@github-actions github-actions Bot temporarily deployed to staging/pull/473 May 14, 2026 20:36 Inactive
@paddybyers paddybyers requested a review from mschristensen May 15, 2026 13:58
paddybyers and others added 3 commits May 28, 2026 14:08
@paddybyers @claude
Add UTS test specs for LiveObjects path-based API (~330 tests)
a77f75a 
Complete portable test suite covering the LiveObjects path-based API:
21 files across unit tests (pure + mock WebSocket), integration tests
(sandbox), and proxy integration tests. Covers PathObject, Instance,
BatchContext, LiveCounter/LiveMap CRDTs, ObjectsPool sync state machine,
value types, subscriptions, and GC. Includes table-driven validation
tests, bytes/binary data coverage, and REST fixture provisioning.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@paddybyers @claude
Delegate proxy port assignment to uts-proxy in all test specs
3de4a4f 
Remove client-side allocated_port/port_base patterns from all proxy
test specs and helper docs. Port is now auto-assigned by the proxy
when omitted from create_proxy_session(). Matches uts-proxy v0.2.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@paddybyers @claude
Update UTS test specs to match LiveObjects path-based API spec (a397e34)
2fba05e 
Align all ~330 LiveObjects UTS test specs with the squashed spec revision
a397e34 (LiveObjects path-based API). Key changes:

- Add parent_references.md (20 tests): RTLO3f, RTLO4g/4h, RTLO4f, RTO5c10
- Add public_object_message.md (13 tests): PAOM1-3, PAOOP1-3
- Thread ObjectMessage through all CRDT operations and LiveObjectUpdate
- Add RTO25 (access preconditions) and RTO26 (write preconditions)
- Update subscription model: subscribe returns Subscription object
- Add RTO24 (PathObjectSubscriptionRegister) dispatch tests
- Add parentReferences maintenance tests to live_map.md (+8 tests)
- Add post-sync parentReferences rebuild tests to objects_pool.md (+3)
- Rename "consume"/"consumption" to "evaluate"/"evaluation" in value_types
- Remove batch.md (Batch API deferred from current spec revision)
- Remove subscribeIterator and LiveObject#unsubscribe tests
- Update PLAN.md to reflect new file structure and test counts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 May 28, 2026 13:09 Inactive
sacOO7
sacOO7 reviewed Jun 1, 2026
View reviewed changes
Comment thread uts/realtime/integration/helpers/proxy.md
sacOO7
sacOO7 reviewed Jun 1, 2026
View reviewed changes
Comment thread uts/docs/integration-testing.md
sacOO7
sacOO7 reviewed Jun 1, 2026
View reviewed changes
Comment thread uts/realtime/integration/helpers/proxy.md
sacOO7
sacOO7 reviewed Jun 1, 2026
View reviewed changes
Comment thread uts/objects/integration/objects_batch_test.md Outdated
sacOO7
sacOO7 reviewed Jun 1, 2026
View reviewed changes

@sacOO7 sacOO7 left a comment •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • We also need to update Spec File Counts section at uts/README.md.
  • Looks like the Directory structure section needs an update as well. We could potentially remove it altogether—I’m not sure it serves much purpose anymore. WDYT?

sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/integration/objects_lifecycle_test.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/integration/objects_sync_test.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/integration/objects_gc_test.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/integration/objects_batch_test.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/unit/realtime_object.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/unit/live_counter_api.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/unit/live_map_api.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/unit/path_object_subscribe.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/unit/live_object_subscribe.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/unit/realtime_object.md Outdated
sacOO7
sacOO7 reviewed Jun 2, 2026
View reviewed changes
Comment thread uts/objects/helpers/standard_test_pool.md Outdated
@paddybyers @claude
UTS: add Protocol Variants to LiveObjects integration tests
eaa6983 
Data-path integration tests must run with both JSON and msgpack per G1.
Add ## Protocol Variants section and useBinaryProtocol to ClientOptions
in objects_lifecycle_test.md and objects_sync_test.md. Update the
annotated specs list in integration-testing.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 June 5, 2026 20:56 Inactive
@paddybyers @claude
UTS: fix root.increment() → root.get("score").increment()
99421b7 
root is the PathObject at / (a LiveMap). Calling increment/decrement on
it would throw 92007 (RTPO17e). The tests intend to operate on the
counter at root["score"]. Fixed in realtime_object.md (10 sites) and
live_counter_api.md (5 sites). path_object_mutations.md is unchanged —
those are intentional negative-path tests for RTPO17e.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 June 5, 2026 21:00 Inactive
@paddybyers @claude
UTS: fix InstanceSubscriptionEvent assertions in live_object_subscribe
8f52447 
Tests use instance.subscribe() (RTINS16) which emits
InstanceSubscriptionEvent { object, message }, not LiveObjectUpdate
{ tombstone, objectMessage }. Fix all assertions to use .message
instead of .objectMessage, and check .message.operation.action instead
of .tombstone. Also fix noop test serial from "01" to "02" so the
newness check (RTLO4a6) doesn't suppress it before the noop path fires.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 June 5, 2026 21:19 Inactive
@paddybyers @claude
UTS: move appliedOnAckSerials/bufferedObjectOperations to RealtimeObject
0b15af8 
Per RTO7a/RTO7b, these attributes belong on RealtimeObject, not
ObjectsPool. Add `realtime_object = RealtimeObject(pool: pool)` to
affected test setups and redirect all references (8 tests, ~11 lines).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 June 5, 2026 21:23 Inactive
@paddybyers @claude
UTS: fix RTPO19b subscribe-on-detached test
f7ace4c 
Replace non-existent getRoot() with get(), and restructure so root is
obtained while attached then channel is detached before subscribe().
The previous setup responded to ATTACH with DETACHED, so get() would
throw 90001 before reaching subscribe().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 June 5, 2026 21:25 Inactive
@paddybyers @claude
UTS: rewrite RTO20f to use observable behaviour
d7d75e7 
Replace private _liveObject.siteTimeserials access with an observable
test: after a local increment (applied via ACK with source LOCAL),
send an inbound COUNTER_INC with the same siteCode and serial. If
LOCAL incorrectly wrote to siteTimeserials, the newness check would
reject the inbound message. The counter reaching 120 proves LOCAL
did not pollute siteTimeserials.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 June 5, 2026 21:32 Inactive
@paddybyers @claude
UTS: rewrite RTO5c9 re-sync test to use observable behaviour
e3614cd 
Instead of just checking score == 100 after re-sync (which proves nothing
since OBJECT_SYNC resets the pool regardless), replay a previously-applied
serial after re-sync. If appliedOnAckSerials was properly cleared, the
replayed message applies normally and score reaches 110. If not cleared,
dedup (RTO9a3) rejects it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 June 5, 2026 21:39 Inactive
@paddybyers @claude
UTS: add OBJECT, OBJECT_SYNC, ANNOTATION to proxy action numbers table
6bcf593 
Add missing protocol message actions from TR2: OBJECT (19),
OBJECT_SYNC (20), and ANNOTATION (21).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/473 June 5, 2026 21:41 Inactive
@paddybyers paddybyers mentioned this pull request Jun 5, 2026
Open
2 tasks
@paddybyers paddybyers marked this pull request as ready for review June 5, 2026 21:48
@paddybyers

paddybyers commented Jun 5, 2026

Copy link
Copy Markdown
Member Author

@ttypic @sacOO7 I've updated in response to the comments; PTAL

sacOO7
sacOO7 reviewed Jun 9, 2026
View reviewed changes
Comment thread uts/objects/unit/live_object_subscribe.md
@@ -0,0 +1,385 @@
# LiveObject Subscribe Tests

Spec points: `RTLO4b`, `RTLO4b3`, `RTLO4b4c1`, `RTLO4b4c3a`, `RTLO4b4c3c`, `RTLO4b4d`, `RTLO4b4e`, `RTLO4b6`, `RTLO4b7`

@sacOO7 sacOO7 Jun 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current test spec uses instance.subscribe, so it can also include relevant spec item RTINS16
instead of just liveobjects.subscribe spec RTLO4b.
Currently, instance#subscribe sub-spec RTINS16d calls underlying liveobjects.subscribe at spec RTLO4b.

sacOO7
sacOO7 reviewed Jun 9, 2026
View reviewed changes
Comment thread uts/objects/integration/objects_batch_test.md Outdated
sacOO7
sacOO7 approved these changes Jun 9, 2026
View reviewed changes

@sacOO7 sacOO7 left a comment •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍
( Apart from above comments )

sacOO7 and others added 6 commits June 26, 2026 16:02
@sacOO7 @claude
fix(uts/objects): align REST fixture provisioning with V2 objects API
051bb95 
The `provision_objects_via_rest` helper described the legacy pre-V2 REST shape
(`POST .../objects` with a `{ "messages": [ { operation: { action, ... } } ] }`
envelope), which no longer matches the LiveObjects objects REST API.

Update it to the V2 contract (per the LiveObjects OpenAPI specification):
- endpoint is singular `POST .../object`;
- the body is a single operation object, or a bare JSON array (batch) — no
  `messages` envelope;
- an operation is identified by its payload key (`mapSet`/`mapRemove`/`mapCreate`/
  `counterInc`/`counterCreate`) with an `objectId` or `path` target (and optional
  idempotency `id`); values are `{string|number|boolean|bytes|objectId}`;
- `mapCreate.semantics` is the integer 0 (LWW) with `{data:}`-wrapped entries.

Also update the only call site (RTPO15 in objects_lifecycle_test.md) to the V2
operation shape so the spec stays internally consistent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sacOO7 @claude
fix(uts): align provision_objects_via_rest host with the nonprod sandbox
9883347 
The REST fixture-provisioning helper still POSTed to https://sandbox-rest.ably.io,
the legacy prod-sandbox host. Every other objects integration spec — and the whole
UTS integration corpus (realtime + rest) — provisions apps and routes clients via
https://sandbox.realtime.ably-nonprod.net (raw HTTP) / endpoint: "nonprod:sandbox"
(SDK clients). The objects integration specs were migrated to the nonprod host in
e57d340, but standard_test_pool.md was missed, leaving provision_objects_via_rest
posting fixtures to a different backend than the app/key were provisioned on — so
RTPO15 (rest-provisioned-data-sync) could never resolve.

Point the helper at the canonical nonprod host to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sacOO7
fix(uts): route objects integration clients to the nonprod sandbox
356b57f 
The Realtime client option blocks in objects_lifecycle_test.md (10) and
objects_sync_test.md (5) set no host, so the SDK falls back to the production
endpoint (realtime.ably.io, REC1a) — meaning the clients never reach the nonprod
sandbox the "Sandbox Setup" provisions the app on. Every other UTS integration
spec (realtime + rest) configures clients with endpoint: "nonprod:sandbox"; add
it here to match so the tests actually exercise the sandbox.

Added missing auto-connect false to clientOptions
@sacOO7
Removed non-required objects_batch_spec.md from the liveobjects uts…
4eb5e0d 
… spec
@sacOO7
Added missing endpoint and autoConnect to objects_faults spec
e10eefb
@sacOO7
Merge pull request #497 from ably/fix/uts-liveobjects-rest-provisioning
61f16f8 
[AIT-1071] fix(uts/objects): Fix liveobjects UTS test spec
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sacOO7 sacOO7 sacOO7 approved these changes
@ttypic ttypic ttypic approved these changes
@mschristensen mschristensen Awaiting requested review from mschristensen

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@paddybyers @ttypic @sacOO7