Skip to content

UTS: fix spec errors — wildcard-clientId presence setups, corrupt RSP5g fixture, RTN15h1 error code#507

Open
paddybyers wants to merge 1 commit into
mainfrom
uts/fix-rtp15c-rsp5g-rtn15h1-spec-errors
Open

UTS: fix spec errors — wildcard-clientId presence setups, corrupt RSP5g fixture, RTN15h1 error code#507
paddybyers wants to merge 1 commit into
mainfrom
uts/fix-rtp15c-rsp5g-rtn15h1-spec-errors

Conversation

@paddybyers

Copy link
Copy Markdown
Member

Fixes three classes of UTS spec error found while deriving tests in two SDKs (the ably-js test/uts audit and the ably-rust UTS suite). Per the spec-first policy in uts/docs/writing-derived-tests.md (#1da26476), these are fixed at source rather than adapted around in each SDK.

1. clientId: "*" in ClientOptions is invalid per RSA4c/RSA7c — presence setups were unrunnable

RSA7c: a clientId in ClientOptions "cannot contain only a wildcard '*' string value as that client ID value is reserved" — construction must reject it. Every ClientOptions(clientId: "*") setup therefore cannot run on a compliant SDK. Both ably-js (which rejects "*" at construction, per spec) and ably-rust had independently adapted these tests with local workarounds — exactly the silent divergence UTS exists to prevent.

  • enterClient-only tests (realtime_presence_enter.md ×5, incl. both RTP4 bulk tests): now use an unidentified client (key auth, no clientId), which is the legitimate way to act on behalf of arbitrary clientIds.
  • RTP8j wildcard test: the wildcard state RTP8j describes arises from token auth with a clientId: "*" token (cf. RSA7a), not from ClientOptions — setup changed accordingly. The obsolete "some SDKs may reject at construction" note is gone (all compliant SDKs must).
  • RTP15c (enterclient-no-side-effects-0): with the wildcard gone this test was doubly contradictory — per RTP8j a wildcard/null clientId means plain enter() errors immediately, and per RTP15f an identified client's enterClient() with a different clientId indicates an error. No single clientId permits both operations, so the test now uses two clients: an identified member doing the normal enter(), and an unidentified key-auth client doing enterClient/leaveClient — which is also the real-world shape (end user + server-side process). ably-js's current adaptation (identified "admin" + enterClient("other-user")) only passes because ably-js does not implement the client-side RTP15f check; SDKs that do (e.g. ably-rust) can never pass it.
  • realtime_presence_reentry.md: same RTP15f contradiction (identified "admin" calling enterClient("alice")) — now unidentified.
  • The file's top-level wildcard note is rewritten to state the RSA7c/RTP8j/RTP15f rules instead of sanctioning per-SDK adaptations.

2. RSP5g cipher fixture is a corrupt truncation (rest_presence.md)

The encrypted_data string HO4…4U0= (32 bytes) is a truncation of the canonical ably-common fixture (test-resources/crypto-data-128.json) value HO4…LQFt (48 bytes) — it cuts off mid-ciphertext, and the plaintext comment ({"secret":"data"}) didn't match the fixture either ({"example":{"json":"Object"}}). Any SDK implementing this test decrypts garbage: ably-rust hit exactly this and had to source the canonical string; ably-js currently has the test skipped (TODO: implement when cipher infrastructure is available) and would hit it on implementation. Restored the canonical string, corrected the comment, and added a value-level assertion on the decrypted JSON so future corruption is detectable.

3. RTN15h1 asserts the wrong errorReason code (connection_failures_test.md)

The unit spec asserted errorReason.code == 40142 (pass-through of the server's token error). Per RSA4a2, when the SDK has no means to renew it "should indicate an error with error code 40171 … and transition the connection to the FAILED state". This also removes the last internal inconsistency: the proxy integration spec (connection_resume.md, RTN15h1) already asserts 40171 with an explanatory note, and error_reason_test.md (RTN25) was already corrected to 40171 following the ably-js audit — this file was missed in that sweep. (ably-js's unit test currently omits the code assertion entirely; ably-rust asserts 40171.)

🤖 Generated with Claude Code

…TN15h1 code

Three classes of UTS spec error, each surfaced by SDKs deriving tests
(ably-js's test/uts audit and the ably-rust UTS suite):

1. Wildcard clientId in ClientOptions (realtime_presence_enter.md,
   realtime_presence_reentry.md). RSA7c reserves "*" as a
   ClientOptions#clientId value — construction must reject it — so every
   `clientId: "*"` setup was unrunnable on a compliant SDK, and both ably-js
   and ably-rust had silently adapted. enterClient-only tests now use an
   unidentified (key-auth) client; the RTP8j wildcard test obtains the
   wildcard state the way it actually arises, via a wildcard token; RTP15c
   now uses two clients, since per RTP8j + RTP15f no single clientId permits
   both a plain enter() and an enterClient() for another user. The reentry
   setup's identified "admin" + enterClient(other) also contradicted RTP15f
   and is now unidentified.

2. RSP5g cipher fixture (rest_presence.md) was a 32-byte truncation of the
   canonical 48-byte ably-common fixture (test-resources/
   crypto-data-128.json), cutting off mid-ciphertext, with a wrong plaintext
   comment. Any SDK implementing the test decrypts garbage. Restored the
   canonical string and added a value-level assertion on the decrypted JSON.

3. RTN15h1 errorReason code (connection_failures_test.md) asserted
   pass-through of the server's 40142. Per RSA4a2 the SDK substitutes 40171
   ("no means to renew the token") and transitions to FAILED. This aligns
   the unit spec with the proxy integration spec (connection_resume.md) and
   with error_reason_test.md (RTN25), which was already fixed to 40171.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@paddybyers

Copy link
Copy Markdown
Member Author

Downstream re-translation work for ably-js once this merges is tracked in ably/ably-js#2265 (RTN15h1 40171 assertion, RTP15c two-client shape, RTP8j wildcard-token setup, comment refreshes).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the UTS unit-test specifications to resolve spec/test mismatches discovered during derived-test work in SDKs, keeping UTS aligned with the authoritative RSA/RTP/RTN requirements.

Changes:

  • Replaces invalid ClientOptions(clientId: "*") setups with compliant auth patterns (unidentified key auth for enterClient flows; wildcard token for RTP8j).
  • Fixes a corrupted encrypted presence fixture (RSP5g) and adds a decrypted-value assertion to detect future fixture drift.
  • Corrects RTN15h1’s expected errorReason.code from 40142 to 40171 per RSA4a2 behavior when token renewal is impossible.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
uts/rest/unit/presence/rest_presence.md Restores canonical cipher fixture and asserts decrypted JSON value for RSP5g.
uts/realtime/unit/presence/realtime_presence_reentry.md Removes invalid wildcard ClientOptions usage; uses unidentified client for enterClient flows.
uts/realtime/unit/presence/realtime_presence_enter.md Reworks wildcard/clientId presence setups (including RTP8j token auth and RTP15c two-client approach).
uts/realtime/unit/connection/connection_failures_test.md Updates RTN15h1 expected error code to 40171 with clarifying rationale.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 938 to +942
install_mock(mock_ws)

# Wildcard clientId to allow both enter() and enterClient() on the same connection.
# See note in Purpose section about SDK-level wildcard validation.
# Note: clientId "*" may not be accepted by all SDKs at construction time.
# See top-level note for alternative auth patterns (e.g., key auth without clientId).
client = Realtime(options: ClientOptions(key: "fake.key:secret", clientId: "*", autoConnect: false))
channel = client.channels.get(channel_name)
# Two clients: no single clientId permits both operations on a compliant SDK.
# Per RTP8j, a wildcard (or null) clientId means plain enter() errors
# immediately; per RTP15f, an identified client's enterClient() with a

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

uts/realtime/unit/presence/realtime_presence_enter.md:944

  • This test now uses two Realtime clients concurrently, but the setup still installs a single MockWebSocket. The mock WebSocket infrastructure is described in terms of a single active connection, and onMessageFromClient has no connection context, so messages/ACKs from the second client can be misrouted (or the second connection may not be handled at all). Use one mock per client/connection (as done later in this file for “different connections”) and install both, while appending both clients' PRESENCE messages into the same captured_presence list.
# Two clients: no single clientId permits both operations on a compliant SDK.
# Per RTP8j, a wildcard (or null) clientId means plain enter() errors
# immediately; per RTP15f, an identified client's enterClient() with a
# different clientId indicates an error. So the normally-entered member is an
# identified client, and the enterClient/leaveClient operations come from a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants