[graphql-stream] SSE transport + SSE-aware GraphiQL [11/n] by tpham-mysten · Pull Request #26425 · MystenLabs/sui

tpham-mysten · 2026-04-29T16:45:58Z

Summary

Replace the WebSocket subscription transport with Server-Sent Events. SSE is the standard transport for GraphQL subscriptions over HTTP. WebSocket was added in 1/n (#26019) as a starting point and is not in production yet, so this PR removes it outright.

Why

For one-way streaming (server pushes events, client mostly listens — exactly our subscription shape), SSE is closer to current industry best practice than WebSocket: simpler infra (no upgrade dance, plays naturally with HTTP/2/3, standard HTTP caching/proxy semantics), debuggable with curl, native browser support. The same pattern is used by OpenAI/Anthropic completion streaming, Cloudflare Workers AI, GitHub live activity, Vercel AI SDK, etc.

GraphiQL story

Upstream GraphiQL has no SSE fetcher today. Rather than block on the in-flight upstream PR (graphql/graphiql#4218), this PR ships a small static HTML template (assets/graphiql.html) that loads GraphiQL via UMD plus the graphql-sse UMD client and wires a custom fetcher: subscriptions go through graphqlSse.iterate(), queries through fetch. CDN deps pinned; subscription detection uses parse + getOperationAST so it handles multi-operation documents with an explicit operationName. Once #4218 lands, this collapses to a one-line createGraphiQLFetcher({ url, sseUrl }) call.

Test plan

How did you test the new or updated feature?

unit + e2e tests
Start local GraphQL Server

Stack

Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

vercel · 2026-04-29T16:46:00Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
sui-docs	Ready	Preview, Comment	May 11, 2026 0:46am

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
multisig-toolkit	Ignored	Preview	May 11, 2026 0:46am
sui-kiosk	Ignored	Preview	May 11, 2026 0:46am

… [12/n] Replace the WebSocket subscription transport with Server-Sent Events. SSE is the standard transport for GraphQL subscriptions over HTTP and is what the rest of the JS/Rust GraphQL ecosystem (graphql-yoga, graphql-helix, the graphql-sse reference protocol) is converging on. The previous WebSocket route was added in 1/n as a working starting point but is not in production yet, so this PR removes it outright rather than running the two transports in parallel. Server changes (`sui-indexer-alt-graphql/src/lib.rs`) * New endpoint: `POST /graphql/subscriptions`. Accepts a GraphQL request with `Accept: text/event-stream`, returns a graphql-sse "distinct connections" stream (`event: next` frames carrying GraphQL responses). Implemented by `graphql_subscriptions` handler which wraps `schema.execute_stream()` in `axum::response::sse::Sse` with a 15 s keep-alive comment. * Removed: `handle_ws`, the GET-multiplexer `graphql_get`, and the imports `ALL_WEBSOCKET_PROTOCOLS`, `GraphQLProtocol`, `GraphQLWebSocket`, `WebSocketUpgrade`. GET on `/graphql` now serves the GraphiQL IDE directly via the `graphiql` handler (gated on `IdeEnabled`), matching the pre-1/n shape. GraphiQL host page (`sui-indexer-alt-graphql/assets/graphiql.html`) `async_graphql::http::GraphiQLSource::build()` only knows how to wire the WebSocket subscription URL, so we ship a small static HTML template that loads upstream GraphiQL via UMD plus the `graphql-sse` UMD client. The template substitutes endpoint paths at serve time via two `.replace()` calls in the `graphiql` handler. * All CDN dependencies pinned to specific patch versions (`graphiql@3.9.0`, `graphql-sse@2.6.0`, `react@18.3.1`, `react-dom@18.3.1`, `graphql@16.14.0`) so the served page does not break when upstreams cut a major. * Subscription detection uses `parse` + `getOperationAST` from the `graphql` ESM module, so multi-operation documents with an explicit `operationName` route correctly. Falls back to non-subscription on parse error. * Subscriptions are routed through `graphqlSse.iterate(params)`; queries and mutations go through a plain `fetch` POST to `/graphql`. This matches the shape that graphql/graphiql#4218 will eventually land upstream, so the future swap to `createGraphiQLFetcher({ url, sseUrl })` will be a one-line change once a UMD build of `@graphiql/toolkit` is available. Test infrastructure (`sui-indexer-alt-e2e-tests/tests/graphql_subscription/testing/harness.rs`) Migrated `SubscriptionTestCluster::subscribe_with_variables` from `tokio-tungstenite` (WebSocket + `graphql-transport-ws` protocol) to `reqwest` HTTP POST + a small inline SSE event-stream parser. The parser reads `event: next` frames, parses their `data:` field as JSON, and yields each one to the test until the server sends `event: complete` or closes the connection. `subscription_url` now points at `http://.../graphql/subscriptions` instead of `ws://.../graphql`. Test bodies (`checkpoint_subscription.rs`, `transaction_subscription.rs`, `event_subscription.rs`) are unchanged; they only consume the public `subscribe()` / `subscribe_with_variables()` methods. Dependency changes: * `sui-indexer-alt-e2e-tests/Cargo.toml`: drop `tokio-tungstenite`, add `async-stream` and `bytes` as dev-deps for the SSE parser. Verified locally * `cargo nextest run -p sui-indexer-alt-graphql --features staging --lib`: 158/158 * `cargo nextest run -p sui-indexer-alt-e2e-tests --features staging --test graphql_subscription`: 15/15 * `cargo clippy -p sui-indexer-alt-graphql -p sui-indexer-alt-e2e-tests --features staging --all-targets -- -D warnings`: clean * `cargo fmt --check`: clean * Manual smoke: ran the local server against testnet via `--checkpoint-stream-url https://fullnode.testnet.sui.io:443`, opened GraphiQL in the browser, confirmed `subscription { checkpoints { sequenceNumber } }` streams real testnet checkpoints over SSE while a regular query (`{ chainIdentifier }`) routes through `fetch`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

amnn

Nice!

amnn · 2026-05-11T13:01:37Z

+  <div id="graphiql" style="height: 100vh;"></div>
+  <script crossorigin src="https://unpkg.com/react@18.3.1/umd/react.production.min.js"></script>
+  <script crossorigin src="https://unpkg.com/react-dom@18.3.1/umd/react-dom.production.min.js"></script>
+  <script src="https://unpkg.com/graphiql@3.9.0/graphiql.min.js"></script>


Note for follow-up. As long as we're customising this, we might as well upgrade it to the latest version (5.2.2)

## Description Streaming subscribers want to take an object ID that surfaces inside an event payload and immediately ask "how was this object referenced by the transaction that emitted the event?" without round-tripping to the indexed Query API. The object is also sometimes an unchanged shared input (e.g. a clock or registry), so it never appears in `objectChanges`; the resolver therefore needs to expose both the change case and the read-only consensus input case. This PR adds: ```graphql type Address { asTransactionObject(transactionDigest: String): TransactionObjectRef } union TransactionObjectRef = ObjectChange | ConsensusObjectRead ``` `ObjectChange` and `ConsensusObjectRead` are reused as-is from the schema; no duplication. The resolver scans `effects.object_changes` first, then `effects.unchanged_consensus_objects` filtered to the ReadOnlyRoot variant. Other unchanged-input markers (cancelled, stream-ended, per-epoch) resolve to null since they do not house an object the user can navigate to. In an events subscription, the transactionDigest argument may be omitted; the field then resolves against the transaction that emitted the parent event, via a new Scope::tx_digest_viewed_at() helper. Passing an explicit transactionDigest other than the parent event's transaction in subscription context is intentionally not supported (the document states this); use the indexed Query API for arbitrary transaction lookups. ## Test plan How did you test the new or updated feature? - unit + e2e tests ## Stack - #26019 - #26094 - #26117 - #26170 - #26194 - #26202 - #26414 - #26453 - #26476 - #26487 - #26425 - #26495 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [x] GraphQL: add asTransactionObject which allows user to query the status of address that is involved as an object in a particular transaction - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework: --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

## Description - Expose cursor: String! on Checkpoint — encoded JsonCursor<u64> of the sequence number, matching the format Query.checkpoints pagination already uses. - Clients (especially streaming subscribers) read this on each yielded checkpoint and pass it back as afterCursor to resume from this point. ## Test plan How did you test the new or updated feature? - e2e + unit tests --- ## Stack - #26019 - #26094 - #26117 - #26170 - #26194 - #26202 - #26414 - #26453 - #26476 - #26487 - #26425 - #26495 - #26714 - #26731 ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [x] GraphQL: Add cursor field to Checkpoint object - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

## Description - Add `afterCursor` and `afterCheckpoint` to the checkpoints subscription so clients can resume from a known point. On subscribe, a LedgerService scan covers the gap to the live tip in parallel batches, then the live broadcast takes over. - Gaps or Lagged events during the live phase re-enter the scan, bounded by `resume_max_recovery_attempts` consecutive retries before disconnecting the subscriber. ## Test plan How did you test the new or updated feature? - cargo nextest run --features staging -p sui-indexer-alt-graphql --lib - Schema snapshot regenerated and checked in. - Manual: connect to a streaming server with afterCheckpoint set --- ## Stack - #26019 - #26094 - #26117 - #26170 - #26194 - #26202 - #26414 - #26453 - #26476 - #26487 - #26425 - #26495 - #26731 - #26714 ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [x] GraphQL: Support resumable checkpoints subscription in features `staging` - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

## Description Part of the GraphQL streaming series (follows #26714, checkpoints [14/n]). Adds a resumable `transactions` subscription: it backfills the filter's matching transactions from a resume point via the scanning API, then hands off to the live checkpoint broadcast at a pinned seam, so delivery is contiguous with no gap or duplicate. The backfill scan retries transient errors (e.g. a rolling indexer deploy) with bounded exponential backoff before it gives up and disconnects. Staging-gated and not yet exposed. Each payload is a batch of transaction edges (`[TransactionEdge!]!`) rather than a single edge: the backfill packs matches up to a fixed batch size so a deep scan coalesces its navigation reads, while live delivers each checkpoint's matches as one batch. Per-transaction cursors keep resume exact even when a batch splits a checkpoint. The scan-then-live handoff, sparse-filter coverage, cursor unification, batching, and retry are documented at the top of `api/subscription/transactions.rs`. ## Test plan - e2e parity test (`test_transaction_subscription_live_backfill_parity`): the same transactions resolve identically whether delivered live or through the backfill scan, plus the existing streaming/ordering/resume subscription tests. - Batching e2e tests: `test_transaction_subscription_backfill_batches_matches` (a deep backfill packs matches into multi-edge payloads) and `test_transaction_subscription_live_batches_per_checkpoint` (each live payload is one checkpoint's matches). - Unit tests for the scan retry: recovers within the budget, gives up once it is exhausted. --- ## Stack - #26019 - #26094 - #26117 - #26170 - #26194 - #26202 - #26414 - #26453 - #26476 - #26487 - #26425 - #26495 - #26731 - #26714 - #27140 ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [x] GraphQL: Support transaction subscription from historical cursor - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

…27537) ## Description Resolve subscription payloads concurrently rather than one at a time. A single `max_concurrent_resolutions` config drives both the resolve window (async-graphql's `subscription_resolution_concurrency`, now [merged](amnn/async-graphql#2)) and the backfill scan page, so a batch's matches resolve within the concurrency budget and coalesce their content reads into one `KvLoader` round trip. ## Test plan - e2e parity test (`test_transaction_subscription_live_backfill_parity`): the same transactions resolve identically whether delivered live or through the backfill scan, plus the existing streaming/ordering/resume subscription tests. - Batching e2e tests: `test_transaction_subscription_backfill_batches_matches` (a deep backfill packs matches into multi-edge payloads) and `test_transaction_subscription_live_batches_per_checkpoint` (each live payload is one checkpoint's matches). - Unit tests for the scan retry: recovers within the budget, gives up once it is exhausted. --- ## Stack - #26019 - #26094 - #26117 - #26170 - #26194 - #26202 - #26414 - #26453 - #26476 - #26487 - #26425 - #26495 - #26731 - #26714 - #27140 - #27537 ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

…17/n] (#27564) ## Description A per-subscriber delivery throttle for GraphQL subscriptions. The SSE handler paces each payload by its cost (`cost / rate` seconds), so a subscriber's sustained delivery stays within `per_subscriber_max_output_nodes_per_second` output nodes per second (default ~off, an operator opts in by lowering it). A payload's cost is its output nodes plus a query-depth surcharge. The throttle lives in the handler, so it wraps every subscription type identically. ## Test plan Unit tests cover the cost model and `Throttle::wrap` pacing (deterministic via tokio's paused clock). `throttle_subscription.rs` adds e2e for checkpoint pacing/ordering, the disable path, richer-payload cost, and the transaction backfill path. ## Stack - #26019 - #26094 - #26117 - #26170 - #26194 - #26202 - #26414 - #26453 - #26476 - #26487 - #26425 - #26495 - #26731 - #26714 - #27140 - #27537 - #27564 ## Release notes - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

tpham-mysten temporarily deployed to sui-typescript-aws-kms-test-env April 29, 2026 16:46 — with GitHub Actions Inactive

vercel Bot deployed to Preview – sui-docs April 29, 2026 16:47 View deployment

tpham-mysten force-pushed the poc-graphql-sse-graphiql branch from be094dd to 71b276e Compare May 11, 2026 02:12

tpham-mysten temporarily deployed to sui-typescript-aws-kms-test-env May 11, 2026 02:12 — with GitHub Actions Inactive

vercel Bot deployed to Preview – sui-docs May 11, 2026 02:15 View deployment

tpham-mysten marked this pull request as ready for review May 11, 2026 02:26

tpham-mysten requested a review from a team as a code owner May 11, 2026 02:26

tpham-mysten requested review from amnn, emmazzz, evan-wall-mysten, nickvikeras and wlmyng May 11, 2026 02:26

tpham-mysten temporarily deployed to sui-typescript-aws-kms-test-env May 11, 2026 02:26 — with GitHub Actions Inactive

tpham-mysten changed the title ~~[POC] graphql-stream: SSE transport + SSE-aware GraphiQL~~ [POC] graphql-stream: SSE transport + SSE-aware GraphiQL [12/n] May 11, 2026

tpham-mysten changed the title ~~[POC] graphql-stream: SSE transport + SSE-aware GraphiQL [12/n]~~ [graphql-stream]: SSE transport + SSE-aware GraphiQL [12/n] May 11, 2026

tpham-mysten force-pushed the poc-graphql-sse-graphiql branch from 71b276e to 3417cb7 Compare May 11, 2026 12:40

tpham-mysten temporarily deployed to sui-typescript-aws-kms-test-env May 11, 2026 12:40 — with GitHub Actions Inactive

bmwill approved these changes May 11, 2026

View reviewed changes

vercel Bot deployed to Preview – sui-docs May 11, 2026 12:46 View deployment

amnn approved these changes May 11, 2026

View reviewed changes

tpham-mysten merged commit 4e5667d into main May 12, 2026
63 checks passed

tpham-mysten deleted the poc-graphql-sse-graphiql branch May 12, 2026 01:19

tpham-mysten mentioned this pull request May 12, 2026

Revert "[graphql-stream]: SSE transport + SSE-aware GraphiQL [12/n] (… #26592

Closed

8 tasks

tpham-mysten changed the title ~~[graphql-stream]: SSE transport + SSE-aware GraphiQL [12/n]~~ [graphql-stream]: SSE transport + SSE-aware GraphiQL [11/n] May 20, 2026

This was referenced May 20, 2026

[graphql-stream] Resumable checkpoints subscription [14/n] #26714

Merged

[graphql-stream] Add Address.asTransactionObject [12/n] #26495

Merged

tpham-mysten mentioned this pull request May 21, 2026

[graphql-stream] Add cursor field to Checkpoint [13/n] #26731

Merged

8 tasks

tpham-mysten mentioned this pull request Jul 3, 2026

[graphql-stream] Resumable transactions subscription [15/n] #27140

Merged

8 tasks

tpham-mysten mentioned this pull request Jul 31, 2026

[graphql-stream] resolve subscription payloads concurrently [16/n] #27537

Merged

8 tasks

This was referenced Aug 4, 2026

[graphql-stream] per-subscriber delivery throttle for subscriptions [17/n] #27564

Merged

[graphql-stream] Resumable events subscription [18/n] #27580

Open

tpham-mysten changed the title ~~[graphql-stream]: SSE transport + SSE-aware GraphiQL [11/n]~~ [graphql-stream] SSE transport + SSE-aware GraphiQL [11/n] Aug 7, 2026

Navigation Menu

Uh oh!

[graphql-stream] SSE transport + SSE-aware GraphiQL [11/n] - #26425

[graphql-stream] SSE transport + SSE-aware GraphiQL [11/n]#26425
tpham-mysten merged 1 commit into
mainfrom
poc-graphql-sse-graphiql

tpham-mysten commented Apr 29, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Apr 29, 2026 •

edited

Loading

Uh oh!

amnn left a comment

Uh oh!

amnn May 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

tpham-mysten commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

GraphiQL story

Test plan

Stack

Release notes

Uh oh!

vercel Bot commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amnn left a comment

Choose a reason for hiding this comment

Uh oh!

amnn May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tpham-mysten commented Apr 29, 2026 •

edited

Loading

vercel Bot commented Apr 29, 2026 •

edited

Loading