feat(studio-events): one push channel to the portal, and stop polling background runs - #131
Open
AndrejK666 wants to merge 3 commits into
Open
feat(studio-events): one push channel to the portal, and stop polling background runs#131AndrejK666 wants to merge 3 commits into
AndrejK666 wants to merge 3 commits into
Conversation
Every consumer of background work polls: the prototype's repository sync
every 1.2 s for five minutes, its Background work view every 4 s while
anything is live. A run that fails in 300 ms still reads "queued…" until
the next tick.
This adds the channel that replaces those loops — one per assembly, not
one per producer:
GET /cf/studio-events/v1/stream SSE, scoped to the caller's tenant
GET /cf/studio-events/v1/events?after_seq=&limit= the same, by cursor
An event is { seq, at_ms, kind, subject_type, subject_id, source, payload }
and nothing in that shape knows about tasks, repositories or IDE sessions.
A producer states what happened, to what, and who says so; everything
type-specific is payload. `seq` and `at_ms` are the channel's, so a
producer cannot forge ordering or backdate an event.
Three properties are deliberate:
* Tenant isolation is structural — a broadcaster per tenant rather than a
filter someone has to remember.
* A reconnect loses nothing: a bounded per-tenant window (500 events) is
replayable by cursor, and `latest_seq` running ahead of the last
delivered `seq` is how a client learns it fell out of that window.
* Frames are unnamed and the event type is the JSON's `kind`, because the
frontend SDK's SSE protocol binds only `onmessage`, which never fires
for a named frame.
Publishing is `dyn StudioEventPublisher` from the ClientHub: synchronous,
infallible, resolved lazily by producers so gear init order does not
matter, and absent in an assembly without this gear — which then behaves
exactly as before.
The platform's event-broker will replace the inside of this: its gear is
still a skeleton (every REST handler is `todo!("lands with #4346")`) while
its SDK is done. When it lands this becomes a consumer→SSE bridge and the
two endpoints above do not change.
Signed-off-by: Andrej Kuchma <Andrej.Kuchma@constructor.tech>
The dispatcher is the one place every background run passes through, so that is where the announcement belongs: `record` for state transitions and the progress sink for phase reports. Imports, catalogue syncs, notification deliveries and scheduled runs are all covered without a single gear that owns work having to know the channel exists. An event carries the fields the run endpoint answers with — state, attempts, phase, summary, error, result — so a view can be fed by either without a second mapping. Only what the transition actually set is included: an absent field means "unchanged", and a consumer merges rather than overwrites. Sending null for what a patch left alone would tell a client the run had just lost its counts. The publisher is resolved per event rather than held, because gear init order is not guaranteed and an assembly without the channel must lose the announcement and nothing else. studio-theia's default sink moves the same way: a forwarded Theia event is republished as `theia.<kind>` about a workspace, with session id, sequence and the raw callback argument inside `payload`. The bridge's vocabulary stops at that boundary — one producer's protocol has no business becoming the contract every consumer lives with. The event-broker sink stays behind its feature flag for when the broker lands. Verified on a running stack: a repository import with a rejected credential produced running ×5, requeued ×4, 10 progress frames and one terminal task.failed, with the counts arriving as the handler reported them. Signed-off-by: Andrej Kuchma <Andrej.Kuchma@constructor.tech>
…port The frontend had the consuming half ready — useApiStream, StreamDescriptor, SseProtocol — and no way to reach an authenticated stream. `SseProtocol` opens `new EventSource(url)`, which cannot carry `Authorization`, and the gateway takes no token from the query string; `auth.ts` says as much, and its transport binder only accepts REST plugins. So the transport is ours, plugged in where the protocol already allows it: `SseAuthPlugin.onConnect` short-circuits with a fetch-backed `EventSourceLike`, exactly as `SseMockPlugin` does with a mock. It adds the two things the protocol does not do: * Reconnect. `attachHandlers` treats `onerror` as fatal and disconnects, which also defeats the native reconnect, so a dropped connection is retried here with backoff and `onerror` is fired only when the session is over (401/403) and retrying would just hammer the gateway. * Exactly-once, ordered redelivery. The gap opened by a disconnect is replayed from the cursor endpoint, live frames are held until that replay is delivered, and the overlap is dropped by cursor. None of it is visible to useApiStream or to an MFE. The starting cursor rides in the URL (`?resume_from=`), which the server ignores and the transport reads: that makes it part of the descriptor key, so useApiStream opens a fresh connection when it changes, and two subscribers with different starting points cannot fight over one field. `StudioEventsApiService` is registered on the shell so every MFE shares one stream. In the prototype two poll loops become subscriptions: the repository-sync runner (both surfaces that start a sync) and the Background work list, which coalesces a burst of progress into one reload. One rule the prototype proves and the docs now state: read the cursor *before* enqueuing. A run that fails in 300 ms is over before the stream is open, and `fromSeq` is what replays it. Signed-off-by: Andrej Kuchma <Andrej.Kuchma@constructor.tech>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A single SSE channel from the assembly to the portal, and the first consumers of it.
studio-tasksannounces every run transition on it, so the portal is told instead of pollingGET /studio-tasks/v1/runs/{id}. In the prototype the repository-sync runner and the Background work list now follow the stream; the shell (studio-frontend) gets the transport and a registered service, ready for its first screen.Design is written up in ADR-0013 — the short version:
{ seq, at_ms, kind, subject_type, subject_id, source, payload }. A producer says what happened, to what, who says so; everything type-specific ispayload.seq/at_msare the channel's, so a producer cannot forge ordering.studio-tasksdispatcher is the one place every background run passes through, so imports, catalogue syncs, notification deliveries and scheduled runs are all covered without the gears that own the work knowing this exists.studio-theia's sink republishes astheia.<kind>about a workspace, with session id and sequence insidepayload. A bridge to an IDE container that most deployments do not run has no business shaping the stream everything else uses.event-brokergear is still a skeleton (every REST handler istodo!("lands with #4346")) while its SDK is done. When it lands this becomes a consumer→SSE bridge and the two endpoints do not change. Its own/v1/events:ssestill should not go to a browser — JOIN/SEEK/topology frames are an SDK protocol.Frontend transport
SseProtocolopensnew EventSource(url), which cannot sendAuthorization, and the gateway takes no token from the query string —auth.tsdocuments the gap, and its transport binder only accepts REST plugins. SoSseAuthPluginshort-circuitsonConnectwith a fetch-backedEventSourceLike(the same seamSseMockPluginuses) that also owns:onerroras fatal and disconnects, which also defeats the native reconnect, so retries live in the transport andonerrorfires only on 401/403;None of that is visible to
useApiStreamor to an MFE.One rule worth knowing: read the cursor before enqueuing. A run that fails in 300 ms is over before the stream is open;
?resume_from=is what replays it.Verification
fmt,clippy --all-targets -D warnings,checkfor--features theia-bridgeand--no-default-features,cargo test— 504 passed.running×5, requeued ×4, 10task.progressframes and one terminaltask.failed, counts arriving as the handler reported them. Backfill by cursor returns exactly the gap;latest_seqreports the high-water mark; no token → 401.tower::timeout, which covers only producing the response) — connection held for minutes with keep-alives every 15 s. Both nginx configs already pass streams through (proxy_buffering off).tscclean; shell 29 tests, host app 2 new service tests, 6 transport tests, prototype 46 tests — all green. The fourconnections-mfefailures that predate this branch are unchanged.Notes for review
backend.replicas: 1today). More than one needs sticky sessions or the broker.