Repository navigation
Write the video start bitrate hint as one connection-level value, once - #2102
Merged
xianshijing-lk merged 3 commits intoSep 22, 2026
Merged
Conversation
x-google-start-bitrate is connection-scoped in libwebrtc: ApplyChangedParams reads it per m-section but pushes it into the shared Call via SetSdpBitrateParameters, where RtpBitrateConfigurator holds one config for the whole peer connection. Differing per-section values were last-writer-wins on m-section order, so a camera plus a screen share could seed the estimator from either one depending on SDP layout. Every video section now carries the same value: the largest hint among the sections that map to a published track. Write it only on the first offer that carries local video. libwebrtc retains start_bitrate_bps and re-applies it on network route changes, so rewriting it later is at best a no-op and at worst a restart of a converged bandwidth estimator. The latch is set only once the offer carrying the hint is accepted locally, so a rejected munge retries on the next offer. Add a 300 kbps target floor, matching the Rust SDK: below that, seeding above the real capacity costs more than the ramp it saves. applyVideoStartBitrate no longer matches the section to a track; that moves to findTrackCodecPayload so the dependent DD extension munging still runs on every offer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 04f3556 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
size-limit report 📦
|
xianshijing-lk
marked this pull request as ready for review
September 14, 2026 17:31
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
`trackBitrates` is append-only and an unpublished section keeps its `a=msid`, so matching a section to a track by msid alone still paired a stale entry with the section it used to occupy. Two consequences, both reachable while `hasAppliedVideoStartBitrate` is still unset (a superseded offer, or a munge the browser rejected): an uncapped screen-share target could seed a connection that now carries only a camera, and the unpublish renegotiation itself could consume the one-shot hint on a section that sends nothing, leaving every later publish with no hint at all. `a=sendonly` separates a live send from an unpublished or pre-populated section: every publish creates its transceiver with `direction: 'sendonly'`, `unpublishTrack` sets it to `inactive` (and `removeTrack` does the same transition for the simulcast senders), and the pre-populated placeholders are `recvonly`. The matching in the munge loop is left as it was: it now writes the correct connection-level value, and suppressing it on reverted sections could leave them without an fmtp line the live section has, which is the bundled payload type collision `conformBundledCodecFmtp` works around. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Matching `a=sendonly` exactly dropped the legacy `addTrack` fallback: `createSender` uses it when `addTransceiver` is unavailable, and `addTrack` reuses a transceiver rather than creating a sendonly one, so a published camera lands on a `sendrecv` section. A section with no direction attribute has the same problem, since SDP defaults it to sendrecv. Those clients got no start bitrate hint at all. Invert the test: skip `recvonly` and `inactive`, the only two directions that cannot carry local media, and count everything else. That still excludes both shapes a dead section takes -- `inactive` from a sendonly transceiver, `recvonly` from the sendrecv one `addTrack` reuses -- and the pre-populated placeholders, which are `recvonly`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
xianshijing-lk
deleted the
sxian/CLT-3068/video-start-bitrate-connection-level
branch
September 22, 2026 00:17
This was referenced Sep 22, 2026
Merged
xianshijing-lk
added a commit
to livekit/client-sdk-flutter
that referenced
this pull request
Sep 24, 2026
#1218) Ports [client-sdk-js#2102](livekit/client-sdk-js#2102) to Flutter. Same change already landed for Rust in [rust-sdks#1430](livekit/rust-sdks#1430). ## Why the JS PR is the reference, not the Rust one `lib/src/core/transport.dart` is a near-literal Dart transliteration of the pre-fix JS code — same `TrackBitrateInfo` shape, same append-only tracker list, same per-section msid match inside `negotiate()`, same `setMungedSDP` fallback that restores the original SDP on rejection. The JS diff maps onto it almost line for line, so this PR mirrors its structure: the logic moves out of `negotiate()` into top-level `findTrackCodecPayload` / `computeTrackStartBitrate` / `computeConnectionStartBitrate` / `applyVideoStartBitrate` functions, which is also what makes it testable without a PeerConnection. Rust could not be a template. It keeps a single `max_send_bitrate_bps` slot on the transport and munges the SDP as a string over video payload types, with no m-section or msid concept at all — there is nothing to port across. ## `x-google-start-bitrate` is connection-scoped, not per-section libwebrtc *reads* it per m-section but doesn't *apply* it per m-section: ```cpp bitrate_config_ = GetBitrateConfigForCodec(send_codec()->codec); // webrtc_video_engine.cc ... call_->GetTransportControllerSend()->SetSdpBitrateParameters(bitrate_config_); ``` That lands in `RtpBitrateConfigurator`, which holds **one** `BitrateConstraints` for the entire peer connection. Every video m-section writes the same slot, last writer wins, and the order is just m-section order. Today each track writes its own value, so publishing a camera and a screen share together seeds the estimator from whichever section happens to be last. `computeConnectionStartBitrate` now takes the largest hint among the video sections that can send, and every video section gets that same number. ## Written once per connection, not on every offer libwebrtc retains `start_bitrate_bps` in `RtpBitrateConfigurator` and re-applies it on network route changes (`RtpTransportControllerSend::OnNetworkRouteChanged`), so a WiFi→cellular handover re-seeds the estimator from this hint with no renegotiation. Rewriting it on a later offer is at best a no-op and at worst restarts a converged estimator. `_hasAppliedVideoStartBitrate` latches only after `setMungedSDP` has accepted the offer — it falls back to the unmunged SDP on rejection — so a rejected munge retries on the next offer. A full reconnect builds a new `Transport` and seeds the new estimator again. ## Only sections that can send The bitrate tracker list is append-only and an unpublished section keeps its `a=msid`, so matching a section to a track by msid alone would still pair a stale entry with the section it used to occupy. That lets an uncapped screen-share target seed a connection that now carries only a camera, or consumes the one-shot hint on a section that sends nothing — which would leave every later publish with no hint at all. The direction is the discriminator, as an exclusion rather than a match: `recvonly` and `inactive` are the only directions that cannot carry local media, and they are exactly where a section lands once its sender is removed (`removeTrack` moves sendonly → inactive and sendrecv → recvonly). `sendonly`, `sendrecv` and a section with no direction attribute — which SDP defaults to sendrecv — all send. ## Two calculation changes **Cap and floor.** Camera is now capped at 1 Mbps so the estimator does not open too aggressively; screen share is exempt, because its content needs the bitrate immediately to stay legible. Targets below 300 kbps get no hint. Both match client-sdk-js and the Rust SDK; Flutter had neither. **Target derivation.** `computeStartTargetBitrate` replaces `encodings[0].maxBitrate`. Simulcast rids are ordered `q`, `h`, `f`, so reading the first encoding took the **lowest** layer — for a 720p publish that is ~150 kbps, giving a ~135 kbps "hint" that did nothing. This is not optional cleanup: with the new 300 kbps floor, the old derivation would have suppressed the hint entirely for most camera publishes. A single SVC stream still uses its first encoding, since it declares the whole budget there; plain simulcast and SVC-published-as-simulcast (`L1T*`) sum. ## Tests `test/core/start_bitrate_test.dart`, 13 cases mirroring the JS suite: section/codec matching, the cap and its screen-share exemption, the floor, the connection-level max, stale-entry filtering, exclusion of a section that stopped sending but kept its msid (both `inactive` and `recvonly`), inclusion of `sendrecv` and direction-less sections, and the target derivation. Verified locally on Flutter 3.44.9 / Dart 3.12.2: `flutter analyze` clean, `flutter test` 437 passed / 1 skipped, `dart format --set-exit-if-changed` clean, `import_sorter` clean. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
xianshijing-lk
pushed a commit
to livekit/client-sdk-swift
that referenced
this pull request
Oct 12, 2026
…itrate`) (#1121) Swift counterpart of livekit/client-sdk-js#1987 and livekit/rust-sdks#1226, reworked from the earlier opt-in draft after the review comment, then aligned (commit 2) with the connection-level, write-once design from livekit/client-sdk-android#973 that merged in livekit/client-sdk-js#2102 and livekit/rust-sdks#1430. ## Problem libwebrtc starts every new video send stream at roughly 300 kbps and ramps up from there, regardless of the encodings' `maxBitrate`. On a healthy network we measured about 30 s to reach a 2.3 Mbps ceiling, with the first 10-12 s at QP 39-42 — a visibly blocky picture. The ObjC API exposes no way to set that starting point, so the handle is libwebrtc's own SDP-level `x-google-start-bitrate` fmtp parameter. ## Change Same behaviour as JS and Rust, no public API: - When a video sender is added, the start bitrate is derived from its encodings: sum of the active layers' `maxBitrate`, × 0.9, capped at 1 Mbps unless the track is a screen share, and no hint under 300 kbps (`Transport.startBitrateKbps(targetBps:isScreenShare:)`). - **One connection-level value.** libwebrtc reads `x-google-start-bitrate` per m-section but applies it to the shared `Call` (`WebRtcVideoSendChannel::ApplyChangedParams` → `SetSdpBitrateParameters`), where `RtpBitrateConfigurator` holds one config per peer connection, so per-sender values were last-writer-wins in m-section order. The offer's value is the largest hint among its *sending* video sections whose `a=msid` track id is a recorded sender (`Transport.connectionStartBitrateKbps(for:kbpsBySenderId:)`), and it is declared on every video codec's fmtp (VP8/VP9/AV1/H264/H265) of **every** sending video section. Receive-only / inactive sections neither contribute nor receive it. An fmtp line is inserted for payloads that have none (VP8); an existing value is replaced. - **Written once per peer connection.** libwebrtc re-reads the hint when the send codec changes and restarts a converged estimator if the value changed, and it retains `start_bitrate_bps` across network route changes (`RtpTransportControllerSend::OnNetworkRouteChanged`), so the munge runs only until the first offer carrying the hint is accepted by `setLocalDescription` (`_hasAppliedVideoStartBitrate`). A rejected munge retries on the next offer; a full reconnect builds a new `Transport`. Offers before any video is published (audio / data only) find no target and leave the latch unset. - The munge is the last, optional entry of the `set(localDescription:munging:)` composition from #1068, in both single- and dual-PC negotiation, built on the `SDP` module from #1078. `Transport.addTransceiver(with:transceiverInit:startBitrateKbps:)` records each sender's hint before returning, so the offer libwebrtc requests can't be created without it; `remove(track:)` clears it. Both `LocalParticipant` publish paths (primary and backup codec) pass it. - `x-google-max-bitrate` is deliberately not written (same `Call`-level promotion would cap the connection's total send bandwidth; libwebrtc's own TODO concedes it). Per-track caps stay in `maxBitrateBps`. - New SDP helper `SDPMediaSection.setFmtpParameter(_:value:forPayload:)` (replace / append / insert, other parameters kept verbatim). - Default `degradationPreference` by source, the other half of the JS PR, is already in `main` (#1083), so nothing to do there. - Drive-by, surfaced by review: the backup-codec publish path now computes its encodings with `isScreenShare`, as the primary path does. It had defaulted to the camera encoding since #275, so a screen share's backup sender got camera bitrates; the new formula's screen-share branch made that visible. ## Tests `SDPTests`, `TransportSDPMungeTests` (formula, encodings sum, connection-level max with the sending filter, the munge on every sending section: codec case, replace/append/insert, rtx/audio/recvonly untouched, no-op identity, latch condition), `VideoEncodingsTests` (backup codec's screen-share encodings) and, against the shipped libwebrtc in `TransportMungeFallbackTests`: a send-only video section's msid carries the sender id, `setLocalDescription` accepts the munged offer (including the inserted VP8 line), and two offers on one peer connection — answered by a second local peer connection — where only the first carries the hint. ## Results These numbers are from the earlier, answer-side version of this branch (H.264 only, hint fixed at 1000 kbps), measured on iOS with a software H.264 encoder, 800x1120@30, single layer, `maxBitrate` 2.3 Mbps: the estimate opened at ~1000 kbps instead of ~370, peak QP dropped from ~42 to ~28, and the 2.3 Mbps ceiling was reached in ~13 s instead of ~31 s. Zero NACKs. The offer-side variant derives the identical 1000 kbps for that track, but I haven't re-measured it on a device yet — will add a log run. ## Open questions - Camera-then-screen-share: with write-once, a screen share published after the camera gets no new seed (the connection stays at the camera's ≤1000). Same trade-off as JS/Rust/Android — the estimator has converged by then anyway. - Only exercised end-to-end with H.264 on iOS on a healthy network. The other codecs are covered by unit tests plus the libwebrtc acceptance test, not by a live session. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1987, bringing the JS start-bitrate munging in line with what came out of the review on client-sdk-android#973. No change to the formula: still 90% of the target, capped at 1 Mbps for camera, uncapped for screen share.
x-google-start-bitrateis connection-scoped, not per-sectionlibwebrtc reads it per m-section but doesn't apply it per m-section:
That lands in
RtpBitrateConfigurator, which holds oneBitrateConstraintsfor the entireRTCPeerConnection. Every video m-section writes the same slot, last writer wins, and the order is just m-section order.Today each track writes its own value. The camera/screen-share split is what makes that bite: camera capped at 1000, screen share uncapped at e.g. 2700, both in the same offer. Whichever section applies last seeds the single estimator — so either the screen share overrides the camera cap for the whole connection, or the camera clobbers the screen share and the exemption does nothing. It depends on SDP layout, so it looks stable in testing and flips in the field.
Fix:
computeConnectionStartBitratetakes the largest hint among the video m-sections that map to a published track, and every video section gets that same number. Only sections present in the current SDP count, sincetrackBitratesis append-only and outlives an unpublish.Written once per connection, not on every offer
libwebrtc guards re-application three ways: it only re-reads the fmtp when the send codec changes (
webrtc_video_engine.cc:1338-1342), ignores a value equal to the stored one (rtp_bitrate_configurator.cc:66-73— "setting the same remote description twice shouldn't restart bandwidth estimation"), and returns-1for "no change" while retaining the real value.The gap: when the value changes between offers and the codec changed too, it really does restart a converged estimator. With per-track values, publishing a screen share mid-session changes the last-writer value.
The reason once is sufficient, not just safe: the seed persists.
RtpBitrateConfiguratorkeepsstart_bitrate_bps, andRtpTransportControllerSend::OnNetworkRouteChangedre-applies it fromGetConfig()(rtp_transport_controller_send.cc:390). A WiFi→cellular handover re-seeds the estimator from this hint automatically, with no renegotiation. A full reconnect builds a new peer connection and seeds the new estimator again.hasAppliedVideoStartBitratelatches only aftersetMungedSDPaccepts the offer, so a rejected munge retries on the next one.300 kbps target floor
Matches the Rust SDK. Below that, seeding above the real capacity costs more than the ramp it saves.
Refactor note
applyVideoStartBitratewas doing double duty as section-matcher, and the DD-extension munging depended on that match. Matching moved tofindTrackCodecPayload, so DD extension still runs on every offer while the bitrate write happens once.Tests
PCTransport.test.ts: 28 passing (6 new — the floor, the cap and its screen-share exemption, the connection-level max, stale-entry filtering, and section matching).All libwebrtc references are against
m144_release.Note
13 test files plus
lintandtscfail on this branch from a missingmachinapackage — reproduced on a clean checkout ofmain, unrelated to this change. All 4tscerrors are inSignalClient*.ts.