Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions docs/guide/media/constraints.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,89 @@ The `scaleResolutionDownBy` parameter specifies how much to scale down the video
Note that these constraints are applied without requiring SDP renegotiation, making them suitable for dynamic adaptation to changing network conditions.
:::

To cap the resolution at an absolute size instead, set `scaleResolutionDownTo`. The video is scaled down to fit, whatever size the source delivers; it takes precedence over `scaleResolutionDownBy` if both are set.

```java
parameters.encodings.get(0).scaleResolutionDownTo = new RTCResolutionRestriction(1280, 720);
sender.setParameters(parameters);
```

## Degradation Preference

When the network or the CPU cannot keep up, a video sender lowers either its frame rate or its resolution. `degradationPreference` on the send parameters decides which:

```java
RTCRtpSendParameters parameters = sender.getParameters();

// Keep text sharp when sharing a screen: lower the frame rate instead.
parameters.degradationPreference = RTCDegradationPreference.MAINTAIN_RESOLUTION;

sender.setParameters(parameters);
```

| Value | Gives up |
|---|---|
| `MAINTAIN_FRAMERATE` | Resolution; suits motion, such as camera video |
| `MAINTAIN_RESOLUTION` | Frame rate; suits detail, such as screens with text |
| `BALANCED` | Both, in turns |
| `MAINTAIN_FRAMERATE_AND_RESOLUTION` | Neither; frames are dropped instead |

If unset, WebRTC keeps the resolution of screen shares and of tracks hinted as detailed or text, and the frame rate of other video.

## Simulcast

With simulcast, a video sender encodes the same video several times, at different resolutions, so that a server can forward to each receiver the one its connection can take. The encodings are given when the transceiver is created, each with an RTP stream ID (`rid`):

```java
RTCRtpTransceiverInit init = new RTCRtpTransceiverInit();
init.direction = RTCRtpTransceiverDirection.SEND_ONLY;

RTCRtpEncodingParameters quarter = new RTCRtpEncodingParameters();
quarter.rid = "q";
quarter.scaleResolutionDownBy = 4.0;
quarter.maxBitrate = 150_000;

RTCRtpEncodingParameters half = new RTCRtpEncodingParameters();
half.rid = "h";
half.scaleResolutionDownBy = 2.0;
half.maxBitrate = 500_000;

RTCRtpEncodingParameters full = new RTCRtpEncodingParameters();
full.rid = "f";
full.maxBitrate = 1_500_000;

init.sendEncodings = Arrays.asList(quarter, half, full);

RTCRtpTransceiver transceiver = peerConnection.addTransceiver(videoTrack, init);
```

The `rid`s cannot be changed afterwards; the other parameters of each encoding can, through `setParameters()`, e.g. `active = false` to pause one of them. An encoding may also send a different codec than the others, by setting its `codec` to one of the negotiated codecs.

## Scalability Modes (SVC)

Instead of separate encodings, one encoding can carry layers, which a server can drop to lower the rate: temporal layers (fewer frames per second) and, with VP9 and AV1, spatial layers (lower resolutions). `scalabilityMode` takes the mode names of the [WebRTC SVC specification](https://www.w3.org/TR/webrtc-svc/), e.g. `L1T3` for three temporal layers, or `L3T3_KEY` for three spatial layers with three temporal layers each:

```java
RTCRtpSendParameters parameters = sender.getParameters();
parameters.encodings.get(0).scalabilityMode = "L1T3";
sender.setParameters(parameters);
```

The codec has to support the mode; `setParameters()` throws otherwise. `getScalabilityModes()` of a codec's capability, from `factory.getRtpSenderCapabilities(MediaType.VIDEO)`, lists the modes it supports. VP8 supports temporal layers only.

## Priorities

`bitratePriority` sets the share of the available bitrate a sender gets relative to the other senders of the peer connection (default 1.0), and `networkPriority` how its packets are marked on the network (DSCP), where the network honors it. Both apply to the whole sender, so only the first encoding may set them:

```java
RTCRtpSendParameters parameters = sender.getParameters();
parameters.encodings.get(0).bitratePriority = 2.0;
parameters.encodings.get(0).networkPriority = RTCPriorityType.HIGH;
sender.setParameters(parameters);
```

For audio, `adaptivePtime` lets the encoder send longer packets when the bitrate is low, which saves the overhead of many small packets.

## Conclusion

In this guide, we've explored several important techniques for controlling media quality and bandwidth usage in WebRTC applications.
Expand Down
47 changes: 47 additions & 0 deletions webrtc-jni/src/main/cpp/include/api/RTCResolutionRestriction.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
* Copyright 2026 Alex Andres
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#ifndef JNI_WEBRTC_API_RTC_RESOLUTION_RESTRICTION_H_
#define JNI_WEBRTC_API_RTC_RESOLUTION_RESTRICTION_H_

#include "JavaClass.h"
#include "JavaRef.h"

#include "api/video/resolution.h"

#include <jni.h>

namespace jni
{
namespace RTCResolutionRestriction
{
class JavaRTCResolutionRestrictionClass : public JavaClass
{
public:
explicit JavaRTCResolutionRestrictionClass(JNIEnv * env);

jclass cls;
jmethodID ctor;
jfieldID maxWidth;
jfieldID maxHeight;
};

JavaLocalRef<jobject> toJava(JNIEnv * env, const webrtc::Resolution & resolution);
webrtc::Resolution toNative(JNIEnv * env, const JavaRef<jobject> & restriction);
}
}

#endif
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ namespace jni
jfieldID clockRate;
jfieldID channels;
jfieldID sdpFmtp;
jfieldID scalabilityModes;
};

JavaLocalRef<jobject> toJava(JNIEnv * env, const webrtc::RtpCodecCapability & capability);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,14 @@ namespace jni
jfieldID maxBitrate;
jfieldID maxFramerate;
jfieldID scaleResolution;
jfieldID rid;
jfieldID scaleResolutionDownTo;
jfieldID scalabilityMode;
jfieldID numTemporalLayers;
jfieldID bitratePriority;
jfieldID networkPriority;
jfieldID adaptivePtime;
jfieldID codec;
};

JavaLocalRef<jobject> toJava(JNIEnv * env, const webrtc::RtpEncodingParameters & parameters);
Expand Down
1 change: 1 addition & 0 deletions webrtc-jni/src/main/cpp/include/api/RTCRtpSendParameters.h
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ namespace jni
jmethodID ctor;
jfieldID transactionId;
jfieldID encodings;
jfieldID degradationPreference;
};

JavaLocalRef<jobject> toJava(JNIEnv * env, const webrtc::RtpParameters & parameters);
Expand Down
4 changes: 4 additions & 0 deletions webrtc-jni/src/main/cpp/src/WebRTCContext.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@

#include "api/environment/environment_factory.h"
#include "api/peer_connection_interface.h"
#include "api/priority.h"
#include "api/rtp_parameters.h"
#include "modules/desktop_capture/desktop_capturer.h"
#include "rtc_base/ssl_adapter.h"

Expand Down Expand Up @@ -83,6 +85,8 @@ namespace jni
JavaEnums::add<webrtc::PeerConnectionInterface::SignalingState>(env, PKG"RTCSignalingState");
JavaEnums::add<webrtc::PeerConnectionInterface::TlsCertPolicy>(env, PKG"TlsCertPolicy");
JavaEnums::add<webrtc::RtpTransceiverDirection>(env, PKG"RTCRtpTransceiverDirection");
JavaEnums::add<webrtc::Priority>(env, PKG"RTCPriorityType");
JavaEnums::add<webrtc::DegradationPreference>(env, PKG"RTCDegradationPreference");
JavaEnums::add<webrtc::SdpType>(env, PKG"RTCSdpType");
JavaEnums::add<webrtc::AudioDeviceModule::AudioLayer>(env, PKG_AUDIO"AudioLayer");
JavaEnums::add<webrtc::AudioProcessing::Config::GainController1::Mode>(env, PKG_AUDIO"AudioProcessingConfig$GainController$Mode");
Expand Down
62 changes: 62 additions & 0 deletions webrtc-jni/src/main/cpp/src/api/RTCResolutionRestriction.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
/*
* Copyright 2026 Alex Andres
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#include "api/RTCResolutionRestriction.h"
#include "JavaClasses.h"
#include "JavaObject.h"
#include "JavaUtils.h"
#include "JNI_WebRTC.h"

namespace jni
{
namespace RTCResolutionRestriction
{
JavaLocalRef<jobject> toJava(JNIEnv * env, const webrtc::Resolution & resolution)
{
const auto javaClass = JavaClasses::get<JavaRTCResolutionRestrictionClass>(env);

jobject object = env->NewObject(javaClass->cls, javaClass->ctor,
static_cast<jint>(resolution.width), static_cast<jint>(resolution.height));

ExceptionCheck(env);

return JavaLocalRef<jobject>(env, object);
}

webrtc::Resolution toNative(JNIEnv * env, const JavaRef<jobject> & restriction)
{
const auto javaClass = JavaClasses::get<JavaRTCResolutionRestrictionClass>(env);

JavaObject obj(env, restriction);

webrtc::Resolution resolution;
resolution.width = obj.getInt(javaClass->maxWidth);
resolution.height = obj.getInt(javaClass->maxHeight);

return resolution;
}

JavaRTCResolutionRestrictionClass::JavaRTCResolutionRestrictionClass(JNIEnv * env)
{
cls = FindClass(env, PKG"RTCResolutionRestriction");

ctor = GetMethod(env, cls, "<init>", "(II)V");

maxWidth = GetFieldID(env, cls, "maxWidth", "I");
maxHeight = GetFieldID(env, cls, "maxHeight", "I");
}
}
}
29 changes: 29 additions & 0 deletions webrtc-jni/src/main/cpp/src/api/RTCRtpCodecCapability.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@
#include "JavaUtils.h"
#include "JNI_WebRTC.h"

#include "api/video_codecs/scalability_mode.h"
#include "modules/video_coding/svc/scalability_mode_util.h"

#include <string>
#include <vector>

namespace jni
{
namespace RTCRtpCodecCapability
Expand Down Expand Up @@ -59,6 +65,16 @@ namespace jni

ExceptionCheck(env);

if (!capability.scalability_modes.empty()) {
std::vector<std::string> modes;

for (webrtc::ScalabilityMode mode : capability.scalability_modes) {
modes.emplace_back(webrtc::ScalabilityModeToString(mode));
}

env->SetObjectField(object, javaClass->scalabilityModes, JavaString::createArray(env, modes).get());
}

return JavaLocalRef<jobject>(env, object);
}

Expand Down Expand Up @@ -90,6 +106,18 @@ namespace jni
codecCapability.num_channels = Integer::getValue(env, channels);
}

JavaLocalRef<jobjectArray> modes = obj.getObjectArray(javaClass->scalabilityModes);
const jsize count = modes.get() != nullptr ? env->GetArrayLength(modes.get()) : 0;

for (jsize i = 0; i < count; i++) {
JavaLocalRef<jstring> mode(env, static_cast<jstring>(env->GetObjectArrayElement(modes.get(), i)));
auto scalabilityMode = webrtc::ScalabilityModeFromString(JavaString::toNative(env, mode));

if (scalabilityMode.has_value()) {
codecCapability.scalability_modes.push_back(*scalabilityMode);
}
}

return codecCapability;
}

Expand All @@ -104,6 +132,7 @@ namespace jni
clockRate = GetFieldID(env, cls, "clockRate", INTEGER_SIG);
channels = GetFieldID(env, cls, "channels", INTEGER_SIG);
sdpFmtp = GetFieldID(env, cls, "sdpFmtp", MAP_SIG);
scalabilityModes = GetFieldID(env, cls, "scalabilityModes", "[" STRING_SIG);
}
}
}
Loading
Loading