feat: add a hardware video encoder factory, with Media Foundation H.264 - #310
Merged
devopvoid merged 1 commit intoOct 1, 2026
Merged
Conversation
HardwareVideoEncoderFactory puts the encoders of the GPU in front of the built-in ones where the platform has them. On Windows, H.264 is encoded by the Media Foundation encoder the GPU driver provides (NVIDIA, AMD, Intel), driven through its asynchronous event model, with frames passed in system memory as NV12. Low-latency mode, no B-frames, CBR, forced IDR frames on request and bitrate changes while encoding are set through ICodecAPI; the parameter sets are put in front of every key frame. The hardware encoder takes over only formats the software encoder has too (H.264 Constrained Baseline and Baseline, packetization mode 1), so the factory offers the same codecs as DefaultVideoEncoderFactory and hardware never changes what is negotiated. A FallbackVideoEncoder switches to the software encoder when the hardware one fails to start or gives up while encoding, and restarts the stream with a key frame. WebRTC's own fallback wrapper is not part of the linked WebRTC library. Hardware encoding is opt-in: DefaultVideoEncoderFactory, and a PeerConnectionFactory without an encoder factory, encode as before. Linux has no hardware encoder yet; on macOS both factories use VideoToolbox.
devopvoid
added this pull request to stack #311
September 30, 2026 00:20
This was referenced Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #303: this PR targets its branch and should be merged after it.
Adds
HardwareVideoEncoderFactory, which encodes on the GPU where the platform supports it. On Windows it encodes H.264 through the Media Foundation encoder of the GPU driver. Hardware encoding is opt-in:DefaultVideoEncoderFactoryis unchanged, and so is aPeerConnectionFactorywithout an encoder factory.Behavior
HardwareVideoEncoderFactoryDefaultVideoEncoderFactory: H.264 through VideoToolboxDefaultVideoEncoderFactory, all software; VA-API and NVENC are plannedDefaultVideoEncoderFactory, and what gets negotiated never depends on the GPU.encoderImplementationstat ofoutbound-rtp, e.g.MediaFoundation (AMDh264Encoder).Native side
MFH264Encoderdrives the asynchronous hardware transform through its event model.MFTransformEventspumps the events on a Media Foundation thread, which is also where encoded frames are handed to WebRTC.ICodecAPI: low-latency mode, no B-frames, CBR, a forced IDR frame when WebRTC requests a key frame, and bitrate changes while encoding.FallbackVideoEncoderhandles the switch to software. WebRTC's own software fallback wrapper isn't part of the prebuiltwebrtc.lib. Adding it would mean a new WebRTC build target, and with it a full WebRTC rebuild on every platform, for one small wrapper.HardwareVideoEncoderFactory(native) puts a platform's hardware encoders in front of the software ones. Platforms plug in throughCreatePlatformHardwareVideoEncoderFactory(); Linux has a stub for now.Testing
HardwareVideoEncoderIntegrationTest:Without a hardware encoder the GPU test is skipped, as on CI runners.
-Dwebrtc.test.hardwareEncoder=truemakes it fail instead.On Windows with an AMD Radeon RX 9070 XT, with that property set, frames go through
MediaFoundation (AMDh264Encoder)and decode on the receiving side.mvn -pl webrtc test: 197 tests pass.mvn -pl webrtc test -Pjni-check: 197 tests pass, with noFATAL ERROR in native method.Not verified yet:
Docs
The video codecs guide (
docs/guide/advanced/video-codecs.md) has a new "Hardware Encoding" section.