feat: encode H.264 with NVENC on NVIDIA GPUs, on Windows and Linux

docs/guide/advanced/video-codecs.md

@@ -26,13 +26,15 @@ PeerConnectionFactory factory = PeerConnectionFactory.builder()
 
 | Platform | Hardware encoding |
 |---|---|
-| Windows | H.264 through the Media Foundation encoder of the GPU driver (NVIDIA, AMD, Intel) |
+| Windows | H.264 with NVENC on NVIDIA GPUs, otherwise through the Media Foundation encoder of the GPU driver (AMD, Intel) |
+| Linux | H.264 with NVENC on NVIDIA GPUs; other GPUs encode in software |
 | macOS | H.264 through VideoToolbox, as with `DefaultVideoEncoderFactory` |
-| Linux | Not yet; encoding is in software |
 
-On Windows, the hardware encoder takes over H.264 Constrained Baseline and Baseline with packetization mode 1, formats the software encoder offers too, so encoding in hardware never changes what is negotiated. When the hardware encoder fails to start, for example because the GPU has no encoder sessions left, or fails while encoding, the stream switches to the software encoder and continues with a key frame. On a machine without a hardware encoder, the factory encodes like `DefaultVideoEncoderFactory`.
+NVENC needs an NVIDIA driver of version 522 or newer on Windows, 520 or newer on Linux; it is loaded at run time, so nothing needs to be installed on machines without one.
 
-Which encoder a stream uses shows in the `encoderImplementation` statistic of its `outbound-rtp` stats, e.g. `MediaFoundation (AMDh264Encoder)` or `OpenH264`.
+The hardware encoders take over H.264 Constrained Baseline and Baseline with packetization mode 1, formats the software encoder offers too, so encoding in hardware never changes what is negotiated. When a hardware encoder fails to start, for example because the GPU has no encoder sessions left, or fails while encoding, the stream switches to the next encoder in line (on Windows, from NVENC to Media Foundation), and finally to the software encoder, and continues with a key frame. On a machine without a hardware encoder, the factory encodes like `DefaultVideoEncoderFactory`.
+
+Which encoder a stream uses shows in the `encoderImplementation` statistic of its `outbound-rtp` stats, e.g. `NVENC (NVIDIA GeForce RTX 4070)`, `MediaFoundation (AMDh264Encoder)` or `OpenH264`.
 
 ### Native Codecs
 

webrtc-jni/src/main/cpp/CMakeLists.txt

@@ -57,6 +57,12 @@ file(GLOB SOURCES_MEDIA_AUDIO_OS "src/media/audio/${SOURCE_TARGET}/*.cpp")
 file(GLOB SOURCES_MEDIA_VIDEO "src/media/video/*.cpp")
 file(GLOB SOURCES_MEDIA_VIDEO_CODEC "src/media/video/codec/*.cpp")
 file(GLOB SOURCES_MEDIA_VIDEO_CODEC_OS "src/media/video/codec/${SOURCE_TARGET}/*.cpp")
+
+# NVENC is loaded at run time from the NVIDIA driver, so building it needs its
+# header only. The driver exists for Windows and Linux.
+if(WIN32 OR LINUX)
+    file(GLOB SOURCES_MEDIA_VIDEO_CODEC_NVENC "src/media/video/codec/nvenc/*.cpp")
+endif()
 file(GLOB SOURCES_MEDIA_VIDEO_DESKTOP "src/media/video/desktop/*.cpp")
 file(GLOB SOURCES_MEDIA_VIDEO_DESKTOP_OS "src/media/video/desktop/${SOURCE_TARGET}/*.cpp")
 file(GLOB SOURCES_MEDIA_VIDEO_OS "src/media/video/${SOURCE_TARGET}/*.cpp")
@@ -73,6 +79,7 @@ list(APPEND SOURCES
     ${SOURCES_MEDIA_VIDEO}
     ${SOURCES_MEDIA_VIDEO_CODEC}
     ${SOURCES_MEDIA_VIDEO_CODEC_OS}
+    ${SOURCES_MEDIA_VIDEO_CODEC_NVENC}
     ${SOURCES_MEDIA_VIDEO_DESKTOP}
     ${SOURCES_MEDIA_VIDEO_DESKTOP_OS}
     ${SOURCES_MEDIA_VIDEO_OS}
@@ -100,6 +107,10 @@ target_include_directories(${PROJECT_NAME}
         include/rtc
 )
 
+if(WIN32 OR LINUX)
+    target_include_directories(${PROJECT_NAME} PRIVATE dependencies/nvenc/include)
+endif()
+
 set_target_properties(${PROJECT_NAME} PROPERTIES
     OUTPUT_NAME "${PROJECT_NAME}-${OUTPUT_NAME_SUFFIX}"
     CXX_STANDARD 20
@@ -151,3 +162,11 @@ install(FILES
     DESTINATION "${CMAKE_INSTALL_PREFIX}/META-INF/licenses/webrtc"
     COMPONENT Runtime
 )
+
+if(WIN32 OR LINUX)
+    install(FILES
+        "${CMAKE_CURRENT_SOURCE_DIR}/dependencies/nvenc/LICENSE"
+        DESTINATION "${CMAKE_INSTALL_PREFIX}/META-INF/licenses/nvenc"
+        COMPONENT Runtime
+    )
+endif()

webrtc-jni/src/main/cpp/dependencies/nvenc/LICENSE

@@ -0,0 +1,25 @@
+nvEncodeAPI.h, the NVIDIA Video Codec SDK header this library is built with
+(API version 12.0), is licensed as follows:
+
+Copyright (c) 2010-2022 NVIDIA Corporation
+
+Permission is hereby granted, free of charge, to any person
+obtaining a copy of this software and associated documentation
+files (the "Software"), to deal in the Software without
+restriction, including without limitation the rights to use,
+copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the software, and to permit persons to whom the
+software is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.

webrtc-jni/src/main/cpp/dependencies/nvenc/README.md

@@ -0,0 +1,11 @@
+# NVENC header
+
+`include/nvEncodeAPI.h` is the NVIDIA Video Codec SDK header for NVENC, API
+version 12.0, taken unchanged from FFmpeg's `nv-codec-headers` at tag
+`n12.0.16.1`:
+
+https://github.com/FFmpeg/nv-codec-headers/blob/n12.0.16.1/include/ffnvcodec/nvEncodeAPI.h
+
+The library loads NVENC and the CUDA driver at run time, so the header is all
+the build needs. API 12.0 runs on NVIDIA drivers 522 and newer on Windows, and
+520 and newer on Linux. Moving to a newer header raises that minimum.