feat: add encoded frame transforms and a media recorder

README.md

@@ -17,6 +17,8 @@ The library provides a comprehensive set of Java classes that map to the WebRTC
 - **Media capabilities** - Audio and video capture from cameras and microphones
 - **Desktop capture** - Screen and application window sharing
 - **Media file playback** - Send video and audio files over a peer connection in place of a camera and microphone, with the optional FFmpeg-based `webrtc-java-media` module
+- **Call recording** - Record what a peer connection sends or receives into MKV, WebM or MP4 files, without re-encoding, with the `webrtc-java-media` module
+- **Encoded transforms** - Read, change or drop encoded audio and video frames on their way through a sender or receiver, e.g. for end-to-end encryption, like insertable streams in the browser
 - **Data channels** - Bidirectional peer-to-peer data exchange
 - **Statistics API** - Detailed metrics for monitoring connection quality
 - **Simple integration** - Available as a Maven dependency
@@ -30,6 +32,8 @@ For more detailed information, check out the documentation:
 - [Guides](https://jrtc.dev/guide/) - Comprehensive documentation on using the library
 - [Examples](https://jrtc.dev/guide/examples) - Sample code demonstrating various features
 - [Media Files](https://jrtc.dev/guide/media/media-files) - Sending video and audio files with the media module
+- [Media Recording](https://jrtc.dev/guide/media/media-recording) - Recording calls into media files with the media module
+- [Encoded Transforms](https://jrtc.dev/guide/advanced/encoded-transforms) - Reading and changing encoded frames, e.g. for end-to-end encryption
 - [Build Notes](https://jrtc.dev/guide/build) - Instructions for building the library from source
 
 ## License

docs/.vitepress/sidebar.ts

@@ -23,6 +23,7 @@ function sidebarGuide(): DefaultTheme.SidebarItem[] {
 				{ text: 'Media Constraints', link: '/media/constraints' },
 				{ text: 'Media Directionality', link: '/media/directionality' },
 				{ text: 'Media Files', link: '/media/media-files' },
+				{ text: 'Media Recording', link: '/media/media-recording' },
 			],
 		},
 		{
@@ -72,6 +73,7 @@ function sidebarGuide(): DefaultTheme.SidebarItem[] {
 			collapsed: false,
 			items: [
 				{ text: 'Field Trials', link: '/advanced/field-trials' },
+				{ text: 'Encoded Transforms', link: '/advanced/encoded-transforms' },
 			],
 		},
 	]

docs/guide/advanced/encoded-transforms.md

@@ -0,0 +1,106 @@
+# Encoded Transforms
+
+This guide explains how to read and change the encoded frames of a call as they pass through an `RTCRtpSender` or an `RTCRtpReceiver`, the way [WebRTC Encoded Transforms](https://www.w3.org/TR/webrtc-encoded-transform/) (also known as insertable streams) do in a browser. It covers:
+
+- Setting a transform on a sender or a receiver
+- Reading and replacing a frame's payload
+- Dropping frames
+- What each frame tells about itself
+- Asking for key frames
+- End-to-end encryption as a worked example
+- Threading and performance
+
+A sender's transform sees every frame after the encoder and before the packetizer; a receiver's sees every frame after the depacketizer and before the decoder. Everything between the two, whether that is the network, a TURN server or an SFU, only ever carries what the transform produced. That is what makes end-to-end encryption possible, and also frame metadata, watermarks, or analysis of the encoded stream.
+
+## Setting a Transform
+
+An `RTCEncodedFrameTransformer` is a functional interface with a single method, `transform(RTCEncodedFrame frame)`. It changes the frame in place, and the frame is sent on when the method returns:
+
+```java
+RTCRtpSender sender = peerConnection.addTrack(videoTrack, List.of("stream"));
+
+sender.setTransform(frame -> {
+    System.out.println(frame);   // mime type, size, key frame, timestamp, SSRC
+});
+```
+
+On the receiving side, the receivers are there once the remote description is set:
+
+```java
+for (RTCRtpReceiver receiver : peerConnection.getReceivers()) {
+    receiver.setTransform(frame -> inspect(frame));
+}
+```
+
+`setTransform(null)` removes the transform again, after which frames pass unchanged.
+
+::: tip
+Set the transform of a video sender before negotiating when you can. The first transform set on a running video sender restarts its send stream, which costs a key frame. Later changes, including removing it, cost nothing.
+:::
+
+The transform belongs to the native sender or receiver, not to the Java object: every `RTCRtpSender` instance for the same sender shares it, setting it through one replaces what was set through another, and disposing of an instance leaves it in place.
+
+## Reading and Changing the Payload
+
+`getData()` returns the payload in a `ByteBuffer`, from position zero to its size. The buffer holds a copy, which may be changed freely; `setData(...)` then writes it back:
+
+```java
+receiver.setTransform(frame -> {
+    ByteBuffer data = frame.getData();
+
+    for (int i = 0; i < data.limit(); i++) {
+        data.put(i, (byte) (data.get(i) ^ 0x5A));
+    }
+
+    frame.setData(data);
+});
+```
+
+`setData` takes a `ByteBuffer` (its remaining bytes) or a `byte[]`, and the new payload may be larger or smaller than the old one. The payload is only copied into Java when `getData()` is called, so a transform that looks at the metadata alone costs no copy at all.
+
+A frame, and the buffer `getData()` returned, is valid only while `transform` runs and only on its thread. The buffer's memory is reused for the next frame, so copy what you need to keep. Reading or changing a frame after the transform returned throws an `IllegalStateException`; its metadata stays readable.
+
+## Dropping Frames
+
+`frame.drop()` releases the frame instead of sending it on. A transform that throws drops the frame as well, and the exception goes to the thread's uncaught exception handler. It is dropped rather than sent on unchanged because an encryption transform that failed must never send the frame in the clear.
+
+A receiver cannot decode a video frame that depends on one it never got, so dropping video frames usually means waiting for the next key frame.
+
+## Frame Metadata
+
+Every `RTCEncodedFrame` tells its `getMimeType()` (e.g. `video/VP8`, `audio/opus`), `getSize()`, RTP `getTimestamp()`, `getSsrc()`, `getPayloadType()` and `getCaptureTimeUs()`. The subclasses add what is particular to their kind:
+
+| `RTCEncodedVideoFrame` | `RTCEncodedAudioFrame` |
+| --- | --- |
+| `isKeyFrame()` | `getSequenceNumber()` (received frames) |
+| `getWidth()`, `getHeight()` (key frames) | `getAudioLevel()` in -dBov |
+| `getRid()` (simulcast layer) | `getContributingSources()` |
+| `getFrameId()`, `getSpatialIndex()`, `getTemporalIndex()` | |
+
+## Key Frames
+
+A transform that needs a key frame, e.g. to start over after a key change, can ask for one:
+
+```java
+videoSender.generateKeyFrame();    // the local encoder makes the next frame a key frame
+videoReceiver.requestKeyFrame();   // asks the remote sender for one
+```
+
+Both do nothing for audio.
+
+## End-to-End Encryption
+
+A complete example with AES-GCM is in [`EncryptedRecordingExample`](https://github.com/devopvoid/webrtc-java/blob/master/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/EncryptedRecordingExample.java). Two points matter for any encrypting transform:
+
+- **Keep the codec header in the clear.** The packetizer and depacketizer read the start of each frame: a VP8 receiver finds key frames and the frame size in the first bytes. Browsers leave 10 bytes of a VP8 key frame, 3 of a VP8 delta frame and 1 of an Opus frame unencrypted, and authenticate them as associated data instead. H.264 is split at its NAL unit start codes before it is sent, so encrypting it whole breaks the packetizer; prefer VP8, VP9 or AV1 for encrypted calls, or leave the NAL unit headers in the clear.
+- **Make each frame self-contained.** Frames are lost, so each needs what it takes to be decrypted, such as its IV, and a receiver must drop a frame that fails to authenticate rather than pass it on.
+
+## Threading and Performance
+
+The transform never runs on a thread that carries media. Each sender and receiver with a transform gets a thread of its own. Frames are queued to that thread and transformed there one at a time, in order, and are handed back to WebRTC from it. This means that:
+
+- a slow transform delays only its own frames, never the connection;
+- a transform may call back into the peer connection, even close it, without deadlocking;
+- a transform that falls several seconds behind has frames dropped until it catches up, rather than having them queue up in memory.
+
+Per frame, the cost is one small Java object and, only if the payload is read, one copy into a buffer that is reused from frame to frame. Senders and receivers without a transform are not touched at all.

docs/guide/examples.md

@@ -95,6 +95,24 @@ mvn exec:java -D"exec.mainClass=dev.onvoid.webrtc.examples.MediaFilePlayerExampl
 ```
 :::
 
+## Encrypted Recording
+
+The [`EncryptedRecordingExample`](https://github.com/devopvoid/webrtc-java/blob/master/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/EncryptedRecordingExample.java) encrypts a call end to end with encoded frame transforms, and records what the receiving side decrypted into a media file. See the [Encoded Transforms](/guide/advanced/encoded-transforms) and [Media Recording](/guide/media/media-recording) guides for the APIs it uses.
+
+**Key features demonstrated:**
+- Encrypting every encoded audio and video frame with AES-GCM in an `RTCEncodedFrameTransformer` on the senders, and decrypting it on the receivers
+- Keeping the codec header of each frame in the clear for the packetizer, and authenticating it instead
+- Dropping frames that fail to encrypt or to authenticate, rather than passing them on
+- Recording the receivers with a `MediaRecorder`, without decoding or re-encoding
+
+::: info
+Like the [Media File](#media-file) example, this one needs the `webrtc-java-media` module. The output file is optional; its extension picks the container.
+
+```bash
+mvn exec:java -D"exec.mainClass=dev.onvoid.webrtc.examples.EncryptedRecordingExample" -D"exec.args=call.mkv"
+```
+:::
+
 ## Web Client
 
 The [`WebClientExample`](https://github.com/devopvoid/webrtc-java/blob/master/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/web/WebClientExample.java) demonstrates how to combine WebSocket signaling with WebRTC peer connections for real-time communication between web and Java clients.

docs/guide/index.md

@@ -8,6 +8,7 @@ This section provides detailed guides for various features of the webrtc-java li
 - [Bitrate and Framerate Constraints](/guide/media/constraints) - Controlling media quality
 - [Send-only and Receive-only](/guide/media/directionality) - Configure transceiver directions (send-only, receive-only or inactive)
 - [Media Files](/guide/media/media-files) - Sending video and audio files instead of a camera and microphone
+- [Media Recording](/guide/media/media-recording) - Recording what a call sends or receives into a media file, without re-encoding
 
 ## Audio
 
@@ -36,6 +37,11 @@ This section provides detailed guides for various features of the webrtc-java li
 - [RTC Stats](/guide/monitoring/rtc-stats) - Monitoring connection quality and performance
 - [Logging](/guide/monitoring/logging) - Configuring and using the logging system
 
+## Advanced
+
+- [Field Trials](/guide/advanced/field-trials) - Enabling experimental features and tuning WebRTC internals
+- [Encoded Transforms](/guide/advanced/encoded-transforms) - Reading and changing encoded frames, e.g. for end-to-end encryption
+
 ## Additional Resources
 
 For a complete API reference, check the [JavaDoc](https://javadoc.io/doc/dev.onvoid.webrtc/webrtc-java/latest/index.html).

docs/guide/media/media-recording.md

@@ -0,0 +1,87 @@
+# Media Recording
+
+This guide explains how to record what a peer connection sends or receives into a media file with `MediaRecorder`, from the `webrtc-java-media` module. It covers:
+
+- Recording senders and receivers
+- Choosing the container
+- Following a recording with a listener
+- How tracks are started, timed and left out
+- Recording an end-to-end encrypted call
+
+`MediaRecorder` writes the encoded frames of each sender or receiver into the file as they are, without decoding or re-encoding them. A recording therefore costs next to no CPU, keeps exactly the quality that went over the network, and needs no codec at all, only the container. The frames are taken in native code, straight from WebRTC, and written by a thread of the recorder's own, so neither Java nor a slow disk ever holds up the call.
+
+See [Media Files](/guide/media/media-files) for how to add the module to a project.
+
+## Recording a Call
+
+Add the senders and receivers to record, then start:
+
+```java
+try (MediaRecorder recorder = new MediaRecorder(Paths.get("call.mkv"))) {
+    recorder.addTrack(videoReceiver);
+    recorder.addTrack(audioReceiver);
+    recorder.start();
+
+    // ... the call goes on ...
+
+    recorder.stop();
+}
+```
+
+A sender's frames are recorded as they leave the encoder, a receiver's as they go to the decoder. Senders and receivers of any number of peer connections can go into one file, e.g. both sides of a call.
+
+`stop()` waits until everything received so far is written and the file is finished, and returns whether the file holds a recording. A recorder that never got any media to write deletes its file rather than leave an unplayable one behind. `close()` stops a recording that still runs.
+
+::: warning
+Keep the senders and receivers undisposed while recording. The recorder asks them for key frames, and a disposed one cannot be asked.
+:::
+
+## Containers
+
+The container follows the file name:
+
+| Extension | Video | Audio | Notes |
+| --- | --- | --- | --- |
+| `.mkv` | VP8, VP9, AV1, H.264, H.265 | Opus, G.711 | Holds every codec WebRTC sends; the safe choice. |
+| `.webm` | VP8, VP9, AV1 | Opus | Plays in browsers. |
+| `.mp4` | VP9, AV1, H.264, H.265 | Opus | Written fragmented, so it plays up to where a recording was cut short. |
+
+A name FFmpeg does not know gets Matroska. A track whose codec the file cannot hold, VP8 in MP4 for example, is left out and reported as a warning, and the rest is recorded.
+
+## Following a Recording
+
+A `MediaRecorderListener` hears when the file begins, what was left out, and whether writing failed:
+
+```java
+recorder.setListener(new MediaRecorderListener() {
+
+    @Override
+    public void onStarted() {
+        System.out.println("Recording");
+    }
+
+    @Override
+    public void onWarning(String message) {
+        System.out.println("Warning: " + message);
+    }
+
+    @Override
+    public void onError(String message) {
+        System.out.println("Failed: " + message);
+    }
+});
+```
+
+Calls arrive in order on a thread the recorder keeps for them, so a listener may take its time and may even stop the recorder.
+
+## How Tracks Are Recorded
+
+- **Video starts at a key frame.** A decoder can do nothing with the frames before one, so they are skipped. The recorder asks the sender or receiver for a key frame, so recording starts within moments instead of waiting until WebRTC sends one of its own accord, which it rarely does.
+- **The file begins once the tracks are known.** The file header describes every stream, and a stream is only known from its first frames. The file begins once every track has sent some, or once the tracks that have waited three seconds for the rest. A track that sends nothing by then is left out, with a warning.
+- **Tracks share one timeline.** Each track is placed by when its first frame arrived, and follows its own RTP timestamps from there, so its timing is exact. The tracks of one sender line up within the jitter of the network.
+- **One simulcast layer.** A sender with simulcast is recorded at the layer whose frames reach the recorder first.
+- **Nothing is lost to a slow disk, up to a point.** Frames queue up in memory for the writer, up to 64 MB. Beyond that, frames are dropped, and video starts over at the next key frame.
+
+## Recording an Encrypted Call
+
+A sender's frames are recorded before an [encoded transform](/guide/advanced/encoded-transforms) runs on them, and a receiver's after it. An end-to-end encrypted call is therefore recorded in the clear, on either side. [`EncryptedRecordingExample`](https://github.com/devopvoid/webrtc-java/blob/master/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/EncryptedRecordingExample.java) shows both together.

docs/index.md

@@ -28,6 +28,10 @@ features:
     details: Audio and video capture from cameras and microphones devices, with support for custom media sources for flexible streaming solutions.
   - title: Media File Playback
     details: Send video and audio files over a peer connection. The optional media module decodes with FFmpeg in native code and paces playback in real time, keeping audio and video in sync.
+  - title: Call Recording
+    details: Record what a peer connection sends or receives into MKV, WebM or MP4 files. Encoded frames go into the file as they are, without re-encoding, so recording costs next to no CPU and keeps the exact quality of the call.
+  - title: End-to-End Encryption
+    details: Encoded transforms let Java code read, change or drop every encoded frame between encoder and network, like insertable streams in the browser; the building block for end-to-end encryption, frame metadata and stream analysis.
   - title: Screen Sharing
     details: Share application windows or the full desktop with minimal setup; integrate screen capture streams like any other media stream.
   - title: Data Channels