Clipper

The problem

I started Clipper because Medal isn't available on Linux, and I wanted a way to save gameplay clips there. The moment worth saving often happens before you think to press record. Clipper is designed to keep recent gameplay ready: capture and encode video and audio as they arrive, hold the encoded packets in a rolling buffer, then write that recent window to a clip when requested. The current branch gets as far as capture, encoding, and buffering. The F9 key sends a clip command, but handling that command is still a todo!; writing the buffered packets to a finished clip remains work to do.

Why these choices

• Encode during capture. The buffer holds compressed packets rather than raw image frames. That keeps its memory use lower than retaining full-resolution frames, though the current code does not enforce a hard memory limit.
• Use a time-based window. The first buffer used a frame-count limit. The implementation moved to timestamps so eviction follows elapsed time rather than assuming a steady frame rate.
• Use ffmpeg-next for the media pipeline. The project began with rusty_ffmpeg and direct FFmpeg calls. After repeated work around packet ownership and cleanup, it moved to ffmpeg-next for higher-level Rust types for frames, encoders, and packets. The migration reduced the amount of code written directly against the C API, but did not remove unsafe FFmpeg calls from packet cleanup.
• Keep capture platform-specific and muxing shared. Linux and Windows have separate capture modules; both feed buffers to the shared muxer. Linux video capture uses PipeWire, while Windows video capture uses Windows Graphics Capture.
• Track audio and video time separately. Video PTS advances per frame; audio PTS advances by samples. They run on different time bases, so the muxer keeps separate counters.

The rolling buffer

Each encoded packet is stored in a VecDeque with the time it arrived. The target is a 30-second replay window; the current implementation uses a three-second setting while I test the pipeline and work through packet memory use. Both audio and video packets are added to the same queue, with stream indices set during encoding.

This design is intended to avoid seeking through a long recording: the muxer can work with the recent packets already in memory. The current clip-writing path is unfinished, so the branch does not yet demonstrate a complete saved clip.

fn encode_video(&mut self, frame: frame::Video) {
    let video_encoder = self.video_encoder.as_mut().unwrap();
    video_encoder.send_frame(&frame).unwrap();

    loop {
        let mut packet = packet::Packet::empty();
        if video_encoder.receive_packet(&mut packet).is_err() {
            break;
        }
        packet.set_stream(0);
        self.replay_buffer.add_frame(packet);
    }

    self.video_pts += 1;
}

Working through packet ownership

The earlier implementation stored raw AVPacket pointers. The history shows several attempts to address memory use and cleanup, including tracking packet sizes, changing how packets are released, and experimenting with a frame pool. Those iterations made resource ownership a recurring part of the project.

The move to ffmpeg-next replaced much of the raw encoder and frame handling with Rust wrapper types. The replay buffer still uses ffmpeg-sys-next to call av_packet_unref and av_packet_free, so that boundary still needs care. A wrapper makes the ordinary path easier to manage; it does not make every interaction with FFmpeg safe automatically.

impl Drop for ReplayBuffer {
    fn drop(&mut self) {
        for (mut packet, _) in self.frames.drain(..) {
            unsafe {
                let mut raw = &mut packet as *mut _
                    as *mut ffmpeg_sys_next::AVPacket;
                sys::av_packet_unref(raw);
                sys::av_packet_free(&mut raw);
            }
        }
    }
}

Current limitations

The current branch exposes several concrete gaps between the intended replay flow and a finished clip.

• Saving a clip is unfinished. F9 sends a command, but the muxer’s handler is still todo!("rewrite write_clip for safe ffmpeg"). The previous raw-FFmpeg writer is commented out.
• The current eviction loop needs correction. It removes the queue’s first packet before checking its age. If that packet is still within the time window, the loop breaks after removing it; if it is old, the code also pops the next queue entry. By inspection, this can discard packets that should remain buffered.
• The time window is not a memory cap. The buffer tracks packet bytes, but eviction is based on age. Variable packet sizes mean memory use can vary, and no byte limit is enforced.
• Capture queues differ by platform. Windows video capture uses a bounded channel with capacity one; Linux video capture and both audio capture paths use unbounded channels. That means their behavior under a slow encoder is not uniform.

What I’ve learned

The rolling buffer is a small feature sitting on top of several hard problems: capture timing, separate audio and video clocks, packet ownership, and platform-specific input. Moving to ffmpeg-next made the media pipeline more approachable, while the remaining raw cleanup calls show where the C API still matters.

The next work is concrete: correct and verify eviction, finish writing buffered packets as a clip, and check the result end to end.