WebCodecs API Tutorial: Decode and Encode Video Natively
A complete WebCodecs API tutorial: decode an MP4 to a canvas, apply frame-level processing, encode the frames back to H.264, and mux them into a playable MP4 with MediaBunny — plus the VideoFrame.close() memory contract every guide skips.
ffmpeg.wasm is roughly 20 megabytes of WebAssembly before you process a single pixel. It boots up a software codec, warms up for seconds on a phone, and eats your user's battery while it works. Meanwhile, the browser already ships with hardware H.264, VP9, and AV1 decoders sitting idle — you just never had a direct line to them. That's what the WebCodecs API gives you: the platform's own codecs, frame by frame, with no download tax. In this WebCodecs API tutorial you'll decode an MP4 to a canvas, draw on the frames, encode them back to H.264, and mux everything into a playable file — all without a single WASM byte. What you'll build and learn Decode an MP4 file frame-by-frame with VideoDecoder and paint each frame to a canvas Encode canvas frames to H.264 with VideoEncoder , including keyframe intervals and bitrate control Mux encoded chunks into a playable MP4 using MediaBunny (the 2026 successor to the deprecated mp4-muxer) Master the VideoFrame.close() memory contract that crashes apps when ignored Probe codec support safely — and why isConfigSupported() deserves a try/catch around configure() Decide when to reach for WebCodecs vs MediaRecorder vs ffmpeg.wasm WebCodecs API Tutorial: The Full Pipeline — Decode an MP4 to Canvas WebCodecs is deliberately low-level. It doesn't know what an MP4 is; it deals in EncodedVideoChunk (compressed data) and VideoFrame (raw pixels). So a decode pipeline has three stages: demux the file into chunks, decode the chunks into frames, and do something with the frames. For demuxing, the 2026 pick is MediaBunny ( npm install mediabunny ) — pure TypeScript, zero dependencies, and the official replacement for the now-deprecated mp4-muxer by the same author. Its EncodedPacketSink hands us packets we convert…