Filmstrips that keep up

12 stills at 240 px in 24 to 36 ms, 9× to 11× faster than ffmpeg, which runs 12 processes, 8 at a time.

What it does

frames_at takes stills at any times, from the keyframe at or before each one, and reads only what it needs. In MKV and WebM it seeks through the file's Cues, so a still late in a long recording costs the same as one at the start. frames_at_seek with Seek::Exact decodes on to the exact frame.

The strip above was made from twelve stills taken by frames_at (a GIF pito-codecs' own encoder wrote in 27 ms).

Against ffmpeg, pito-codecs takes all 12 stills in one process; ffmpeg runs 12 processes (-noaccurate_seek), 8 at a time, as a filmstrip would call it.

  • A handle that keeps its decoders

    An app that asks for stills again and again keeps a Stills handle: Stills::open opens the file once, and stills.frames reuses its decoders across calls and hands each still over as soon as it is decoded, in any order.

  • Memory you can cap

    A handle runs and keeps up to 16 decoders; stills.keep(n) caps both and stills.trim() drops the kept ones when the app goes idle. With three handles open, keeping 4 decoders each brings the process's peak from 916 MB to 442 MB.

  • Contact sheets from the command line

    pcod stills writes N evenly spaced stills or stills at given times, as PNGs into a folder and as one contact sheet, through pito-codecs' own PNG encoder.

Measured

12 stills at 240 px, 9× to 11× faster than ffmpeg running 12 processes, 8 at a time
24–36 ms
How it was measured
stills of a 1080p H.264 file on the CPU; ffmpeg 57, running 12 processes, 8 at a time
510 / s
How it was measured
stills of a 1440p HEVC file, GPU first; ffmpeg 33, running 12 processes, 8 at a time
369 / s
How it was measured
a 24-still filmstrip at 90 px of a 120 s 1080p file on a kept handle; 22.5 ms through separate calls
10.8 ms
How it was measured
a 24-still contact sheet of a 10-minute 1080p file on the GPU route; ffmpeg's filter chain 5.2 to 6.6 s
0.54–0.62 s
How it was measured

How to use it

  1. A contact sheet and the PNGs behind it:

    pcod stills clip.mp4 --count 24 --cols 6 --sheet sheet.png --out stills/
  2. Stills at given times, each listed with its time, size and path:

    pcod stills clip.mp4 --at 90,1:30,1:02:03.5 --out stills/ --json
  3. In Rust, stills at 60 s and 90 minutes, 120 px tall:

    let strip = frames_at("recording.mkv".as_ref(), &[60.0, 5400.0], 120, &prefs)?;
Paired bars, stills per second, higher is better. H.264 1080p: 510 against ffmpeg's 56.9 on the CPU, 507 against 57.2 GPU first. H.264 1440p: 332 against 38.9, 340 against 38.8. HEVC 1080p: 486 against 51.0, 470 against 50.2. HEVC 1440p: 360 against 33.1, 369 against 33.3.
Filmstrip, 12 stills at 240 px 510 stills a second against 57 for 1080p H.264, and 9× to 11× more across the rows.Ours runs in one process; ffmpeg runs 12 processes (-noaccurate_seek), 8 at a time. How it was measured: against ffmpeg

Read it. Run it. It's MIT.

The code is MIT licensed, © Catalin Ilinca: read it, fork it, ship it in your own work. Its name and logos, and PITO MD's, are © Catalin Ilinca, all rights reserved, and are not covered by the MIT licence.

Credits

The clip comes from pito-codecs v1.16.3: a picture its examples/readme_shots.rs makes with its own decoders and GIF encoder from public-domain NASA footage ("A Message of Resilience from the International Space Station", NASA Johnson).

The charts are drawn by pito-compare from pito-codecs' data files and from numbers its README states.

Set in Archivo by Omnibus-Type and Atkinson Hyperlegible Mono by the Braille Institute, both under the SIL Open Font License.