mirror of
https://github.com/storytold/filmcraft.git
synced 2026-10-22 18:22:22 +00:00
T3: golden-image tests for the renderer + GPU parity; fix stale GPU uploads
crates/golden (test-only, L3): nine procedural 320x180 scenes (transform and opacity, blend modes, Gaussian Blur, Lumetri basic, Crop, Cross Dissolve / Dip to Black / Wipe mid-transition, timecode/clip-name text) rendered on the CPU and compared with committed goldens (PSNR >= 45 dB, max <= 12, p99 <= 2); FILMCRAFT_BLESS=1 regenerates them with attribution sidecars. testkit::golden adds PNG I/O, diff metrics and failure images. The same scenes are composited on the GPU and compared with the CPU. GPU parity exposed a real bug: GpuCompositor cached uploads by pixel-buffer address without owning the buffer, so a new frame allocated at a freed frame's address was drawn with a stale texture (whole wrong frames, PSNR ~12 dB). The cache now keeps the buffer alive; regression test upload_cache_keeps_buffers_alive. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Brandon ThomasandClaude Opus 5.5 committed
1 parent
3704002f89
commit
7655447233
30 files changed
+879
-8
No files matched your search
@@ -23,6 +23,24 @@ iconography, images or other Adobe assets; open licences only; every asset attri
|
||||
| `docs/images/filmcraft-effects.png` | FilmCraft contributors | Original work: FilmCraft screenshot crop; NASA Earth Views (US Government work, public domain) | MIT OR Apache-2.0; footage public domain |
|
||||
| `docs/images/filmcraft-timeline.png` | FilmCraft contributors | Original work: FilmCraft UI crop; thumbnails from Night of the Living Dead (1968) and Carnival of Souls (1962), CC0 Chopin waveform | MIT OR Apache-2.0; film frames public domain |
|
||||
|
||||
## Golden test images
|
||||
|
||||
Reference frames for the renderer golden tests (`crates/golden`), rendered by FilmCraft itself from
|
||||
procedural demo footage (`crates/media/src/generators.rs`); no external media. Regenerate with
|
||||
`FILMCRAFT_BLESS=1 cargo test -p filmcraft-golden`.
|
||||
|
||||
| Asset | Author | Source | Licence |
|
||||
|---|---|---|---|
|
||||
| `crates/golden/goldens/blend_modes.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `blend_modes`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/crop.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `crop`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/gaussian_blur.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `gaussian_blur`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/lumetri_basic.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `lumetri_basic`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/text_burnin.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `text_burnin`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/transform_opacity.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transform_opacity`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/transition_cross_dissolve.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transition_cross_dissolve`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/transition_dip_to_black.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transition_dip_to_black`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/transition_wipe.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transition_wipe`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
|
||||
## Third-party media shown in screenshots
|
||||
|
||||
The README screenshots show frames from these works. The media files themselves are not in the
|
||||
|
||||
Generated
+15
@@ -1451,6 +1451,21 @@ dependencies = [
|
||||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-golden"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"filmcraft-geom",
|
||||
"filmcraft-gpu",
|
||||
"filmcraft-media",
|
||||
"filmcraft-project",
|
||||
"filmcraft-render",
|
||||
"filmcraft-testkit",
|
||||
"filmcraft-time",
|
||||
"pollster 0.4.0",
|
||||
"wgpu",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-gpu"
|
||||
version = "0.1.0"
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
[package]
|
||||
name = "filmcraft-golden"
|
||||
description = "Golden-image tests for the CPU renderer/compositor and GPU-vs-CPU parity (test-only crate)"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
rust-version.workspace = true
|
||||
publish = false
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
|
||||
# Test-only: the library is empty and everything is a dev-dependency, so the crate adds no edges
|
||||
# to the layer graph. Goldens live in `goldens/` (see docs/testing.md §3).
|
||||
[dependencies]
|
||||
|
||||
[dev-dependencies]
|
||||
filmcraft-testkit = { workspace = true }
|
||||
filmcraft-time = { workspace = true }
|
||||
filmcraft-geom = { workspace = true }
|
||||
filmcraft-media = { workspace = true }
|
||||
filmcraft-project = { workspace = true }
|
||||
filmcraft-render = { workspace = true }
|
||||
filmcraft-gpu = { workspace = true }
|
||||
pollster = "0.4"
|
||||
wgpu = { version = "30", features = ["wgsl", "metal", "vulkan", "dx12", "gles"] }
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 37 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: blend_modes.png
|
||||
title: Blend modes (Multiply, Screen, Overlay, Difference)
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (blend_modes). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 7.5 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: crop.png
|
||||
title: Crop effect
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (crop). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 7.8 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: gaussian_blur.png
|
||||
title: Gaussian Blur effect
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (gaussian_blur). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: lumetri_basic.png
|
||||
title: Lumetri Color basic correction
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (lumetri_basic). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 4.4 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: text_burnin.png
|
||||
title: Timecode and Clip Name burn-in text
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (text_burnin). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: transform_opacity.png
|
||||
title: Motion transform and opacity
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (transform_opacity). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 29 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: transition_cross_dissolve.png
|
||||
title: Cross Dissolve at 50%
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (transition_cross_dissolve). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 17 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: transition_dip_to_black.png
|
||||
title: Dip to Black at 25%
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (transition_dip_to_black). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: transition_wipe.png
|
||||
title: Wipe at 50%
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (transition_wipe). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
@@ -0,0 +1,8 @@
|
||||
//! Golden-image tests for FilmCraft's renderer (test-only crate; the library is empty).
|
||||
//!
|
||||
//! `tests/golden.rs` builds small procedural projects (no media files), renders one frame of each
|
||||
//! through the CPU reference compositor (`filmcraft-render`) and compares it with a committed PNG
|
||||
//! in `goldens/` using `filmcraft_testkit::golden`. The same scenes are composited on the GPU
|
||||
//! (`filmcraft-gpu`) when an adapter is available and compared with the CPU result.
|
||||
//!
|
||||
//! Regenerate references with `FILMCRAFT_BLESS=1 cargo test -p filmcraft-golden`.
|
||||
@@ -0,0 +1,345 @@
|
||||
//! Golden-image tests: procedural projects rendered through the CPU compositor and compared with
|
||||
//! committed reference PNGs (`goldens/*.png`, made by this test with `FILMCRAFT_BLESS=1`), plus
|
||||
//! GPU-vs-CPU parity on the same scenes when a GPU adapter is available.
|
||||
//!
|
||||
//! Tolerances (8-bit sRGB levels, see `filmcraft_testkit::golden`):
|
||||
//! - CPU vs golden: `Tolerance::RENDER` — PSNR ≥ 45 dB, max abs ≤ 12, 99th percentile ≤ 2.
|
||||
//! - GPU vs CPU: 99th percentile of the per-pixel max channel difference ≤ 6 and mean < 1.5
|
||||
//! (antialiased edges and half-float textures differ slightly; the CPU is the reference).
|
||||
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
|
||||
use filmcraft_geom::Vec2;
|
||||
use filmcraft_media::generators::GeneratorSource;
|
||||
use filmcraft_media::{DemoScene, Generator, MediaSource, SharedSource};
|
||||
use filmcraft_project::{
|
||||
ClipId, EffectInstance, ItemId, ItemKind, Label, MediaClip, MediaRef, ParamValue, Project, SequenceSettings, TrackItem, TrackKind, Transition,
|
||||
TransitionId, find_effect,
|
||||
};
|
||||
use filmcraft_render::{RenderOptions, SourceMap, render_sequence};
|
||||
use filmcraft_testkit::golden::{Rgba8, Tolerance, assert_golden, diff};
|
||||
use filmcraft_time::{FrameRate, TICKS_PER_SECOND, Tick, TimeRange};
|
||||
|
||||
const W: u32 = 320;
|
||||
const H: u32 = 180;
|
||||
const RATE: FrameRate = FrameRate::FPS_24;
|
||||
|
||||
/// A procedurally generated project and the frame to render.
|
||||
struct Scene {
|
||||
project: Project,
|
||||
seq: ItemId,
|
||||
sources: SourceMap,
|
||||
frame: i64,
|
||||
}
|
||||
|
||||
struct Builder {
|
||||
p: Project,
|
||||
map: SourceMap,
|
||||
seq: ItemId,
|
||||
next_transition: u64,
|
||||
}
|
||||
|
||||
impl Builder {
|
||||
fn new(video_tracks: usize) -> Self {
|
||||
let mut p = Project::new("golden");
|
||||
let seq = p.new_sequence("golden", SequenceSettings { width: W, height: H, frame_rate: RATE, ..Default::default() }, video_tracks, 0, None);
|
||||
Builder { p, map: SourceMap::default(), seq, next_transition: 1 }
|
||||
}
|
||||
|
||||
/// A generated media item (640×360, 24 fps, 10 s).
|
||||
fn media(&mut self, g: Generator) -> ItemId {
|
||||
let src = GeneratorSource::new(g, 640, 360, RATE, Tick(10 * TICKS_PER_SECOND));
|
||||
let info = src.info().clone();
|
||||
let id = self.p.add_item(
|
||||
&info.name.clone(),
|
||||
Label::Iris,
|
||||
ItemKind::Media(MediaClip {
|
||||
media: MediaRef::Generator(src.generator.clone()),
|
||||
info,
|
||||
interpret: Default::default(),
|
||||
mark_in: None,
|
||||
mark_out: None,
|
||||
markers: vec![],
|
||||
offline: false,
|
||||
proxy: None,
|
||||
}),
|
||||
None,
|
||||
);
|
||||
self.map.0.insert(id, Arc::new(src) as SharedSource);
|
||||
id
|
||||
}
|
||||
|
||||
fn demo(&mut self, scene: DemoScene) -> ItemId {
|
||||
self.media(Generator::Demo(scene))
|
||||
}
|
||||
|
||||
/// Place `item` on video track `track` (frames), scaled to the frame size.
|
||||
fn place(&mut self, track: usize, item: ItemId, start: i64, dur: i64) -> ClipId {
|
||||
let mut ti = self.p.make_track_item(item, TrackKind::Video, RATE.tick_of(start), TimeRange::new(Tick::ZERO, RATE.tick_of(dur)), RATE).unwrap();
|
||||
ti.scale_to_frame = true;
|
||||
let id = ti.id;
|
||||
let s = self.p.sequence_mut(self.seq).unwrap();
|
||||
s.video_tracks[track].items.push(ti);
|
||||
s.video_tracks[track].sort();
|
||||
id
|
||||
}
|
||||
|
||||
fn clip(&mut self, id: ClipId) -> &mut TrackItem {
|
||||
self.p.sequence_mut(self.seq).unwrap().find_item_mut(id).unwrap().1
|
||||
}
|
||||
|
||||
/// Set intrinsic-effect parameters (`motion`, `opacity`).
|
||||
fn fixed(&mut self, id: ClipId, effect: &str, params: &[(&str, ParamValue)]) {
|
||||
let e = self.clip(id).effect_mut(effect).unwrap();
|
||||
set(e, params);
|
||||
}
|
||||
|
||||
/// Append a standard effect with parameters.
|
||||
fn effect(&mut self, id: ClipId, effect: &str, params: &[(&str, ParamValue)]) {
|
||||
let mut e = find_effect(effect).unwrap_or_else(|| panic!("no effect {effect}")).instance();
|
||||
set(&mut e, params);
|
||||
self.clip(id).effects.push(e);
|
||||
}
|
||||
|
||||
/// A transition of `dur` frames centred on the cut at `cut` between `a` and `b` on `track`.
|
||||
fn transition(&mut self, track: usize, effect: &str, a: ClipId, b: ClipId, cut: i64, dur: i64) {
|
||||
let id = TransitionId(self.next_transition);
|
||||
self.next_transition += 1;
|
||||
let tr = Transition {
|
||||
id,
|
||||
effect: find_effect(effect).unwrap_or_else(|| panic!("no transition {effect}")).instance(),
|
||||
start: RATE.tick_of(cut - dur / 2),
|
||||
duration: RATE.tick_of(dur),
|
||||
from: Some(a),
|
||||
to: Some(b),
|
||||
align: Default::default(),
|
||||
reverse: false,
|
||||
};
|
||||
self.p.sequence_mut(self.seq).unwrap().video_tracks[track].transitions.push(tr);
|
||||
}
|
||||
|
||||
fn at(self, frame: i64) -> Scene {
|
||||
Scene { project: self.p, seq: self.seq, sources: self.map, frame }
|
||||
}
|
||||
}
|
||||
|
||||
fn set(e: &mut EffectInstance, params: &[(&str, ParamValue)]) {
|
||||
let id = e.effect.clone();
|
||||
for (k, v) in params {
|
||||
e.param_mut(k).unwrap_or_else(|| panic!("{id}: no param {k}")).value = v.clone();
|
||||
}
|
||||
}
|
||||
|
||||
fn fl(v: f64) -> ParamValue {
|
||||
ParamValue::Float(v)
|
||||
}
|
||||
|
||||
fn pt(x: f64, y: f64) -> ParamValue {
|
||||
ParamValue::Vec2(Vec2::new(x, y))
|
||||
}
|
||||
|
||||
fn blend(name: &str) -> ParamValue {
|
||||
ParamValue::Choice(filmcraft_project::effect::BLEND_MODES.iter().position(|b| *b == name).unwrap_or_else(|| panic!("blend {name}")) as u32)
|
||||
}
|
||||
|
||||
// ---- scenes ----
|
||||
|
||||
/// Motion (scale, rotation, position) and 70 % opacity over a full-frame background.
|
||||
fn transform_opacity() -> Scene {
|
||||
let mut b = Builder::new(2);
|
||||
let bg = b.demo(DemoScene::OceanSunset);
|
||||
let fg = b.demo(DemoScene::Aurora);
|
||||
b.place(0, bg, 0, 48);
|
||||
let top = b.place(1, fg, 0, 48);
|
||||
b.fixed(top, "motion", &[("scale", fl(45.0)), ("rotation", fl(20.0)), ("position", pt(215.0, 70.0))]);
|
||||
b.fixed(top, "opacity", &[("opacity", fl(70.0))]);
|
||||
b.at(12)
|
||||
}
|
||||
|
||||
/// Four quadrant layers over a background in Multiply, Screen, Overlay and Difference.
|
||||
fn blend_modes() -> Scene {
|
||||
let mut b = Builder::new(5);
|
||||
let bg = b.demo(DemoScene::OceanSunset);
|
||||
let fg = b.demo(DemoScene::Dunes);
|
||||
b.place(0, bg, 0, 48);
|
||||
for (i, (mode, x, y)) in [("Multiply", 80.0, 45.0), ("Screen", 240.0, 45.0), ("Overlay", 80.0, 135.0), ("Difference", 240.0, 135.0)].into_iter().enumerate()
|
||||
{
|
||||
let c = b.place(i + 1, fg, 0, 48);
|
||||
b.fixed(c, "motion", &[("scale", fl(50.0)), ("position", pt(x, y))]);
|
||||
b.fixed(c, "opacity", &[("blend", blend(mode))]);
|
||||
}
|
||||
b.at(12)
|
||||
}
|
||||
|
||||
/// Gaussian Blur on colour bars (sharp edges show the kernel).
|
||||
fn gaussian_blur() -> Scene {
|
||||
let mut b = Builder::new(1);
|
||||
let bars = b.media(Generator::BarsAndTone);
|
||||
let c = b.place(0, bars, 0, 48);
|
||||
b.effect(c, "gaussian_blur", &[("blurriness", fl(12.0))]);
|
||||
b.at(12)
|
||||
}
|
||||
|
||||
/// Lumetri Color basic correction: temperature, exposure, contrast, highlights/shadows, saturation.
|
||||
fn lumetri_basic() -> Scene {
|
||||
let mut b = Builder::new(1);
|
||||
let bg = b.demo(DemoScene::OceanSunset);
|
||||
let c = b.place(0, bg, 0, 48);
|
||||
b.effect(
|
||||
c,
|
||||
"lumetri",
|
||||
&[
|
||||
("temperature", fl(25.0)),
|
||||
("exposure", fl(0.6)),
|
||||
("contrast", fl(30.0)),
|
||||
("highlights", fl(-30.0)),
|
||||
("shadows", fl(25.0)),
|
||||
("saturation", fl(135.0)),
|
||||
],
|
||||
);
|
||||
b.at(12)
|
||||
}
|
||||
|
||||
/// Crop (left/top/right/bottom) of a top layer revealing the background.
|
||||
fn crop() -> Scene {
|
||||
let mut b = Builder::new(2);
|
||||
let bg = b.demo(DemoScene::OceanSunset);
|
||||
let bars = b.media(Generator::BarsAndTone);
|
||||
b.place(0, bg, 0, 48);
|
||||
let c = b.place(1, bars, 0, 48);
|
||||
b.effect(c, "crop", &[("left", fl(15.0)), ("top", fl(10.0)), ("right", fl(25.0)), ("bottom", fl(20.0))]);
|
||||
b.at(12)
|
||||
}
|
||||
|
||||
/// Two clips with a 12-frame transition centred on the cut at frame 24, rendered at `frame`.
|
||||
fn transition(effect: &str, frame: i64) -> Scene {
|
||||
let mut b = Builder::new(1);
|
||||
let a = b.demo(DemoScene::OceanSunset);
|
||||
let c = b.demo(DemoScene::Aurora);
|
||||
let ca = b.place(0, a, 0, 24);
|
||||
let cb = b.place(0, c, 24, 24);
|
||||
b.transition(0, effect, ca, cb, 24, 12);
|
||||
b.at(frame)
|
||||
}
|
||||
|
||||
/// Timecode and Clip Name burn-ins (FilmCraft's built-in bitmap text).
|
||||
fn text_burnin() -> Scene {
|
||||
let mut b = Builder::new(1);
|
||||
let bg = b.demo(DemoScene::CityNight);
|
||||
let c = b.place(0, bg, 0, 48);
|
||||
b.effect(c, "timecode", &[("position", pt(160.0, 40.0)), ("size", fl(20.0))]);
|
||||
b.effect(c, "clip_name", &[("position", pt(160.0, 145.0)), ("size", fl(12.0))]);
|
||||
b.at(30)
|
||||
}
|
||||
|
||||
/// (name, title, scene).
|
||||
fn scenes() -> Vec<(&'static str, &'static str, fn() -> Scene)> {
|
||||
vec![
|
||||
("transform_opacity", "Motion transform and opacity", transform_opacity),
|
||||
("blend_modes", "Blend modes (Multiply, Screen, Overlay, Difference)", blend_modes),
|
||||
("gaussian_blur", "Gaussian Blur effect", gaussian_blur),
|
||||
("lumetri_basic", "Lumetri Color basic correction", lumetri_basic),
|
||||
("crop", "Crop effect", crop),
|
||||
("transition_cross_dissolve", "Cross Dissolve at 50%", || transition("cross_dissolve", 24)),
|
||||
("transition_dip_to_black", "Dip to Black at 25%", || transition("dip_to_black", 21)),
|
||||
("transition_wipe", "Wipe at 50%", || transition("wipe", 24)),
|
||||
("text_burnin", "Timecode and Clip Name burn-in text", text_burnin),
|
||||
]
|
||||
}
|
||||
|
||||
fn render_cpu(s: &Scene) -> Rgba8 {
|
||||
let img = render_sequence(&s.project, s.seq, RATE.tick_of(s.frame), RenderOptions::default(), &s.sources);
|
||||
Rgba8::new(img.w as u32, img.h as u32, img.over_black_rgba8())
|
||||
}
|
||||
|
||||
fn golden_path(name: &str) -> PathBuf {
|
||||
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("goldens").join(format!("{name}.png"))
|
||||
}
|
||||
|
||||
fn check(name: &str) {
|
||||
let (_, title, make) = scenes().into_iter().find(|(n, _, _)| *n == name).unwrap();
|
||||
let img = render_cpu(&make());
|
||||
assert_eq!((img.w, img.h), (W, H));
|
||||
let d = assert_golden(&golden_path(name), &img, Tolerance::RENDER, title, &format!("crates/golden/tests/golden.rs ({name})"));
|
||||
eprintln!("{name}: {d}");
|
||||
}
|
||||
|
||||
macro_rules! goldens {
|
||||
($($name:ident),* $(,)?) => {
|
||||
mod cpu {
|
||||
$(
|
||||
#[test]
|
||||
fn $name() {
|
||||
super::check(stringify!($name));
|
||||
}
|
||||
)*
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_scene_has_a_test() {
|
||||
let tested = [$(stringify!($name)),*];
|
||||
for (n, _, _) in scenes() {
|
||||
assert!(tested.contains(&n), "scene {n} has no golden test");
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
goldens!(transform_opacity, blend_modes, gaussian_blur, lumetri_basic, crop, transition_cross_dissolve, transition_dip_to_black, transition_wipe, text_burnin);
|
||||
|
||||
/// Sanity: the scenes are not trivially empty or identical to each other.
|
||||
#[test]
|
||||
fn scenes_are_distinct() {
|
||||
let imgs: Vec<(&str, Rgba8)> = scenes().into_iter().map(|(n, _, f)| (n, render_cpu(&f()))).collect();
|
||||
for (n, img) in &imgs {
|
||||
let lit = img.px.chunks_exact(4).filter(|p| p[0] as u32 + p[1] as u32 + p[2] as u32 > 30).count();
|
||||
assert!(lit > img.px.len() / 4 / 4, "{n}: mostly black");
|
||||
}
|
||||
for i in 0..imgs.len() {
|
||||
for j in i + 1..imgs.len() {
|
||||
let d = diff(&imgs[i].1, &imgs[j].1).unwrap();
|
||||
assert!(d.psnr < 30.0, "{} and {} are near-identical ({d})", imgs[i].0, imgs[j].0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- GPU parity ----
|
||||
|
||||
fn device() -> Option<(wgpu::Device, wgpu::Queue)> {
|
||||
let instance = wgpu::Instance::default();
|
||||
let adapter = pollster::block_on(instance.request_adapter(&wgpu::RequestAdapterOptions::default())).ok()?;
|
||||
pollster::block_on(adapter.request_device(&wgpu::DeviceDescriptor::default())).ok()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gpu_matches_cpu_on_golden_scenes() {
|
||||
let Some((dev, q)) = device() else {
|
||||
eprintln!("SKIPPED (gpu parity): no GPU adapter");
|
||||
return;
|
||||
};
|
||||
let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);
|
||||
let mut failures = Vec::new();
|
||||
for (name, _, make) in scenes() {
|
||||
let s = make();
|
||||
let t = RATE.tick_of(s.frame);
|
||||
let cpu = render_cpu(&s);
|
||||
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources);
|
||||
let kind = if matches!(plan, filmcraft_render::plan::FramePlan::Layers { .. }) { "layers" } else { "cpu image" };
|
||||
c.composite(&plan);
|
||||
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
|
||||
px.chunks_exact_mut(4).for_each(|p| p[3] = 255);
|
||||
let gpu = Rgba8::new(gw, gh, px);
|
||||
let d = diff(&cpu, &gpu).unwrap();
|
||||
// mean over RGB only (alpha is equal by construction)
|
||||
let mean_rgb = d.mean_abs * 4.0 / 3.0;
|
||||
eprintln!("{name} ({kind}): GPU vs CPU {d}");
|
||||
if d.p99 > 6 || mean_rgb >= 1.5 {
|
||||
let dir = filmcraft_testkit::golden::failures_dir();
|
||||
let _ = filmcraft_testkit::golden::write_png(&dir.join(format!("{name}.gpu.png")), &gpu);
|
||||
let _ = filmcraft_testkit::golden::write_png(&dir.join(format!("{name}.cpu.png")), &cpu);
|
||||
failures.push(format!("{name} ({kind}): {d}; images in {}", dir.display()));
|
||||
}
|
||||
}
|
||||
assert!(failures.is_empty(), "GPU differs from the CPU reference:\n{}", failures.join("\n"));
|
||||
}
|
||||
@@ -27,6 +27,9 @@ struct Uploaded {
|
||||
code_scale: f32,
|
||||
chroma: (u32, u32),
|
||||
last_used: u64,
|
||||
/// The uploaded pixel buffers, kept alive while cached: the cache key is the buffer address,
|
||||
/// and a freed buffer's address can be reused by a different frame (stale texture).
|
||||
_pixels: PixelData,
|
||||
}
|
||||
|
||||
pub struct GpuCompositor {
|
||||
@@ -245,12 +248,12 @@ impl GpuCompositor {
|
||||
let up = match &f.data {
|
||||
PixelData::Rgba8(d) => {
|
||||
let v = self.plane_texture(w, h, wgpu::TextureFormat::Rgba8UnormSrgb, d, 4);
|
||||
Uploaded { views: [v, dummy(), dummy()], kind: 0, code_scale: 1.0, chroma: (w, h), last_used: self.clock }
|
||||
Uploaded { views: [v, dummy(), dummy()], kind: 0, code_scale: 1.0, chroma: (w, h), last_used: self.clock, _pixels: f.data.clone() }
|
||||
}
|
||||
PixelData::RgbaF32(d) => {
|
||||
let half: Vec<u8> = d.iter().flat_map(|v| f32_to_f16(*v).to_le_bytes()).collect();
|
||||
let v = self.plane_texture(w, h, wgpu::TextureFormat::Rgba16Float, &half, 8);
|
||||
Uploaded { views: [v, dummy(), dummy()], kind: 1, code_scale: 1.0, chroma: (w, h), last_used: self.clock }
|
||||
Uploaded { views: [v, dummy(), dummy()], kind: 1, code_scale: 1.0, chroma: (w, h), last_used: self.clock, _pixels: f.data.clone() }
|
||||
}
|
||||
PixelData::Yuv8 { planes, chroma, .. } => {
|
||||
let (sx, sy) = chroma.shifts();
|
||||
@@ -258,7 +261,7 @@ impl GpuCompositor {
|
||||
let y = self.plane_texture(w, h, wgpu::TextureFormat::R8Unorm, &planes[0], 1);
|
||||
let u = self.plane_texture(cw, ch, wgpu::TextureFormat::R8Unorm, &planes[1], 1);
|
||||
let v = self.plane_texture(cw, ch, wgpu::TextureFormat::R8Unorm, &planes[2], 1);
|
||||
Uploaded { views: [y, u, v], kind: 2, code_scale: 255.0, chroma: (cw, ch), last_used: self.clock }
|
||||
Uploaded { views: [y, u, v], kind: 2, code_scale: 255.0, chroma: (cw, ch), last_used: self.clock, _pixels: f.data.clone() }
|
||||
}
|
||||
PixelData::Yuv16 { planes, chroma, bits, .. } => {
|
||||
let (sx, sy) = chroma.shifts();
|
||||
@@ -270,7 +273,7 @@ impl GpuCompositor {
|
||||
let u = self.plane_texture(cw, ch, wgpu::TextureFormat::R16Float, &pu, 2);
|
||||
let v = self.plane_texture(cw, ch, wgpu::TextureFormat::R16Float, &pv, 2);
|
||||
let _ = Chroma::C420;
|
||||
Uploaded { views: [y, u, v], kind: 2, code_scale: scale, chroma: (cw, ch), last_used: self.clock }
|
||||
Uploaded { views: [y, u, v], kind: 2, code_scale: scale, chroma: (cw, ch), last_used: self.clock, _pixels: f.data.clone() }
|
||||
}
|
||||
};
|
||||
self.uploads.insert(key, up);
|
||||
|
||||
@@ -77,3 +77,21 @@ fn half_float_conversion() {
|
||||
assert!((d - v).abs() <= v.abs() * 1e-3 + 1e-6, "{v} → {d}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The upload cache is keyed by pixel-buffer address, so it must keep the buffer alive: otherwise
|
||||
/// a new frame allocated at a freed frame's address is drawn with the stale texture (seen as
|
||||
/// whole frames from a previous composite in `crates/golden` GPU parity).
|
||||
#[test]
|
||||
fn upload_cache_keeps_buffers_alive() {
|
||||
let Some((dev, q)) = device() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let mut c = GpuCompositor::new(&dev, &q);
|
||||
let px = Arc::new(vec![200u8; 16 * 8 * 4]);
|
||||
let frame = Arc::new(VideoFrame { width: 16, height: 8, data: PixelData::Rgba8(px.clone()), ..(*yuv_frame(16, 8)).clone() });
|
||||
let plan = FramePlan::Layers { width: 16, height: 8, layers: vec![PlanLayer { frame, matrix: Affine::IDENTITY, opacity: 1.0 }] };
|
||||
c.composite(&plan);
|
||||
drop(plan);
|
||||
assert!(Arc::strong_count(&px) > 1, "cached upload must own its pixel buffer");
|
||||
}
|
||||
@@ -0,0 +1,344 @@
|
||||
//! Golden-image testing: RGBA8 images, PNG I/O, difference metrics and reference checks.
|
||||
//!
|
||||
//! A golden test renders an image, then calls [`assert_golden`] with the path of a committed
|
||||
//! reference PNG and a [`Tolerance`]. On mismatch the actual image and an amplified difference
|
||||
//! image are written to `<workspace>/target/golden-failures/` for inspection.
|
||||
//!
|
||||
//! `FILMCRAFT_BLESS=1` (re)writes the reference instead of comparing, plus its `.attribution`
|
||||
//! sidecar when missing (AGENTS.md §1). References must be produced by FilmCraft itself (original
|
||||
//! work), be small (< [`MAX_GOLDEN_BYTES`]) and be listed in `ATTRIBUTION.md`.
|
||||
|
||||
use std::io::{BufReader, Cursor};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// Size limit for a committed golden PNG.
|
||||
pub const MAX_GOLDEN_BYTES: u64 = 50 * 1024;
|
||||
|
||||
/// An 8-bit RGBA image, row-major, no padding.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct Rgba8 {
|
||||
pub w: u32,
|
||||
pub h: u32,
|
||||
pub px: Vec<u8>,
|
||||
}
|
||||
|
||||
impl Rgba8 {
|
||||
pub fn new(w: u32, h: u32, px: Vec<u8>) -> Self {
|
||||
assert_eq!(px.len(), w as usize * h as usize * 4, "pixel buffer size");
|
||||
Self { w, h, px }
|
||||
}
|
||||
pub fn get(&self, x: u32, y: u32) -> [u8; 4] {
|
||||
let i = (y as usize * self.w as usize + x as usize) * 4;
|
||||
[self.px[i], self.px[i + 1], self.px[i + 2], self.px[i + 3]]
|
||||
}
|
||||
fn opaque(&self) -> bool {
|
||||
self.px.chunks_exact(4).all(|p| p[3] == 255)
|
||||
}
|
||||
}
|
||||
|
||||
/// Encode as PNG (RGB when fully opaque, else RGBA), maximum compression.
|
||||
pub fn encode_png(img: &Rgba8) -> Result<Vec<u8>, String> {
|
||||
let mut out = Vec::new();
|
||||
{
|
||||
let mut enc = png::Encoder::new(&mut out, img.w, img.h);
|
||||
let opaque = img.opaque();
|
||||
enc.set_color(if opaque { png::ColorType::Rgb } else { png::ColorType::Rgba });
|
||||
enc.set_depth(png::BitDepth::Eight);
|
||||
enc.set_compression(png::Compression::High);
|
||||
let mut w = enc.write_header().map_err(|e| e.to_string())?;
|
||||
if opaque {
|
||||
let rgb: Vec<u8> = img.px.chunks_exact(4).flat_map(|p| [p[0], p[1], p[2]]).collect();
|
||||
w.write_image_data(&rgb).map_err(|e| e.to_string())?;
|
||||
} else {
|
||||
w.write_image_data(&img.px).map_err(|e| e.to_string())?;
|
||||
}
|
||||
w.finish().map_err(|e| e.to_string())?;
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Decode a PNG (8-bit gray/RGB/RGBA, palette expanded) to RGBA8.
|
||||
pub fn decode_png(bytes: &[u8]) -> Result<Rgba8, String> {
|
||||
let mut dec = png::Decoder::new(BufReader::new(Cursor::new(bytes)));
|
||||
dec.set_transformations(png::Transformations::EXPAND | png::Transformations::STRIP_16);
|
||||
let mut r = dec.read_info().map_err(|e| e.to_string())?;
|
||||
let mut buf = vec![0; r.output_buffer_size().ok_or("PNG too large")?];
|
||||
let info = r.next_frame(&mut buf).map_err(|e| e.to_string())?;
|
||||
let (w, h) = (info.width, info.height);
|
||||
let n = w as usize * h as usize;
|
||||
let px = match info.color_type {
|
||||
png::ColorType::Rgba => buf[..n * 4].to_vec(),
|
||||
png::ColorType::Rgb => buf[..n * 3].chunks_exact(3).flat_map(|p| [p[0], p[1], p[2], 255]).collect(),
|
||||
png::ColorType::GrayscaleAlpha => buf[..n * 2].chunks_exact(2).flat_map(|p| [p[0], p[0], p[0], p[1]]).collect(),
|
||||
png::ColorType::Grayscale => buf[..n].iter().flat_map(|&g| [g, g, g, 255]).collect(),
|
||||
png::ColorType::Indexed => return Err("unexpanded palette PNG".into()),
|
||||
};
|
||||
Ok(Rgba8 { w, h, px })
|
||||
}
|
||||
|
||||
pub fn read_png(path: &Path) -> Result<Rgba8, String> {
|
||||
let bytes = std::fs::read(path).map_err(|e| format!("{}: {e}", path.display()))?;
|
||||
decode_png(&bytes).map_err(|e| format!("{}: {e}", path.display()))
|
||||
}
|
||||
|
||||
pub fn write_png(path: &Path, img: &Rgba8) -> Result<u64, String> {
|
||||
let bytes = encode_png(img)?;
|
||||
if let Some(d) = path.parent() {
|
||||
std::fs::create_dir_all(d).map_err(|e| e.to_string())?;
|
||||
}
|
||||
std::fs::write(path, &bytes).map_err(|e| format!("{}: {e}", path.display()))?;
|
||||
Ok(bytes.len() as u64)
|
||||
}
|
||||
|
||||
/// Per-pixel difference statistics over all four channels (8-bit levels).
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct Diff {
|
||||
/// Largest absolute channel difference.
|
||||
pub max_abs: u8,
|
||||
/// Mean absolute channel difference.
|
||||
pub mean_abs: f64,
|
||||
/// 99th percentile of the per-pixel maximum channel difference.
|
||||
pub p99: u8,
|
||||
/// PSNR in dB over all channels (`f64::INFINITY` when identical).
|
||||
pub psnr: f64,
|
||||
/// Pixels with any channel different.
|
||||
pub differing: usize,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for Diff {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(f, "PSNR {:.2} dB, max {}, p99 {}, mean {:.3}, {} pixels differ", self.psnr, self.max_abs, self.p99, self.mean_abs, self.differing)
|
||||
}
|
||||
}
|
||||
|
||||
/// Compare two images of equal size.
|
||||
pub fn diff(a: &Rgba8, b: &Rgba8) -> Result<Diff, String> {
|
||||
if (a.w, a.h) != (b.w, b.h) {
|
||||
return Err(format!("size {}x{} vs {}x{}", a.w, a.h, b.w, b.h));
|
||||
}
|
||||
let mut hist = [0usize; 256];
|
||||
let (mut sum, mut sq, mut max, mut differing) = (0u64, 0u64, 0u8, 0usize);
|
||||
for (p, q) in a.px.chunks_exact(4).zip(b.px.chunks_exact(4)) {
|
||||
let mut m = 0u8;
|
||||
for k in 0..4 {
|
||||
let d = p[k].abs_diff(q[k]);
|
||||
sum += d as u64;
|
||||
sq += d as u64 * d as u64;
|
||||
m = m.max(d);
|
||||
}
|
||||
hist[m as usize] += 1;
|
||||
max = max.max(m);
|
||||
differing += (m > 0) as usize;
|
||||
}
|
||||
let pixels = (a.w as usize * a.h as usize).max(1);
|
||||
let target = (pixels * 99).div_ceil(100);
|
||||
let mut acc = 0;
|
||||
let mut p99 = 0u8;
|
||||
for (v, n) in hist.iter().enumerate() {
|
||||
acc += n;
|
||||
if acc >= target {
|
||||
p99 = v as u8;
|
||||
break;
|
||||
}
|
||||
}
|
||||
let n = (pixels * 4) as f64;
|
||||
let mse = sq as f64 / n;
|
||||
let psnr = if mse == 0.0 { f64::INFINITY } else { 10.0 * (255.0f64 * 255.0 / mse).log10() };
|
||||
Ok(Diff { max_abs: max, mean_abs: sum as f64 / n, p99, psnr, differing })
|
||||
}
|
||||
|
||||
/// Acceptance thresholds; every set limit must hold.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct Tolerance {
|
||||
pub min_psnr: f64,
|
||||
pub max_abs: u8,
|
||||
pub p99: u8,
|
||||
}
|
||||
|
||||
impl Tolerance {
|
||||
/// Bit-exact.
|
||||
pub const EXACT: Tolerance = Tolerance { min_psnr: f64::INFINITY, max_abs: 0, p99: 0 };
|
||||
/// Default for CPU renders compared with CPU-made references: absorbs float differences
|
||||
/// between platforms (FMA, SIMD widths, thread scheduling of reductions) but not real changes.
|
||||
pub const RENDER: Tolerance = Tolerance { min_psnr: 45.0, max_abs: 12, p99: 2 };
|
||||
|
||||
pub fn accepts(&self, d: &Diff) -> bool {
|
||||
d.psnr >= self.min_psnr && d.max_abs <= self.max_abs && d.p99 <= self.p99
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `FILMCRAFT_BLESS` asks to regenerate references.
|
||||
pub fn blessing() -> bool {
|
||||
std::env::var("FILMCRAFT_BLESS").is_ok_and(|v| matches!(v.trim().to_ascii_lowercase().as_str(), "1" | "true" | "yes" | "on"))
|
||||
}
|
||||
|
||||
/// Where failing comparisons write `<name>.actual.png` / `<name>.diff.png`.
|
||||
pub fn failures_dir() -> PathBuf {
|
||||
crate::workspace_root().join("target").join("golden-failures")
|
||||
}
|
||||
|
||||
/// Difference image: |a − b| per channel ×8, opaque.
|
||||
pub fn diff_image(a: &Rgba8, b: &Rgba8) -> Rgba8 {
|
||||
let px =
|
||||
a.px.chunks_exact(4)
|
||||
.zip(b.px.chunks_exact(4))
|
||||
.flat_map(|(p, q)| {
|
||||
let d = |k: usize| (p[k].abs_diff(q[k]) as u32 * 8).min(255) as u8;
|
||||
let al = d(3);
|
||||
[d(0).max(al), d(1).max(al), d(2).max(al), 255]
|
||||
})
|
||||
.collect();
|
||||
Rgba8 { w: a.w, h: a.h, px }
|
||||
}
|
||||
|
||||
/// Today's date (UTC) as YYYY-MM-DD, for attribution sidecars.
|
||||
fn today() -> String {
|
||||
let secs = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0);
|
||||
// civil-from-days (Howard Hinnant)
|
||||
let z = (secs / 86_400) as i64 + 719_468;
|
||||
let era = z.div_euclid(146_097);
|
||||
let doe = z - era * 146_097;
|
||||
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
|
||||
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
|
||||
let mp = (5 * doy + 2) / 153;
|
||||
let d = doy - (153 * mp + 2) / 5 + 1;
|
||||
let m = if mp < 10 { mp + 3 } else { mp - 9 };
|
||||
let y = yoe + era * 400 + (m <= 2) as i64;
|
||||
format!("{y:04}-{m:02}-{d:02}")
|
||||
}
|
||||
|
||||
/// The attribution sidecar text for a generated golden image.
|
||||
pub fn sidecar_text(file_name: &str, title: &str, generated_by: &str) -> String {
|
||||
format!(
|
||||
"asset: {file_name}\n\
|
||||
title: {title}\n\
|
||||
author: FilmCraft contributors\n\
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)\n\
|
||||
license: MIT OR Apache-2.0\n\
|
||||
license-file:\n\
|
||||
added: {} by FilmCraft contributors (FILMCRAFT_BLESS=1)\n\
|
||||
notes: Golden reference image generated by {generated_by}. Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.\n",
|
||||
today()
|
||||
)
|
||||
}
|
||||
|
||||
/// The `ATTRIBUTION.md` table row for a golden image at workspace-relative `rel`.
|
||||
pub fn attribution_row(rel: &str, generated_by: &str) -> String {
|
||||
format!("| `{rel}` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft ({generated_by}) | MIT OR Apache-2.0 |")
|
||||
}
|
||||
|
||||
/// Compare `actual` with the golden PNG at `golden` (or bless it). `title` and `generated_by`
|
||||
/// fill the attribution sidecar written on bless. Returns the difference on success.
|
||||
pub fn check_golden(golden: &Path, actual: &Rgba8, tol: Tolerance, title: &str, generated_by: &str) -> Result<Diff, String> {
|
||||
let name = golden.file_stem().map(|s| s.to_string_lossy().to_string()).unwrap_or_default();
|
||||
if blessing() {
|
||||
let size = write_png(golden, actual)?;
|
||||
if size > MAX_GOLDEN_BYTES {
|
||||
return Err(format!("{}: {size} bytes exceeds the {MAX_GOLDEN_BYTES}-byte golden limit; render smaller", golden.display()));
|
||||
}
|
||||
let side = PathBuf::from(format!("{}.attribution", golden.display()));
|
||||
if !side.exists() {
|
||||
let file = golden.file_name().map(|s| s.to_string_lossy().to_string()).unwrap_or_default();
|
||||
std::fs::write(&side, sidecar_text(&file, title, generated_by)).map_err(|e| e.to_string())?;
|
||||
}
|
||||
let root = crate::workspace_root();
|
||||
let rel = golden.strip_prefix(&root).unwrap_or(golden).to_string_lossy().replace('\\', "/");
|
||||
let index = std::fs::read_to_string(root.join("ATTRIBUTION.md")).unwrap_or_default();
|
||||
if !index.contains(&format!("`{rel}`")) {
|
||||
eprintln!("golden {rel}: add this row to ATTRIBUTION.md:\n{}", attribution_row(&rel, generated_by));
|
||||
}
|
||||
eprintln!("blessed {rel} ({size} bytes)");
|
||||
return Ok(Diff { max_abs: 0, mean_abs: 0.0, p99: 0, psnr: f64::INFINITY, differing: 0 });
|
||||
}
|
||||
let expected = read_png(golden).map_err(|e| format!("{e}\n(missing or unreadable golden; generate it with FILMCRAFT_BLESS=1)"))?;
|
||||
let d = diff(&expected, actual).map_err(|e| format!("{}: {e}", golden.display()))?;
|
||||
if tol.accepts(&d) {
|
||||
return Ok(d);
|
||||
}
|
||||
let dir = failures_dir();
|
||||
let a = dir.join(format!("{name}.actual.png"));
|
||||
let _ = write_png(&a, actual);
|
||||
let _ = write_png(&dir.join(format!("{name}.diff.png")), &diff_image(&expected, actual));
|
||||
Err(format!(
|
||||
"{}: {d} (tolerance: PSNR ≥ {} dB, max ≤ {}, p99 ≤ {}); actual and diff written to {}",
|
||||
golden.display(),
|
||||
tol.min_psnr,
|
||||
tol.max_abs,
|
||||
tol.p99,
|
||||
dir.display()
|
||||
))
|
||||
}
|
||||
|
||||
/// [`check_golden`] that panics on failure.
|
||||
pub fn assert_golden(golden: &Path, actual: &Rgba8, tol: Tolerance, title: &str, generated_by: &str) -> Diff {
|
||||
match check_golden(golden, actual, tol, title, generated_by) {
|
||||
Ok(d) => d,
|
||||
Err(e) => panic!("golden mismatch: {e}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn ramp(w: u32, h: u32, alpha: u8) -> Rgba8 {
|
||||
let px = (0..w * h).flat_map(|i| [(i % w * 255 / w) as u8, (i / w * 255 / h) as u8, 77, alpha]).collect();
|
||||
Rgba8::new(w, h, px)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn png_round_trip_rgb_and_rgba() {
|
||||
for a in [255, 128] {
|
||||
let img = ramp(37, 21, a);
|
||||
let png = encode_png(&img).unwrap();
|
||||
assert_eq!(decode_png(&png).unwrap(), img);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn diff_metrics() {
|
||||
let a = ramp(10, 10, 255);
|
||||
assert_eq!(diff(&a, &a).unwrap().psnr, f64::INFINITY);
|
||||
let mut b = a.clone();
|
||||
b.px[0] = b.px[0].wrapping_add(10); // one channel of one pixel
|
||||
let d = diff(&a, &b).unwrap();
|
||||
assert_eq!((d.max_abs, d.p99, d.differing), (10, 0, 1));
|
||||
assert!((d.mean_abs - 10.0 / 400.0).abs() < 1e-12);
|
||||
let mse: f64 = 100.0 / 400.0;
|
||||
assert!((d.psnr - 10.0 * (255.0f64 * 255.0 / mse).log10()).abs() < 1e-9);
|
||||
assert!(Tolerance::RENDER.accepts(&d));
|
||||
assert!(!Tolerance::EXACT.accepts(&d));
|
||||
assert!(diff(&a, &ramp(10, 11, 255)).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn check_golden_accepts_within_tolerance_and_reports_failures() {
|
||||
if blessing() {
|
||||
return;
|
||||
}
|
||||
let dir = std::env::temp_dir().join(format!("filmcraft-testkit-golden-{}", std::process::id()));
|
||||
let path = dir.join("ramp.png");
|
||||
let img = ramp(32, 16, 255);
|
||||
write_png(&path, &img).unwrap();
|
||||
assert!(check_golden(&path, &img, Tolerance::EXACT, "t", "unit test").is_ok());
|
||||
let mut near = img.clone();
|
||||
near.px[5] += 1;
|
||||
assert!(check_golden(&path, &near, Tolerance::RENDER, "t", "unit test").is_ok());
|
||||
let mut far = img.clone();
|
||||
far.px.iter_mut().step_by(4).for_each(|v| *v = 255 - *v);
|
||||
let e = check_golden(&path, &far, Tolerance::RENDER, "t", "unit test").unwrap_err();
|
||||
assert!(e.contains("PSNR"), "{e}");
|
||||
assert!(check_golden(&dir.join("missing.png"), &img, Tolerance::RENDER, "t", "unit test").unwrap_err().contains("FILMCRAFT_BLESS"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn date_and_sidecar_fields() {
|
||||
let t = today();
|
||||
assert_eq!(t.len(), 10);
|
||||
assert!(t.starts_with("20"));
|
||||
let s = sidecar_text("x.png", "X", "test y");
|
||||
for f in ["asset:", "title:", "author:", "source:", "license:", "added:"] {
|
||||
assert!(s.lines().any(|l| l.starts_with(f) && l.len() > f.len() + 2), "{f}");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,11 +4,14 @@
|
||||
//! `FILMCRAFT_REQUIRE_ORACLES=1`, fail — when they are missing ([`require_ffmpeg!`] & co.).
|
||||
//! - [`fixtures`]: the shared fixture directory (`<workspace>/target/fixtures/<crate>`) and
|
||||
//! collision-free temporary names for concurrent generators.
|
||||
//! - [`golden`]: RGBA8 images, PNG I/O, image-difference metrics and golden-file checks
|
||||
//! (`FILMCRAFT_BLESS=1` regenerates references).
|
||||
//!
|
||||
//! ffmpeg/ffprobe are external test oracles and fixture generators only; they are never linked
|
||||
//! or shipped (AGENTS.md §2).
|
||||
|
||||
pub mod fixtures;
|
||||
pub mod golden;
|
||||
pub mod oracle;
|
||||
|
||||
pub use fixtures::{fixtures_dir, temp_path, workspace_root};
|
||||
|
||||
@@ -23,7 +23,7 @@ Design principles:
|
||||
L6 apps/filmcraft · apps/filmcraft-cli
|
||||
L5 ui-egui · automation
|
||||
L4 engine
|
||||
L3 render · gpu · export
|
||||
L3 render · gpu · export · golden (test-only)
|
||||
L2 edit · codecs · interchange
|
||||
L1 frame · media · project · audio-dsp
|
||||
L0 foundation: time · geom · color · bitstream · testkit (dev-dependency only)
|
||||
@@ -55,6 +55,7 @@ and `filmcraft-cli`.
|
||||
| `interchange` | L2 | EDL, FCP7 XML, FCPXML and OTIO import/export (no file I/O) |
|
||||
| `render` | L3 | sequence evaluation, CPU compositor, video effects, transitions, audio mix |
|
||||
| `gpu` | L3 | wgpu compositor (WGSL) |
|
||||
| `golden` | L3 | test-only: golden-image tests of the CPU renderer and GPU-vs-CPU parity; empty library, dev-dependencies only |
|
||||
| `export` | L3 | render → encode → mux pipeline, progress/cancel |
|
||||
| `engine` | L4 | `Session`, command registry, undo history, media pool, jobs, interchange glue |
|
||||
| `ui-egui` | L5 | the egui frontend: docking, panels, timeline, monitors, playback, control-channel handlers |
|
||||
|
||||
+20
-3
@@ -82,9 +82,26 @@ Each codec README has the full fixture matrix and the measured results.
|
||||
difference of ≤ 6 and a mean of < 1.5 (8-bit levels). It skips when no GPU adapter is available.
|
||||
- Effects implemented only on the CPU need no GPU test: the plan pre-renders those layers on the
|
||||
CPU.
|
||||
- Golden-image files (rendered PNGs compared against committed references) are not used yet. If you
|
||||
add them, generate the references from FilmCraft itself, give each one an attribution sidecar,
|
||||
and document the tolerance (PSNR in dB, or max/percentile error).
|
||||
- **Golden images** (`crates/golden`, a test-only crate): `tests/golden.rs` builds small procedural
|
||||
projects (demo-generator footage, bars, no media files) at 320×180 and renders one frame of each
|
||||
through the CPU compositor: Motion transform + opacity, four blend modes (Multiply, Screen,
|
||||
Overlay, Difference), Gaussian Blur, Lumetri basic correction, Crop, Cross Dissolve at 50 %, Dip
|
||||
to Black at 25 %, Wipe at 50 %, and Timecode / Clip Name burn-in text. Each frame is compared
|
||||
with `crates/golden/goldens/<scene>.png` by `filmcraft_testkit::golden`:
|
||||
**PSNR ≥ 45 dB, max abs ≤ 12, 99th-percentile per-pixel max channel difference ≤ 2** (8-bit
|
||||
sRGB levels; `Tolerance::RENDER`). On failure the actual frame and a ×8 difference image go to
|
||||
`<workspace>/target/golden-failures/`. `scenes_are_distinct` guards against blank or duplicate
|
||||
scenes.
|
||||
- **Blessing:** `FILMCRAFT_BLESS=1 cargo test -p filmcraft-golden` rewrites the references (and
|
||||
writes a `.attribution` sidecar for any new one; it prints the `ATTRIBUTION.md` row to add).
|
||||
References are original work rendered by FilmCraft, must stay under 50 KB (enforced on bless),
|
||||
and every one needs its sidecar and index row (`cargo xtask assets`). Look at a re-blessed PNG
|
||||
before committing it.
|
||||
- The same scenes run through `plan_frame` + `GpuCompositor` when a GPU adapter exists
|
||||
(`gpu_matches_cpu_on_golden_scenes`, same p99 ≤ 6 / mean < 1.5 criterion; skipped otherwise).
|
||||
This found a real bug: the GPU upload cache was keyed by pixel-buffer address without keeping
|
||||
the buffer alive, so a new frame allocated at a freed frame's address was drawn with the stale
|
||||
texture (fixed; regression test `upload_cache_keeps_buffers_alive`).
|
||||
|
||||
## 4. UI: control channel and screenshots
|
||||
|
||||
|
||||
@@ -46,6 +46,7 @@ const LAYERS: &[(&str, u8)] = &[
|
||||
("interchange", 2),
|
||||
("render", 3),
|
||||
("gpu", 3),
|
||||
("golden", 3),
|
||||
("scopes", 3),
|
||||
("playback", 3),
|
||||
("export", 3),
|
||||
|
||||
Reference in new issue
Block a user