IKANDY Scene Contract
Status: v2 — drafted Phase 2.3 (2026-05-21), revised 2026-05-26 to fold in
the WebGPU compute category as a first-class, fully-built category. The v1
draft listed webgpu as "reserved but not implemented"; that is now stale — the
ComputeRuntime (dispatch orchestrator, audio UBO, storage-buffer pool, pipeline
manager) is live and 8 compute scenes ship on it. Documents the architectural
contract the curated pipeline scenes implement collectively.
Relationship to scene-schema.md: that doc is the data spec — what
fields a scene may declare. This doc is the runtime contract — what a
scene must do behind those fields, how it relates to the pipeline runtime,
who owns which GPU resources, and what each category's render pattern
looks like in code. Read both for a complete picture; a scene that
satisfies the schema but violates the contract still won't ship.
Section 0 — Foundational principle
IKANDY is a music visualizer. Every scene must be demonstrably audio-reactive.
Time-only animations, static visuals, and ambient backdrops are out of scope by definition — not "discouraged" or "low priority," but not part of the product category. A beautiful scene that doesn't visibly respond to audio belongs in ShaderToy, not IKANDY.
This principle drives downstream decisions:
- Every scene declares
audio: {…}consuming at least one audio signal (bass,mid,treble,intensity,fft,timeDomain, or an event likebeat). - Every scene declares
hints.audioReactive: true(or the harness skips its reactivity test, which is itself a sign the scene shouldn't ship). - Reactivity is enforced, not requested — see Section 9 (Audio-reactivity enforcement) for the harness check that gates new scenes.
- The library is curated against this rule. The 18 v1 scenes all pass; new scenes that fail are rejected.
The remaining sections describe the engineering contract that lets a scene satisfy this rule without breaking the runtime.
Section 1 — What "compliant" means
A scene is contract-compliant when it:
- Satisfies Section 0 — visibly responds to audio under the harness reactivity check
- Declares the required identity fields (
id,name,credit,category,schema) - Implements an
init(runtime) → SceneInstancematching its category's pattern - Returns a
SceneInstancewith the required lifecycle methods (update,resize,dispose) - Renders into
runtime.hdrTarget(or the appropriate bridge for its category) - Respects the declared color-space contract (
selfTonemappedflag truthfully describes its output) - Cleans up the GPU resources it created on
dispose() - Does not retain references to the runtime, GL context, or HDR target after
dispose()returns - Does not touch any DOM element outside its own offscreen canvas (canvas2d) or render target (three)
The pipeline runtime trusts items 2–9. Item 1 is enforced by the harness, not the runtime. Violating 2–9 silently corrupts state across scene transitions or leaks GPU resources; violating 1 means the scene isn't an IKANDY scene at all and shouldn't have been merged.
Section 2 — Lifecycle (all categories)
Compute (
category: 'webgpu') scenes use aframe(ctx)hook, notupdate(audio, dt)— see §5. The flow below is the pipeline categories (fragment / canvas2d / three).
register(scene) ← module load
↓
setActive(id) ← user picks scene (or harness drives)
→ instance = scene.init(runtimeApi) // sync OR async; runtime awaits
↓
loop:
instance.update(audio, dt) // ~once per frame
↓
resize(w, h) ← viewport change
→ instance.resize(w, h)
↓
clearActive() / setActive(otherId)
→ instance.dispose()
Once dispose() returns, the runtime treats the scene as fully gone. It may then activate the next scene and reuse the GL context for new programs/textures/FBOs without any state cleanup. If your scene leaks (uncleared gl.useProgram, dangling textures bound to active units, modified blend/depth state without restore), the next scene inherits the corruption.
The Three.js runtime is the one exception — it calls _renderer.resetState() after every update() to fence Three.js's internal state changes from leaking. Fragment and canvas2d scenes don't get this safety net; they must leave GL state clean themselves. In practice the adapters handle this — see "Per-category patterns" below.
Section 3 — The runtime object passed to init()
Source: src/render/pipeline-runtime.js near L879.
runtime = {
gl, // WebGL2RenderingContext, shared with every scene
canvas, // the pipeline canvas (#pipeline-canvas); category:'three'
// scenes wrap this with THREE.WebGLRenderer
hdrTarget, // { fbo, tex, width, height } — the HDR FBO scenes render INTO
width, height, // backing-pixel dimensions (CSS px × dpr)
dpr, // devicePixelRatio used to size the FBO
audioGetter: () => _audio, // CURRENT audio frame; canvas2d scenes typically
// don't call this — audio arrives via update(audio, dt)
helpers, // { VS_FULLSCREEN, compileShader, makeProgram, drawFullscreen,
// bindTargetForDraw } — minimal GL utilities scenes can use
bindHDR(), // shortcut for bindTargetForDraw(gl, hdrTarget)
controls, // schema:2 — current values keyed by ControlSpec.target
// (populated from defaults + localStorage before init runs)
};
gl is shared. Every scene uses the same WebGL2 context. There is no second context for a scene to opt into. (Pre-Phase-1.4 each scene owned its own context; the Chromium ~16-context cap was the architectural ceiling that motivated the pipeline.)
hdrTarget is a RGBA16F half-float FBO in linear color space. Scenes write linear-light values; the pipeline tonemaps + applies sRGB at the output pass (passes 3 + 9 of the chain). Exception: selfTonemapped: true scenes write sRGB-encoded [0,1] directly into the same FBO and the pipeline's tonemap pass is bypassed for them. See "Color space" below.
helpers.makeProgram(gl, fragmentSource [, vertexSource]) compiles a fragment shader against the shared fullscreen vertex shader (VS_FULLSCREEN). Scenes that need a custom vertex shader pass it as the third arg; the GLSL adapter handles this transparently for the 14 fragment scenes.
Section 4 — The SceneInstance returned from init()
Every category returns the same shape:
interface SceneInstance {
update(audio: AudioFrame, dt: number): void;
resize(w: number, h: number): void;
dispose(): void;
}
audio shape is defined in scene-schema.md § AudioFrame. dt is wall-clock seconds since the previous tick; scenes that want time scaled by the user's speed slider should use audio.scaledDt instead (already = dt × audio.speed).
update() must:
- Render to the HDR FBO (directly for fragment, via the bridge for canvas2d, via
renderToSceneTarget+blitToHdrfor three) - Not modify pipeline-owned GL state without restoring it (current FBO binding, viewport, blend/depth enables)
- Be fast — target ~16ms total budget shared with the post-FX chain
resize(w, h) may be a no-op if the scene has nothing to resize. Most fragment scenes return resize() {}; canvas2d scenes typically call bridge.resize(w, h); three scenes update their render target.
dispose() must free everything init() allocated. This is the only hard rule that the runtime cannot enforce for you. Audit a scene's init() for each gl.create*()/new THREE.*()/document.createElement('canvas') and ensure each has a matching cleanup in dispose().
Section 5 — Per-category patterns
The curated scenes fall into four categories — fragment, canvas2d, three, and webgpu (WebGPU compute). Each has a different runtime relationship; new scenes must implement the same pattern as the existing scenes in their category.
One structural distinction up front: the first three categories render into runtime.hdrTarget (directly for fragment, via a bridge for canvas2d/three) on the shared WebGL2 context. The fourth — webgpu — runs on a separate GPUDevice and renders to its own #wgpu-canvas; the pipeline pulls that canvas in as a source (setWgpuSourceCanvas()) rather than receiving FBO writes. Read its subsection for why the contract there is shaped differently.
category: 'fragment' — GLSL (13 scenes — 12 fullscreen-quad + ferrofluid, a 3D mesh; see §11)
Scenes: aurora, eeu, fic, hol, hypnotise, ikandy-eye, jaf, lmi, prism, singularity, sw, voyager, ferrofluid, plus the four canvas2d adjacent scenes that are GLSL-rendered (see canvas2d section).
Pattern: never write a fragment scene by hand. Use the GLSL adapter.
import { createGlslScene } from './glsl-adapter.js';
export const myScene = createGlslScene({
id: 'my-scene',
name: 'My Scene',
credit: 'Author Name',
category: 'fragment',
schema: 2,
hints: { selfTonemapped: true, audioReactive: true, /* … */ },
audio: { bass: true, mid: true, treble: true, intensity: true, time: true },
controls: [ /* ControlSpec[] */ ],
fragmentSource: `#version 300 es
precision highp float;
uniform float u_time;
uniform float u_bass;
// …
out vec4 outColor;
void main() { outColor = vec4(/* … */); }
`,
});
The adapter (src/render/scenes/glsl-adapter.js) handles:
- Compiling the shader against the shared fullscreen vertex shader
- Looking up every standard uniform (
u_time,u_bass,u_intensity, etc.) and silently skipping those the shader doesn't declare - Binding declared schema:2 control uniforms (
u_density, etc.) each frame - Maintaining per-scene smoothed bass/mid/treble state (
u_bassSmoothetc., instant attack with band-specific exponential release) - Loading + binding declared image textures (Phase 2.11) at
u_<name>Texwith au_<name>Loadedflag for fallback rendering during async decode - Pinning time + mouse + audio to deterministic values when
window.__IKANDY_HARNESS__is set - Disposing the GL program + textures on
dispose()
Standard uniforms the adapter binds (declared = bound; absent = skipped):
| Uniform | Source | Range |
|---|---|---|
iTime / u_time |
scaledTime accumulator (+= dt × audio.speed) |
seconds |
iResolution / u_res |
vec2(hdrTarget.width, hdrTarget.height) |
backing px |
iMouse / u_mouse |
mouse position × dpr, Y-flipped to GL bottom-origin | px |
u_bass / u_mid / u_treble |
audio.bass/mid/treble |
[0, 1] |
u_intensity / u_sensitivity |
audio.intensity |
[0, ~2] |
u_volume |
volume slider value | [0, 2] |
u_bassSmooth / u_midSmooth / u_trebSmooth |
smoothed bands | [0, 1] |
u_beat / u_transient / u_onset / u_energy / u_tempo |
audio events (Phase 2.7) | [0, 1] |
Two naming conventions are accepted in parallel: iTime/iResolution/iMouse (ShaderToy / mrange CC0 ports) and u_time/u_res/u_mouse (IKANDY originals). Declare either; both routes write the same value.
Don't write a fragment scene that bypasses the adapter. A scene that compiles its own shader, binds uniforms directly, and writes to the FBO outside the adapter loses harness time-pinning, the smoothed-bands state, persistent control plumbing, and texture-load fallback. Future contract changes (new standard uniforms, new validation, new compile-time checks) will land in the adapter and miss bypassed scenes.
category: 'canvas2d' — Canvas 2D drawn into HDR FBO (4 scenes)
Scenes: fire, pawrticles, ripple, waveray.
Pattern: use the canvas2d bridge.
import { createCanvas2dBridge } from '../runtimes/canvas2d-runtime.js';
export const myScene = {
id: 'my-scene', name: 'My Scene', credit: 'Author',
category: 'canvas2d', schema: 2,
hints: { selfTonemapped: true, audioReactive: true },
audio: { /* … */ },
controls: [ /* … */ ],
init(runtime) {
const { gl, helpers, hdrTarget } = runtime;
const bridge = createCanvas2dBridge(gl, helpers, hdrTarget.width, hdrTarget.height);
const ctx = bridge.ctx;
return {
update(audio, dt) {
// Draw with ctx (CanvasRenderingContext2D)
ctx.fillStyle = 'rgba(0,0,0,0.5)';
ctx.fillRect(0, 0, bridge.canvas.width, bridge.canvas.height);
// … scene-specific drawing …
bridge.blitToHdr(runtime.hdrTarget); // upload + sample into HDR FBO
},
resize(w, h) { bridge.resize(w, h); },
dispose() { bridge.dispose(); },
};
},
};
The bridge (src/render/runtimes/canvas2d-runtime.js) handles:
- Allocating a per-scene offscreen
<canvas>(not in DOM) sized to the HDR FBO - Allocating a GPU texture matching the canvas
- Each
blitToHdr()uploads the current canvas contents to the GPU texture, then draws a fullscreen quad sampling it into the HDR FBO UNPACK_FLIP_Y_WEBGL = trueon upload so the canvas's top-left maps to the FBO's top-left- One shared blit program across all canvas2d scenes (compiled once, reused)
- Reallocating canvas + texture on
resize() - Releasing the GPU texture on
dispose()
Per-scene canvases are intentional (vs. one shared canvas). Two canvas2d scenes might want different blend modes / composite operations / global alpha; isolated canvases prevent cross-contamination.
Canvas2d scenes are always selfTonemapped: true. Canvas 2D outputs sRGB pixels by design; the bridge uploads them as-is and the pipeline's tonemap pass is bypassed. Writing linear values via a canvas2d scene is not supported.
audio.scaledDt vs dt: canvas2d scenes accumulate time manually for animation (no equivalent of the adapter's _scaledTime). Use audio.scaledDt ?? dt for animation that should respond to the speed slider; use raw dt for physics that should stay wall-clock.
category: 'three' — Three.js scene + camera (1 scene)
Scenes: trails.
Pattern: use the three runtime.
import * as THREE from 'three';
import {
ensureRenderer, ensureSceneTarget,
renderToSceneTarget, blitToHdr,
} from '../runtimes/three-runtime.js';
export const myScene = {
id: 'my-scene', name: 'My Scene', credit: 'Author',
category: 'three', schema: 2,
hints: { selfTonemapped: false, /* see Color Space */ },
audio: { /* … */ },
controls: [ /* … */ ],
init(runtime) {
const { gl, canvas, helpers, hdrTarget } = runtime;
const renderer = ensureRenderer(gl, canvas, helpers);
const target = ensureSceneTarget(hdrTarget.width, hdrTarget.height);
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(50, hdrTarget.width / hdrTarget.height, 0.1, 1000);
// sRGB-authored colors MUST be converted to linear at construction —
// see "Color input contract" below for the full rationale.
const mat = new THREE.MeshBasicMaterial({
color: new THREE.Color('#ff5900').convertSRGBToLinear(),
});
// … build scene graph using `mat`, other geometries, etc. …
return {
update(audio, dt) {
// animate scene graph using audio + dt
renderToSceneTarget(scene, camera); // renders into the shared HalfFloat target
blitToHdr(runtime.hdrTarget, helpers); // copies that target into the pipeline FBO
},
resize(w, h) {
ensureSceneTarget(w, h);
camera.aspect = w / h;
camera.updateProjectionMatrix();
},
dispose() {
// Free YOUR geometries / materials / textures here. Do NOT call
// renderer.dispose() — the renderer is shared across three scenes
// and lives for the lifetime of the pipeline runtime.
scene.traverse(obj => {
if (obj.geometry) obj.geometry.dispose();
if (obj.material) {
const mats = Array.isArray(obj.material) ? obj.material : [obj.material];
for (const m of mats) m.dispose();
}
});
},
};
},
};
The three runtime (src/render/runtimes/three-runtime.js) handles:
- Constructing one shared
THREE.WebGLRendererwrapping the pipeline'sgl+canvas(so Three.js renders into the SAME WebGL2 context — no second canvas, no DOM-level compositing) - Renderer configured with
toneMapping: NoToneMapping+outputColorSpace: LinearSRGBColorSpaceso the pipeline's ACES tonemap is the only one - One shared HalfFloat render target every three scene draws into
- After
renderer.render(), callingrenderer.resetState()so Three.js's internal GL state mutations don't leak into the post-FX chain - A blit program that copies the render target's texture handle into the pipeline's HDR FBO
- Full dispose only happens when the pipeline itself disposes — scene
dispose()only frees its own geometries/materials
The renderer is shared. Do not call renderer.dispose() from a scene. That would tear down the pipeline's GL context.
Color space for three scenes is special. With renderer toneMapping: NoToneMapping + outputColorSpace: LinearSRGBColorSpace, the scene writes linear values to the HalfFloat target. The blit copies them into the HDR FBO unchanged, where the pipeline's ACES pass tonemaps + sRGBs them. So three scenes are selfTonemapped: false. Trails sets this; future three scenes should as well. (Setting selfTonemapped: true on a three scene means the pipeline will skip its tonemap pass, leaving linear values in an sRGB output — wrong, and visually obvious as washed-out shadows.)
Color input contract — read this before writing a three scene
DO NOT pass sRGB-encoded hex colors directly into THREE.Color and use them as material inputs. Under our renderer's outputColorSpace: LinearSRGBColorSpace configuration, Three.js does NOT auto-convert THREE.Color values from sRGB to linear during shader uniform binding (it would under the default SRGBColorSpace). An unconverted sRGB hex triplet flows into the shader as if it were already linear — quietly wrong, and downstream tonemap + bloom math compounds the error.
This is the specific bug that almost shipped with trails. The original port (Phase 2.9) carried 5 hex color constants ('#004c94', '#2e89ff', '#003994', '#004bad', '#ff5900') authored as sRGB by the original CodePen artist. They were passed straight to new THREE.Color(hex) and bound as uColor uniforms. With LinearSRGBColorSpace the green channel of #ff5900 rode into the shader at 3.5× its correct linear value (0.349 instead of 0.099); the compensating overbright was buried in CONFIG.brightness = 4.0131 and inflated hints.bloomHint calibration. The math was wrong; the output happened to look approximately right because of accidental tuning. Phase 3 (2026-05-22) corrected this — trails now reads better than the empirically-tuned version because the natural character of the scene comes through once the underlying math is correct.
The canonical pattern — copy this for every new three scene
// sRGB-authored hex (artist-readable, e.g. from a CodePen / Figma)
const COLOR_HEX = '#ff5900';
// At material construction — decode sRGB to linear at uniform-bind time:
const uColor = new THREE.Color(COLOR_HEX).convertSRGBToLinear();
// Inside the material:
material.uniforms.uColor = { value: uColor };
// Equivalent (Three.js r152+ idiomatic form):
const uColorAlt = new THREE.Color().setHex(0xff5900, THREE.SRGBColorSpace);
// If you author colors as linear values directly (rare — designers think in sRGB):
const uColorLinear = new THREE.Color(1.0, 0.099, 0.0); // no conversion needed
Use the same pattern for material.color on MeshBasicMaterial / MeshStandardMaterial / etc:
const mat = new THREE.MeshBasicMaterial({
color: new THREE.Color('#ff5900').convertSRGBToLinear(),
});
What this does NOT apply to
- Pure black or pure white (
#000000/#ffffff) are identical in sRGB and linear — no conversion needed (but converting them is also harmless) - Linear-authored values (decimal triplets like
new THREE.Color(1.0, 0.099, 0.0)) already ARE linear — don't double-convert - Texture maps with
colorSpace: THREE.SRGBColorSpace— Three.js handles colorspace for textures separately via thecolorSpaceproperty; that's already correct in the sRGB → linear direction. Different topic fromTHREE.Color.
Why this matters now (and not in classic Three.js workflows)
In a "default" Three.js setup (outputColorSpace: SRGBColorSpace, no custom render targets), Three.js handles sRGB → linear → light-math → sRGB conversion transparently in the standard shader chunks. Authors pass sRGB hex, get sRGB-correct output, never think about it.
We're not in that setup. We use LinearSRGBColorSpace because the renderer writes to a HalfFloat target the pipeline tonemaps later — the renderer must NOT pre-encode to sRGB (that would double-encode after the pipeline's ACES + linear-to-sRGB). The price is that the automatic color-space handling Three.js does for material colors is also off — every sRGB color the scene declares needs explicit .convertSRGBToLinear() to land correctly in the linear render path.
See src/render/scenes/trails.js:281 for the working pattern (new THREE.Color(CONFIG[\color${colorIdx}`]).convertSRGBToLinear()`). New three scenes should match this idiom.
category: 'webgpu' — WebGPU compute (8 scenes)
Scenes: compute-chromosphere, compute-fluid, compute-gravity, compute-membrane,
compute-particles, compute-spectra, compute-tesla-orb (Plasma Globe),
compute-tesseract. (compute-grid and compute-smoke are dev/test entry points
— IKANDY_COMPUTE_GRID() / IKANDY_COMPUTE_SMOKE() — not user-facing scenes.)
This is no longer speculative. WebGPU compute is a first-class category with its own runtime stack:
ComputeRuntime(src/render/runtimes/compute-runtime.js) — the category's peer toThreeRuntime. Owns a single sharedGPUDevice(acquired once, lazily), the#wgpu-canvas+ itsGPUCanvasContext, the per-scene activation loop, and the per-framedt/time/Speed handling. Public API mirrorsThreeRuntime:register(name, factory),setActive(name),clearActive(),resize(w,h),setAudio(...),setSpectrum(...),setSpeed(mul),getAudioDebug().DispatchOrchestrator(runtimes/dispatch-orchestrator.js) — drives the declarative path: per frame it opens a compute encoder, runs each declared compute pass inorder, ping-pong-swaps the storage buffers, then opens a render encoder that draws the freshly-swapped buffers. Split encoders are deliberate (compute results must be visible to the render pass).AudioUBO(runtimes/audio-ubo.js) — one GPU uniform buffer written once per frame, bound at@group(0) @binding(0)of every compute and render pass. Packstime, bass, mid, treble, intensity, dt, kick, energy, lo, midBand, hiplus the 32-bin FFT. This is how WGSL shaders read audio (see Section 7).StorageBufferPool(runtimes/storage-buffer-pool.js) — per-scene; a fresh pool is created onsetActiveand disposed onclearActive, so scene buffers are freed cleanly. Holds the ping-pong slots ('pos'/'pos:out', etc.).ComputePipelineManager(runtimes/compute-pipeline-manager.js) — per-device; caches compiled WGSL modules + pipelines across scene switches.
Two authoring paths. A scene factory factory(runtimeApi) → instance returns
either a declarative instance (no frame(), but declares
storageBuffers + computePasses + renderPass — the orchestrator runs the
compute-pass→swap→render-pass chain for you) or an imperative instance
(implements frame(ctx) and records/submits its own encoders by hand). Prefer
declarative; reach for imperative only when the pass graph is irregular
(compute-tesla-orb and compute-fluid are imperative; the rest are declarative).
Lifecycle (parallels every other category's init → SceneInstance outer shape):
register(name, factory) ← module load
↓
setActive(name) ← user picks scene (or harness)
→ bufferPool = new StorageBufferPool(device)
→ instance = await factory(runtimeApi) // sync OR async; runtime awaits
runtimeApi = { device, queue, format, context, canvas, width, height,
audio, bufferPool, audioUBO, pipelines }
↓
loop (per frame):
instance.update?(audio, dt, frameIndex) // optional CPU-side per-frame logic
instance.frame(ctx) // REQUIRED — record GPU work
ctx = { device, queue, view, format, width, height,
time, dt, frameIndex, audio, bufferPool, audioUBO, pipelines }
↓
resize(w, h) → instance.resize?(w, h)
↓
clearActive() / setActive(other)
→ instance.dispose?() // free buffers/textures/pipelines;
// do NOT destroy the device
→ bufferPool.dispose()
audio on runtimeApi/ctx is the same live object each frame — read it inside
frame()/update(); never cache values across frames. ctx.audio.kick is the
runtime-derived hybrid kick (Section 7).
Declarative pattern — copy compute-particles.js:
export function createMyComputeScene(/* api */) {
return {
id: 'my-compute', name: 'My Compute', credit: 'Author',
category: 'webgpu', schema: 2,
hints: { selfTonemapped: true, audioReactive: true },
audio: { bass: true, mid: true, treble: true, time: true },
// Ping-pong storage buffers; the pool allocates 'name' (front) + 'name:out' (back).
storageBuffers: [
{ name: 'pos', count: 'particleCount', stride: 16, usage: ['storage'],
pingPong: true, init: 'random-cube' },
{ name: 'vel', count: 'particleCount', stride: 16, usage: ['storage'],
pingPong: true, init: 'zero' },
],
// Compute pass(es): read front ('pos'/'vel'), write back ('pos:out'/'vel:out').
computePasses: [
{ name: 'integrate', order: 0, workgroupSize: 64,
dispatch: { x: 'particleCount' },
storageBuffers: ['pos', 'vel', 'pos:out', 'vel:out'],
shader: PHYS_WGSL },
],
// Render pass: reads buffers POST-swap (fresh), draws into #wgpu-canvas.
renderPass: {
shader: RENDER_WGSL, topology: 'triangle-list',
vertexCount: 6, instanceCount: 'particleCount',
storageBuffers: ['pos', 'vel'],
blend: 'additive', clear: [0, 0, 0, 1],
},
};
}
Imperative pattern — copy compute-tesla-orb.js when the graph is irregular:
init(runtimeApi) {
// allocate buffers/pipelines here (or let the scene own them directly)
return {
frame(ctx) {
const { device, queue, view, audio, audioUBO } = ctx;
// CPU-side per-frame work: read audio.kick / audio.bass etc., write scene UBO
// ... beat-onset detection, physics impulses, control packing ...
const enc = device.createCommandEncoder();
const cpass = enc.beginComputePass(); /* dispatch */ cpass.end();
const rpass = enc.beginRenderPass({ colorAttachments: [{ view, /* ... */ }] });
/* draw */ rpass.end();
queue.submit([enc.finish()]); // imperative scenes own submission
},
resize(w, h) { /* reallocate size-dependent targets */ },
dispose() { /* destroy your buffers/textures/pipelines — NOT the device */ },
};
}
How the output reaches the screen. Compute scenes do not write into
runtime.hdrTarget. They render to #wgpu-canvas (a screen-blended overlay), and
PipelineRuntime.setWgpuSourceCanvas(canvas, selfTonemapped, renderFn, edgeTaper)
registers that canvas as the pipeline's source — the post-FX chain (bloom, CA,
LUT, grain, vignette…) then runs over it each tick, just as it does for the FBO
categories. Two args matter for compute authors:
selfTonemapped— same contract as every other category (Section 6). All current compute scenes sethints.selfTonemapped: true(they tonemap internally / are LDR), so the pipeline's tonemap pass is bypassed.edgeTaper— the 4th arg tosetWgpuSourceCanvas, settruefor compute scenes. Compute scenes render bright content to the literal frame edge; the CA pass's radial R/B shift fringes that hard edge.edgeTaper: truetells the CA pass to fade its scale → 0 across the outer border band. GLSL/WGSL fragment scenes leave it false (pipeline-runtime.js:990/:1518).
Resource ownership. dispose() must free every GPUBuffer / GPUTexture /
pipeline the scene created, but must never call device.destroy() — the
device is shared and owned by ComputeRuntime for the lifetime of the runtime
(same rule as "don't call renderer.dispose()" for three scenes). The per-scene
StorageBufferPool is disposed by the runtime on clearActive.
Color hazard. Compute scenes commonly composite with blend: 'additive'.
Additive blending into a non-tonemapped 8-bit target silently clamps to white —
see Section 6's v8 additive-blend hazard and the per-channel diagnostic before
shipping any layered-additive compute scene.
Reference scenes: compute-particles.js (canonical declarative scene), compute-tesla-orb.js (canonical imperative scene — see Section 11 for why it's the heavily-iterated reference point).
category: 'milkdrop' — not a pipeline category
Butterchurn owns its own preset lifecycle, render loop, post-FX, and DOM canvas (#viz + #mdpp-canvas). It runs as a peer to the pipeline, not under it. See CLAUDE.md → Scene engines for the full architecture.
Section 6 — Color space
Two output modes, declared by hints.selfTonemapped.
selfTonemapped: true (17 of 18 scenes)
Scene writes sRGB-encoded [0, 1] pixels into the HDR FBO. The pipeline's tonemap pass (position 3) is bypassed; output pass (position 9) bypasses linearToSRGB(). Effectively: what the scene draws is what reaches the screen, after intermediate passes (bloom, LUT, grain, CA, scanlines, vignette).
Use this when the scene was authored to look right at full brightness in sRGB — i.e., it has its own tonemap baked in, or it's a low-dynamic-range design (canvas2d).
All 13 fragment scenes (ferrofluid included — the 3D-mesh exception, §11) and all 4 canvas2d scenes set this true (13 + 4 = 17). The fragment scenes were ported from ShaderToy / CodePen sources that author for sRGB display; canvas2d output is inherently sRGB.
selfTonemapped: false (or absent) (1 of 18 scenes — trails)
Scene writes linear-light HDR values into the FBO. Bright pixels are allowed above 1.0 (bloom uses this headroom). The pipeline's ACES tonemap pass curves them down to display range, then the output pass applies sRGB encoding.
Use this for new physically-based scenes that want the pipeline's bloom + tonemap to handle their highlights correctly.
Why bloom looks different across the two modes: the bloom pass runs before tonemap (position 0). It thresholds at 1.0 by default. For selfTonemapped: true scenes, bloom is thresholding already-tonemapped sRGB values — so most pixels are below 1.0 and only the brightest get bloomed. For selfTonemapped: false scenes, bloom is thresholding linear values — so HDR highlights well above 1.0 get bloomed naturally. Per-scene bloomHint.threshold adjustments compensate. (Phase 8.1 documented these per-scene; Phase 2.2 spot-check confirmed the library reads coherent.)
The v8 additive-blend hazard (compute / layered-additive scenes)
8-bit additive blending into a non-tonemapped target silently clamps to white.
When a scene composites many bright contributions with blend: 'additive'
(one, one, add) into an 8-bit (u8) render target that the pipeline does not
tonemap, overlapping hues sum past 255 in each channel and clamp to
(255, 255, 255). The result looks lit, animated, and audio-reactive — but
every overlapping element has collapsed into a white blob, destroying all hue
information. This is the v8 hazard (named for the Tesla Orb v8 regression:
the shockwave's intended rainbow burst read as a featureless white ring).
The failure is insidious because it passes every coarse check: pixels are non-zero, the frame changes with audio, brightness tracks the beat. Only the color is wrong. Mitigations live in the scene's WGSL, not the runtime:
- Keep additive hues saturated, not white — e.g.
hsv2rgb(vec3(hue, 0.95*falloff, 1.0))rather than summing towardvec3(1,1,1)(compute-tesla-orb.js:487–499). - Use a thin, sharp bright front (pow falloff) so fewer hues sum additively at any pixel — narrows a wide white band into a narrow rainbow arc (
pow(1.0 - wt, 2.5), compute-tesla-orb.js:487). - Bound total energy with distance + age falloff so the additive sum can't run away.
Recommended verification — the per-color-channel diagnostic. A pixel-count or
mean-diff smoke test cannot catch this (a white blob has plenty of non-zero,
audio-varying pixels). Use the pattern in src/test/compute-tesla-orb-diag.html:
render the scene, force the worst-case strike frame (max bolts + kick), read the
frame back from the GPU, and classify every pixel by per-channel ratios —
count "bright", "white" (all channels > 200), and the scene's signature hue (e.g.
magenta = R & B high, G much lower). A regression of this class shows up as the
signature-hue pixel count collapsing toward zero while "white" climbs. Any compute
scene using layered additive composition should pass a per-channel check like this
before shipping (Section 10).
Section 7 — Standard uniforms / audio surface
The audio surface is documented in detail in scene-schema.md § AudioFrame and § Uniform contract. Quick reference:
- Bands (
audio.bass,audio.mid,audio.treble): pre-multiplied by the user's per-band gain sliders,[0, 1]typical range - Master scalars (
audio.intensity,audio.volume,audio.speed): user-facing slider values,[0, 2]range - Time (
audio.time,audio.scaledDt): monotonic seconds and per-frame Δt × speed - Bulk data (
audio.fft,audio.timeDomain):Uint8Arraysnapshots, may benulluntil first audio frame arrives - Smoothed bands (
audio.bassSmooth,audio.midSmooth,audio.trebSmooth): instant attack, exponential decay (0.82 / 0.85 / 0.78release factors) - Events (
audio.beat,audio.transient,audio.onset,audio.energy,audio.tempo): decaying[0, 1]signals from the audio-events detector (src/render/audio-events.js)
Kick + the audio debug surface (WebGPU compute path)
The compute path derives a beat signal in AudioUBO and exposes a debug probe via
ComputeRuntime.getAudioDebug() (console: IKANDY_COMPUTE_RUNTIME.getAudioDebug(),
delegates to AudioUBO.getDebug()). The fields it surfaces:
| Field | Meaning |
|---|---|
bass |
RAW detection signal (pre-user-slider, clamped [0,1]) — what the kick detector runs on |
bassVisual |
the visual bass value scenes actually render (post-slider) |
mid / treble |
visual band values |
mainKick |
Path 1 — transient (rectified bass rise above its slow EMA) OR beat-onset cadence during sustained-loud sections (≤145 BPM, ×0.6 weight) |
subKick |
Path 2 — sharp bass-derivative hits on their own latch (~600 BPM ceiling, capped 0.7) for EDM fills / drum-roll subdivisions |
kick |
combined — max(mainKick, subKick); the value exposed to scenes as audio.kick |
energy, lo, midBand, hi |
mean of all / low / mid / high thirds of the 32-bin FFT |
window |
min/max/peak of bass/mid/kick/energy since the previous getDebug() call (the right instrument for catching a brief kick spike a one-shot read misses) |
Dual-mode kick detection contract. kick is the max of two independent
detectors with separate refractory latches, so they can't lock each other out:
mainKick catches musical beats (clean transients + sustained-section cadence),
subKick catches fast subdivisions. Either path firing wins. The detector runs on
the raw band (bassRaw), never the visual band — a hot bass-gain slider
(e.g. 2.0) must not shift the detector's operating point or break beat timing.
What scenes should read. Read audio.kick for beats, and
audio.bass / audio.mid / audio.treble for band amplitude (these are the
visual, slider-scaled values). bassRaw / midRaw / trebleRaw are
detection-internal — fed to the kick detector, not for scenes to render. Don't
re-implement beat detection off the raw bands in a scene; consume the
runtime-derived audio.kick.
Fragment scenes consume all of these via standard uniform names (u_bass, u_time, u_beat, etc.). Canvas2d/three scenes read them from the audio object passed to update().
Declare what you read in the audio: field of the scene metadata. The runtime delivers every field regardless, but the declaration tells the harness what to synthesize during capture and tells the UI which gain sliders are load-bearing.
Section 8 — Resource ownership
The simplest way to stay compliant: every gl.create*() / new THREE.*() / document.createElement('canvas') in init() must have a matching cleanup in dispose().
| Resource | Allocated by | Disposed by |
|---|---|---|
WebGL context (gl) |
pipeline runtime | pipeline runtime |
| HDR FBO + texture | pipeline runtime | pipeline runtime |
| GLSL program (fragment scene) | glsl-adapter via helpers.makeProgram() |
glsl-adapter on dispose() |
| Image texture (fragment scene) | glsl-adapter when scene declares textures: {…} |
glsl-adapter on dispose() |
| Schema:2 control state | pipeline runtime | pipeline runtime |
| Offscreen canvas + GPU texture (canvas2d) | createCanvas2dBridge() |
bridge.dispose() |
THREE.WebGLRenderer |
three-runtime, lazily on first three scene activation | three-runtime on pipeline dispose |
Shared WebGLRenderTarget (three) |
three-runtime via ensureSceneTarget() |
three-runtime on pipeline dispose |
THREE.Scene / Camera / geometries / materials |
the scene's init() |
the scene's dispose() |
| Custom render targets (e.g. trails' floor blur) | the scene's init() |
the scene's dispose() |
setTimeout / setInterval / event listeners |
the scene's init() |
the scene's dispose() |
The ~16-context cap is no longer at risk under the pipeline (one shared context). But texture and buffer counts still matter — a scene that allocates a 10K-particle position texture in init() and forgets to dispose it accumulates ~1MB per scene activation. Over a day of scene cycling that adds up.
Section 9 — Audio-reactivity enforcement
Section 0 says every scene must be audio-reactive. This section says how that's checked — automatically, before merge, for every scene.
Existing mechanism: harness audio-reactive check
src/test/visual-harness.js already runs a silent-vs-loud pixel-diff per scene:
- Capture two frames at
t=1.0with identical mouse + time, differing only in audio input: silent (bass=mid=treble=intensity=0) vs. loud (bass=1.0, mid=0.5, treble=0.5, intensity=0.67) - Compute mean per-pixel diff between the two frames
- Pass requires
diff ≥ AUDIO_DIFF_MIN(default1.5) - Per-scene override via
hints.audioReactive: falseskips the check; this should be considered a code smell for any scene in the curated library
This is the enforcement mechanism. A scene that ships without satisfying it is a bug — either the scene isn't actually reactive (rejected from the library) or the test is wrong (fix the test). Both are blockers.
Look at any recent npm run test:visual log: each scene's report includes a line
like v audio-reactive: diff(silent,audio)=49.38 (passing scenes show v with
the actual diff magnitude). The check has been live since Phase 2.12.
Wire-band sweeps (additional reactivity coverage)
The harness also runs per-band sweeps (wire-bass, wire-mid, wire-treble,
wire-intensity, wire-volume) that vary each audio band min→max with the
others held constant. A scene that responds only to bass but declares
audio: { mid: true } will fail the wire-mid check even if it passes the
overall audio-reactive check — catches mis-declared audio surfaces.
Pass requires per-band pixel-diff ≥ WIRE_AUDIO_MIN (default 1.5).
Visible-effect smoke test (compute scenes — complement to pixel-diff)
Pixel-diff alone is insufficient for compute scenes. A flat white blob passes the silent-vs-loud diff (and the wire sweeps) as easily as a correct render — it has plenty of non-zero, audio-varying pixels while being visually broken (the v8 hazard, Section 6). The diff confirms something changed, not that the right thing changed.
The complement is a visible-effect smoke test that drives real control levers
through the live ComputeRuntime and asserts the rendered frame changes in the
expected way — see src/test/compute-audio-controls-page.html. It runs each
lever and checks a magnitude:
- Band magnitude lever — render with low bands vs high bands; the two frames must differ by mean-absolute-difference above a threshold (band-reactive scenes show a large frame delta between quiet and loud).
- Speed lever — render at slow vs fast Speed; the evolution rate must change
(motion ratio above a floor), proving the scene actually consumes
dt/time. - Smoke floor — a minimum non-zero pixel count, so a black/dead frame fails outright.
It runs across both authoring paths (imperative Tesla Orb + declarative Particles
/ Gravity) so the contract — not just "did anything render" — is what's verified.
Treat this as the compute-scene analogue of the fragment-scene audio-reactive
check: pixel-diff proves reactivity exists; the visible-effect test proves the
output is the intended visual rather than a degenerate blob. Pair it with the
per-channel diagnostic (Section 6) for any layered-additive scene.
v1.x extensions to the reactivity gate
The existing harness covers the curated 18 scenes well. v1.x extensibility work needs richer enforcement:
test:capture:reactivityfor import path — when users import their own GLSL via the schema:2 template, the same silent-vs-loud diff must run before the scene enters their library. Failing scenes get a "this scene doesn't respond to audio — IKANDY only accepts audio-reactive visuals" rejection in the import UI. Estimated 1-2 hours wrapping the existing harness check.- Reactivity gate on AI-generated scenes — the AI generation path runs the same check against its output. A generated scene that fails goes back to the generator with the failure reason (e.g., "diff was 0.4, threshold 1.5; ensure the shader actually samples u_bass / u_intensity"). Foundation for the v1.x AI work that's currently undecided.
test:capture:milkdropreactivity scoring — same idea applied to the ~717 MilkDrop presets. Already on the v1.x backlog (CLAUDE.md → MilkDrop preset reactivity scoring). Outputs a CSV ranking; presets failing the threshold are hidden from default rotation or surfaced as an "ambient" category.- Stochastic-scene reactivity verification — the simple pixel-diff is defeated by per-frame randomness. See "Stochastic scenes" under Section 11.
All four extensions are documented in the v1.x backlog. None block the v1 launch — the curated library is fully verified by the existing harness check.
Section 10 — Validation checklist for new scenes
Before merging a new scene, confirm:
- Section 0 — audio-reactive. Scene visibly responds to audio. The
harness
audio-reactivecheck produces adiff(silent,audio) ≥ 1.5result for it. If not, the scene isn't an IKANDY scene — reject. - Wire-band coverage. Every audio band the scene reads passes its
per-band sweep check (
wire-bass,wire-mid,wire-treble,wire-intensity,wire-volume— whichever are declared inaudio: {…}). - Metadata:
id,name,credit,category,schema: 2 - Hints:
selfTonemappedset truthfully for the category (true for fragment/canvas2d and current webgpu scenes, false for three) - Hints:
audioReactive: truedeclared;mouseReactive/timeAnimatedset if applicable - Audio: every field consumed in
update()is declared inaudio: {…} - Controls (schema:2): every user-tunable parameter has a
ControlSpec(range, default, target) - Lifecycle:
init()returns{ update, resize, dispose } -
update()renders into the HDR FBO via the category's standard pattern (adapter / bridge / three runtime), not by callinggl.bindFramebuffer()directly -
dispose()frees every resourceinit()allocated (auditgl.create*,new THREE.*,setTimeout, event listeners) - Harness:
npm run test:visualpasses for the new scene end-to-end - Live: a scene-cycling session shows no console errors when activating/deactivating the scene repeatedly
WebGPU compute scenes (category: 'webgpu') additionally:
- Declares
category: 'webgpu'(the current label — never'compute') - Implements the right authoring path: declarative (
storageBuffers+computePasses+renderPass) or imperative (frame(ctx)owning its own encoder + submission) -
dispose()frees everyGPUBuffer/GPUTexture/ pipeline it created and never callsdevice.destroy() -
edgeTaper: truepassed in thesetWgpuSourceCanvas(...)runtime call when the scene renders bright content to the frame edge (kills the CA edge fringe) — the default for compute scenes - Passes the visible-effect smoke test (
compute-audio-controls-page.html), not just pixel-diff — band + speed levers produce the expected frame change - If it uses layered additive composition: passes per-channel diagnostic verification (
compute-tesla-orb-diag.htmlpattern) — signature hues survive, no white-blob clamp (Section 6 v8 hazard)
Section 11 — Open issues and category exceptions
This section lists deviations from the canonical pattern. Each entry says whether it's a permanent exception or an open issue with a planned resolution.
Stochastic scenes (pawrticles, fire) — open issue, special verification needed
These scenes use Math.random() for per-frame particle state (ember spawn
positions, ripple-burst seeds, etc.). The harness currently sets
hints.stochasticRender: true on them, which skips the pixel-diff
comparison against the reference PNG — because two identical runs produce
different per-pixel output, so any reference comparison hits noise.
This is an escape hatch from the pixel-diff method, NOT from the
music-visualizer requirement. The scenes might be reactive — the simple
pixel-diff just can't prove it. The wire-band sweeps + the audio-reactive
check still run against them, and currently pass (recent harness output:
fire audio-reactive=85.31, pawrticles audio-reactive=172.23). So they do
satisfy Section 0 today.
The gap is that we don't have a robust verification method for stochastic scenes — a scene could be reactive in non-stochastic content (bar heights, overall brightness) while the stochastic layer (embers, particles) drowns the diff in noise. v1.x reactivity work needs one of:
- Frame-averaged diff — average N frames of silent + N frames of loud, compare the averages. Per-frame randomness cancels out in the mean.
- Energy-of-motion comparison — compute total optical flow magnitude per frame, compare distributions. Reactive scenes show different motion energy between silent and loud.
- Region-of-interest diff — declare a non-stochastic region (e.g., fire's
bar baseline at
y > 0.7) and run pixel-diff only there.
None is implemented yet. For v1 launch, the existing wire + simple-diff checks are good enough — stochastic scenes pass them. The richer method is v1.x.
Ferrofluid — permanent exception (3D mesh in category: 'fragment')
src/render/scenes/ferrofluid.js declares category: 'fragment' but does
not use createGlslScene. Instead it builds its own GL pipeline directly:
two compiled programs (background fullscreen quad + foreground 3D mesh), a
256×256 vertex grid with indexed draws, a depth renderbuffer attached to
the pipeline FBO at init and detached at dispose, per-vertex sampling of
a 128-bin R8 audio texture, and mouse drag-to-rotate event listeners on
runtime.canvas. Seven gl.create* calls live in init().
Why the bypass is correct: the GLSL adapter is designed for fullscreen- quad fragment shaders. Ferrofluid is a mesh visual — 3D geometry with depth testing, two cooperating programs, and per-vertex (not per-fragment) audio data. None of that fits the adapter's surface. The scene file's header comment (lines 1–32) documents the rationale explicitly.
Why category: 'fragment' instead of inventing 'mesh': the category
is a semantic best-fit until we have a second mesh-style scene. The scene
still produces sRGB pixels into the standard pipeline path with
selfTonemapped: true, so the post-FX chain treats it identically to a
fragment scene. Inventing a single-instance 'mesh' category +
createMeshScene adapter for one scene is over-engineering.
Resource cleanup is still compliant. dispose() deletes both programs,
both buffers, the audio texture, the depth renderbuffer attachment, and
removes the canvas event listeners. Audit this when changing the scene.
Trigger condition for revisiting: when (if) a second mesh-style scene
is proposed, that's the point to formalize. Pattern: invent
category: 'mesh', extract the shared mesh-runtime bits from ferrofluid's
init into a createMeshScene adapter (mirroring createGlslScene and
createCanvas2dBridge), migrate ferrofluid as the reference scene, then
the new scene plugs into the same adapter. Until then, ferrofluid is a
one-off and the bypass stays documented here.
MilkDrop — permanent exception (not a category)
Butterchurn owns its own preset lifecycle, render loop, post-FX, and DOM
canvas (#viz + #mdpp-canvas). It runs as a peer system to the pipeline,
not a category under it. Future MilkDrop integration extensions (mdpp
post-FX, texture injection) belong in src/mdpp/* and main/*, not in this
contract.
The reactivity gate still applies to MilkDrop presets — they ARE music
visualizer content. The v1.x test:capture:milkdrop work is the
enforcement mechanism there.
WebGPU compute (category: 'webgpu') — built (was: reserved)
No longer an open item. The ComputeRuntime stack is live and 8 compute
scenes ship on it — see Section 5's category: 'webgpu' subsection for the full
contract. This entry now only tracks the residual deviations.
Legacy WGSL scenes — open issue, planned unification. Three older WGSL scenes
(nebula, ikandy-singularity, ikandy-aurora) still run their own device +
start/stop loops on #wgpu-canvas, not through ComputeRuntime. Their loop
and the ComputeRuntime loop are mutually exclusive (the host page stops one
before starting the other), so reconfiguring the canvas context per setActive
is safe. Folding them onto the single shared device/context owner is the planned
unification goal (compute-runtime.js header, "Day-7"). Until then they bypass
the harness/visible-effect gates because they aren't ComputeRuntime scenes — the
reactivity requirement still applies to them in principle.
Plasma Globe (compute-tesla-orb) — canonical heavily-iterated compute scene
src/render/scenes/compute-tesla-orb.js (id: 'compute-tesla-orb',
name: 'Plasma Globe', category: 'webgpu', imperative frame(ctx)) is the
reference point for a heavily-iterated compute scene — shipped through ~v9 with
behavior that goes well past the declarative template. Look here when authoring a
compute scene that needs custom reactivity:
- Custom audio reactivity — bass → tube displacement + attractor pull; mid →
ripple detail + rotation/swirl; treble → bolt count (1–3) + core flicker; kick →
explosion impulse + shockwave brightness. Reads the runtime-derived
ctx.audio.kickrather than re-detecting beats. - Beat-onset detection — fires on a substantial rise above a stable baseline
(
_kickFloorsnaps down fast, creeps up slow, sitting in the between-beats trough), gated by a hard refractory interval (MIN_BOLT_INTERVAL). This is what stops the decay tail's micro-rises from re-firing 3–6 bolts per beat (compute-tesla-orb.js:721–729). - Shockwave physics — on a strike, particles within a radius get a kick-scaled radial impulse; each reassigns its wave phase to the most-recent strike; a wave front sweeps outward at constant speed with a thin saturated-hue rainbow burst (Section 6 v8 mitigation), then spring-damper settles back home (compute-tesla-orb.js:227 + 235–242 impulse/spring-damper pass, 487–499 wave burst).
It's also the scene that motivated the v8 additive-blend hazard documentation
(Section 6) and its per-channel diagnostic (compute-tesla-orb-diag.html). When
in doubt about how far a compute scene may deviate while staying compliant, this is
the high-water mark: imperative encoding, bespoke detection, and physics are all
fine — resource cleanup (Section 8), the color contract (Section 6), and the
reactivity gates (Section 9) still bind.
Section 11.5 — Post-FX pass authoring requirements
Different scope from scene authoring — this section is for contributors writing new entries in src/render/passes/*.js (Bloom, CA, Vignette, etc.). Captured here because post-FX passes and scenes interact through the same runtime contract and bugs in one class affect the other.
Visibility floor
Every post-FX pass MUST produce a CLEARLY visible effect at slMul=1.0 on at least one reference scene. Subtle is fine; invisible is a bug.
The perceptual minimums for "user can register that the slider does something":
- Positional effects (CA chromatic shift, FXAA edge sample offsets, lens distortion): 8–15 pixels of UV displacement at 1080p. Anything ≤ 5 pixels reads as "the slider does nothing" to the typical viewer.
- Intensity effects (Vignette darkening, Bloom intensity, Grain amount, LUT mix): 20–40% change in pixel brightness or color saturation at the affected region. Anything ≤ 5% reads as no change.
- Region effects (Vignette spread, scanline density): affect at least 50% of visible area. A vignette confined to the literal four corners reads as "no vignette" to most users — the eye dwells on the middle of the frame.
Authoring checklist
- Test at
slMul=0andslMul=1.0side-by-side on hol, fire, and aurora (sharp content + bright content + volumetric content). Visible difference at all three. - Run
npm run test:visual— the post-FX visibility harness section asserts mean-diff > 0.5 between slMul=0 and slMul=1.0 for the tested pass. Add your new pass to that harness check (seesrc/test/visual-harness.js,passesToCheckarray). New passes that don't pass the visibility harness can't merge. - Compare to mdpp's equivalent —
src/render/passes/mdpp.jscontains the working reference for CA, Vignette, Bloom, Grain, Scanlines as visibility magnitudes. If your pass produces less visible effect than mdpp's at equivalent slider values, the tuning is too conservative. - Calibrate the magic numbers against pixel-space expectations, not against the individual values in the per-scene hint × pass-default × slider math. The FINAL product is what hits the screen.
- Document the magic numbers inline with what they target. Example:
* 0.20 // produces ~8-15 px UV offset at slMul=1, caHint=0.10, 1080p— leaves a paper trail for the next contributor.
What NOT to do
- Don't infer visibility from a single screenshot. If you can only TELL the pass is on by toggling it and watching for change, the pass is sub-perceptual. The pre-Thread-1 CA and Vignette both passed this test (they "looked the same" with and without, because they were invisible).
- Don't tune against a single reference scene. Hol's heart edges, fire's bars, and aurora's curtains all have different content characteristics. A pass tuned only against one scene may be invisible on the others.
- Don't trust math-based reasoning that the pass should work. The original
* 0.030in CA was correctly producing the math it was authored to produce — but the math itself was producing sub-pixel offsets that no human could see. Always validate visually. - Don't quote code identifiers with backticks inside shader-source comments. The shader is a JS template literal; backticks terminate it early. Use parens, quotes, or bare words. (Cost one investigation round during Thread 1.)
v1.x extension
When the import / AI authoring paths land, user-uploaded and AI-generated post-FX passes (if we allow them — currently out of scope) must satisfy this section. The harness post-FX visibility check is the enforcement.
Section 12 — Future contract evolution
When v1.x extensibility work (import templates, AI generation) begins, this is the target. Specifically:
- Import UI: a user-uploaded scene file must satisfy this contract. The import path likely wraps user code in a
createGlslScene/createCanvas2dBridge-style adapter so user-written code can be minimal. - AI generation: the generator's prompts must reference this contract. A generated scene that misses
dispose()or violates color space is a regression worth catching in the generator. - WebGPU bring-in: ✅ done — the
category: 'webgpu'contract is built and documented in Section 5 (ComputeRuntime+ dispatch orchestrator + audio UBO, samefactory → instanceouter shape, declarative + imperative paths). Remaining: migrate the three legacy WGSL scenes (nebula, ikandy-singularity, ikandy-aurora) off their standalone loops onto the sharedComputeRuntimedevice/context (Section 11). - Schema:3 (if ever): any new top-level metadata field or runtime API addition lands first in
scene-schema.md, with this doc updated to document the runtime relationship for it.
Compliance audits should re-run after each major contract change.
← Back to the Artist Program