Zestial: visualizing music in the browser with WebGL

How Zestial analyses audio, composes layers and exports video, all in the browser on your own device.

Zestial started as a weekend toy, an FFT feeding a fragment shader. It became a node-based composition tool because I kept wanting to bind any audio feature to any visual parameter.

The audio analysis pipeline

An AudioWorklet extracts bass, mids, highs, spectral flux, RMS and beat confidence in real time and publishes them to the render thread as a small typed-array frame. Nothing allocates per frame, because every buffer is a ring buffer.

Layers

A Zestial composition is a stack of layers. Particles, audio bars, mesh flow and orb cores are four of the many types. Each layer has its own blend mode and its own bindings. The node editor compiles bindings to a flat instruction list, so nothing traverses a graph at render time.

Exporting video without a server

The hardest constraint was one I set myself, no upload. Export renders offline, frame by frame, into a WebCodecs encoder, and WASM code muxes the result. It exports 1080p, 1440p or 4K at 60 frames per second on anything that runs a browser, phones included. Your track never leaves your device.

Written by Alopex ↗Developer, maker, explorer. I build tools, games and systems for creators and players.