Guide

Guide Recipes Audio reactive scene

Audio reactive scene

Map audio analysis to movement, particles, and camera response.

Intermediate ~2 min read

Audio reactive scene

This recipe combines BeatDetector, ParticleSystem, tween-driven camera shake, and audio-triggered visual effects into one scene. It builds on Beat detection for analysis and Audio-reactive visualization for the visual patterns — this chapter is about orchestration, not re-teaching the individual APIs.

Approach

Three systems run in parallel, each driven by a different facet of the audio analysis:

  1. Beat → particle burst: onBeat fires a BurstSpawn and triggers camera shake.
  2. Bar → tween chain: onBarStart triggers a timed sequence of scale/rotation tweens on a central element.
  3. Frequency → continuous motion: Low-band energy from AudioAnalyser.getLowMidHigh() drives the background tint or a sprite’s idle animation.

Setup

One analyser for frequency data, one beat detector for rhythm, one particle system for visual feedback:

import {
    Application, AudioStream,
    Scene, View, Texture, Vector, Color, Ease,
} from '@codexo/exojs';
import { AudioAnalyser, BeatDetector } from '@codexo/exojs-audio-fx';
import {
    AlphaFadeOverLifetime, BurstSpawn, ConeDirection, Constant,
    particlesExtension, ParticleSystem,
} from '@codexo/exojs-particles';

const app = new Application({ extensions: [particlesExtension] });

class AudioReactiveScene extends Scene {
    async load(loader) {
        await loader.load(AudioStream, { track: 'audio/track.ogg' });
        await loader.load(Texture, { particle: 'image/particle.png' });
    }

    init(loader) {
        this.app.audio.play(loader.get(AudioStream, 'track'), { loop: true, volume: 0.8 });

        // Analysis — both taps read the music bus the track plays through.
        this.analyser = new AudioAnalyser({ source: this.app.audio.music, fftSize: 512 });
        this.detector = new BeatDetector({ source: this.app.audio.music });
        this._bg = new Color(20, 24, 40, 1);

        // Particles — burst on beat
        this.particles = new ParticleSystem(loader.get(Texture, 'particle'), { capacity: 5000 });
        this.burst = new BurstSpawn({
            schedule: [{ time: 0, count: 120 }],
            lifetime: new Constant(0.8),
            velocity: ConeDirection.omni(100, 360),
            scale: new Constant(new Vector(0.2, 0.2)),
        });
        this.particles.addSpawnModule(this.burst);
        this.particles.addUpdateModule(new AlphaFadeOverLifetime());

        // Camera shake
        this.view = new View(400, 300, 800, 600);

        // Beat → burst + shake
        this.detector.onBeat.add(() => {
            this.burst.reset();
            this.view.shake(14, 200, { frequency: 30, decay: true });
        });
    }

    update(delta) {
        this.view.update(delta.milliseconds);
        this.particles.update(delta);

        // Continuous frequency → background tint
        const { low } = this.analyser.getLowMidHigh();
        this._bg.set(
            20 + low * 26,
            24 + low * 15,
            40 + low * 52,
            1,
        );
    }

    draw(context) {
        context.backend.clear(this._bg);
        context.render(this.particles, { view: this.view });
    }
}

Layering effects

The pattern scales by adding more independent listeners, each reacting to a different audio facet:

// Bar start → tween chain on a central logo
this.detector.onBarStart.add(() => {
    this.app.tweens.create(this.logo.scale)
        .to({ x: 1.3, y: 1.3 }, 0.15)
        .easing(Ease.cubicOut)
        .yoyo()
        .repeat(1)
        .start();
});

// Mid-band → particle tint cycling
this.detector.onBeat.add(info => {
    if (info.beatInBar === 2 || info.beatInBar === 4) {
        this.burst.config.tint = new Constant(
            info.beatInBar === 2 ? Color.orange : Color.skyBlue
        );
        this.burst.reset();
    }
});

Camera shake from audio

View.shake(intensity, durationMs, { frequency, decay }) displaces the view center by a decaying sinusoidal offset for durationMs milliseconds. decay: true reduces the shake intensity over the duration; frequency controls the oscillation rate. Call view.shake() inside onBeat for a bass hit, or onDownbeat for a stronger effect on the first beat of each bar.

For frequency-driven shake (low-end rumble instead of discrete hits), lerp the view center by the low-band energy in update:

update(delta) {
    const { low } = this.analyser.getLowMidHigh();
    const shakeX = (Math.random() - 0.5) * low * 20;
    const shakeY = (Math.random() - 0.5) * low * 20;
    this.view.setCenter(
        400 + shakeX,
        300 + shakeY,
    );
}

This produces a continuous subtle shake proportional to bass energy, suitable for engine rumble, earthquake effects, or low-frequency ambience.

Orchestration principles

  • One source, many listeners: One BeatDetector can feed particle bursts, camera shake, HUD flash, and tween triggers simultaneously — each subscriber acts independently.
  • Separate analyser for continuous data: AudioAnalyser.getLowMidHigh() gives band energies every frame without event overhead. Use it for smooth, continuous visuals like tint shifts, bar graphs, or procedural animation.
  • Beat detector for discrete events: onBeat/onDownbeat/onBarStart are for one-shot triggers. They fire at discrete moments and carry timing metadata (isDownbeat, beatInBar) for per-beat variation.
  • Keep the per-frame work bounded: A particle system at 120 particles per beat with 120 BPM spawns 240 particles/second — well within a 5000-capacity system. Check system.aliveCount occasionally to confirm you’re not overrunning capacity.

Examples

Audio Reactive Particles Audio Open in Playground View source

Preview is paused until you click Play.

 
import { Application, AudioStream, Color, Scene, Text, Texture, Vector, type Voice } from '@codexo/exojs';
import { AudioAnalyser, BeatDetector } from '@codexo/exojs-audio-fx';
import {
    AlphaFadeOverLifetime,
    ConeDirection,
    Constant,
    particlesExtension,
    ParticleSystem,
    RateSpawn,
} from '@codexo/exojs-particles';
import { mountControls } from '@examples/runtime';

const app = new Application({
    canvas: {
        width: 1280,
        height: 720,
        mount: document.body,
        sizingMode: 'fit',
    },
    clearColor: Color.black,
    extensions: [particlesExtension],
});

const colors = [new Color(255, 120, 140), new Color(120, 220, 255), new Color(130, 255, 170), new Color(255, 220, 120)];

class AudioReactiveParticlesScene extends Scene {
    private music!: AudioStream;
    private musicVoice!: Voice;
    private analyser!: AudioAnalyser;
    private detector!: BeatDetector;
    private ps!: ParticleSystem;
    private spawn!: RateSpawn;
    private rate!: Constant<number>;
    private cone!: ConeDirection;
    private hud!: ReturnType<typeof mountControls>;
    private tapPrompt!: Text;

    override async load(loader): Promise<void> {
        await loader.load(AudioStream, { track: assets.demo.audio.musicLoop });
        await loader.load(Texture, { particle: assets.demo.textures.particleLight });
    }

    override init(loader): void {
        const { width, height } = this.app.canvas;

        this.music = loader.get(AudioStream, 'track');

        // Two parallel taps of the same track: the analyser gives per-band
        // energy (drives emission), the detector gives beats (recolours).
        this.analyser = new AudioAnalyser({ fftSize: 1024, source: this.app.audio.music });
        this.detector = new BeatDetector();
        this.detector.source = this.app.audio.music;

        this.ps = new ParticleSystem(loader.get(Texture, 'particle'), { capacity: 6000 });
        this.ps.setPosition(width / 2, height / 2);

        // The rate (density) and the cone speed range (spread) are mutated every
        // frame from live audio energy. Starting both near zero means a silent
        // track emits (almost) nothing — the field is genuinely data-driven, not
        // a timed fountain dressed up as "reactive".
        this.rate = new Constant(0);
        this.cone = ConeDirection.omni(20, 40);
        this.spawn = new RateSpawn({
            rate: this.rate,
            lifetime: new Constant(0.9),
            position: new Constant(new Vector(0, 0)),
            velocity: this.cone,
            scale: new Constant(new Vector(0.22, 0.22)),
            tint: new Constant(colors[0]),
        });
        this.ps.addSpawnModule(this.spawn);
        this.ps.addUpdateModule(new AlphaFadeOverLifetime());

        // Beats only recolour the stream; they do not fake emission on their own.
        this.detector.onBeat.add(() => {
            this.spawn.config.tint = new Constant(colors[(Math.random() * colors.length) | 0]);
        });

        this.hud = mountControls({
            title: 'Audio Reactive Particles',
            controls: [{ keys: 'Audio', action: 'bass → density · treble → spread' }],
            status: 'Listening…',
            hint: 'Density follows low-band energy; spread follows high-band energy. Silence = still.',
        });

        // Shown while the browser still blocks audio (`app.audio.locked`); the
        // first click or keypress unlocks it and the queued music starts.
        this.tapPrompt = new Text('Click or press any key to start the music', { fillColor: Color.white, fontSize: 22, align: 'center' })
            .setAnchor(0.5, 0.5)
            .setPosition(width / 2, height - 64);

        // Core defers playback until the AudioContext unlocks on the first
        // gesture, then starts automatically — play() returns the Voice now.
        this.musicVoice = this.app.audio.play(this.music, { loop: true, volume: 0.8 });
    }

    override update(delta): void {
        // Low band (bass) drives how MANY particles spawn this second.
        const low = this.analyser.getBandEnergy(20, 180);
        // High band (treble) drives how WIDE the velocity cone fans out.
        const high = this.analyser.getBandEnergy(2000, 16000);

        // bass → density: 0 in silence, up to ~1200 particles/s on heavy bass.
        this.rate.value = low * low * 1200;

        // treble → spread: a tight slow core grows into a fast wide burst.
        this.cone.minSpeed = 40 + high * 120;
        this.cone.maxSpeed = 90 + high * 360;

        if (this.musicVoice) {
            this.hud.setStatus(`bass ${(low * 100) | 0}%  treble ${(high * 100) | 0}%`);
        }

        this.ps.update(delta);
    }

    override draw(context): void {
        context.backend.clear();
        context.render(this.ps);

        if (this.app.audio.locked) {
            context.render(this.tapPrompt);
        }
    }
}

app.start(new AudioReactiveParticlesScene());

Particles burst on every beat with alpha fade — the canonical beat → particles pattern.

Low Band Camera Shake Audio Open in Playground View source

Preview is paused until you click Play.

 
import { Application, AudioStream, Color, Scene, Sprite, Text, Texture, View, type Voice } from '@codexo/exojs';
import { AudioAnalyser } from '@codexo/exojs-audio-fx';
import { mountControls } from '@examples/runtime';

const app = new Application({
    canvas: {
        width: 1280,
        height: 720,
        mount: document.body,
        sizingMode: 'fit',
    },
    clearColor: Color.black,
});

class LowBandCameraShakeScene extends Scene {
    private music!: AudioStream;
    private musicVoice!: Voice;
    private analyser!: AudioAnalyser;
    private view!: View;
    private sprite!: Sprite;
    private hud!: ReturnType<typeof mountControls>;
    private tapPrompt!: Text;

    override async load(loader): Promise<void> {
        await loader.load(AudioStream, { track: assets.demo.audio.musicLoop });
        await loader.load(Texture, { ship: assets.demo.textures.shipA });
    }

    override init(loader): void {
        const { width, height } = this.app.canvas;

        this.music = loader.get(AudioStream, 'track');
        this.analyser = new AudioAnalyser({ fftSize: 1024, source: this.app.audio.music });
        this.view = new View(width / 2, height / 2, width, height);
        this.sprite = new Sprite(loader.get(Texture, 'ship')).setAnchor(0.5).setScale(3).setPosition(width / 2, height / 2);

        this.hud = mountControls({
            title: 'Low Band Camera Shake',
            controls: [{ keys: 'Audio', action: 'low-band energy → shake' }],
            status: 'Listening…',
            hint: 'Shake amplitude tracks bass energy only — in silence the camera is perfectly still.',
        });

        // Shown while the browser still blocks audio (`app.audio.locked`); the
        // first click or keypress unlocks it and the queued music starts.
        this.tapPrompt = new Text('Click or press any key to start the music', { fillColor: Color.white, fontSize: 22, align: 'center' })
            .setAnchor(0.5, 0.5)
            .setPosition(width / 2, height - 64);

        // Core defers playback until the AudioContext unlocks on the first
        // gesture, then starts automatically — play() returns the Voice now.
        this.musicVoice = this.app.audio.play(this.music, { loop: true, volume: 0.8 });
    }

    override update(delta): void {
        const low = this.analyser.getBandEnergy(20, 180);

        // No constant floor: amplitude is purely low-band energy, so a quiet
        // passage produces zero shake. A small deadzone keeps faint noise still.
        const amplitude = low > 0.04 ? low * 28 : 0;
        this.view.shake(amplitude, 90, { decay: true, frequency: 22 });

        // Advance the shake oscillation (the View only animates when updated).
        this.view.update(delta.milliseconds);

        if (this.musicVoice) {
            this.hud.setStatus(`bass ${(low * 100) | 0}%`);
        }
    }

    override draw(context): void {
        context.backend.clear(new Color(22, 24, 34));
        context.backend.setView(this.view);
        context.render(this.sprite);
        context.backend.setView(null);

        if (this.app.audio.locked) {
            context.render(this.tapPrompt);
        }
    }
}

app.start(new LowBandCameraShakeScene());

Bass-frequency energy drives continuous camera shake — low-end rumble as visual motion.

Where to go next

The next recipe, Game feel, covers general feedback techniques — damage flashes, screen shake, audio cues, and tween-driven response that make interaction feel responsive.