Map audio analysis to movement, particles, and camera response.
Audio reactive scene
This recipe combines BeatDetector, ParticleSystem, tween-driven camera shake, and audio-triggered visual effects into one scene. It builds on Beat detection for analysis and Audio-reactive visualization for the visual patterns — this chapter is about orchestration, not re-teaching the individual APIs.
Approach
Three systems run in parallel, each driven by a different facet of the audio analysis:
- Beat → particle burst:
onBeat fires a BurstSpawn and triggers camera shake.
- Bar → tween chain:
onBarStart triggers a timed sequence of scale/rotation tweens on a central element.
- Frequency → continuous motion: Low-band energy from
AudioAnalyser.getLowMidHigh() drives the background tint or a sprite’s idle animation.
Setup
One analyser for frequency data, one beat detector for rhythm, one particle system for visual feedback:
import {
Application, AudioAnalyser, BeatDetector, Music,
Scene, View, Texture, Vector, Color, Ease,
} from '@codexo/exojs';
import {
AlphaFadeOverLifetime, BurstSpawn, ConeDirection, Constant,
particlesExtension, ParticleSystem,
} from '@codexo/exojs-particles';
const app = new Application({ extensions: [particlesExtension] });
class AudioReactiveScene extends Scene {
async load(loader) {
await loader.load(Music, { track: 'audio/track.ogg' });
await loader.load(Texture, { particle: 'image/particle.png' });
}
init(loader) {
const music = loader.get(Music, 'track').setLoop(true).setVolume(0.8).play();
// Analysis
this.analyser = new AudioAnalyser({ source: music, fftSize: 512 });
this.detector = new BeatDetector({ source: music });
this._bg = new Color(20, 24, 40, 1);
// Particles — burst on beat
this.particles = new ParticleSystem(loader.get(Texture, 'particle'), { capacity: 5000 });
this.burst = new BurstSpawn({
schedule: [{ time: 0, count: 120 }],
lifetime: new Constant(0.8),
velocity: ConeDirection.omni(100, 360),
scale: new Constant(new Vector(0.2, 0.2)),
});
this.particles.addSpawnModule(this.burst);
this.particles.addUpdateModule(new AlphaFadeOverLifetime());
// Camera shake
this.view = new View(400, 300, 800, 600);
// Beat → burst + shake
this.detector.onBeat.add(() => {
this.burst.reset();
this.view.shake(14, 200, { frequency: 30, decay: true });
});
}
update(delta) {
this.view.update(delta.milliseconds);
this.particles.update(delta);
// Continuous frequency → background tint
const { low } = this.analyser.getLowMidHigh();
this._bg.set(
20 + low * 26,
24 + low * 15,
40 + low * 52,
1,
);
}
draw(context) {
context.backend.clear(this._bg);
context.render(this.particles, { view: this.view });
}
}
Layering effects
The pattern scales by adding more independent listeners, each reacting to a different audio facet:
// Bar start → tween chain on a central logo
this.detector.onBarStart.add(() => {
this.app.tweens.create(this.logo.scale)
.to({ x: 1.3, y: 1.3 }, 0.15)
.easing(Ease.cubicOut)
.yoyo()
.repeat(1)
.start();
});
// Mid-band → particle tint cycling
this.detector.onBeat.add(info => {
if (info.beatInBar === 2 || info.beatInBar === 4) {
this.burst.config.tint = new Constant(
info.beatInBar === 2 ? Color.orange : Color.skyBlue
);
this.burst.reset();
}
});
Camera shake from audio
View.shake(intensity, durationMs, { frequency, decay }) displaces the view center by a decaying sinusoidal offset for durationMs milliseconds. decay: true reduces the shake intensity over the duration; frequency controls the oscillation rate. Call view.shake() inside onBeat for a bass hit, or onDownbeat for a stronger effect on the first beat of each bar.
For frequency-driven shake (low-end rumble instead of discrete hits), lerp the view center by the low-band energy in update:
update(delta) {
const { low } = this.analyser.getLowMidHigh();
const shakeX = (Math.random() - 0.5) * low * 20;
const shakeY = (Math.random() - 0.5) * low * 20;
this.view.setCenter(
400 + shakeX,
300 + shakeY,
);
}
This produces a continuous subtle shake proportional to bass energy, suitable for engine rumble, earthquake effects, or low-frequency ambience.
Orchestration principles
- One source, many listeners: One
BeatDetector can feed particle bursts, camera shake, HUD flash, and tween triggers simultaneously — each subscriber acts independently.
- Separate analyser for continuous data:
AudioAnalyser.getLowMidHigh() gives band energies every frame without event overhead. Use it for smooth, continuous visuals like tint shifts, bar graphs, or procedural animation.
- Beat detector for discrete events:
onBeat/onDownbeat/onBarStart are for one-shot triggers. They fire at discrete moments and carry timing metadata (isDownbeat, beatInBar) for per-beat variation.
- Keep the per-frame work bounded: A particle system at 120 particles per beat with 120 BPM spawns 240 particles/second — well within a 5000-capacity system. Check
system.aliveCount occasionally to confirm you’re not overrunning capacity.
Examples
import {
Application,
AudioAnalyser,
BeatDetector,
Color,
Music,
Scene,
Text,
Texture,
Vector,
} from '@codexo/exojs';
import {
AlphaFadeOverLifetime,
ConeDirection,
Constant,
particlesExtension,
ParticleSystem,
RateSpawn,
} from '@codexo/exojs-particles';
import { mountControls } from '@examples/runtime';
const app = new Application({
canvas: {
width: 1280,
height: 720,
mount: document.body,
sizingMode: 'fit',
},
clearColor: Color.black,
extensions: [particlesExtension],
});
const colors = [new Color(255, 120, 140), new Color(120, 220, 255), new Color(130, 255, 170), new Color(255, 220, 120)];
class AudioReactiveParticlesScene extends Scene {
private music!: Music;
private analyser!: AudioAnalyser;
private detector!: BeatDetector;
private ps!: ParticleSystem;
private spawn!: RateSpawn;
private rate!: Constant<number>;
private cone!: ConeDirection;
private hud!: ReturnType<typeof mountControls>;
private tapPrompt!: Text;
override async load(loader): Promise<void> {
await loader.load(Music, { track: assets.demo.audio.musicLoop });
await loader.load(Texture, { particle: assets.demo.textures.particleLight });
}
override init(loader): void {
const { width, height } = this.app.canvas;
this.music = loader.get(Music, 'track');
// Two parallel taps of the same track: the analyser gives per-band
// energy (drives emission), the detector gives beats (recolours).
this.analyser = new AudioAnalyser({ fftSize: 1024, source: this.music });
this.detector = new BeatDetector();
this.detector.source = this.music;
this.ps = new ParticleSystem(loader.get(Texture, 'particle'), { capacity: 6000 });
this.ps.setPosition(width / 2, height / 2);
// The rate (density) and the cone speed range (spread) are mutated every
// frame from live audio energy. Starting both near zero means a silent
// track emits (almost) nothing — the field is genuinely data-driven, not
// a timed fountain dressed up as "reactive".
this.rate = new Constant(0);
this.cone = ConeDirection.omni(20, 40);
this.spawn = new RateSpawn({
rate: this.rate,
lifetime: new Constant(0.9),
position: new Constant(new Vector(0, 0)),
velocity: this.cone,
scale: new Constant(new Vector(0.22, 0.22)),
tint: new Constant(colors[0]),
});
this.ps.addSpawnModule(this.spawn);
this.ps.addUpdateModule(new AlphaFadeOverLifetime());
// Beats only recolour the stream; they do not fake emission on their own.
this.detector.onBeat.add(() => {
this.spawn.config.tint = new Constant(colors[(Math.random() * colors.length) | 0]);
});
this.hud = mountControls({
title: 'Audio Reactive Particles',
controls: [{ keys: 'Audio', action: 'bass → density · treble → spread' }],
status: 'Listening…',
hint: 'Density follows low-band energy; spread follows high-band energy. Silence = still.',
});
// Shown while the browser still blocks audio (`app.audio.locked`); the
// first click or keypress unlocks it and the queued music starts.
this.tapPrompt = new Text('Click or press any key to start the music', { fillColor: Color.white, fontSize: 22, align: 'center' })
.setAnchor(0.5, 0.5)
.setPosition(width / 2, height - 64);
// Core defers playback until the AudioContext unlocks on the first
// gesture, then starts automatically — just call play().
this.music.setLoop(true).setVolume(0.8).play();
}
override update(delta): void {
// Low band (bass) drives how MANY particles spawn this second.
const low = this.analyser.getBandEnergy(20, 180);
// High band (treble) drives how WIDE the velocity cone fans out.
const high = this.analyser.getBandEnergy(2000, 16000);
// bass → density: 0 in silence, up to ~1200 particles/s on heavy bass.
this.rate.value = low * low * 1200;
// treble → spread: a tight slow core grows into a fast wide burst.
this.cone.minSpeed = 40 + high * 120;
this.cone.maxSpeed = 90 + high * 360;
if (!this.music.paused) {
this.hud.setStatus(`bass ${(low * 100) | 0}% treble ${(high * 100) | 0}%`);
}
this.ps.update(delta);
}
override draw(context): void {
context.backend.clear();
context.render(this.ps);
if (this.app.audio.locked) {
context.render(this.tapPrompt);
}
}
}
app.start(new AudioReactiveParticlesScene());
Particles burst on every beat with alpha fade — the canonical beat → particles pattern.
import { Application, AudioAnalyser, Color, Music, Scene, Sprite, Text, Texture, View } from '@codexo/exojs';
import { mountControls } from '@examples/runtime';
const app = new Application({
canvas: {
width: 1280,
height: 720,
mount: document.body,
sizingMode: 'fit',
},
clearColor: Color.black,
});
class LowBandCameraShakeScene extends Scene {
private music!: Music;
private analyser!: AudioAnalyser;
private view!: View;
private sprite!: Sprite;
private hud!: ReturnType<typeof mountControls>;
private tapPrompt!: Text;
override async load(loader): Promise<void> {
await loader.load(Music, { track: assets.demo.audio.musicLoop });
await loader.load(Texture, { ship: assets.demo.textures.shipA });
}
override init(loader): void {
const { width, height } = this.app.canvas;
this.music = loader.get(Music, 'track');
this.analyser = new AudioAnalyser({ fftSize: 1024, source: this.music });
this.view = new View(width / 2, height / 2, width, height);
this.sprite = new Sprite(loader.get(Texture, 'ship')).setAnchor(0.5).setScale(3).setPosition(width / 2, height / 2);
this.hud = mountControls({
title: 'Low Band Camera Shake',
controls: [{ keys: 'Audio', action: 'low-band energy → shake' }],
status: 'Listening…',
hint: 'Shake amplitude tracks bass energy only — in silence the camera is perfectly still.',
});
// Shown while the browser still blocks audio (`app.audio.locked`); the
// first click or keypress unlocks it and the queued music starts.
this.tapPrompt = new Text('Click or press any key to start the music', { fillColor: Color.white, fontSize: 22, align: 'center' })
.setAnchor(0.5, 0.5)
.setPosition(width / 2, height - 64);
// Core defers playback until the AudioContext unlocks on the first
// gesture, then starts automatically — just call play().
this.music.setLoop(true).setVolume(0.8).play();
}
override update(delta): void {
const low = this.analyser.getBandEnergy(20, 180);
// No constant floor: amplitude is purely low-band energy, so a quiet
// passage produces zero shake. A small deadzone keeps faint noise still.
const amplitude = low > 0.04 ? low * 28 : 0;
this.view.shake(amplitude, 90, { decay: true, frequency: 22 });
// Advance the shake oscillation (the View only animates when updated).
this.view.update(delta.milliseconds);
if (!this.music.paused) {
this.hud.setStatus(`bass ${(low * 100) | 0}%`);
}
}
override draw(context): void {
context.backend.clear(new Color(22, 24, 34));
context.backend.setView(this.view);
context.render(this.sprite);
context.backend.setView(null);
if (this.app.audio.locked) {
context.render(this.tapPrompt);
}
}
}
app.start(new LowBandCameraShakeScene());
Bass-frequency energy drives continuous camera shake — low-end rumble as visual motion.
Where to go next
The next recipe, Game feel, covers general feedback techniques — damage flashes, screen shake, audio cues, and tween-driven response that make interaction feel responsive.