Map audio analysis to movement, particles, and camera response.
Intermediate~2 min read
Audio reactive scene
This recipe combines BeatDetector, ParticleSystem, tween-driven camera shake, and audio-triggered visual effects into one scene. It builds on Beat detection for analysis and Audio-reactive visualization for the visual patterns — this chapter is about orchestration, not re-teaching the individual APIs.
Approach
Three systems run in parallel, each driven by a different facet of the audio analysis:
Beat → particle burst: onBeat fires a BurstSpawn and triggers camera shake.
Bar → tween chain: onBarStart triggers a timed sequence of scale/rotation tweens on a central element.
Frequency → continuous motion: Low-band energy from AudioAnalyser.getLowMidHigh() drives the background tint or a sprite’s idle animation.
Setup
One analyser for frequency data, one beat detector for rhythm, one particle system for visual feedback:
// Bar start -> tween chain on a central logothis.detector.onBarStart.add(() => { this.app.tweens.create(this.logo.scale).to({ x: 1.3, y: 1.3 }, 0.15).easing(Ease.cubicOut).yoyo().repeat(1).start();});// Mid-band -> particle tint cyclingthis.detector.onBeat.add((info: BeatInfo) => { if (info.beatInBar === 2 || info.beatInBar === 4) { this.burst.config.tint = new Constant(info.beatInBar === 2 ? new Color(0xffa500) : new Color(0x87ceeb)); this.burst.reset(); }});
Camera shake from audio
View.shake(intensity, duration, { frequency, decay }) displaces the view center by a decaying sinusoidal offset for duration seconds. decay: true reduces the shake intensity over the duration; frequency controls the oscillation rate. Call view.shake() inside onBeat for a bass hit, or onDownbeat for a stronger effect on the first beat of each bar.
For frequency-driven shake (low-end rumble instead of discrete hits), lerp the view center by the low-band energy in update:
This produces a continuous subtle shake proportional to bass energy, suitable for engine rumble, earthquake effects, or low-frequency ambience.
Orchestration principles
One source, many listeners: One BeatDetector can feed particle bursts, camera shake, HUD flash, and tween triggers simultaneously — each subscriber acts independently.
Separate analyser for continuous data: AudioAnalyser.getLowMidHigh() gives band energies every frame without event overhead. Use it for smooth, continuous visuals like tint shifts, bar graphs, or procedural animation.
Beat detector for discrete events: onBeat/onDownbeat/onBarStart are for one-shot triggers. They fire at discrete moments and carry timing metadata (isDownbeat, beatInBar) for per-beat variation.
Keep the per-frame work bounded: A particle system at 120 particles per beat with 120 BPM spawns 240 particles/second — well within a 5000-capacity system. Check system.aliveCount occasionally to confirm you’re not overrunning capacity.
import { Application, Asset, AudioStream, Color, FixedResolutionCanvasSizing, type RenderingContext, Scene, type Seconds, Sprite, Text, Time, View, type Voice } from '@codexo/exojs';
import { AudioAnalyser } from '@codexo/exojs-audio-fx';
import { mountControls } from '@examples/runtime';
class LowBandCameraShakeScene extends Scene {
private music!: AudioStream;
private musicVoice!: Voice;
private analyser!: AudioAnalyser;
private view!: View;
private sprite!: Sprite;
private hud!: ReturnType<typeof mountControls>;
private tapPrompt!: Text;
override async load(): Promise<void> {
const app = this.app;
const { width, height } = app;
// AudioStream is a non-leaf resource kind (no seamless placeholder), so it
// is loaded directly through `Asset.type('music', ...)` and awaited rather
// than fetched synchronously via `get()`.
this.music = await this.loader.load(Asset.type('music', assets.demo.audio.musicLoop));
this.analyser = new AudioAnalyser({ fftSize: 1024, source: app.audio.music });
this.view = new View(width / 2, height / 2, width, height);
this.sprite = new Sprite(this.loader.get(assets.demo.textures.shipA))
.setAnchor(0.5)
.setScale(3)
.setPosition(width / 2, height / 2);
this.hud = mountControls({
title: 'Low Band Camera Shake',
controls: [{ keys: 'Audio', action: 'low-band energy → shake' }],
status: 'Listening…',
hint: 'Shake amplitude tracks bass energy only — in silence the camera is perfectly still.',
});
// Shown while the browser still blocks audio (`app.audio.locked`); the
// first click or keypress unlocks it and the queued music starts.
this.tapPrompt = new Text('Click or press any key to start the music', { fillColor: Color.white, fontSize: 22, align: 'center' })
.setAnchor(0.5, 0.5)
.setPosition(width / 2, height - 64);
// Core defers playback until the AudioContext unlocks on the first
// gesture, then starts automatically - play() returns the Voice now.
this.musicVoice = app.audio.play(this.music, { loop: true, volume: 0.8 });
}
override update(delta: Seconds): void {
const low = this.analyser.getBandEnergy(20, 180);
// No constant floor: amplitude is purely low-band energy, so a quiet
// passage produces zero shake. A small deadzone keeps faint noise still.
const amplitude = low > 0.04 ? low * 28 : 0;
this.view.shake(amplitude, Time.seconds(0.09), { decay: true, frequency: 22 });
// Advance the shake oscillation (the View only animates when updated).
this.view.update(delta * 1000);
if (this.musicVoice) {
this.hud.setStatus(`bass ${(low * 100) | 0}%`);
}
}
override draw(context: RenderingContext): void {
const app = this.app;
context.backend.setView(this.view);
context.render(this.sprite);
context.backend.setView(null);
if (app.audio.locked) {
context.render(this.tapPrompt);
}
}
}
const app = new Application({
scenes: { LowBandCameraShakeScene },
canvas: {
width: 1280,
height: 720,
mount: document.body,
sizing: new FixedResolutionCanvasSizing(),
},
clearColor: new Color(22, 24, 34),
});
await app.start(LowBandCameraShakeScene);
Bass-frequency energy drives continuous camera shake — low-end rumble as visual motion.
Where to go next
The next recipe, Game feel, covers general feedback techniques — damage flashes, screen shake, audio cues, and tween-driven response that make interaction feel responsive.