Mouse and pointer

ExoJS unifies mouse, touch, and pen input under a single pointer system built on the browser’s Pointer Events API. Every pointer is represented by a Pointer instance with a canvas-local design-space position, button state, pressure, tilt, and a slot index. The InputSystem tracks up to 16 simultaneous pointers — enough for ten-finger multitouch, plus a mouse, plus a pen — with no additional configuration.

Pointer signals

The application’s input system emits these pointer signals:

Each signal’s callback receives the Pointer instance:

import { type Application, Sprite } from '@codexo/exojs';

declare const application: Application;
declare const crosshair: Sprite;
declare function placeBuilding(x: number, y: number): void;

application.input.onPointerMove.add(pointer => {
    crosshair.position.set(pointer.x, pointer.y);
});

application.input.onPointerTap.add(pointer => {
    placeBuilding(pointer.x, pointer.y);
});

The pointer’s position is in canvas-local design space — (0, 0) is the top-left of the canvas, regardless of CSS size, device pixel ratio, or page scroll.

Primary pointer convenience

Pointer.X, Pointer.Y, and Pointer.Active are channel constants that always reference the primary pointer (normally the mouse, or the first finger to touch). These can be used with the binding API just like keyboard channels:

this.inputs.onActive(Pointer.Active, () => {
  this.isPointing = true;
});
this.inputs.onStop(Pointer.Active, () => {
  this.isPointing = false;
});

The signal-style API (app.input.onPointerMove) is typically more ergonomic for pointer input than the binding API, since pointer data includes position, pressure, and tilt — not just an active/inactive boolean.

Multi-touch

Each pointer gets a slot index (0–15). Per-slot channel constants let you track individual fingers for multi-touch:

class TouchScene extends Scene {
  private touchX = 0;

  override init(): void {
    // Slot 1 (second finger)
    this.inputs.onActive(Pointer.Slot1X, value => {
      this.touchX = value * this.app.width;
    });
  }
}

For most multi-touch use cases, tracking pointer instances by ID via the signal API is simpler:

override init(): void {
  this.pointers = new Map();

  this.app.input.onPointerDown.add(p => {
    this.pointers.set(p.id, { x: p.x, y: p.y });
  });

  this.app.input.onPointerMove.add(p => {
    if (this.pointers.has(p.id)) {
      this.pointers.set(p.id, { x: p.x, y: p.y });
    }
  });

  this.app.input.onPointerUp.add(p => {
    this.pointers.delete(p.id);
  });

  this.app.input.onPointerCancel.add(p => {
    this.pointers.delete(p.id);
  });
}

Gesture recognition

The input system includes built-in gesture recognizers for common multi-touch patterns:

import { type Application, Container } from '@codexo/exojs';

declare const application: Application;
declare const camera: { zoom: number };
declare const map: Container;
declare function showContextMenu(x: number, y: number): void;

application.input.onPinch.add((scale, centerX, centerY) => {
    // scale > 1 = spreading fingers, scale < 1 = pinching.
    // centerX/centerY are the midpoint between the two pointers — zoom
    // towards it to keep the pinched spot under the fingers.
    camera.zoom *= scale;
    map.setPosition(centerX, centerY);
});

application.input.onRotate.add(angleDelta => {
    // angleDelta in radians — positive = clockwise rotation
    map.rotate(angleDelta);
});

application.input.onLongPress.add(pointer => {
    // pointer held without significant movement for >= 500 ms
    showContextMenu(pointer.x, pointer.y);
});

These are high-level events built on top of the raw pointer signals. They track two pointers for pinch/rotate and a single pointer with a 500 ms timer for long-press. No additional setup is required beyond subscribing.

Pointer-to-world coordinates

Pointer positions are in design space (0..app.width × 0..app.height), independent of device pixel ratio or how the canvas is displayed. To map them into scene world coordinates — for placing objects, selecting units, or aiming — pass them through the active view’s screenToWorld, which undoes the camera’s pan / zoom / rotation:

import { Pointer, View } from '@codexo/exojs';

declare const pointer: Pointer;
declare const view: View;

const world = view.screenToWorld(pointer.x, pointer.y);
// world.x, world.y are now in scene world space

The inverse is worldToScreen(worldX, worldY), and the two round-trip. If the default centered camera is active (no pan/zoom), design space already equals world space, so screenToWorld is the identity and no mapping is needed.

For raw canvas backing-store pixels — e.g. an off-screen render target at a different pixel ratio — the four-argument form screenToWorld(x, y, canvasWidth, canvasHeight) additionally accounts for the viewport rectangle and physical pixel size.

Interactive sprites

A Sprite (or any RenderNode) can be made interactive and draggable directly:

this.sprite = new Sprite(this.loader.get('image/hero.png'));
this.sprite.setAnchor(0.5);
this.sprite.setPosition(400, 300);

this.sprite.interactive = true; // respond to pointer events
this.sprite.draggable = true; // enable drag behavior

Setting interactive = true makes the sprite participate in hit testing. Setting draggable = true lets the user drag it with the pointer — the engine handles pointer tracking and position updates.

Custom hit areas

By default a node is picked by its bounds — the axis-aligned box for an unrotated node, the true oriented box once it is rotated or skewed. That is right for a rectangular sprite and wrong for almost anything else: a circular button claims its corners, and an L-shaped panel claims the notch it does not cover. Set hitArea to a Rectangle, Circle, Ellipse or Polygon and that shape decides the pick instead:

import { Circle, Polygon, Sprite, Vector } from '@codexo/exojs';

declare const knob: Sprite;
declare const panel: Sprite;

// A round knob: the bounds corners stop responding.
knob.interactive = true;
knob.hitArea = new Circle(32, 32, 32);

// A concave L-shape - containment is an even-odd ray cast, so the notch
// between the two arms is a miss, not a hit.
panel.interactive = true;
panel.hitArea = new Polygon([new Vector(0, 0), new Vector(60, 0), new Vector(60, 60), new Vector(120, 60), new Vector(120, 120), new Vector(0, 120)]);

The shape is in the node’s local space, the same space its untransformed geometry lives in. The pointer position is mapped back through the inverse of the node’s global transform before the test, so the region rotates, scales and moves with the node and never needs recomputing — the opposite of cullArea, which is world-space and does go stale when its node moves.

Two things to keep in mind:

• hitArea affects picking only. Bounds, culling and rendering ignore it, so shrinking the pick region does not shrink what is drawn or what the renderer considers on-screen.
• Because bounds are what the interaction system uses to find candidates, a hit area is reliable where it overlaps the node’s bounds. Use it to shrink or reshape a pick region rather than to grow one past the node’s own extent.

Setting hitArea back to null restores the bounds test.

Pointer properties

The Pointer instance passed to every signal callback carries the properties you typically need:

import { Pointer } from '@codexo/exojs';

declare const pointer: Pointer;

pointer.x, pointer.y   // design-space pixel position (0..app.width, 0..app.height)
pointer.position       // Vector with current x and y
pointer.buttons        // bitmask: 1=left, 2=right, 4=middle
pointer.isPrimary      // true for mouse or first finger to touch
pointer.type           // 'mouse', 'touch', or 'pen'

For hover-vs-drag discrimination, check pointer.down in onPointerMove — it fires for every movement, including when no button is pressed. The API reference documents the full property set including pressure, tiltX/tiltY (pen tilt), pressPosition/releasePosition, maxDistanceFromPress (the press excursion tap/swipe logic is built on), and currentState.

Every pointer also carries a per-frame snapshot that stays stable for the whole frame: pressed, moved, released, cancelled, entered and exited describe what happened since the last frame boundary, and delta spans that same interval. Several platform events of one kind collapse into a single frame phase, but no phase is lost — a press and release that both land between two frames set pressed and released on the same frame.

When to use which

• Signal API (app.input.onPointerMove etc.) for raw pointer data — position tracking, custom gesture detection, drawing apps.
• Binding API (this.inputs.onActive(PointerButton.Primary) etc.) for simple button/binary state — holding mouse button to fire, right-click context menus.
• Interactive sprites (interactive/draggable) for drag-and-drop and click-on-object behavior without manual hit testing.
• Gesture signals (onPinch/onRotate/onLongPress) for map zoom/rotate and touch-hold menus.

Examples

import { Application, Color, FixedResolutionCanvasSizing, Graphics, type RenderingContext, Scene, Sprite } from '@codexo/exojs';
import { mountControls } from '@examples/runtime';

// Everything the pointer pipeline reports, surfaced at once:
//   - live position (onPointerMove)
//   - pressed-button bitmask (Pointer.buttons: 1=left, 2=right, 4=middle)
//   - frame-to-frame movement delta
//   - a click counter (onPointerTap fires on a press+release without a drag)
//   - a draggable sprite (the engine's built-in drag on an interactive node)
class MouseAndPointerScene extends Scene {
  private ship!: Sprite;
  private crosshair!: Graphics;
  private pointer = { x: 400, y: 300 };
  private previous = { x: 400, y: 300 };
  private deltaX = 0;
  private deltaY = 0;
  private buttons = 0;
  private clicks = 0;
  private hud!: ReturnType<typeof mountControls>;

  override init(): void {
    const app = this.app;
    const { width, height } = app;

    this.pointer = { x: width / 2, y: height / 2 };
    this.previous = { x: width / 2, y: height / 2 };

    this.ship = new Sprite(this.loader.get('image/ship-a.png')).setAnchor(0.5).setPosition(width / 2, height / 2);
    this.ship.interactive = true;
    this.ship.draggable = true;
    this.crosshair = new Graphics();

    app.input.onPointerMove.add(pointer => {
      this.pointer.x = pointer.x;
      this.pointer.y = pointer.y;
      this.buttons = pointer.buttons;
    });
    app.input.onPointerDown.add(pointer => {
      this.buttons = pointer.buttons;
    });
    app.input.onPointerUp.add(pointer => {
      this.buttons = pointer.buttons;
    });
    app.input.onPointerTap.add(() => {
      this.clicks++;
    });

    this.hud = mountControls({
      title: 'Mouse and Pointer',
      controls: [
        { keys: 'Move', action: 'track position + delta' },
        { keys: 'Click', action: 'count taps' },
        { keys: 'Drag', action: 'move the ship sprite' },
      ],
      status: '',
      hint: 'Drag the ship to move it; the crosshair follows the cursor.',
    });
  }

  private buttonLabel(): string {
    const held = [this.buttons & 1 && 'Left', this.buttons & 2 && 'Right', this.buttons & 4 && 'Middle'].filter(Boolean);

    return held.length ? held.join(' + ') : 'none';
  }

  override update(): void {
    this.deltaX = this.pointer.x - this.previous.x;
    this.deltaY = this.pointer.y - this.previous.y;
    this.previous.x = this.pointer.x;
    this.previous.y = this.pointer.y;

    this.hud.setStatus(
      `x ${Math.round(this.pointer.x)}, y ${Math.round(this.pointer.y)} · Δ ${this.deltaX >= 0 ? '+' : ''}${this.deltaX.toFixed(0)}, ${this.deltaY >= 0 ? '+' : ''}${this.deltaY.toFixed(0)} · buttons: ${this.buttonLabel()} · clicks: ${this.clicks}`,
    );
  }

  override draw(context: RenderingContext): void {
    context.render(this.ship);

    this.crosshair.clear();
    this.crosshair.lineWidth = 2;
    this.crosshair.lineColor = new Color(255, 220, 80);
    this.crosshair.drawLine(this.pointer.x - 12, this.pointer.y, this.pointer.x + 12, this.pointer.y);
    this.crosshair.drawLine(this.pointer.x, this.pointer.y - 12, this.pointer.x, this.pointer.y + 12);
    context.render(this.crosshair);
  }
}

const app = new Application({
  scenes: { MouseAndPointerScene },
  canvas: {
    width: 1280,
    height: 720,
    mount: document.body,
    sizing: new FixedResolutionCanvasSizing(),
  },
  clearColor: new Color(10, 12, 20),
  loader: {
    basePath: 'assets/',
  },
});

await app.start(MouseAndPointerScene);

A draggable sprite with a crosshair that follows the primary pointer.

import { Application, Color, FixedResolutionCanvasSizing, Graphics, type RenderingContext, Scene, type Seconds, View } from '@codexo/exojs';
import { mountControls } from '@examples/runtime';

// The camera continuously pans (a slow figure-eight) and breathes its zoom, so
// the same design-space pixel maps to a moving world point every frame.
// `screenToWorld(x, y)` undoes the camera transform - pointer coordinates are
// already in design space (`0..app.width`) - so we never hand-roll the inverse
// projection. Tap to drop a marker in *world* space; it stays pinned to the
// world as the camera moves over it.
class PointerToWorldScene extends Scene {
  private view!: View;
  private grid!: Graphics;
  private markers!: Graphics;
  private cursor = { x: 0, y: 0 };
  private world = { x: 0, y: 0 };
  private markerWorld: { x: number; y: number }[] = [];
  private elapsed = 0;
  private userZoom = 1;
  private hud!: ReturnType<typeof mountControls>;

  override init(): void {
    const app = this.app;
    const width = app.width;
    const height = app.height;

    this.view = new View(width / 2, height / 2, width, height);
    this.grid = new Graphics();
    this.markers = new Graphics();
    this.cursor = { x: width / 2, y: height / 2 };

    // Static world-space grid so the camera motion is visible against it.
    // Extends well beyond the viewport so the panning camera never runs off it.
    this.grid.lineWidth = 1;
    this.grid.lineColor = new Color(60, 66, 82);

    for (let x = -640; x <= width + 640; x += 80) {
      this.grid.drawLine(x, -480, x, height + 480);
    }

    for (let y = -480; y <= height + 480; y += 80) {
      this.grid.drawLine(-640, y, width + 640, y);
    }

    app.input.onPointerMove.add(pointer => {
      this.cursor.x = pointer.x;
      this.cursor.y = pointer.y;
    });

    app.input.onPointerTap.add(pointer => {
      const world = this.view.screenToWorld(pointer.x, pointer.y);

      this.markerWorld.push({ x: world.x, y: world.y });
    });

    // Scroll to nudge a user-controlled zoom that the automatic breath multiplies.
    app.input.onMouseWheel.add((_deltaX, deltaY) => {
      this.userZoom = Math.max(0.4, Math.min(3, this.userZoom + (deltaY < 0 ? 0.1 : -0.1)));
    });

    this.hud = mountControls({
      title: 'Pointer to World',
      controls: [
        { keys: 'Move', action: 'read world coordinate' },
        { keys: 'Click', action: 'drop a world-pinned marker' },
        { keys: 'Wheel', action: 'zoom' },
      ],
      status: '',
      hint: 'The camera pans and zooms on its own — markers stay fixed in the world.',
    });
  }

  override update(delta: Seconds): void {
    const app = this.app;
    const width = app.width;
    const height = app.height;

    this.elapsed += delta;

    // Slow figure-eight pan plus a gentle zoom breath.
    const centerX = width / 2 + Math.sin(this.elapsed * 0.5) * 220;
    const centerY = height / 2 + Math.sin(this.elapsed * 1.0) * 140;

    this.view.setCenter(centerX, centerY);
    this.view.setZoom(this.userZoom * (1 + Math.sin(this.elapsed * 0.35) * 0.25));
    this.view.update(delta * 1000);

    // Live world coordinate under the cursor - recomputed every frame because
    // the mapping changes as the camera moves.
    this.world = this.view.screenToWorld(this.cursor.x, this.cursor.y);

    this.hud.setStatus(
      `Screen ${Math.round(this.cursor.x)}, ${Math.round(this.cursor.y)} → World ${this.world.x.toFixed(0)}, ${this.world.y.toFixed(0)} · zoom ${this.view.zoomLevel.toFixed(2)}`,
    );
  }

  override draw(context: RenderingContext): void {
    context.backend.setView(this.view);

    context.render(this.grid);

    // Rebuild markers each frame in their fixed world positions.
    this.markers.clear();
    this.markers.fillColor = new Color(255, 160, 80);

    for (const marker of this.markerWorld) {
      this.markers.drawCircle(marker.x, marker.y, 7);
    }

    // Highlight the live cursor→world point.
    this.markers.fillColor = new Color(120, 230, 255);
    this.markers.drawCircle(this.world.x, this.world.y, 5);

    context.render(this.markers);
    context.backend.setView(null);
  }
}

const app = new Application({
  scenes: { PointerToWorldScene },
  canvas: {
    width: 1280,
    height: 720,
    mount: document.body,
    sizing: new FixedResolutionCanvasSizing(),
  },
  clearColor: new Color(10, 12, 20),
  loader: {
    basePath: 'assets/',
  },
});

await app.start(PointerToWorldScene);

Tapping on a zoomed, pannable grid to place markers at the correct world coordinates.

Where to go next

The next chapter, Gamepad, covers the four-slot gamepad system — button and axis listeners, vibration, slot strategies, and per-pad connection lifecycle.