dancer.js

dancer.js is a high-level audio API, usable with both Mozilla's Audio Data API and Webkit's Web Audio API with flash fallback, designed to make sweet visualizations.

http://jsantell.github.com/dancer.js

v0.2.1 (6/15/2012)

Features

• Use real-time audio frequency data and map it to any arbitrary visualization
• Leverage beat detection into your visualizations
• Simple API to time callbacks and events to any section of a song
• Supports Web Audio (webkit), Audio Data (mozilla) and flash fallback (v9+)
• Extensible framework supporting plugins and custom behaviours

Dancer Instance Methods

Controls

All controls return this.

• play() plays the audio and begins the dance.
• stop() stops the madness.

Getters

• getTime() returns the current time.
• getWaveform() returns the waveform data array (Float32Array(1024))
• getSpectrum() returns the frequency data array (Float32Array(512)).
• getFrequency( freq [, endFreq ] ) returns the magnitude of a frequency or average over a range of frequencies.
• isLoaded() returns a boolean value for the dancer instance's song load state.
• isPlaying() returns a boolean value indicating whether the dancer instance's song is currently playing or not.

Sections

All section methods return this (CHAIN IT UP) and callbacks executed with this referencing the dancer instance.

• after( t, callback ) fires callback on every frame after time t.
• before( t, callback ) fires callback on every frame before time t.
• between( t0, t1, callback ) fires callback on every frame between time t0 and t1.
• onceAt( t, callback ) fires callback once at time t.

Bindings

Basic pub/sub to tie into the dancer instance. update and loaded are predefined events called within the framework that are published on every frame (update) and on audio file load (loaded). All callbacks executed with this referencing the dancer instance.

• bind( name, callback ) subscribes a callback of name. Can call this method several times to bind several callbacks of the same name.
• unbind( name ) unsubscribes all callbacks of name.
• trigger( name ) calls all callbacks of name.

Beat

Beats are detected when the amplitude (normalized values between 0 and 1) of a specified frequency, or the max amplitude over a range, is greater than the minimum threshold, as well as greater than the previously registered beat's amplitude, which is decreased by the decay rate per frame.

• createBeat( options ) creates a new beat instance tied to the dancer instance, with an options object passed as an argument. Options listed below.
  • frequency the frequency (element of the spectrum) to check for a spike. Can be a single frequency (number) or a range (2 element array) that uses the frequency with highest amplitude. Default: [ 0, 10 ]
  • threshold the minimum amplitude of the frequency range in order for a beat to occur. Default: 0.3
  • decay the rate that the previously registered beat's amplitude is reduced by on every frame. Default: 0.02
  • onBeat the callback to be called when a beat is detected.
  • offBeat the callback to be called when there is no beat on the current frame.

Dancer Static Methods

• addPlugin( name, fn ) registers a plugin of name with initiation function fn -- described in more detail below
• isSupported() returns a string of webaudio, audiodata or flash indicating level of support. Returns an empty string if the browser doesn't support any of the methods. Can also return null when browser does not support typed arrays.
• canPlay( type ) returns either true or false indicating whether the browser supports playing back audio of type type, which can be a string of 'mp3', 'ogg', 'wav', or 'aac'.
• setOptions( options ) takes a set of key-value pairs in an object for options. Options below.

Dancer Options

• flashSWF The path to soundmanager2.swf. Required for flash fallback.
• flashJS The path to soundmanager2.js. Required for flash fallback.

Dancer Constructor

new Dancer( source, [ codecs ] ) returns a new Dancer instance -- takes a string of source as a path to the audio file. Optionally, you may pass in array of codec extensions of the form [ 'mp3', 'ogg' ], where the first supported codec is used and appended to source in the form source + '.' + supportedCodec.

Beat Instance Methods

These methods can be called on a beat instance to turn on and off the registered callbacks

• on() turns on the beat instance's callbacks and detections
• off() turns off the beat instance's callbacks and detections

Example

For simple examples, check out the examples/ folder -- both the FFT and waveform examples are straight forward, leveraging the corresponding plugins for visualizations.

  // To enable flash fallback, specify the paths for the flashSWF and flashJS
  Dancer.setOptions({
    flashJS  : '../../lib/soundmanager2.js',
    flashSWF : '../../lib/soundmanager2.swf'
  });

  var
    dancer = new Dancer( "sickjams.ogg" ),
    beat = dancer.createBeat({
      onBeat: function ( mag ) {
        console.log('Beat!');
      },
      offBeat: function ( mag ) {
        console.log('no beat :(');
      }
    });

  // Let's turn this beat on right away
  beat.on();

  dancer.onceAt( 10, function() {
    // Let's set up some things once at 10 seconds
  }).between( 10, 60, function() {
    // After 10s, let's do something on every frame for the first minute
  }).after( 60, function() {
    // After 60s, let's get this real and map a frequency to an object's y position
    // Note that the instance of dancer is bound to "this"
    object.y = this.getFrequency( 400 );
  }).onceAt( 120, function() {
    // After 120s, we'll turn the beat off as another object's y position is still being mapped from the previous "after" method
    beat.off();
  });

  dancer.play();

Requirements

HTML5 Playback with Web Audio or Audio Data Chrome and Firefox are both supported out of the box -- other browsers will need to leverage the flash fallback until either of these APIs are implemented.

To enable flash You must set Dancer's defaults for flashSWF with the path to the soundmanager2.swf and flashJS to the path to soundmanager2.js, both found in lib/. Flash player 9 is required, and you must provide an mp3 option. Waveform data in Flash is a 1024 Float32Array, but only the first 512 elements have values due to flash's computeSpectrum method.

Uint32Array and Float32Array are required Include a shim if you'd like to support browsers that do not have these typed arrays.

Dependencies

• dsp.js - A subset of dsp.js (fft) is used for Fast Fourier Transformations ( Included in packaged Dancer )
• flash_detect - flash detect is used for immediate flash detection ( Included in packaged Dancer )
• soundmanager2 - soundmanager2 is used for flash fallback ( found in lib/, asynchronously loaded )

Extending/Plugins

You can extend the Dancer prototype by calling the static method addPlugin( name, fn ), which extends the Dancer prototype. A Dancer instance then can call the function provided in its context and subscribe to a preexisting event like update, or make your own. Look in the plugins/ directory for examples.

Development

This project uses smoosh to build and jasmine for testing. A CLI for testing would be awesome, but Mozilla and WebKit implementations differ greatly -- go to spec/index.html in Mozilla/WebKit browsers to test. All tests should pass in Chrome, Firefox, Opera and Safari, although they can get weird. Should consistently pass.

Change Logs

v0.2.1 (6/16/2012)

• Added getWaveform() method and a corresponding visualization for waveforms

v0.2.0 (6/14/2012)

• Added flash support with soundmanager2 -- flash_detect now included in build
• Added static methods isSupported, canPlay and setOptions
• Added multiple audio codecs support (#7)
• Added a new simple FFT examples, both examples having feature detection and controls (#10)
• Fixed several Webkit bugs (#4, #8)

v0.1.0 (6/3/2012)

• Initial Web Audio/ Audio Data release