Low Frequency Operators

The lfo library provides a simple and efficient graph-based javascript API primarily designed for the processing and analysis of signal and event data streams such as audio, audio descriptors and motion sensor data.

A graph of lfo modules can process data streams online (i.e. processing data from audio inputs or event sources) as well as offline (e.g. iterating over recorded data) depending on the used source and sink modules. Many of the operator modules provided by the library (e.g. filters, signal statistics) can also be used for processing data using an alternative API without the lfo formalism.

The library exposes two main entry points, waves-lfo/client and waves-lfo/node, that respectively provide modules to be used in a browser or in a Node.js environment. This architecture allows for adapting the library to virtually any context and platform by only providing adequate source and sink modules.

The library provides three namespaces:

• source modules produce streams and propagate their properties (i.e. frameRate, frameType, etc.) through the graph.
• sink modules are endpoints of the graph such as recorders and visualizers.
• operator modules process an incoming stream and propagate the resulting stream to the next operators.

A graph is a combination of at least a source and a sink with any number of operator modules in between:

Documentation

http://wavesjs.github.io/waves-lfo

Important: in the documentation, all nodes in the common and core namespaces are platform independent and can be used client-side as well as in node (aka from entry points waves-lfo/client and waves-lfo/node).

Usage

Install

$ npm install [--save] waves-lfo

Import the library

To use the library in browser or in node, import the corresponding entry point. These different access allow to use sources and sinks specific to the platform:

// in browser
import * as lfo from 'waves-lfo/client';

// in node
import * as lfo from 'waves-lfo/node';

To create a script that targets any possible environnements (i.e. if no platform specific source or sink is used), the common entry point can be used:

import * as lfo from 'waves-lfo/common';

Create a graph

import * as lfo from 'waves-lfo/common';

const eventIn = new lfo.source.EventIn({
  frameType: 'vector'
  frameSize: 3,
  frameRate: 1,
});

const rms = new lfo.operator.Rms();
const logger = new lfo.sink.Logger({ data: true });

eventIn.connect(rms);
rms.connect(logger);

eventIn.start();
eventIn.process(0, [2, 1, 3]);
// > [2.16024689947]

Terminology

The lfo modules produce and consume data streams composed of frames. This is the terminology used by the library.

• stream - a succession of frames described by a set of stream parameters
• frame - an element of a stream that associates a data element with a time and optional metadata
• data - a generic term to designate the payload of a frame which can be a vector, a signal or a scalar
• vector - an array of values that correspond to different dimensions such as [x y z] or [mean stddev min max]
• signal - an array of time-domain values corresponding to a fragment of a signal
• scalar - a single value that can be arbitrarily considered as a one-dimensional vector or one sample of a signal
• time - a timestamp associating each frame in a stream to a point in time regarding an arbitrary reference common to all modules in a graph
• metadata - additional description data associated to a frame by a module
• stream parameters - parameters defining the nature of a stream at the output of a module (attribute streamParams) that may depend on the stream parameters of the incoming stream
  • frameSize: number of values in the frame data
  • frameRate: number of frames per seconds for regularly sampled streams, 0 otherwise
  • frameType: type of frame data (vector, signal or scalar)
  • sourceSampleRate: number of frames per seconds output by the graph's source
  • sourceSampleCount: number of consecutive discrete time values contained in the data frame output by the graph's source (e.g. the signal block size of an audio source or 1 for streams of sensor data vectors)
  • description: an array of strings describing the output dimensions of vector or scalar frames (e.g. ['mean', 'stddev', 'min', 'max'])

Available nodes and examples

common

core:

• BaseLfo

operators:

• Biquad - example 1, example 2
• Dct
• Fft
• Magnitude
• MeanStddev
• Mel
• Mfcc - example
• MinMax - example
• MovingAverage - example (graphical)
• MovingMedian
• OnOff - example
• Rms
• Segmenter - example
• Select
• Slicer
• Yin - example

sources:

• EventIn - example

sinks:

• Bridge - example 1, example 2
• DataRecorder - example
• Logger
• SignalRecorder - example

client only

sources:

• AudioInBuffer
• AudioInNode - example

sinks:

• BaseDisplay
• BpfDisplay - example
• MarkerDisplay - example
• SignalDisplay - example
• SpectrumDisplay - example
• TraceDisplay - example
• VuMeterDisplay - example
• WaveformDisplay - example

node only

sources:

• AudioInFile

sinks:

• DataToFile

Standalone usage

Most of the operators can be used in a standalone mode which allow to consume the implemented algorithm without the burden of creating a whole graph.

import * as lfo from 'waves-lfo/common';

const rms = new lfo.operator.Rms();
rms.initStream({ frameType: 'signal', frameSize: 1000 });

const results = rms.inputSignal([...values]);

Implementation of an lfo operator

To create a new operator, the BaseLfo must be extended, the class is available in the waves-lfo/core entry point.

import { BaseLfo } from 'waves-lfo/core';
// define class parameters
const parameters = {
  factor: {
    type: 'integer',
    default: 1,
  },
};

class Multiplier extends BaseLfo {
  constructor(options = {}) {
    // set the parameters and options of the node
    super(parameters, options);
  }

  // allow the node to handle incoming `vector` frames
  processVector(frame) {
    const frameSize = this.streamParams.frameSize;
    const factor = this.params.get('factor');

    // transfer data from `frame` (output of previous node)
    // to the current node's frame, data from the incoming frame 
    // should never be modified
    for (let i = 0; i < frameSize; i++)
      this.frame.data[i] = frame.data[i] * factor;
  }
}

const multiplier = new Multiplier({ factor: 4 });

Creating plugins

To contribute and distribute a new lfo module, a good pratice is that the module should not directly depend on lfo (i.e. wavesjs/waves-lfo shouldn't be found in the package.json of the module or as a devDependency). The final application should be responsible for importing the \lfo library as well as the plugin.

This practice should allow to create an ecosystem of module that, in the final application, would all point to the same instance of lfo and thus enforce inter-compatibilies.

If need, all entry points expose a VERSION property that allows a plugin to test the loaded lfo version.

A example of plugin can be found at https://github.com/Ircam-RnD/xmm-lfo

lfo and PiPo

The lfo library is based on the same concepts and very similar formalisms as PiPo. However, the APIs of lfo and PiPo defer in many details due to the very different constraints of the Javascript and C/C++ development and runtime environments.

Credits and License

The lfo library has been developed at Ircam – Centre Pompidou and is released under the BSD-3-Clause license.

The formalisms and API of the library has been designed in the framework of the EU H2020 project Rapid-Mix by Norbert Schnell and Benjamin Matuszewski.

The library has been developed by Benjamin Matuszewski in the framework of the CoSiMa research project funded by the French National Research Agency (ANR).

A first version of the library has been developed by Victor Saiz in the framework of the WAVE ANR research project coordinated by Samuel Goldszmidt.