Low Frequency Operators
The lfo
library provides a simple and efficient graph-based javascript API primarily designed for the processing and analysis of signal and event data streams such as audio, audio descriptors and motion sensor data.
A graph
of lfo
modules can process data streams online (i.e. processing data from audio inputs or event sources) as well as offline (e.g. iterating over recorded data) depending on the used source
and sink
modules. Many of the operator modules provided by the library (e.g. filters, signal statistics) can also be used for processing data using an alternative API without the lfo
formalism.
The library exposes two main entry points, waves-lfo/client
and waves-lfo/node
, that respectively provide modules to be used in a browser or in a Node.js environment. This architecture allows for adapting the library to virtually any context and platform by only providing adequate source
and sink
modules.
The library provides three namespaces:
source
modules produce streams and propagate their properties (i.e.frameRate
,frameType
, etc.) through the graph.sink
modules are endpoints of the graph such as recorders and visualizers.operator
modules process an incoming stream and propagate the resulting stream to the next operators.
A graph
is a combination of at least a source
and a sink
with any number of operator
modules in between:
Documentation
http://wavesjs.github.io/waves-lfo
Important: in the documentation, all nodes in the common
and core
namespaces are platform independent and can be used client-side as well as in node (aka from entry points waves-lfo/client
and waves-lfo/node
).
Usage
Install
$ npm install [--save] waves-lfo
Import the library
To use the library in browser or in node, import the corresponding entry point. These different access allow to use sources and sinks specific to the platform:
// in browser
import * as lfo from 'waves-lfo/client';
// in node
import * as lfo from 'waves-lfo/node';
To create a script that targets any possible environnements (i.e. if no platform specific source
or sink
is used), the common
entry point can be used:
import * as lfo from 'waves-lfo/common';
Create a graph
import * as lfo from 'waves-lfo/common';
const eventIn = new lfo.source.EventIn({
frameType: 'vector'
frameSize: 3,
frameRate: 1,
});
const rms = new lfo.operator.Rms();
const logger = new lfo.sink.Logger({ data: true });
eventIn.connect(rms);
rms.connect(logger);
eventIn.start();
eventIn.process(0, [2, 1, 3]);
// > [2.16024689947]
Terminology
The lfo
modules produce and consume data streams composed of frames. This is the terminology used by the library.
- stream - a succession of frames described by a set of stream parameters
- frame - an element of a stream that associates a
data
element with atime
and optionalmetadata
- data - a generic term to designate the payload of a frame which can be a
vector
, asignal
or ascalar
- vector - an array of values that correspond to different dimensions such as [x y z] or [mean stddev min max]
- signal - an array of time-domain values corresponding to a fragment of a signal
- scalar - a single value that can be arbitrarily considered as a one-dimensional
vector
or one sample of asignal
- time - a timestamp associating each frame in a stream to a point in time regarding an arbitrary reference common to all modules in a graph
- metadata - additional description data associated to a frame by a module
- stream parameters - parameters defining the nature of a stream at the output of a module (attribute
streamParams
) that may depend on the stream parameters of the incoming streamframeSize
: number of values in the frame dataframeRate
: number of frames per seconds for regularly sampled streams,0
otherwiseframeType
: type of frame data (vector
,signal
orscalar
)sourceSampleRate
: number of frames per seconds output by the graph'ssource
sourceSampleCount
: number of consecutive discrete time values contained in the data frame output by the graph'ssource
(e.g. the signal block size of an audio source or 1 for streams of sensor data vectors)description
: an array of strings describing the output dimensions ofvector
orscalar
frames (e.g.['mean', 'stddev', 'min', 'max']
)
Available nodes and examples
common
core:
operators:
- Biquad - example 1, example 2
- Dct
- Fft
- Magnitude
- MeanStddev
- Mel
- Mfcc - example
- MinMax - example
- MovingAverage - example (graphical)
- MovingMedian
- OnOff - example
- Rms
- Segmenter - example
- Select
- Slicer
- Yin - example
sources:
sinks:
client only
sources:
sinks:
- BaseDisplay
- BpfDisplay - example
- MarkerDisplay - example
- SignalDisplay - example
- SpectrumDisplay - example
- TraceDisplay - example
- VuMeterDisplay - example
- WaveformDisplay - example
node only
sources:
sinks:
Standalone usage
Most of the operators can be used in a standalone
mode which allow to consume the implemented algorithm without the burden of creating a whole graph.
import * as lfo from 'waves-lfo/common';
const rms = new lfo.operator.Rms();
rms.initStream({ frameType: 'signal', frameSize: 1000 });
const results = rms.inputSignal([...values]);
Implementation of an lfo
operator
To create a new operator, the BaseLfo
must be extended, the class is available in the waves-lfo/core
entry point.
import { BaseLfo } from 'waves-lfo/core';
// define class parameters
const parameters = {
factor: {
type: 'integer',
default: 1,
},
};
class Multiplier extends BaseLfo {
constructor(options = {}) {
// set the parameters and options of the node
super(parameters, options);
}
// allow the node to handle incoming `vector` frames
processVector(frame) {
const frameSize = this.streamParams.frameSize;
const factor = this.params.get('factor');
// transfer data from `frame` (output of previous node)
// to the current node's frame, data from the incoming frame
// should never be modified
for (let i = 0; i < frameSize; i++)
this.frame.data[i] = frame.data[i] * factor;
}
}
const multiplier = new Multiplier({ factor: 4 });
Creating plugins
To contribute and distribute a new lfo
module, a good pratice is that the module should not directly depend on lfo
(i.e. wavesjs/waves-lfo
shouldn't be found in the package.json
of the module or as a devDependency
). The final application should be responsible for importing the \lfo library as well as the plugin.
This practice should allow to create an ecosystem of module that, in the final application, would all point to the same instance of lfo
and thus enforce inter-compatibilies.
If need, all entry points expose a VERSION
property that allows a plugin to test the loaded lfo
version.
A example of plugin can be found at https://github.com/Ircam-RnD/xmm-lfo
lfo
and PiPo
The lfo
library is based on the same concepts and very similar formalisms as PiPo.
However, the APIs of lfo
and PiPo
defer in many details due to the very different constraints of the Javascript and C/C++ development and runtime environments.
Credits and License
The lfo
library has been developed at Ircam – Centre Pompidou and is released under the BSD-3-Clause license.
The formalisms and API of the library has been designed in the framework of the EU H2020 project Rapid-Mix by Norbert Schnell and Benjamin Matuszewski.
The library has been developed by Benjamin Matuszewski in the framework of the CoSiMa research project funded by the French National Research Agency (ANR).
A first version of the library has been developed by Victor Saiz in the framework of the WAVE ANR research project coordinated by Samuel Goldszmidt.