This Web SDK is currently in BETA, and is subject to change. Some features may not function correctly, or may be removed in future versions of the software.

Introduction

Features

• Spatial Audio
• Metadata
• Near Field Effects
• High Dynamic Range Peak-Limiter
• Noise Gate
• Compatible with WebAudio

Spatialize Audio

High Fidelity's AudioNodes library defines a virtual audio space for 2D or 3D applications. Within the virtual audio space are two types of endpoints: (1) sound sources and (2) listeners. Each of these endpoints has a position within the virtual audio space. The audio from each sound source is fed into High Fidelity’s Head-Related Transfer Function (HRTF) algorithm so that each listener hears the sound as if it was generated from the sound source's relative direction and distance from the listener. The web application can set or change the positions of sound sources and listeners, as well as the direction the listener is facing. Changes to sound source positions can be synchronized across the network to other instances of the application through the use of metadata.

Metadata

The library supports tight coupling and a high update rate for a sound source's positional data. If metadata is enabled, the audio stream from a local sound source will be modified to include the position of the source. Other instances of the application can extract the position of the source along with audio data. This enables near real-time position updates and avoids any distracting desynchronization between the perceived auditory location of a sound source and the visual representation of the sound source. For example, if a sound is coming from an avatar that is speaking as it is walking, the sound will move in sync with the avatar in motion.

Near Field Effects

When a source and listener are within 1 meter of each other, an alternate "near field" HRTF is used. This includes a head occlusion algorithm and can help deliver ASMR-like effects. Nearby audio sources sound convincingly close.

High Dynamic Range Peak-Limiter

All audio within this library is handled as 32-bit floats, which allows for an unlimited dynamic range. This is important in applications where more than one source may be producing sound at the same time. The more sound sources, the more critical this feature is to simulating sound in the natural world. Unlike conventional handling of mixed audio, the High Dynamic Range Peak-Limiter eliminates any perceptible audio clipping, distortion, and other artifacts.

Internally, WebRTC uses a 16-bit audio pipeline. Before audio is sent across the network or delivered to an output device, the Peak Limiter applies a specialized look-ahead function, which results in a high-quality final compression of the audio pipeline.

When it's time to deliver audio from this library to the application, the higher dynamic-range data must be brought within the expected range, which is what the Peak Limiter does. It works by looking ahead several frames and adjusting the overall volume of the output stream, caps the loudness, and can handle huge overloads.

Noise Gate

The Noise Gate node has an adjustable threshold (or can be disabled). It can be placed between a microphone's output stream and the TransportManager's sender. It works by not sending any audio that falls below the set loudness threshold. The Noise Gate eliminates most background noise, frees network resources, and reduces CPU load. Louder sounds, such as a speaker’s voice, are transmitted clearly.

Compatible with WebAudio

The nodes in this package are compatible with standard WebAudio nodes. They can be arranged as needed to take advantage of WebAudio's flexible modular design. If all audio sources use 48 kHz, resampling is avoided and the best audio quality will be delivered.

Installation

To include the High Fidelity Web Audio Nodes library in your project, use npm:

npm install hifi-audio-nodes@latest --registry https://npm.highfidelity.com/