WebAudio

The Web Audio API provides a very versatile system for controlling Audio on the Web, allowing developers to select Audio sources, add effects to Audio, visualize Audio, add spatial effects (such as panning), and more.

1. AudioContext

Constructor: There are three optional latencyHint values. The default values are generally used

  • balancedBalance audio output latency and resource consumption
  • inteactiveDefault provides minimal audio output delay preferably without interference
  • playbackContrast audio output delay, priority playback is not interrupted
const ac = new AudioContext({ latencyHint: "balanced" });
Copy the code

Return an instance of AudioContext (just to get an impression)

console.log(ac);

// audioWorklet: AudioWorklet {}
/ / baseLatency: 0.005333333333333333
/ / currentTime: 1.5946666666666667
// destination: AudioDestinationNode {maxChannelCount: 2, context: AudioContext, numberOfInputs: 1, numberOfOutputs: 0, channelCount: 2,... }
// listener: AudioListener {positionX: AudioParam, positionY: AudioParam, positionZ: AudioParam, forwardX: AudioParam forwardY: AudioParam,... }
// onstatechange: null
// sampleRate: 48000
// state: "running"
Copy the code
  • BaseLatency: Returns the number of seconds of processing delay for the AudioContext to pass the audio from the AudioDestinationNode to the audio subsystem

All other attributes are inherited from the parent class:

  • SampleRate: sampling rate
  • CurrentTime: read-only. The value starts from 0 and is incremented in seconds
  • State: indicates the current status
    • Suspended: callac.suspend()
    • Running: The system runs normally
    • Closed: the callac.close()
  • Onstatechange: listens for status change events
audioCtx.onstatechange = function () {
    console.log(audioCtx.state);
};
Copy the code
  • AudioWorklet: An audioWorklet is related to an AudioWorkletNode (TODO explained below)
  • Destination: Returns an AudioDestinationNode object representing the end node of the context, typically an audio rendering device
  • Listener: Returns an AudioListener object that can be used to spatialize 3D audio

1.1 AudioNode and destination

AudioNode interface is a general module for processing audio. In practical application, we will not directly use AudioNode, but use the subclasses of AudioNode. There are many subclasses of AudioNode

The name of the node Create a way meaning
AudioDestinationNode Destination property, which is also created by audioContext by default Represents the final node of the context, typically an audio rendering device
AudioBufferSourceNode Through audioContext. CreateBufferSource () to create Audio source
GainNode Created by audioContext.creategain () Adjust the volume

Each AudioNode code is a module that processes the sound (such as volume GainNode, audio source AudioSourceNode). After the processing is completed, it is handed over to the next AudioNode for processing. The transmission mode is implemented by Audionode1.connect (audioNode2)

Destination is a special AudioNode that represents the exit for the sound to be played after the processing is done. It’s also necessary for the AudioContext to exist, so it’s already created when you create the AudioContext, This can be obtained directly from audioContext.destination

With the understanding of audionodes above, you can use demo to implement the most basic function: use the AudioContext to play music

1.2 Demo1 Use AudioContext to play music

Added an API: audioContext. DecodeAudioData: ArrayBuffer decoding the audio into AudioBuffer

You can use Ajax or fileReader to read the rawData of a music file and turn it into an ArrayBuffer, because music files are encoded and then packaged, DecodeAudioData decodes compressed music files into AudioBuffer data.

You can restrict the range of playback by adding a range request to headers

const mp3 = "http://127.0.0.1:5500/public/tornado.mp3";
const button = document.querySelector("button");

class Demo1 {
    constructor() {
        this.audioContext = new AudioContext({ latencyHint: "balanced" });
    }

    async start() {
        const musicArrayBuffer = await this.getMp3ArrayBuffer(mp3);
        const decodedAudioData = await this.decode(musicArrayBuffer);
        this.play(decodedAudioData);
    }

    / * * *@param {AudioBuffer} decodedAudioData
     * @returns* /
    async play(decodedAudioData) {
        const sourceNode = this.audioContext.createBufferSource();
        sourceNode.buffer = decodedAudioData;
        sourceNode.connect(this.audioContext.destination);
        sourceNode.start(0);
    }

    / * * *@param {string} url
     * @returns {ArrayBuffer}* /
    async getMp3ArrayBuffer(url) {
        return fetch(url).then((r) = > r.arrayBuffer());
    }

    / * * *@param {ArrayBuffer} arrayBuffer
     * @returns {AudioBuffer}* /
    async decode(arrayBuffer) {
        return this.audioContext.decodeAudioData(arrayBuffer);
    }
}

button.onclick = () = > new Demo1().start();
Copy the code

Demo flow:

Ajax => Raw data (ArrayBuffer) => Decoded data (AudioBuffer) => AudioBufferSourceNode(mount decoded data to audio source) => Audiocontext. destination is sent to the hardware to playCopy the code

2. AudioNode

So a little bit about audionodes, there’s actually a lot of subclasses of Audio, and these audionodes are the most important part of AudioContext

2.1 AudioBufferSourceNode

This node represents the audio source

By audioContext. CreateBufferSource () method to create

This object is used as shown in the demo above. After it is created, you need to mount the data to be played to its buffer property, in addition to a few other properties

  • Loop: Indicates whether to play in a loop
  • LoopStart /loopEnd: If loopplay is set, the loop will be played within this interval
  • PlaybackRate: indicates the playback speedsource.playbackRate.value = 2To modify the value, while also onminValueandmaxValueTo indicate the adjustment interval

There are two key approaches

  • Play: Play parameters ([when][, offest][, duration])
    • When is when the sound is played, and if when is less than AudioContext.currentTime or 0, the sound is played immediately
    • Duration Indicates the duration of playback, if not set, to the end
  • Stop: Stops the playback

Events:

  • Onended: Indicates the end event

2.2 AudioDestinationNode

This node represents the sound output

Created by audioContext. CreateMediaStreamDestination ()

Since this node represents the sound output node, it can no longer connect to other nodes using the connect method, otherwise it will report

Uncaught DOMException: Failed to execute 'connect' on 'AudioNode': output index (0) exceeds number of outputs (0).
Copy the code

DestinationNode will automatically mount audioContext to AudioContext. destination when creating audioContext, so you do not need to create audioContext

Properties:

  • MaxChannelCount (read-only): Returns the maximum number of channels that the device can process.

2.3 GainNode

This node represents the volume control

Created by audioContext. CreateGain ()

The volume can be set by setting gainnode.gain. value to the value [0, 1].

See Demo2 for more details

function play(decodedAudioData) {
    const audioContext = new AudioContext();
    const sourceNode = audioContext.createBufferSource();
    sourceNode.buffer = decodedAudioData;
    const gainNode = audioContext.createGain();
    sourceNode.connect(gainNode);
    gainNode.connect(audioContext.destination);

    gainNode.gain.value = 0.5;
    sourceNode.start(0);
}
Copy the code

2.4 Other ways to create a Source

Against 2.4.1 createMediaElementSource

Receive an audio or video element

2.4.2 createMediaStreamSource

Receives a MediaStream object

The documentation on MediaStream can be found in the navigator. MediaDevices API

2.4.3 createConstantSource

No parameters, no sound

For the use of audio sources, see Demo3

2.5 AnalyserNode

AudioContext’s createAnalyser() method creates an AnalyserNode that can be used to obtain audio time and frequency data (both in the time domain and frequency domain), as well as to visualize the data. It does not modify the audio, just analyze it.

To use it correctly, you need to set some parameters

  • FftSize: FFT is the fast Fourier transform (back to the fear of being dominated by high number signals), and fftSize is the window size of the sample. The value of fftSize must be a non-zero power of 2 (2^5, 2^6, 2^7, etc.) in the range 32-32768. The default value is 2048

Below are two screenshots after analysis, which are 2048*4 and 512 respectively, with a difference of 16 times. The so-called window size can be understood as the horizontal tolerance range

  • FrequencyBinCount (readOnly) : half fftSize creates an instance of Unit8Array for bufferLength and copies the audio data in the time domain or frequency domain

The methods of obtaining time domain and frequency domain are respectively

  • getByteTimeDomainData
  • getByteFrequencyData
const analyser = audioContext.createAnalyser();
analyser.fftSize = 2048;
const dataArray = new Unit8Array(analyser.frequencyBinCount);
// Copy the data in the time domain to dataArray
analyser.getByteTimeDomainData(dataArray);
// You can put the method into rAF to refresh dataArray in real time
// Render the dataArray data
Copy the code

See Demo4 for the usage

X. the appendix

  • Make your web page speak – audioContext API
  • Web Audio API front-end sound processing
  • code