Comp Sound

Sound

p5.js Sound Library

In addition to the basic drawing API, p5.js includes add-on libraries. The parameters chapter introduces the p5.dom library, and the examples in this chapter use the p5.sound library. The p5.sound library builds on the Web Audio API and provides functions for generating tones, playing recorded sounds, and visualizing the waveform and spectrum of sounds. I highly suggest taking a look at all of the sound examples to get an idea of what the sound library can do.

This chapter will focus on playing and visualizing pre-recorded sound assets. The Comp Music chapter talks more about sound synthesis.

Xin Xin

Plant Ensemble, 2018

Plant Ensemble uses an arduino, speaker, and plants to create randomized sounds.

Plant EnsembleVideoxin-xin.infoArtist Site

Jason Sigal

Sonic Scrolls, 2014

In this interactive installation, users created a sonic scroll that comprised of colors and shapes that represented different instruments, tone, and timbre.

Sonic ScrollsProject Pagejasonsigal.ccArtist Site

Luisa Pereira and Manuela Donoso

The Harmonic Series, 2012

This interactive series visualizes sound and harmony through 2D and 3D prints and sculptures.

The Harmonic SeriesProject Pageluisapereira.netArtist Site

Vineet Kapil

Neon Impasse, 2021

In this sound visualizer, users can add guitar riffs and synth sounds using their keyboard and mouse to the first track of the album, "neon impasse" by city girl.

Neon ImpasseProject Pagevineetk.comArtist Site

let myImg;

function preload() {
  myImg = loadImage("images/world.jpg");
}

function setup() {
  createCanvas(500, 500);
}

function draw() {
  image(myImg, 0, 0);
}

let mySound;

function preload() {
  mySound = loadSound("sounds/hack-comp.wav");
}

function setup() {
  mySound.loop(0, 1, 1, 0, 4);
}

Computers, Light, Sound, and People

Light and sound flow through our environment as electromagnetic and air pressure waves. Our eyes and our ears collect data about these waves and our visual and auditory cortexes process that data to create information. Though they work in different ways, both of these sensory systems are powerful. We can take advantage of these systems by choosing the types of forms we create.

Playing Sound with p5.js

Playing Recorded Sounds

p5.js makes it pretty easy to play sound assets, like this audio clip from Hackers (1995).

Changing Rate + Pitch

The second parameter to play() controlls the rate of playback. If you play a sound faster the pitch will go up; play it slower and the pitch will go down. p5 cannot change the pitch of a sound without changing its length.

Looping

This example loads an audio file twice and then plays two loops at once. The looping end time is slightly different so the playback falls out of sync.

Drum Kit

This example uses frameCount to keep time and play a beat with drum samples. This works for a quick sketch, but frameCount isn’t a great timekeeping source, so you might notice hitches in your rhythm. The Tone.js lets you precisely schedule when samples are played, and might be a better choice if you want to make music.

Bounce

This example doesn’t have any sound in it. Add sound as part of the coding challenges below

Capturing Output

p5.sound provides methods for recording the sound generated by your program and saving it to a file. The following function records length milliseconds of audio and saves it as output.wav.

Your audio may sound fine at run time but clipped when you play back the .wav. If this happens, try reducing the volume of the sounds you generate.

If you can’t get good quality audio capture from p5.SoundRecorder, you might get better results using a screen recording program like Quicktime Player.

// uses the p5 SoundRecorder and SoundFile classes to record the audio output.
// begins recording when called. records for _length_ time in milliseconds.
function record(length) {
  const soundRecorder = new p5.SoundRecorder();
  const soundFile = new p5.SoundFile();
  soundRecorder.record(soundFile);
  setTimeout(function () {
    console.log("Recording Complete");
    soundRecorder.stop();
    save(soundFile, "output.wav");
  }, length);
}

Visualizing Sound with p5.js

Cuepoints

This example adds a single cuepoint to the sound at 1.7 seconds. When playback passes that specific time, cueBig() is called.

Volume

This example uses p5.Amplitude() to track and visualize the volume/loudness of the entire sound.

Wave

This example uses p5.FFT() to track the and visualize the waveform of the sound.

FFT

The Fast Fourier Transform transforms a signal from the time domain to the frequency domain. For audio analysis that means the FFT can tell the strength of different frequencies—bass, mids, treble—in an audio buffer.

For a visual exploration of FFT, see this excellent video by 3Blue1Brown But what is the Fourier Transform? A visual introduction.

Making Sounds from Scratch

You can create an empty soundFile object with new p5.SoundFile() and generate the sound data yourself with Javascript. To do so you will create and fill a Float32Array and then attach it to the SoundFile object with setBuffer().

Creating sounds this way lets you work at the lowest possible level: individual samples. This can be fun, and it gives you complete control, but you will probably need to reinvent some basics.

P5.js has functions for working with oscillators, envelopes, and effects if you want to work at a little higher level. You might also consider using a dedicated Javascript sound synthesis library like Tone.js, which is a little more powerful and better documented.

Explore

.