Music is organized sound.


p5.js p5.sound


Composers have long explored creating music with algorithms and chance. These works fall into a few rough categories.

In algorithmic music, an algorithm is employed at the time of composition resulting in a fixed score.

In aleatoric music, important aspects of a composition are intended to be decided at the time of the performance. Aleatoric music allows for variation according to the performer or chance within a specific structure. For example, a musical system published in 1792, Anleitung zum Componieren von Walzern so viele man will vermittelst zweier Würfel, ohne etwas von der Musik oder Composition zu verstehen (Instructions for the composition of as many waltzes as one desires with two dice, without understanding anything about music or composition) employed chance to arrange pre-composed measures of music.

Brian Eno used the term Generative Music to describe music that is created by a system and that constantly changes.

Six Marimbas Come Out Its Gonna Rain
Three compositions by Steve Reich
John Zorn’s “game piece” music.
Music for Airports
Ambient music created by Brian Eno
Jeremy Wentworth: WebKitSynth
Generates a short sequence, with lots of knobs to tweak.
Happy Songs in Sad Versions
Popular songs in Minor Keys.
Neural Network Music
Academic paper (with examples) on “Music Language Modeling with Recurrent Neural Networks”.

This chapter covers some specific tactics for generating a short piece of music. Our result will be a simplified version of this demo.

We’ll look at how to plan this project, the pieces that need to be built, and how to represent data. The final generator will be implemented in JavaScript using the p5.sound library.

Synthesizing a Note

Generating a melody isn’t much use if there is no way to hear it. We could export our melody in an interchange format like MIDI, and play it back in an audio software package, but it will be more fun to build our own synthesizer. A synthesizer is a computer program or circuit that generates sounds. Synthesizers can simulate the sounds of natural instruments like pianos, guitars, and drums or create entirely new sounds.

Wikipedia: Synthesizer


When a note is played on two different instruments—a piano and guitar—the note will sound different even if it is the same pitch, length, and volume. Instruments sound different because of a variety of characteristics: spectral envelope; noisiness; attack, sustain, and decay; vibrato; tremolo; and others. Considered together these characteristics are called timbre or tone.

One of the key contributors to the timbre of sounds created by a synthesizer is the shape of the generated wave. For example a sine wave has a clear, smooth sound and a square wave has a brassier, buzzier sound. Natural instruments produce varying overtones resulting in more complicated wave shapes.

[[illustrations: sine wave, square, guitar ]]

There are many ways to synthesize tones. Some of the most common are additive, subtractive, frequency modulation, amplitude modulation, wavetable, and sampled synthesis.

Additive Synthesis
Combine simple sine wave signals together to create complexity. Fourier theory shows us that any periodic wave shape can be generated by adding sine waves of various frequencies and amplitudes.

[[illustration: fundamental, harmonic overtone, sum]]

Subtractive Synthesis
In subtractive synthesis a tone with rich overtones is filtered with frequency-pass filters that subtract different overtones to create new timbres. Square and sawtooth waves have strong overtones and are often used as the source in subtractive synthesis. The Minimoog and other 1970’s synthesizers often used subtractive synthesis.
Wavetable Synthesis
In wavetable synthesis the wave isn’t generated with a periodic function or math. It is simply stored as data which is used by the synthesizer.
Sampled Synthesis
Sampled synthesis is a type of wavetable synthesis where the stored wave data is recorded from physical instruments.
FM and AM Synthesis
In FM and AM synthesis simple square, sine, or other oscillators are chained together. This allows one oscillator to change the frequency or amplitude of the other, usually at a rate higher than the fundamental frequency being played. This results in tremolo and vibrato effects.

Designing the Synth

Most synthesizers offer a great deal of customization through parameters. What qualities do we want control of?

Qualities of our Synth

P5.sound has built-in monophonic and polyphonic synthesizers. They are a little underdocumented, so we’ll build our own using the p5.sound Oscillator and Env classes. Building our own is a good way to understand what is going on anyway.

The p5.sound Oscillator class generates a periodic signal with customizable frequency, amplitude, and waveform.

We can shape the amplitude of the Oscillator using the p5.sound Evn class. This class can control the amplitude of an oscillator using an ADSR envelope. An ADSR envelope controls the attack, decay, sustain and release of a sound and can be used to simulate these characteristics of acoustic instruments.

[whiteboard diagram ADSR Envelope] [whiteboard diagram [Evn] -> (amp)[Oscillator]]

By combining a p5 Oscillator and Env we’ll have a synth with these parameters:

  • Frequency
  • Basic Waveform Tone
  • Attack Time
  • Decay Time
  • Sustain Level
  • Release Time

Our synth will be very basic. We won’t have control of amplitude at the note level. We also won’t be able to play more than one note at a time. Playing a note will cut off any other note currently playing and chords won’t work at all. This type of synth was pretty common on cheap 80’s musical keyboards.


The code above creates a small synthesizer system. SimpleSynth is a small, reusable class that encapsulates this system and gives it an easy to use programming interface.

SimpleSynth Source Try it with MIDI

Playing a Melody

Now that we can play individual musical notes, we need to create some code to play a series of notes—a melody.

What does a Melody Look Like?

A melody is a series of notes. What information is necessary to describe a melody? How would you represent that information in code?

Our Melody Format

The p5.sound library has a set of classes—Phrase, Part, and Score—for representing and playing musical sequences. That system assumes each “note” in the song is the same length, and doesn’t specifically support precise timing, so we’ll build our own representation of a song.

Since our synth can’t play more than one note at a time our format can be a simple sequence, without support for chords. And because our synth isn’t velocity sensitive we don’t need to represent that either. We do need to represent the pitch of each note and its length.


We could represent our pitch with the raw frequency in hertz: e.g. 261.63.

We could use scientific pitch notation: e.g. C4.

We could use the MIDI note value: e.g. 60.

Hertz is a little fussy, and scientific pitch notation is harder to parse than MIDI note values, so we’ll use MIDI note values. P5.sound’s midiToFreq() will translate from midi values to frequencies for us.

A B♭ B C4 D♭ D E♭ E F G♭ G A♭
33 34 35 36 37 38 39 40 41 42 43 44
In musical notation, note lengths are usually provided in relation to the tempo and time signature of the piece: e.g. quarter note. We’ll use time in seconds. This is a little less flexible and maybe a little less musical but very straightforward.
Notes have two values: Pitch and Length. We could use an object {pitch: 60, length: .25} or an array [60, .25] to represent our note. The object is clearer but the array is more compact. I think a melody will look better with arrays: let’s use them.
Rests are gaps between notes. They have a length, but no pitch. We could have a different format for rests, or we could use notes but set the pitch to a special value. For simplicity, let’s use notes with a special pitch value. We might use 0 or undefined to represent a rest, but those aren’t quite semantically correct. Since we can mix types in JavaScript we could use a string too: rest.

Here is how our melody will look.

const melody = [
    [64, 3/8],
    [62, 1/8],
    [60, 1/4],
    [62, 1/4],

    [64, 1/4],
    [64, 1/4],
    [64, 1/2],

    ['rest', 1],

Playing our Melody Format

SimpleSynth’s noteOn and noteOff methods accept a parameter time to schedule events in the future. To play back our melody, we can loop through each note and schedule it on the SimpleSynth. SimpleSynth has the playNote and playNotes methods to do this.

playNote(note, length, time = 0) {
    this.noteOn(note, time);
    this.noteOff(note, time + length - this.spacing);

playNotes(notes) {
    let time = 0;
    for (let i = 0; i < notes.length; i++) {
        const freq = notes[i][0];
        const length = notes[i][1];
        this.playNote(freq, length, time);
        time += length;

Generating a Melody

Now we have a format to represent a melody and a function to play it. Our final challenge is the code to generate one. We can’t jump into coding yet: we need a much more detailed plan that we can translate into code. You really can’t write computer code without first deciding exactly what you want the computer to do. You can try, but you’ll end up making decisions anyway. You’ll just make them as you go along, without a plan.

One approach we might consider is picking random notes and random lengths and placing them in a sequence. But that would be like generating images by randomly assigning colors to pixels: the result would be noise—in many senses. We don’t want noise, we want to make music.

What Kind of Melody do We Want?

A melody is an organized series of notes, but how do we want our melody organized? What qualities do we want to ensure? What qualities do we want to leave to chance?

Our Target Characteristics

  • We are going to use the C-Major key.
  • We are going to use 4/4 time.
  • We will use only half-notes and quarter-notes.
  • Our melody will have no rests!
  • Our melody will have 4 measures.
  • These measures will have an A B A C structure.
  • Our melody will start at a random place in the scale, and move up and down the scale in random steps.
  • Our melody should contain mostly conjunct (single) pitch steps: e.g. C to D, F to E
  • Our melody should have occasional disjunct (bigger) jumps: e.g. C to E.
  • Our melody will end on the tonic: C.

Our Melody Generator

Compare the Code to Spec

Codebases often diverge from their initial specifications. Compare the target characteristics with the melody generator above. How are they different?

Keep Sketching!


Continuing to explore generating and visualizing sound and music.

Challenge: Write and Record a Song

It doesn’t have to be good. It doesn’t have to use code.

Melody in Songwriting :Melody in Songwriting is an excellent book for developing melody-writing skills.

C-sharp vs D-flat :Music theory article addressing note naming within different keys.

Kahn Academy: Music Basics :Khan Academy offers interactive tutorials on basic music theory, including rhythm and musical notation.

MDN: Web Audio API :Documentation for the Web Audio API.

HTML5Rocks: Getting Started with Web Audio

HTML5Rocks: Audio Scheduling :A tutorial on audio timing and scheduling in the Web Audio API.

I don’t know who the Web Audio API is designed for

UNSW: Midi Notes and Math

Teoria.js Music Theory Library :A library for music software that supports musical structures and language, including chords, scales, and notes.

Ocenaudio: :Ocenaudio is a free sound editor available for both OS X and Windows. It’s beginner-friendly while still offering more advanced features.

ChucK :ChucK is a programming language for real-time music and audio generation. Watch creator Ge Wang discuss his design and use of the program in this video.