DETAILS, DETAILS...

3D Layouts of Music

Maps of Musical Worlds

Click above to view our two main layouts for music animation. The vertical display mirrors music notation perfectly, but shows so much more.

If you aren't familiar with reading music, here’s another way to think of what we’re showing.

In car and home stereos there is often a tool to control the Bass, Treble, and Middle frequencies of the system called a Graphic Equalizer. Sometimes there is a meter to show the frequencies playing called a Spectrum Analyzer. Lowest Bass frequencies are shown on the Left side. The highest Treble frequencies are on the far Right.

Imagine we have a frequency meter with a thousand bands instead of ten or twenty.

If we use this to take a “snapshot” of the music’s frequencies every few milliseconds and line up thousands of these snapshots together they form a 3D topography of the notes and tones from the music.

Reading Animated Music

In these 3D scenes of music there are always three aspects of the music that are displayed in three different dimensions. Illustrating these aspects of the music reveals notes, chords and other important parts clearly.

The first direction is Time. In Landscape layouts earlier music often is shown in the foreground and later music recedes into the background. In scenes using a Lattice layout earlier audio is usually shown on the left side with later audio on the right side just as text or musical notation would be displayed.

The second direction of representation is the music's Frequencies. The pitches, how high or low they are, may be shown as individual music notes or include the entire frequency spectrum to show the overtones of pitched instruments and the inharmonics of percussion.

The third direction shows the music’s Amplitude or Volume, so across a passage of music, hills and valleys are created by the amount of volume for the various frequencies at that point in the recording.

There are two main ways to present the overall layout:

Music As a Landscape

The Landscape style of layout resemblesa horizontal topography or terrain. This layout is often best for displaying drumsand other complex instruments with lots of high frequency.

Music as a Vertical Lattice

The Vertical Lattice stands up. Higher notes are visually higher in this display. When viewed from the side, the Vertical Lattice mimics musical notation. But it displays many details of the music that musical notation does not.

Combined Display Methods

At times a combination of both display methods is used to show separate parts more clearly or make better use of the available space in the portrait.

We often color and texture different parts or players separately. When someone views animation of music they are familiar with, they can see the melody, chords, and other aspects they know from the tune. Backgrounds like flat ground, sky, or water are usually not a part of the musical portrayal. Only the 3D shapes rising above represent the musical parts.

from SOUND to SIGHT

To create the 3D images used for the animation and artwork a passage of music is analyzed for its frequency content (what we hear as the pitches and tone of the music) thousands of times. Up to one hundred thousand “slices” of the analysis are strung together to assemble a 3D terrain.

After creating a landscape from the audio analysis, we spend hours exploring the unique, detailed terrain. Then the artistic side takes over. The technology is only a tool for the human creative process.

We work to find the best layout, angles of view, colors, textures, backgrounds, and lighting to achieve a final piece that reflects the character and spirit of the music. We try many approaches before determining the look that is best.

ART & MUSIC - the Relationship

There are many examples of composers and musicians who described music in visual terms and were inspired by works of art. And hundreds of cases of artists whose work was inspired by music. Beethoven, Liszt, Messiaen, Rimsky-Korsakov, Sibelius, Bernstein, Varèse make up just a short incomplete list of composers who have spoken on the topic.

Here’s a quote from Messiaen on the phenomenon:

"When I hear music, I see in the mind's eye colors which move with the music. This is not imagination, nor is it a psychic phenomenon. It is an inward reality."

Synaesthesia

Kandinsky, widely considered to be the father of abstract art was inspired by music to paint the invisible. He had a condition, widely recognized today as synaesthesia.

Kandinsky was overwhelmed by the effect of synaesthesia while viewing a Wagner opera in Moscow: "I saw all my colors in spirit, before my eyes. Wild, almost crazy lines were sketched in front of me."

Later, as he developed his thoughts and theories on the relationship between art and music he filled his work with shapes and colors that would “sing together.” He wrote,“Color is the keyboard. The eye is the hammer. The soul is the piano with its many strings.”

The list of artists, musicians, and other creative types who have experienced synaesthesia number in the hundreds; from Van Gogh to Van Halen.

More from Beethoven on his Vision of Music:

“I change many things, discard, and try again until I am satisfied. Then, however, there begins in my head the development in every direction, and, in as much as I know exactly what I want, the fundamental idea never deserts me,—it arises before me, grows,—I see and hear the picture in all its extent and dimensions stand before my mind like a cast, and there remains for me nothing but the labor of writing it down, which is quickly accomplished when I have the time.” 1822

Quotes about Music & Architecture

“Music is liquid architecture; Architecture is frozen music.” - Johann Wolfgang von Goethe

“I see music as fluid architecture.” - Joni Mitchell

“If architecture is frozen music then music must be liquid architecture.” - Quincy Jones

DETAILS, DETAILS...

3D Layouts of Music

Maps of Musical Worlds

​

from SOUND to SIGHT

ART & MUSIC - the Relationship