All basic clefs, note values and accidentals exist as single wood elements. Whole, half, quarter and eighth notes differ not only in their form, but also in their weight: Long note values are heavier than short ones.

The table has two modes: A standard mode, where you can place notes on the table in a playful and experimental way and explore the related music outcome. And an excercise mode, where excercises and tutorials sort by topic and difficulty have to be mastered.

To activate “Noteput”, simply put the treble clef on the table. As soon as a note is placed on the staves, the respective sound is heard. That serves as a kind of preview and an orientation while putting the notes. If several notes are on the table, you can hit the play button and listen to the notes in relation to each other and considering note values. In addition to piano other instruments like guitar, flute, vibraphon or e-piano can be chosen. It is also possible to play the notes in a loop. That way it is easy to compare how the current note sequence sounds like and how changes of the notation immediatly influence the music.

The theoretical basis for the Output project was the theses, that a combination of as much as many senses leads to the best learning results. One’s motivation plays a key role in the process of learning and remembering. With the fundamental approach of tangible notes Output does not only want to reach music interested children and pupils, but mostly those who have not yet enjoyed music lessons and have thought of learning the notation as a burden.

As proof of concept we built and programmed a functional prototype of the table with slightly reduced functions. All elements were cut out of thin wood plates, weighted differently with lead balls and painted black matte. The technical part was done with the visual programming language vvvv and the so-called Fiducial Marker. Each object has a unique marker at its bottom, which is tracked with a camera placed inside the table. The software recognizes the marker and is therefore able to define which note with what note value is at what position on the staves. With this data a vvvv-patch computes the respective note sequence and sends it via MIDI to a sound software, which then can play these singals for different instruments.

Some pictures of the prototype:

The vvvv-patch tracking the positions of the notes and computing the respective note sequence.

The visual concept of Clavilux 2000 is quite simple. For every note played on the keyboard a new visual element appears in form of a stripe, which follows in its dimensions, position and colour the way the particular key was stroke: The length and vertical position show the velocity, the stripe’s width reflects the length of each note.
By mapping the color wheel on the circle of fifths, the colours finally give the viewer and listener an impression of the harmonic relations. Notes belonging to one specific tonality always get colors from one specific area of the color wheel. Therefore each key gets it’s own color scheme and “wrong” notes stand out in contrasting colors. The more different tonalities a piece has, the more colorful the visualization will be.

Unlike most other visualized music, the visualization of Clavilux 2000 does not disappear when the music fades away. All stripes stay and overlap each other in an additive way, so at the end a kind of pattern remains – a summary of the music – which will be always unique. Since not only the notes played but also the interpretation of the piano player are influencing the outcome, visualizations for one single piece of music will vary, too. Therefore the stripes allow one to say something about the composition aswell as the specific performance: Which notes were played the most? Which were the loudest notes? Which range of the keys was played mostly? How harmonically constant was the music?

Furthermore the system allows the piano player to switch between the standart 2d view and an additional 3d view of the visualization while playing. Also the visibility of a background grid, as well as the „deepness“ of the 3d view can be adjusted with sliders on the piano.
The technical part was done with the visual programming language vvvv. The MIDI signals sent by the piano to the computer get processed in real time and directly transfered into the visualization of the Clavilux.

The Clavilux 2000 project was part of my 3rd semester at the Hochschule für Gestaltung Schwäbisch Gmünd and was supervised by Prof. Michael Götte and Veldana Sehic.

Claude Debussy’s Childrens Corner - Dr Gradus ad Parnassum (rendering)

Claude Debussy’s Childrens Corner - Dr Gradus ad Parnassum (3d view rendering)

Rimski-Korsakow’s Hummelflug (3d view rendering)

More screenshots on flickr.

Listen to the track without video above or on soundcloud.

The projection showed the FORMAT logo, constantly changing in its appearance and sliced into 20 layers. These slices were combined with randomly chosen and also sliced pages of the actual magazine and bright colors.

Music was aswell randomly chosen from a collection of about 150 tracks. The idea was to create a radio-like impression with noise and glitches between the songs - as if someone was tuning an old school radio. Tracks have been played for at least 8 seconds and 2 minutes at most.

Again, vvvv was the best choice for both video and audio…

Typography, Images and Colors changing randomly… more pictures on flickr.com