Music Interaction Design: Midterm / by Maya Pruitt


My project is a music learning tool for singers. Inspired by the phenomenon of synesthesia, the project combines color with notes as a way to provide a visual to sound. For an untrained ear, one may not be able to hear if she is singing incorrectly, but perhaps she could more easily adjust if she sees it.

I believe it is pretty entertaining. For the initial prototypes it certainly feels game like and participants become determined to see how their voice can change things. I think it is a good entry point to music learning because its simply exploring your own ability to listen to and produce sound.

Technology. Briefly, how does it work?

The project mainly uses the getCentroid function of FFT in p5 sound to identify pitch. A recording (with tuning tones) are analyzed and parsed for pitch. Live microphone input, to capture voice pitch, is analyzed the same way. Voice input is mapped in certain ways to give the user a better sense of what’s happening.

In action.

PROTOTYPE 1: This first attempted tried to map pitch to the width or height of the voice circle. Meaning that a flat note would squish the circle making is squat and stretched horizontally. A sharp note would become tall and narrow. Thus, the user is meant to aim for a perfectly round circle.

However, with all the ranges/bins of centroid values, it ends up producing more than 2 circles. This is confusing. My goal is to have one circle that’s static to represent the given pitch and an ever changing circle for voice input.

PROTOTYPE 2: Gone away with mapping pitch to width and height of the ellipse, and instead, this version asks the user to match the diameter of their voice circle to the diameter of the pitch circle. This is most similar to my initial modular prototypes. I also added a “confidence” mapping. This links volume to voice input. The louder the user sings, they create a bolder color.

PROTOTYPE 3: I still feel compelled to illustrate the nuances of pitch. So in this version, the circle travels vertically depending on whether the voice note is flat or sharp. This metaphor works well, I think, because you can think about producing sound as going below or above the target. Volume mapping is included in this version as well and as an exciting artifact, the color mixing for harmony is especially prominent.

Next steps.

  • Although tones are associated with unique frequency values hertz…getting p5 to recognize them in the same way is almost impossible….in addition, something like the human voice, has way more idiosyncrasies than a computerized singular tone. Thus, my main battle is with noise. I tried different filtering techniques but still have a weird strobing effect that I don’t like. My next step is to really figure this out so that the aesthetics can be purer.

  • I would like to continue building the interface to figure out the best way to help people learn to sing.

  • title?

On another note, this project has really encouraged me to play with Ableton. I don’t think it relates to my project yet…but I wanted to also share my 2nd real attempt at a song: