May 12, 2003
Seeing Sound
By Diana Phillips Mahoney (Computer Graphics World Aug, 2001):
Thanks to advanced simulation and visualization techniques, sound has never looked so good
What does sound look like, and who wants to know? Mostly it looks like funky abstract art, and everyone from urban planners to VR developers to concert pianists wants to see it.
Actually, depending on who is looking at it and why, sound has many different visual guises. At Lucent Technologies' Bell Laboratories, for example, sound appears as beams of light that vary in length and intensity relative to their distance from the point of origin and the obstacles encountered along their path. At the Center for New Music and Audio Technologies (CNMAT) at the University of California at Berkeley, sound looks like oceans of alternately vibrant and muted colors with waves emanating from multiple geometric objects. And in Vancouver, students and faculty at the University of British Columbia see sound as expanding and contracting balloons.
Of course, actual, physical sound doesn't "look" like anything. Our perception of soundwaves is related to a variety of acoustic characteristics, including decibel level, spread patterns (propagation), and intensity changes over time-none of which is tangible or visible. To enhance our understanding of sound, a variety of techniques exist that simulate acoustic phenomena. These result in complex numerical models, which, frankly, are not much to look at. In addition, the computational data, because of its vastness, is difficult to synthesize and comprehend. A picture, on the other hand, is worth a thousand numbers.
Comments
MetaSynth is a Mac-only sound design program which offers the unique capability of transforming any picture into a sound, or any sound into a picture, and permits manipulation of audio elements entirely in the graphic domain, manipulating MIDI files, sound and music as images in any Mac based paint or graphics application. The program literally allows the user to paint with sound, or compose with color and light.
Since its first public demonstration at last year's New York AES conference, MetaSynth has become the most talked about and widely used, new signal processing and sound design software in years.
MetaSynth generated sounds and MIDI music samples are heard daily on MTV commercials, on dozens of electronic, dance and underground CDs, and in new computer game soundtracks.
Posted by: will at May 13, 2003 06:36 AM
This looks interesting but was thinking more along the lines of how to visualize audio info in our WEM system setup - ie how to visualize and browse sound samples in realtime in an environmental setting.
Posted by: sfisher at May 13, 2003 11:25 AM
Right. metasynth is the reverse - making sound out of images rather than making image out of sound data. I've done a lot of stuff with Signal Analysis in MSP - but haven't played around much with mapping that analysis in an interesting way to perhaps, Jitter objects. The thing that's kind of cool about Jitter is that you could do really sophisticated image processing in real time - and then you could have the arguments to these processes supplied by an audio signal analysis. But besides these image filters, jitter has added particle systems and a number of OpenGL objects to the mix. I'd really like to try some stuff with this...if only I had a license ;-)
Posted by: will at May 13, 2003 03:32 PM
This may be off-topic, but another aspect of "seeing sound" is experienced by synesthetes, people who, in many cases, involuntarily and consistently associate letters, numbers, words with color. Current theory speculates that these cross sensory experiences result from "cross activation" among areas of the brain.. Research on synesthsia is just gaining respectabilty with fMRI data. The MAY 2003 Scientific American has an article by Ramachandran & Hubbard; HEARING COLORS, TASTING SHAPES
http://www.sciam.com/article.cfm?colID=1&articleID=0003014B-9D06-1E8F-8EA5809EC5880000
Posted by: Peggy at May 16, 2003 09:28 AM
i would like to know how i can transform a music file into a digital image/graphics, to clarify,i want to know how would jazz music for example transform into graphics, what it would look like as an image.i would really appreciate it if anyone can help me out.dont know if that metasynth software that u r discussing would be of any use but i dont have a Mac so its not really of any help to me......
Posted by: maggie at October 30, 2003 09:24 AM
