Found a nifty article about a piece of digital music software called Photosounder, posted on a blog called Create Digital Music. Photosounder is an image-sound editing program-- that is, music creation is done visually, by drawing and editing the sound's spectrogram. The videos in the CDM article show some of the ways in which this software is being used; it's pretty impressive stuff. I also found this plugin for winamp which produces a simple spectrogram of your music as a visualization, if you're just curious to see what the music you're listening to would look like.
The spectrogram is actually a good representation of how sound is coded in the brain-- the cochlea in your ear breaks down sound input into narrow frequency bands, just as we see on the X axis of the spectrogram, and cells in each frequency band fire in proportion with the intensity of sound at that frequency (so, you have a physical structure in your ear which performs a Fourier transform-- how cool is that?) As seen in this video, a single sound object usually consists of several harmonics, and a full spectrogram can be quite complex-- and yet our brain can easily segment that spectrogram to identify different instruments, even when there's a lot of frequency overlap. We are even able to focus our attention on one specific instrument, which means selectively responding to one particular batch of signals as they move up and down across frequency channels/cell populations. Brains are pretty awesome, guys.
(And on another note: as you can see in the aforelinked video, one of the easiest ways to pick out one instrument from a spectrogram is to look for elements which "move together" in time/across the spectrum-- this notion drives a lot of work in both auditory processing and the corresponding problem of object recognition in computer vision.)
No comments:
Post a Comment