Wednesday, May 6, 2009

Visualizing Music and Looking for Patterns

Found this and many other impressive videos on one Stephen Malinowski's YouTube channel. I really like the way the colored bar visualization separates out the different voices in a piece, especially the fugues.

The opening to Gödel, Escher, Bach has a fun discussion on the structure of the fugue-- the gist of it is that the composer develops the piece out of one short theme (a few measures of some simple melody), carried by a fixed number of voices. Starting with one voice expressing the theme, each additional voice chimes in repeating the theme until all are present. The theme is further explored and varied throughout the piece via transformations of the original melody: inverting, reversing, transposing, compressing. Soooo the video above is really cool, because the visualizations make it that much easier to pick out all the transformations that are taking place. Yay!

I wonder if you could make other visualization methods which help you pick out recurring themes in a piece, and are robust to transformations from the original theme (or measure distance from the original). It seems like a problem that crops up a lot, in problems from network analysis to predicting structural motifs in proteins. For instance, all integral membrane proteins will have a hydrophobic region which crosses the lipid bilayer-- this requires an extended sequence of hydrophobic amino acids, which will be reflected in the genetic code. There would be variation in sequence (not all membrane proteins would have the same arrangement of hydrophobic amino acids), but there might still be trends which might be picked up. Fourier/Laplace transforms can break a signal down into its periodic components; is there some way to transform a signal to visualize it in the space of its recurrent themes and their variations?

No comments: