The Moodbar audio visualisation method

One of the niftiest legacies of the old Amarok music player (version 1) is the Moodbar visualisation method. The idea behind it is simple: take the source audio signal, analyse it, and create a colourful representation of the sound as a timeline.

[Image showing Exaile with Moodbar visualisation, in which the original seek bar is replaced by a colourful-looking bar]

Exaile with Moodbar replacing the standard seek bar

Back in 2009, Solyanov Michael created an external plugin to add Moodbar support to Exaile, and at some point it was imported into the Exaile tree. Now that we’re upgrading Exaile to GTK+ 3, I have been rewriting the plugin’s UI code to use Cairo instead of GDK drawing functions, which don’t exist anymore. (Unfortunately I don’t think I’ll have the time and motivation to implement all features of the existing plugin.) The above screenshot shows a sneak peek of how the new UI currently looks like.

What do the colours mean?

Moodbar is explained in a 2005 paper by Gavin Wood and Simon O’Keefe, which presents a number of algorithms for visualising audio signals. Specifically, the Bandwise Spectral Magnitude method is the one implemented as Moodbar.

Long story short, for each point in the timeline, the red channel represents low frequencies, green represents mid frequencies, and blue represents high frequencies. In practice, however, it is often difficult to imagine what an audio segment will sound like just based on its colour. The only thing you can pretty accurately guess is the audio level based on colour intensity (bright=loud, dark=quiet).

I’ve prepared a few Moodbar examples with playable audio for you to see this in action. Some of the samples (Hypocrisy, What Is Love) are pretty good and show all sorts of colours differentiating song sections. Some are less exciting, either because the song mostly uses a single instrument (Spring) or because the song is relatively monotonous (Max).

Current support in media players

To my knowledge, there are only three actively-developed media players with Moodbar support, all of which are based on Amarok 1 to some degree: Exaile, Amarok 2, and Clementine. Ironically, Amarok 2’s Moodbar support is the least integrated; you need to manually scan your files, whereas Exaile and Clementine do this for you as they play individual files.

Currently Exaile and Amarok 2 still rely on the old, GStreamer 0.10-based Moodbar program. Clementine seems to have integrated Moodbar into their own code and is using it as a library. Their latest release is still using GStreamer 0.10, but their development tree already contains a GStreamer 1.x-based Moodbar.

Future

Right now we are focusing on finishing the GTK+ 3 port of Exaile, which includes getting the Moodbar plugin to a reasonable state. The basic Moodbar functionality is ready; I just have to implement a fallback UI for tracks that have not been or cannot be scanned for whatever reason.

Other Moodbar-related tasks on my radar (read: may or may not happen):

  • Steal split Clementine’s new Moodbar code into a separate library so that Exaile and other media players can make use of it. I have the code sitting ready to be published, but it still needs more testing, especially since I’ve just found out that it produces different output to the original Moodbar program.
  • Add some of the old features that don’t make the cut to the GTK+ 3 port (darken should be doable; waveform will be tricky, given the way the new drawing is done).
  • Add support to share the Moodbar cache with Clementine. Not terribly useful, to be honest.
  • Do more research to find a better timeline visualisation method.
Advertisements

Note: By commenting, you grant me permission to freely republish your comment.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s