SpectroGraph 2.0 for iTunes
by Alexander Thomas (aka Dr. Lex)
What is it?
SpectroGraph is a visual plug-in for iTunes for Mac OS X. Visual plug-ins were introduced in version 1.1 of iTunes. The plug-in started life as a simple modification of Apple's iTunesXPlugIn sample code, but has grown beyond that in newer versions. Of course it's completely free!
The plug-in displays a plain spectrogram (hence the name) of the music. A spectrogram is a graph of the frequency content of a sound in function of time. The left channel is plotted in red, the right channel in green. This means that a mono sound will show up in pure yellow (because in RGB space, red + green = yellow).
I made this plug-in because I needed something which represented the actual spectrum of the sound as realistically as possible. Amidst the existing visual plug-ins for iTunes, I only found funny gizmos which produce a fancy display which moves to the music. That's nice to look at, but not handy if for some reason you need an accurate display of the frequency content of the music.
The immediate reason why I made this plug-in, was to be able to see hidden images in the frequency content of certain songs. Some songs by Aphex Twin contain hidden images (for instance, the ‘Windowlicker’ EP and especially the ‘Equation’ song), but they may look distorted because he used a harmonic frequency scale while this plug-in has the usual linear scale. The song ‘The Warning’ from the Year Zero album by Nine Inch Nails also contains a hidden image, originally to be found in a pre-release version of ‘My Violent Heart’ (it is hard to see in Spectrograph though). But even with a lot of ‘ordinary’ songs, the frequency display is quite interesting and fun to look at. To get the best renditions of hidden images in songs, you should use the original CD or a high-bitrate encoding.
You can change the look of the spectrogram either by accessing the ‘Options’ dialog (in iTunes versions prior to 7, you can use the button in the top-right corner when the plug-in is active, from version 7 on you need to use the ‘View’ menu), or by using the keyboard. You can choose between horizontal and vertical scanning or scrolling, color and grayscale display, anti-banding, inverting of the spectrogram and change the maximum drawing speed. You can also force the plug-in to re-start drawing at the left or top at any moment.
Mind that the maximum drawing speed is not only limited by the software setting, but also by how much processor time the plug-in gets. If you run heavy background tasks or stretch the display all over the screen, playback may become slow and irregular. Run the plug-in in full screen mode for optimum performance.
Version 2.0 is a major rewrite of the plug-in. The drawing routines have been converted from the archaic QuickDraw to OpenGL. This was necessary to make the plug-in compatible with OS X Leopard, but it has a lot of nice side efects. The quality of the display is improved and the drawing speed is much more stable. Moreover, it allows to do more fancy things, for instance scrolling the whole image (and more). The only drawback is that a recent graphics card is required to get similar maximum drawing speeds as with the older version. On older Macs which can't run Leopard anyway, I recommend using version 1.2.1 because the new version won't even be able to run at 'normal' speed.
Note 1: SpectroGraph runs extremely slow in iTunes 10.0. It runs faster in iTunes 10.0.1 and newer, but the maximum speed is still limited to about one third of what would be required to view all detail in the spectrogram. There's nothing I can do about this, iTunes 10 has a built-in limit on how many chunks of spectrum data it can deliver per second. If you want SpectroGraph to run fast, reinstall or do not upgrade from iTunes 9 unless you really need any of iTunes 10's new functionality. Or, ask Apple to either drop the data frame rate limit or at least increase it to 180 frames per second. You can also file a bug report and refer to bug #8393710.
Note 2: the plug-in will display nothing if a file is played which is not MP3, AAC, or uncompressed audio. This is a ‘feature’ of iTunes, and it's also the case with all other visual plug-ins (they might display something, but will not react to the music). If you want the visual plug-ins to react to other files too, the only thing you can do is ask Apple and hope they'll listen. By the way, the equalizer, audio enhancer and sound check features also do not work for other formats, so you might want to ask to fix that too while you're at it.
The download includes an mp3 file which will produce an interesting result when viewed with the plug-in!
To install, unzip the file, and drag the ‘SpectroGraph.bundle’ into the folder “Library/iTunes/iTunes Plug-ins”, either in your home directory (the plug-in will only be available to you) or in your root directory (the plug-in will then be available to all users). Then re-open iTunes and choose "SpectroGraph" from the 'Visuals' menu.
Remember, this plug-in is Mac OS X only. I didn't have the time to work on a Windows version yet, and given the fact that I have zero experience in programming for Windows, it's unlikely that I will do so in the near future. If you are a Windows developer with some spare time, download the source code below and get crackin'.
SpectroGraph works in Mac OS X Lion, but only if iTunes runs in 32-bit mode (this can be set through the ‘Get Info’ window in the Finder). Unfortunately, upgrading SpectroGraph to 64-bit proves to be more difficult than a simple recompile, so it may take a while before a 64-bit compatible version is ready.
This version will only work in OS X 10.4.x or newer. You can download older versions of SpectroGraph to use on older systems.
Download SpectroGraph 1.2.1 (Gzip archive, 157KiB) (universal binary: will only work in OS X 10.4.x)
Download SpectroGraph 1.1 (ZIP archive, 122KiB) (works from OS X 10.3.0/iTunes 4.0 on)
Download SpectroGraph 1.0 (StuffIt archive, 130KiB) (works almost anywhere)
Version 1.0: initial release.
Version 1.1: made the source code available and added 'anti-banding'. The latter is an attempt to reduce the banding artefacts which often occur with deep bass sounds. It provides a cleaner output and is on by default, but can be toggled with the 'b' key or in the settings dialog.
Version 1.2: converted to universal binary, added speed limiter.
Version 1.2.1: fixed an overflow bug in the speed limiter which could cause the plugin to stop drawing.
Version 2.0: complete rewrite, using OpenGL for rendering, added scrolling.
Ideas for improvements
If you didn't already notice it from the huge delay between version 1.2.1 and 2.0: I have very little time to work on this plug-in. Moreover, the following pile-up of issues have been eating away at the little motivation I had left:
- The silly limitation on the number of spectrum updates per second that has been introduced in iTunes 10, crippling SpectroGraph's potential.
- The apparent total lack of information and sample code for making a 64-bit visual plug-in. It seems impossible to build a 64-bit iTunes Visual Plug-in, but I know it must be possible because 64-bit plug-ins are available. It would be a great help if anyone could give me some pointers on this.
- A large part of my music collection is in OGG format, and despite sending suggestions to Apple for years the visualizer still does not react to files played through QuickTime. Of course, what I really would want to see is native OGG support in iTunes, just playback only would be great already.
2012/01/11: Good news! Someone took the effort in his spare time to rewrite SpectroGraph with the latest iTunes visuals SDK, making it 64-bit compatible. Moreover, this seems to allow a higher update rate as well. When this new version has been polished, I will make it available here and the source will be available on GitHub.
On my TODO list are the following points, in decreasing order of importance/feasibility:
- Add 64-bit support for iTunes in OS X Lion. Spectrograph works in Lion, but only if iTunes is launched in 32-bit mode.
- Add some kind of grid to allow a more quantitative analysis of the data, and the possibility to ‘zoom in’ on certain ranges of frequencies.
- Create a Windows version. With the conversion to OpenGL, this has become much more viable. However, there's still a lot of work to do.
- Implement my own FFT and logarithmic filter bank, and calculate them from waveform data. This would allow higher resolutions, a proper fix for the banding issue, and offer the option of a true logarithmic frequency analysis. The big problem is that it requires full continuous waveform data. Until iTunes 10 it was still theoretically possible (albeit very problematic and error-prone) to reconstruct it from the visualiser data frames. However, due to the forced cap on the update rate, it has become completely impossible.
- Start from scratch and write a stand-alone program that can capture audio from any application (in a similar fashion as AudioHijack) and visualise it. This would solve every problem currently caused by limitations of the iTunes plug-in framework, but I really don't have the time to start a project like this. Although I could recycle a lot of the current SG code, finding out how to obtain raw audio from OS X' innards is a total mystery for me at this point.
If you think you can tackle one or more of these problems on your own, you can simply download the source code and give it a shot. Remember, this source code is provided on a “share your improvements”-basis: if you release an updated version of the plug-in, you should also release the updated source code. The project will only open in XCode 2.4 or newer, but you can easily reconstruct the project in older versions by taking the iTunesPlugin sample source code from the Apple site and replacing its source code files with the ones from this project.
Honto.com has created a custom plug-in, based on the source code of this one. The display is mono, but it supports multiple color schemes and constant drawing speed.
The author of this software is not responsible for possible damage of any kind caused by the use of this program. There is no warranty of fitness for any purpose. Use at your own risk!
This software is free and may not be sold. No explicit permission of the author is required for reproduction and distribution of the software. The compiled SpectroGraph bundle may only be distributed if the original files, i.e. the ReadMe file and the SpectroGraph.mp3 sample, are included unmodified.