Fun with Line Art
December 23, 2008 — Robert Raguet-Schofield, User Interface Group
I’m constantly amazed by the wide variety of tasks people accomplish with Mathematica, everything from serious scientific research and development to fun games and puzzles. This one is more on the fun side.
A few days ago I was trying to convert a raster image to a vector image. I remembered seeing some online service to do this in the past and I was trying to dig up the URL. In the back of my mind I thought I could probably do this with Mathematica, but it wasn’t immediately clear how. I spent a minute or two contemplating various algorithms one could use before realizing Mathematica already has a built-in visualization function that could do most of the work for me: ListContourPlot. This function was meant to handle elevation-like data, but a two-dimensional list of grayscale values is essentially the same thing.
The first step is to get a suitable raster image into Mathematica 7. This is easy enough: just drag a JPEG file into the notebook window and assign it to a variable. Here is a picture of my handlebars after a muddy bike race.
ColorConvert turns the image to grayscale:
ImageData extracts the numerical values:
With this “elevation” data (grayscale pixel values) ready, it only takes one ListContourPlot command to convert the raster image into a vector image. I pass a number of options to ListContourPlot here, but most of the options are only there to make the output look like an image rather than a data plot. The options to pay attention to here are ColorFunction and ContourStyle (and to a lesser extent Contours).
Using slightly different values for the ColorFunction and ContourStyle options to ListContourPlot one can get line art.
If there’s one thing computers are good at, it’s performing repetitive tasks. Now that we have done something interesting with a single image, the next step is to repeatedly do something interesting with a sequence of dozens, hundreds, even thousands of images.
Movies are essentially a sequence of still images. Mathematica can directly import many movies; however, doing so will load all the movie frame images into memory at once. This is fine if your computer is capable of holding all the uncompressed movie frame images in memory at the same time, but most computers will quickly run out of memory. A common workflow for manipulating movies with Mathematica is to convert the movie into a sequence of image files (PNG or TIFF work best), import an individual image, process the image, export the image, then move on to the next movie frame.
This movie was recorded by my wife while she cheered me through the sand pit at a recent cyclocross race.
QuickTime Player Pro provides a convenient interface to generate a sequence of images from a movie. Open the movie in QuickTime Player and choose File > Export… from the menu bar. Select “Movie to Image Sequence” from the “Export:” popup menu and choose a location to save the files. (In this example I have saved them to a folder called “FramesIn” located in the same directory as this notebook.)
Now we have a sequence of 512 image files.
Next, define a function to process a single frame. This involves importing the image, performing the line art conversion, then exporting the frame.
At this point I could just run a simple Do command to process all the frames, but I’m on a multicore machine and I want this to go as quickly as possible. So I’ll use ParallelDo instead. First I will explicitly launch the default number of subkernels (determined by the number of cores on my machine).
Once the subkernels are launched I want to be sure the definition for my new function is properly distributed to all the subkernels, so I use DistributeDefinitions.
Now that my ProcessFrame function is properly distributed to each subkernel I can call ParallelDo.
The processed image files will begin appearing in the destination folder. Despite the simplicity of the code it is rather CPU and memory intensive, so processing hundreds of frames will take a bit of time.
Once all the frames have been exported it is a simple matter to reassemble them into a movie, again using QuickTime Player Pro. Choose File > Open Image Sequence… from the menu bar. Then choose the first image in the sequence (“MVI_0879 001.png” in this case). Use the same frame rate as the original movie (in this case 30 frames/second).
For the sake of completeness we can also add the audio track from the original movie to this new movie. Choose Window > Show Movie Properties from the menu bar. Select the Sound Track. Click the Extract button to create a new audio-only movie. With this new movie choose Edit > Select All, followed by Edit > Copy from the menu bar. Finally, select the new line-art movie and choose Edit > Add to Movie.
Voilà. Now we have a line-art movie.
I should add that while ContourPlot is a very nice way to convert images into true vector art (not bitmaps, actual lines), there are a lot of other image processing functions you might use if you actually just want bitmaps as output. For example, here’s a slightly different function that also gives a line-like effect.
One big advantage of raster-based image processing functions is speed: this is about a hundred times faster than the ContourPlot-based version. In fact, it’s so fast I can use Manipulate to look at the processed frames in real time. This lets me preview the filter effects in real time for any frame in the movie.
These image manipulation tasks, like most other things, can be accomplished in a variety of ways with Mathematica. With ContourPlot, we get a resolution-independent vector graphic, but with image processing functions like LaplacianFilter we get much faster performance. I could probably come up with a few more ways to achieve similar results, but I’m not interested in spending all day on this fun little side project. This was just a situation where I wondered “what if?”, and I was able to take the idea from conception to actual results in just minutes. The complete integration of Mathematica makes it better than any other language for this type of rapid prototyping.