Professional cameras offer a resolution of 50 megapixels and more. In addition, projects like GigaPan allow one to create gigapixel panoramas with billions of pixels. How can we process these images on a desktop computer with 8 GB of RAM? One of Mathematica 9's new and exciting features is out-of-core image processing. What does the out-of-core term really mean? It is a way to process very large images that are too big to fit into main memory. Let's say we have a machine with 8 GB of RAM, and let's assume that Mathematica can use up to 7.2 GB of that memory (the remaining 0.8 GB will be used by the operating system). Freshly started, Mathematica 9 on Windows 8 takes up about 200 MB of memory, so the kernel can use about 7 GB of RAM. What is the maximal size of the image that we can load into the kernel (we don't want to visualize it at this point)? If we assume that the image is in the RGB color space and a single byte encoding, then the following formula gives a maximal width (and height) of an image that can be loaded at once into the memory:

Mathematica 9 has just been released with many new or enhanced capabilities for image processing. You can perform morphological operations, color manipulation, segmentation analysis, feature detection, and more, most of which can be applied to the new Image3D object as well. A byproduct of this whole ecosystem is that now it is easier than ever to use Mathematica to create and apply effects to your images. Two Mathematica super functions that can be used to apply transformations directly to an image are ImageApply, which is a pixel operator, and ImageFilter, which considers the pixel as well as a neighborhood of pixels around it and works as a local filter. For example, you can remove the blue channel and perform a gamma correction only on the green channel by doing the following:

With Mathematica, you can bring new ideas into focus. No one knows that better than Fritz Lebowsky. He's a senior principal engineer who does image-quality-related algorithm development for STMicroelectronics, a global manufacturer of electronics and semiconductors with advanced image processing technologies. Thanks to Mathematica's advanced programming language and computational power, Lebowsky is seeing major advancements in his image processing development work, with his latest color imaging project set to double performance while reducing cost by half. About the development, he says, "For the very first time in my research career I could combine several simple non-linear functions/dimensions to overcome some fundamental weaknesses in today's linear mathematics applied to image processing."

We just attended the International Conference on Image Processing (ICIP) in Brussels and the 14th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in Toronto. Both events proved to be great occasions to demonstrate how Mathematica can aid research in image processing, from investigation and prototyping to deployment. Last but not least, we seized the opportunity to listen to the experts in the field to extend the functionality of Mathematica in the right direction. Last year, Ruogu Fang, Kevin D. Tang, Noah Snavely, and Tsuhan Chen received the best paper award at ICIP for their study of kinship detection from pairs of images. I decided to build a detector by following their proposed framework with Mathematica. A lot has been said and studied about our human ability to recognize faces; computer programs can be quite good at it, too. Similarly, when looking at faces, we may display a canny ability to detect an ancestry relationship, or kinship. In these pictures, is the person on the left the father of the person on the right? Yes, and we can figure it out with Mathematica.

Creating an interactive app could be a complex and painstaking task. Not with Mathematica. Here I will present how I created a photo booth program in three easy steps—mostly during my lunch breaks. Step One: Architecture The application will have four main stages. Stage one: We show live webcam images with different image effects applied (possibly multiple pages of them) as a preview, and let the user choose one. Stage two: The chosen image takes up the window, waiting for the user to click a button. Stage three: Count down. Stage four: Capture an image, apply the effect, and add it in a film strip. Repeat.

When last seen in the whereabouts of the Marlborough Maze, I was slinking off stage left, having been upstaged by Jon McCloone and his mix of image processing and graph theory alchemy. In a comment on my post, Jaebum Jung showed similar methods. Me, I only wanted to compute a bunch of distances from the entrance, then walk the maze. But I was not at that time able to show which was the shortest path, or even to prune off the dead ends. I'm over that lapse now. In this post I will provide brief Mathematica code to take the grid of maze pathway distances that I computed, and get the hopeless paths to melt away. Technically this is referred to as a retraction—not in the sense of an apology, but, rather, topology.