Wolfram Computation Meets Knowledge

How to Count Cells, Annihilate Sailboats, and Warp the Mona Lisa

In a recent series of Image Processing with Mathematica workshops held at universities across the United States, we presented Mathematica‘s new image processing functionality and applied it on the spot to attendees’ real-world problems. It was amazing to me to see how rapidly and flexibly Mathematica could be applied to solve complex image processing problems. For example, it might seem like writing a program to automatically count cells in an image would be a master’s research project, but amazingly you can do it with a few lines of Mathematica code.

Below I am using morphological operations and measurement tools to segment and analyze red blood cells in a microscopy image.

Using morphological operations and measurement tools to segment and analyze red blood cells in a microscopy image

Cell segmentation and hole filling can be done with an intensity thresholding using Binarize and FillingTransform:

b = FillingTransform[ColorNegate[Binarize[rbc]]]

Using Binarize and FillingTransform to show cell segmentation and hole filling

As you may notice, some cells are overlapping, and this will eventually cause misleading measurements. In order to separate the overlapping cells, I extract a set of markers by finding local maximas of the DistanceTransform and use these as markers in WatershedComponents:

distT = DistanceTransform[b, Padding → 0]; marker = MaxDetect[ImageAdjust[distT], 0.02]; w = WatershedComponents[GradientFilter[b, 3], marker, Method → "Rainfall"]; Colorize[w]

Finding local maximas of the DistanceTransform and using these as markers in WatershedComponents

Next, I select cells whose pixel counts fall between a reasonable cell size and use our powerful ComponentMeasurements function to locate and analyze the cells:

cells = SelectComponents[w, "Count", 500 < # < 3000 &]; measures = ComponentMeasurements[cells, {"Centroid", "EquivalentDiskRadius", "Label"}]; Show[rbc, Graphics[{Blue, Circle @@ # & /@ (measures[[All, 2, 1 ;; 2]]), MapThread[Text, {measures[[All, 2, 3]], measures[[All, 2, 1]]}]}]]

Using ComponentMeasurements function to locate and analyze the cells

I can also use ComponentMeasurements to take measurements for all selected components. Here I plot the size distribution of the cells:

Histogram[ComponentMeasurements[cells, "Count"][[All, 2]], 20, AxesLabel → {"pixels"}, PlotLabel → "Histogram of cells' pixel counts"]

Histogram of cells' pixel counts

Image restoration is another major topic in image processing. It is sometimes used to visually enhance the image quality or make the details more visible.

Below, I use our new Inpaint function to retouch a crack in an old picture:

Inporting a photo of Abraham Lincoln

In order to locate the crack, I create and use an oriented derivative filter:

OrientedDerivativeFilter[img_, α_, σ_: 1] := Module[{Lxx = ImageData[DerivativeFilter[img, {0, 2}, σ]], Lxy = ImageData[DerivativeFilter[img, {1, 1}, σ]], Lyy = ImageData[DerivativeFilter[img, {2, 0}, σ]]}, Image[Cos[\α]^2 Lxx + 2 Cos[α] Sin[α] Lxy + Sin[α]^2 Lyy]];

This function takes the image derivative along the specified orientation. Here you can see the result of the function on an image of a disk:

Bulding a Manipulate to show the result of an oriented derivative filter on an image of a disc

Snowflakes

To create the crack mask, I take the oriented derivative of my Lincoln image along 120º and compute straight and strong edges in that image:

crack = EdgeDetect[ImageAdjust[OrientedDerivativeFilter[lincoln, 120°]], 2.5, 0.11, "StraightEdges" → 0.25]

Mask for the crack in the image

In order to completely cover the crack, I use morphological Dilation and then use DeleteSmallComponents to delete components that do not belong to the large crack:

mask = DeleteSmallComponents[Dilation[crack, DiskMatrix[2]]]

Mask for the entire crack

Now, removing the crack is as simple as running Inpaint on the original image and the created mask:

result = Inpaint[lincoln, mask]

Repaired Abraham Lincoln image

Let’s take a look at the before and after images side by side:

Grid[{{ImageTake[lincoln, 150]}, {ImageTake[result, 150]}}]

Side-by-side comparison of the two Abraham Lincoln images

Inpaint by default uses a texture synthesis algorithm and therefore can also be used to remove large objects from an image. Here is an example:

Using Inpaint to remove large objects from an image

Image of a harbor with two sailboats removed using Inpaint

Here is again a side-by-side look at the before and after images:

Side-by-side comparison of before and after photos

Restoration techniques such as smoothing, denoising, and deblurring are sometimes essential as preprocessing steps in an image processing task. We have introduced many smoothing functions in Mathematica 8. Below I have an example where an edge-preserving smoothing can help a segmentation task. The image is a kidney ultrasound, and we want to segment the angiomyolipoma (the large, light blob at the upper left):

Importing an image of a kidney ultrasound

I am using our new PeronaMalikFilter, which is an inhomogeneous diffusion method used for smoothing the image while preserving the edges:

p = PeronaMalikFilter[ultrasound, 20, 0.02, 2]

Result of using PeronaMalikFilter

Once the image is smoothed, an intensity thresholding can detect the bright tumor, and then we apply a selection based on the size of the bright object. Below is the tumor highlighted in the original image:

ImageApply[# {1, .5, 1} &, ultrasound, Masking → SelectComponents[Binarize[p], "Area", # > 5000 &]]

The tumor highlighted in the original image

I also demoed Mathematica‘s transformation and registration capabilities starting with ImageTransformation, our new and optimized image transformation that can take any function to spatially transform image pixel coordinates.

Here is an example where a sine wave is added to the x coordinates of the Mona Lisa.

Using ImageTransformation on an image of the Mona Lisa

Image processing
Or a custom function to create a fisheye effect:

f[x_, y_] := With[{r = N@Sqrt[(x - .5)^2 + (y - .5)^2], a = ArcTan[x - .5, y - .5], R = 0.5}, rn = r*r/R; {rn*Cos[a] + .5, rn*Sin[a] + .5}]; ImageTransformation[mona, f[#[[1]], #[[2]]] &, Padding → Transparent]

Fisheye effect on the Mona Lisa

I also showed the integration of Wolfram|Alpha into Mathematica and its current image processing capabilities. We are expanding the extent of these capabilities on a daily basis. Here is a quick example:

apply embossing effect to mona

The Mona Lisa with an embossing effect

In addition, using the Compile function, DLL linking, and new CUDA or OpenCL linking (which can all increase the performance of a program) were among the most popular topics.

Below I have an example where I would like to apply a function to all pixel values in an image using ImageApply. While the two versions give exactly the same result, the computation is ten times faster when using the CompiledFunction:

f[x_] := {Sin[Pi x*4], Sin[Pi x], Sin[Pi x * 3]}

c = Compile[{{x, _Real}}, {Sin[Pi x*4], Sin[Pi x], Sin[Pi x * 3]}];

Using AbsoluteTiming to compare both functions

Comparing two images and the time it took to create them

Here is what some attendees had to say about the Image Processing with Mathematica events:

[The] talk and Workshop focus on my thermography problem were an “eye opener,” even for a long-term user like myself. The imaging processing functions built in to Mathematica are so powerful and comprehensive that I will now probably require my students to use it on their Senior Project….

As someone who has given quite a few technical presentations, I was very impressed with [the seminar]. This just confirms my belief in the quality of the product put out by Wolfram Research… I think that Mathematica is the single most impressive product I have ever used or seen. It is a bonus that Wolfram believes in its product enough to offer free seminars.

We discussed many more image processing applications in the university workshops. If you were not able to attend, I encourage you to explore our image processing functions by starting from the main guide page for image processing and analysis. We also have several training courses and seminars available regularly.

Would you like a Wolfram developer to give a seminar near you? Request a Mathematica presentation, workshop, or event in your area or at your organization.

Thank you to all of the attendees and other presenters for your participation and support. We enjoyed meeting all of you and hope to see you at future Wolfram events.

Download this post as a Computable Document Format (CDF) file.

Comments

Join the discussion

!Please enter your comment (at least 5 characters).

!Please enter your name.

!Please enter a valid email address.

6 comments

  1. aren’t these features included in mathematika 7……nonetheless these are amazing pieces of genius coupled with creativity!!…:-)

    Reply
  2. Weren’t these techniques already covered in the 2011 conference?

    Reply
  3. Thanks. Really powerful functionality.

    Reply
  4. Would be really cool if could just upload a picture to Wolfram Alpha mobile and have it count the objects in a picture for me.

    Reply