Wolfram Blog
Jon McLoone

Building a Microscopy Application in Mathematica

September 9, 2011 — Jon McLoone, International Business & Strategic Development

As a change from my usual recreational content, today I thought I would describe a real Mathematica application that I wrote. The project came from my most important Mathematica user—not because she spends a lot of money with Wolfram Research, but because I am married to her!

Her company, Particle Therapeutics, works on needle-free injection devices that fire powdered drug particles into the skin on a supersonic gas shock wave. She was trying to analyze the penetration characteristics on a test medium by photographing thin slices of a target under a microscope and measuring the locations of the particles.

Photograph of thin slices of a target under a microscope and measuring the locations of the particles

The problem was that her expensive image processing software was doing a poor job of identifying overlapping particles and gave her no manual override for its mistakes.

Faced with the alternative of holding rulers up to her screen and recording each value by hand, I promised that I could do better in Mathematica, with the added advantage that now her image processing tool would be integrated into her analysis code to go from image file to report document in a single workflow.

Here is a detail from a typical image:

image = ImageTake[Import["022108shot06.jpg"], All, {1, 800}]

Detail from typical microscopy image

The task breaks into three parts:

  1. Image processing
  2. An optimal GUI for adjusting parameters
  3. Report generation

Step 1 seems like it should be the hardest, but it can be achieved in very little code. First I need to clean up the image by deleting dirt and texture. DeleteSmallComponents removes small white dots within the particles; inverting and doing it again with a parameter lets me delete large black components up to the parameter size.

image2 = DeleteSmallComponents[ColorNegate[DeleteSmallComponents[Binarize[image]]], 200]

Image after removing small white dots within the particles, then inverting and doing it again with a parameter lets us delete large black components up to the parameter size

Once cleaned, I need to erode the components back to a single pixel. But I need multiple particles that slightly overlap to break apart and erode back to separate points. This “ultimate erosion” can be done by finding the local maxima of the DistanceTransform of the image.

Using ImageAdjust to find single pixels

Image showing two particles that slightly overlap

Using MaxDetect to find local maxima of two points

Two single pixels after finding local maxima

I can then use ComponentMeasurements to get information on the identified points—in this case the coordinates of the center.

ComponentMeasurements[MaxDetect[DistanceTransform[image2]], "Centroid"]

Coordinates of the center

As a final cleanup, I use DeleteDuplicates to remove spurious points that come from badly shaped particles. I give a user parameter for how close points should be before they are considered the same. Here is all of that packaged into a function:

Using identifyBlobs and DeleteDuplicates to remove extra points

It turns out that, already, this does better than her image processing software, which was only capable of standard Erosion that treats all touching particles as one.

The hardest part is the GUI, because I want to give lots of control. Here is a video of the final application in action:

Let’s walk through each of those elements.

First I want this to be in a palette of its own, not in a notebook. But I want multiple instances to be able to be open at once. So my code starts by localizing variables into the window instance (not globally to the Mathematica session).

Using CreatePalette to localize variables into the window instance

The main guts of the user interface (UI) are then just a Manipulate that first calls the identifyBlobs command, and then displays the results.

Using Manipulate to call the identifyBlobs command

identifyBlobs will be called automatically when any of the parameters change, and the image will update automatically without any extra work.

The clever bit is the use of locators, which means that the “+” signs for the identified positions are not just for display, but can be changed by dragging them. With the LocatorAutoCreate option, I can Alt-click to add or remove points and automatically update the particle location data variable. That’s a lot of UI for one line of code.

Using locators to identify positions

The sliders for the left and right side of the surface line are just Manipulate controls, except that I choose vertical sliders rather than horizontal and place them in specific order. (It is possible to auto-detect the surface with ImageLines, but in some images the surface is so faint that this manual control is more practical.)

Setting vertical sliders

Then we have a button to load the data. It’s mostly error trapping code, but note the Method → “Queued” that prevents the Button from timing out if I spend time dithering about which file to open.

Setting a button to load the data

I hide the advanced controls using OpenerView, and I specify the layout of the controls inside that.

Hiding the advanced controls using OpenerView and laying out the controls

Finally, to make the scrollable area, all I have to do is set the ContentSize to manual, with an initial size:

Setting the ContentSize to manual with an initial size

Now on to reporting: I can’t share the details of the analysis she was doing, but here is a rough outline for how reporting works. Mathematica provides a complete document description language, so all I have to do is describe the layout and style of the document in terms of the individual visualizations and analysis that I have done.

Using CreateDocument to describe the layout and style of the document

This creates a new document that starts with three cells: first a static title, then some text summarizing results, and then a chart.

And that’s it.

In a sensible width window, UI, image processing, analysis, and report generation, including a few extra charts, amount to about 30 lines of code and can be deployed to Mathematica or Mathematica Player Pro.

To view the entire application put together, download this post as a Mathematica notebook.

Leave a Comment

20 Comments


Pedro Fonseca

Since I have complained in the past that there aren’t a lot of posts on real world applications, I couldn’t let this one pass without a comment.

This one is awesome!

Simple and short code, but with great results.

Thank you,

Posted by Pedro Fonseca    September 9, 2011 at 10:48 am
Dan Dill

Your wife is a very lucky gal! I hope she takes you to dinner at the least. Very, very impressive application of the power and vision of Mathematica.

Posted by Dan Dill    September 9, 2011 at 11:23 am
Todd Allen

As a molecular biologist that uses Mathematica, I am thrilled to see this!

There is so much that Mathematica can be used for in biology, but Mathematica is not well known in the biology community. Please keep these examples coming!

Posted by Todd Allen    September 9, 2011 at 1:59 pm
Murta

Very cool real world example!..

Posted by Murta    September 9, 2011 at 6:43 pm
Thales Fernandes

Personally, I only see this blog because of you.
Your posts are simple hard things to do on other languages, but easy to do on mathematica, and it isn’t just how to make a simple plot.

Many Thanks.

Continue the hard work :)

Posted by Thales Fernandes    September 9, 2011 at 8:49 pm
Lucifer

Great post! Thanks for the code. You guys must have some bright kids between the two of you. Well done.

Posted by Lucifer    September 10, 2011 at 11:56 pm
Eric Hat

Could you please provide examples for Visual Basic.

Thank you.

Posted by Eric Hat    September 11, 2011 at 7:10 am
Lucile Lichtblau

I do not know anything about biology, codes, waves, gases and only a limited amount about powder but I do know about husbands and wives being one and having one. You did good. Congratulations! Order champagne when she takes you out to dinner and drink to each other’s health. Most impressive!

Posted by Lucile Lichtblau    September 11, 2011 at 2:45 pm
david

As usual, your blog entries are exceptional. Great job! You have a great way of showing others how to use Mathematica to solve real world problems.

Posted by david    September 11, 2011 at 5:45 pm
Mooniac

Very educational, illustrative example! Although I don’t know what an “optimal GUI” is supposed to be, this is another great example to show the vast applicability of M in very diverse fields of study.

Marketing idea: instead of showing various toy and hobby examples at several places on the web, one could research how much taxpayer money was spent on governmental research projects and what the features of the resulting applications were, and then contrast that with a M application that does the same or has even more features, in x programmer days. Maybe people will start seeing the suitability of M when you FORCE them to acknowledge the waste of taxpayer money by showing the cost difference in $ and c, and demonstrate the corresponding M application.

Posted by Mooniac    September 12, 2011 at 1:14 pm
Jon McLoone

Thanks for all the positive comments – if you like it, please do post it to your social media of choice!

@ Eric Hat. I suspect it would be much harder in Visual Basic, aside from the fact that I haven’t written Basic since I was a child! Probably the easiest approach would be to link to Mathematica from your Visual Basic application for image processing and document generation. Take a look at MathLink and ,NETLink

@Mooniac, I am currently doing a little “competitive analysis” and would be most interested in any such examples that would make good case studies.

Posted by Jon McLoone    September 12, 2011 at 1:32 pm
Luboš Motl

Dear Jon, these are great tools and a great topic.

Are there simple enough ways to reverse engineer graphs and maps?

For example, how much time would you need with Mathematica to find the shape of the curves on the Higgs graphs e.g. here

http://blog.vixra.org/2011/08/31/did-the-higgs-signal-fade/

from the GIF/JPG… images? Similarly, what about extracting temperatures as a function of latitude and longitude at

http://www.osdpd.noaa.gov/data/sst/anomaly/anomnight.current.gif

If you have some cool answer and time to answer, it would be great if you could send me an e-mail as well.

Posted by Luboš Motl    September 13, 2011 at 1:36 am
Jon McLoone

@Lubos
The second one is relatively easy and would look something like
temperature[lat_, lon_] :=
rgbToTemperature[
ImageValue[ImageTake[image, cropRange], lat*scale1, lon*scale2]];

Where the two scale values are related to the image dimensions, cropRange would be just the map area and rgbToTemperature would take a triple and decode the color map (sending black to inderterminate).

The first one is a bit harder as there is some cleaning needed. But generally this is trying to implement a software tool like Un-scan-it in Mathematica. That has been on my “possible blog topics” list for at least a year, but I haven’t got round to seeing how easy it would be yet.

Posted by Jon McLoone    September 13, 2011 at 12:26 pm
Isaac Abraham

What is that “fishy” looking thing in the top right of the raw image? Its just looks so odd in an otherwise spherule and strand dominated picture.

Posted by Isaac Abraham    September 14, 2011 at 10:58 am
David

Very clever. Something similar to your earlier blog, by witch I was also attracted because of incredible idea, just to mention here:

http://blog.wolfram.com/2009/06/23/musical-archaeology-with-mathematica/

Thank you for your genius blogs!!!

David

Posted by David    September 15, 2011 at 5:12 am
Julio de la Yncera

Jon McLoone, the ideas you use in this article will be similar to the ones used for finding coordinates for motion capture the only thing missing is the splitting of the movie into frames and the mechanism for exporting the data into a motion capture file format.

Posted by Julio de la Yncera    November 10, 2011 at 5:26 pm
Kevin McCann

Very nice application. However, the response to the sliders is VERY slow. It appears that mma is redrawing the image each time. Is there a way around this?

Posted by Kevin McCann    March 31, 2012 at 4:09 pm
Jon McLoone

@Kevin McCann
You are correct, it is redrawing each time, and would be a lot better if it only redrew the overlay line. You can make it do that by replacing the Show command in line 5 of the final code with

Show[img, Epilog -> {Thickness[.001], Cyan, Dynamic[Line[{{0, h l}, {w, h r}}]]}]

Wrapping Dynamic around the part of the image that is expected to change allows Mathematica to identify that none of the rest of the image has changed and needs redrawing.

Posted by Jon McLoone    April 3, 2012 at 5:31 am
Jose E. Calderon

Do you know of any themal camara that can be connected directly to the Mathematica software?

Posted by Jose E. Calderon    March 24, 2013 at 12:01 am
    Jon McLoone

    Mathematica should support any USB video device class cameras.

    Posted by Jon McLoone    April 9, 2013 at 8:13 am


Leave a comment

Loading...

Or continue as a guest (your comment will be held for moderation):