Building a Microscopy Application in Mathematica

September 9, 2011 — Jon McLoone, Director, Technical Communication & Strategy

As a change from my usual recreational content, today I thought I would describe a real Mathematica application that I wrote. The project came from my most important Mathematica user—not because she spends a lot of money with Wolfram Research, but because I am married to her!

Her company, Particle Therapeutics, works on needle-free injection devices that fire powdered drug particles into the skin on a supersonic gas shock wave. She was trying to analyze the penetration characteristics on a test medium by photographing thin slices of a target under a microscope and measuring the locations of the particles.

The problem was that her expensive image processing software was doing a poor job of identifying overlapping particles and gave her no manual override for its mistakes.

Faced with the alternative of holding rulers up to her screen and recording each value by hand, I promised that I could do better in Mathematica, with the added advantage that now her image processing tool would be integrated into her analysis code to go from image file to report document in a single workflow.

Here is a detail from a typical image:

The task breaks into three parts:

1. Image processing
2. An optimal GUI for adjusting parameters
3. Report generation

Step 1 seems like it should be the hardest, but it can be achieved in very little code. First I need to clean up the image by deleting dirt and texture. DeleteSmallComponents removes small white dots within the particles; inverting and doing it again with a parameter lets me delete large black components up to the parameter size.

Once cleaned, I need to erode the components back to a single pixel. But I need multiple particles that slightly overlap to break apart and erode back to separate points. This “ultimate erosion” can be done by finding the local maxima of the DistanceTransform of the image.

I can then use ComponentMeasurements to get information on the identified points—in this case the coordinates of the center.

As a final cleanup, I use DeleteDuplicates to remove spurious points that come from badly shaped particles. I give a user parameter for how close points should be before they are considered the same. Here is all of that packaged into a function:

It turns out that, already, this does better than her image processing software, which was only capable of standard Erosion that treats all touching particles as one.

The hardest part is the GUI, because I want to give lots of control. Here is a video of the final application in action:

Let’s walk through each of those elements.

First I want this to be in a palette of its own, not in a notebook. But I want multiple instances to be able to be open at once. So my code starts by localizing variables into the window instance (not globally to the Mathematica session).

The main guts of the user interface (UI) are then just a Manipulate that first calls the identifyBlobs command, and then displays the results.

identifyBlobs will be called automatically when any of the parameters change, and the image will update automatically without any extra work.

The clever bit is the use of locators, which means that the “+” signs for the identified positions are not just for display, but can be changed by dragging them. With the LocatorAutoCreate option, I can Alt-click to add or remove points and automatically update the particle location data variable. That’s a lot of UI for one line of code.

The sliders for the left and right side of the surface line are just Manipulate controls, except that I choose vertical sliders rather than horizontal and place them in specific order. (It is possible to auto-detect the surface with ImageLines, but in some images the surface is so faint that this manual control is more practical.)

Then we have a button to load the data. It’s mostly error trapping code, but note the Method → “Queued” that prevents the Button from timing out if I spend time dithering about which file to open.

I hide the advanced controls using OpenerView, and I specify the layout of the controls inside that.

Finally, to make the scrollable area, all I have to do is set the ContentSize to manual, with an initial size:

Now on to reporting: I can’t share the details of the analysis she was doing, but here is a rough outline for how reporting works. Mathematica provides a complete document description language, so all I have to do is describe the layout and style of the document in terms of the individual visualizations and analysis that I have done.

This creates a new document that starts with three cells: first a static title, then some text summarizing results, and then a chart.

And that’s it.

In a sensible width window, UI, image processing, analysis, and report generation, including a few extra charts, amount to about 30 lines of code and can be deployed to Mathematica or Mathematica Player Pro.

To view the entire application put together, download this post as a Mathematica notebook.

 Since I have complained in the past that there aren’t a lot of posts on real world applications, I couldn’t let this one pass without a comment. This one is awesome! Simple and short code, but with great results. Thank you, Posted by Pedro Fonseca    September 9, 2011 at 10:48 am
 Your wife is a very lucky gal! I hope she takes you to dinner at the least. Very, very impressive application of the power and vision of Mathematica. Posted by Dan Dill    September 9, 2011 at 11:23 am
 As a molecular biologist that uses Mathematica, I am thrilled to see this! There is so much that Mathematica can be used for in biology, but Mathematica is not well known in the biology community. Please keep these examples coming! Posted by Todd Allen    September 9, 2011 at 1:59 pm
 Very cool real world example!.. Posted by Murta    September 9, 2011 at 6:43 pm
 Personally, I only see this blog because of you. Your posts are simple hard things to do on other languages, but easy to do on mathematica, and it isn’t just how to make a simple plot. Many Thanks. Continue the hard work :) Posted by Thales Fernandes    September 9, 2011 at 8:49 pm
 Great post! Thanks for the code. You guys must have some bright kids between the two of you. Well done. Posted by Lucifer    September 10, 2011 at 11:56 pm
 Could you please provide examples for Visual Basic. Thank you. Posted by Eric Hat    September 11, 2011 at 7:10 am
 I do not know anything about biology, codes, waves, gases and only a limited amount about powder but I do know about husbands and wives being one and having one. You did good. Congratulations! Order champagne when she takes you out to dinner and drink to each other’s health. Most impressive! Posted by Lucile Lichtblau    September 11, 2011 at 2:45 pm
 As usual, your blog entries are exceptional. Great job! You have a great way of showing others how to use Mathematica to solve real world problems. Posted by david    September 11, 2011 at 5:45 pm
 Very educational, illustrative example! Although I don’t know what an “optimal GUI” is supposed to be, this is another great example to show the vast applicability of M in very diverse fields of study. Marketing idea: instead of showing various toy and hobby examples at several places on the web, one could research how much taxpayer money was spent on governmental research projects and what the features of the resulting applications were, and then contrast that with a M application that does the same or has even more features, in x programmer days. Maybe people will start seeing the suitability of M when you FORCE them to acknowledge the waste of taxpayer money by showing the cost difference in \$ and c, and demonstrate the corresponding M application. Posted by Mooniac    September 12, 2011 at 1:14 pm
 Thanks for all the positive comments – if you like it, please do post it to your social media of choice! @ Eric Hat. I suspect it would be much harder in Visual Basic, aside from the fact that I haven’t written Basic since I was a child! Probably the easiest approach would be to link to Mathematica from your Visual Basic application for image processing and document generation. Take a look at MathLink and ,NETLink @Mooniac, I am currently doing a little “competitive analysis” and would be most interested in any such examples that would make good case studies. Posted by Jon McLoone    September 12, 2011 at 1:32 pm
 Dear Jon, these are great tools and a great topic. Are there simple enough ways to reverse engineer graphs and maps? For example, how much time would you need with Mathematica to find the shape of the curves on the Higgs graphs e.g. here http://blog.vixra.org/2011/08/31/did-the-higgs-signal-fade/ from the GIF/JPG… images? Similarly, what about extracting temperatures as a function of latitude and longitude at http://www.osdpd.noaa.gov/data/sst/anomaly/anomnight.current.gif If you have some cool answer and time to answer, it would be great if you could send me an e-mail as well. Posted by Luboš Motl    September 13, 2011 at 1:36 am
 @Lubos The second one is relatively easy and would look something like temperature[lat_, lon_] := rgbToTemperature[ ImageValue[ImageTake[image, cropRange], lat*scale1, lon*scale2]]; Where the two scale values are related to the image dimensions, cropRange would be just the map area and rgbToTemperature would take a triple and decode the color map (sending black to inderterminate). The first one is a bit harder as there is some cleaning needed. But generally this is trying to implement a software tool like Un-scan-it in Mathematica. That has been on my “possible blog topics” list for at least a year, but I haven’t got round to seeing how easy it would be yet. Posted by Jon McLoone    September 13, 2011 at 12:26 pm
 What is that “fishy” looking thing in the top right of the raw image? Its just looks so odd in an otherwise spherule and strand dominated picture. Posted by Isaac Abraham    September 14, 2011 at 10:58 am
 Very clever. Something similar to your earlier blog, by witch I was also attracted because of incredible idea, just to mention here: http://blog.wolfram.com/2009/06/23/musical-archaeology-with-mathematica/ Thank you for your genius blogs!!! David Posted by David    September 15, 2011 at 5:12 am
 Jon McLoone, the ideas you use in this article will be similar to the ones used for finding coordinates for motion capture the only thing missing is the splitting of the movie into frames and the mechanism for exporting the data into a motion capture file format. Posted by Julio de la Yncera    November 10, 2011 at 5:26 pm
 Very nice application. However, the response to the sliders is VERY slow. It appears that mma is redrawing the image each time. Is there a way around this? Posted by Kevin McCann    March 31, 2012 at 4:09 pm
 @Kevin McCann You are correct, it is redrawing each time, and would be a lot better if it only redrew the overlay line. You can make it do that by replacing the Show command in line 5 of the final code with Show[img, Epilog -> {Thickness[.001], Cyan, Dynamic[Line[{{0, h l}, {w, h r}}]]}] Wrapping Dynamic around the part of the image that is expected to change allows Mathematica to identify that none of the rest of the image has changed and needs redrawing. Posted by Jon McLoone    April 3, 2012 at 5:31 am
 Do you know of any themal camara that can be connected directly to the Mathematica software? Posted by Jose E. Calderon    March 24, 2013 at 12:01 am
 Mathematica should support any USB video device class cameras. Posted by Jon McLoone    April 9, 2013 at 8:13 am
 How do I get this particle analyzer program? Posted by Xiaxing Xiong    July 27, 2015 at 8:10 pm
 Thank you for your comment, the Mathematica notebook attached at the end of the blog post is available for download. Posted by The Wolfram Team    July 31, 2015 at 9:30 am
 I’m having trouble with adding an InputField. I tried to follow the instruction for Manipulate[] function by adding: {value, InputField} inside Manipulate[]. It did give me a blank input box, but I cannot type anything inside the box. I’d appreciate if you can give me some suggestions, because I want to plug in the number of pixels per unit length in order to find the size distribution (I can do the slide bar, but that would not be as convenient as plugging in the number). I’m new to Mathematica, so this blog is a great help. Thank you so much for posting this! Posted by Hieu Nguyen    September 1, 2016 at 9:46 pm
 The problem us my use of CreatePalette which sets a collection of Notebook options that prevent editing. Change that to CreateDocument and the InputField will work, but you will have to specify some of the options like WindowSize, ShowCellBracket, Editable etc to get it to look as nice as it does with CreatePalette. Posted by Jon McLoone    September 7, 2016 at 3:30 am