Wolfram Computation Meets Knowledge

Slicing Silhouettes of Jupiter: Processing JunoCam Images

Juno images processing

With the images from the Juno mission being made available to the public, I thought it might be fun to try my hand at some image processing with them. Though my background is not in image processing, the Wolfram Language has some really nice tools that lessen the learning curve, so you can focus on what you want to do vs. how to do it.

The Juno mission arose out of the effort to understand planetary formation. Jupiter, being the most influential planet in our solar system—both literally (in gravitational effect) and figuratively (in the narrative surrounding our cosmic origin)—was the top contender for study. The Juno spacecraft was launched into orbit to send back high-res images of Jupiter’s apparent “surface” of gases back to Earth for study in order to answer some of the questions we have about our corner of the universe.

The images captured by the Juno spacecraft give us a complete photographic map of Jupiter’s surface in the form of color-filtered, surface patch images. Assembling them into a complete color map of the surface requires some geometric and image processing.

Preprocessing the Images

Images from the JunoCam were taken with four different filters: red, green, blue and near-infrared. The first three of these are taken on one spacecraft rotation (about two revolutions per minute), and the near-infrared image is taken on the second rotation. The final image product stitches all the single-filter images together, creating one projected image.

NASA has put together a gallery of images captured through the JunoCam that contains all the pieces used for this procedure, including the raw, unsorted image; the red, green and blue filtered images; and the final projected image.

Let’s first import the specific red, green and blue images:

Input 1

imgBlue = 
  Import["~/Desktop/JunoImages/ImageSet/JNCE_2017192_07C00061_V01-\
blue.png"];
imgGreen = 
  Import["~/Desktop/JunoImages/ImageSet/JNCE_2017192_07C00061_V01-\
green.png"];
imgRed = Import[
   "~/Desktop/JunoImages/ImageSet/JNCE_2017192_07C00061_V01-red.png"];

Input 2

{imgRed, imgGreen, imgBlue}

To assemble an RGB image from these bands, I use ColorCombine:

Input 3

jup = ColorCombine[{imgRed, imgGreen, imgBlue}] //
  ImageResize[#, Scaled[.25]] &

To clear up some of the fogginess in the image, we need to adjust its contrast, brightness and gamma parameters:

Input 4

jupInit = ImageAdjust[IMAGE,{.14(*contrast*), .3(*brightness*), 2.(*gamma*)}]

You can see that there’s a shadowing effect that wasn’t as prominent to begin with in the initial color-combined image. To prevent the shadowing on the foreground image from disturbing any further analysis, the brightness needs to be uniform throughout the image. I first create a mask that limits the correction to the white area:

Input 5

newMask = Binarize[jupInit, {0.01, 1}]

When I apply this mask, I get:

Input 6

jupBright = BrightnessEqualize[jupInit, Masking -> newMask]

It’s much darker now, so I have to readjust the image. This time, I’m doing it interactively using a Manipulate:

Input 7

stretchImage[image_] := Block[{thumbnail},
   
   thumbnail = ImageResize[image, Scaled[.7]];
   With[{t = thumbnail},
    Manipulate[
     ImageAdjust[t, {c, b, g}],
     {{c, 0, "Contrast"}, -5.0, 5.0, 0.01},
     {{b, 0, "Brightness"}, -5.0, 5.0, 0.01},
     {{g, 2.0, "Gamma"}, 0.01, 5.0, 0.001},
     ControlPlacement -> {Bottom, Bottom, Bottom}
     
     ]
    ]];

Input 8

stretchImage[IMAGE]

I use the parameter values I found with the Manipulate to create an adjusted image:

Input 9

jupadj = ImageAdjust[IMAGE,{-.16, 3.14, 1.806}];

Any time an image is captured on camera, it’s always a little bit blurred. The Wolfram Language has a variety of deconvolution algorithms available for immediate use in computations—algorithms that reduce this unintended blur.

Most folks who do image processing, especially on astronomical images, have an intuition for how best to recover an image through deconvolution. Since I don’t, it’s better to do this interactively:

Input 10

deconvolveImage[image_] := Block[{thumbnail},
   
   thumbnail = ImageResize[image, Scaled[.7]];
   With[{t = thumbnail},
    Manipulate[
     ImageDeconvolve[t, GaussianMatrix[n], Method -> "RichardsonLucy"],
     {{n, 0, "Blur Correction Factor"}, 1, 3.0, 0.1},
     ControlPlacement -> Bottom
     
     ]
    ]];

Input 11

deconvolveImage[jupadj]

Again, I use the blur correction I found interactively to make an unblurred image:

Input 12

jupUnblur = 
  ImageDeconvolve[jupadj, GaussianMatrix[1.7], 
   Method -> "RichardsonLucy"];

And as a sanity check, I’ll see how these changes look side by side:

Input 13

table = Transpose@
   {{"Original", jup},
    {"Initial Correction", jupInit},
    {"Uniform Brightness", jupBright},
    {"Better Adjustment", jupadj},
    {"Deconvolved Image", jupUnblur}};
Row[
 MapThread[
  Panel[#2, #1, ImageSize -> Medium] &,
  table]]

Processing the Image

Now that the image has been cleaned up and prepared for use, it can be analyzed in a variety of ways—though it’s not always apparent which way is best. This was a very exploratory process for me, so I tried a lot of methods that didn’t end up working right, like watershed segmentation or image Dilation and Erosion; these are methods that are great for binarized images, but the focus here is enhancing colorized images.

With Jupiter, there is a lot of concentration on the Great Red Spot, so why not highlight this feature of interest?

To start, I need to filter the image in a way that will easily distinguish three different regions: the background, the foreground and the Great Red Spot within the foreground. In order to do this, I apply a MeanShiftFilter:

Input 14

filtered = MeanShiftFilter[jupadj, 1, .5, MaxIterations -> 10]

This is useful because this filter removes the jagged edges of the Great Red Spot. Additionally, this filter preserves edges, making the boundary around the Great Red Spot smoother and easy for a computer to detect.

Using Manipulate once again, I can manually place seed points that indicate the locations of the three regions of the image (you can see how much the filter above helped separate out the regions):

Input 15Input 15Juno images processing

Manipulate[seeds = pts;
 Row[
  {Image[jupadj, ImageSize -> All], 
   Image[ImageForestingComponents[jupadj, pts] // Colorize, 
    ImageSize -> All],
   Image[ImageForestingComponents[filtered, pts] // Colorize, 
    ImageSize -> All]
   }
  ],
  {
  {pts, RandomReal[Min[ImageDimensions[jupadj]], {3, 2}]},
  {0, 0},
  ImageDimensions[jupadj],
  Locator,
  Appearance -> Graphics[{Green, Disk[{0, 0}]}, 
    ImageSize -> 10],
  LocatorAutoCreate -> {2, 10}
  }
 ]

The values of the seeds at these places are stored within a variable for further use:

Input 16

seeds

Using these seeds, I can do segmentation programmatically:

Input 17

Colorize[label = ImageForestingComponents[filtered, seeds, 2]]

With the regions segmented, I create a mask for the Great Red Spot:

Input 18

mask = Colorize[DeleteBorderComponents[label]]

I apply this mask to the image:

Input 19

ImageApply[{1, 0, 0} &, jupadj, Masking -> mask]

This is great, but looking at it more, I wish I had an approximate numerical boundary for the Great Red Spot region in the image. Luckily, that’s quite straightforward to do in the Wolfram Language.

Our interactive right-click menu helped me navigate the image to find necessary coordinates for creating this numerical boundary:

Coordinates tool 1

It’s a handy UI feature within our notebook front end—intuitively guiding me through finding roughly where the y coordinate within the Great Red Spot is at a minimum:

Coordinates tool 2

As well as where the x coordinate within that same region is at a minimum:

Coordinates tool 3

I also did this for the maximum values for each coordinate. Using these values, I numerically generate ranges of numbers with a step size of .1:

Input 20

x = Range[144, 275, .1];
y = Range[264, 350, .1];

I construct the major and minor axes:

Input 21

xRadius = (Max[x] - Min[x])/2;
yRadius = (Max[y] - Min[y])/2;

And I approximate the center:

Input 22

center = {Min[x] + xRadius, Min[y] + yRadius}

And finally, I create the bounding ellipse:

Input 23

bounds = Graphics[{Thick, Blue,
   RegionBoundary[
    Ellipsoid[center, {xRadius, yRadius}]
    ]}]

This bounding ellipse is applied to the image:

Input 24

HighlightImage[jupadj, bounds]

More Neat Analysis on Jupiter

Aside from performing image processing on external JunoCam images in order to better understand Jupiter, there are a lot of built-in properties for Jupiter (and any other planet in our solar system) already present in the language itself, readily available for computation:

Input 25

\!\(\*
NamespaceBox["LinguisticAssistant",
DynamicModuleBox[{Typeset`query$$ = "Jupiter", Typeset`boxes$$ = 
      TemplateBox[{"\"Jupiter\"", 
RowBox[{"Entity", "[", 
RowBox[{"\"Planet\"", ",", "\"Jupiter\""}], "]"}], 
        "\"Entity[\\\"Planet\\\", \\\"Jupiter\\\"]\"", "\"planet\""}, 
       "Entity"], 
      Typeset`allassumptions$$ = {{
       "type" -> "Clash", "word" -> "Jupiter", 
        "template" -> "Assuming \"${word}\" is ${desc1}. Use as \
${desc2} instead", "count" -> "3", 
        "Values" -> {{
          "name" -> "Planet", "desc" -> "a planet", 
           "input" -> "*C.Jupiter-_*Planet-"}, {
          "name" -> "Mythology", "desc" -> "a mythological figure", 
           "input" -> "*C.Jupiter-_*Mythology-"}, {
          "name" -> "GivenName", "desc" -> "a given name", 
           "input" -> "*C.Jupiter-_*GivenName-"}}}}, 
      Typeset`assumptions$$ = {}, Typeset`open$$ = {1, 2}, 
      Typeset`querystate$$ = {
      "Online" -> True, "Allowed" -> True, 
       "mparse.jsp" -> 0.926959`6.418605518937624, "Messages" -> {}}}, 
DynamicBox[ToBoxes[
AlphaIntegration`LinguisticAssistantBoxes["", 4, Automatic, 
Dynamic[Typeset`query$$], 
Dynamic[Typeset`boxes$$], 
Dynamic[Typeset`allassumptions$$], 
Dynamic[Typeset`assumptions$$], 
Dynamic[Typeset`open$$], 
Dynamic[Typeset`querystate$$]], StandardForm],
ImageSizeCache->{149., {7., 17.}},
TrackedSymbols:>{
        Typeset`query$$, Typeset`boxes$$, Typeset`allassumptions$$, 
         Typeset`assumptions$$, Typeset`open$$, Typeset`querystate$$}],
DynamicModuleValues:>{},
UndoTrackedVariables:>{Typeset`open$$}],
BaseStyle->{"Deploy"},
DeleteWithContents->True,
Editable->False,
SelectWithContents->True]\)["Properties"] //
 Take[#, 30] &

Included here is a textured equirectangular projection of the surface of Jupiter: perfect for 3D reconstruction!

Input 26

surface = 
 Entity["Planet", "Jupiter"][
    EntityProperty["Planet", "CylindricalEquidistantTexture"]] //
   NestList[Sharpen, #, 2] & //
  #[[-1]] &

Using this projection, I can map it to a spherical graphic primitive:

Input 27

sphere[image_] := Block[{plot},
  
  plot = SphericalPlot3D[1, {theta, 0, Pi}, {phi, 0, 2 Pi}, 
    Mesh -> None, TextureCoordinateFunction -> ({#5, 1 - #4} &), 
    PlotStyle -> Directive[Texture[image]], Lighting -> "Neutral", 
    Axes -> False, Boxed -> False, PlotPoints -> 30]
  ]

Input 28Jupiter Juno image post-processing animation

sphere[surface]

Final Thoughts…

I started out knowing next to nothing about image processing, but with very few lines of code I was able to mine and analyze data in a fairly thorough way—even with little intuition to guide me.

The Wolfram Language abstracted away a lot of the tediousness that would’ve come with processing images, and helped me focus on what I wanted to do. Because of this, I’ve found some more interesting things to try, just with this set of data—like assembling the raw images using ImageAssemble, or trying to highlight features of interest by color instead of numerically—and feel much more confident in my ability to extract the kind of information I’m looking for.


If you’d like to work with the code you read here today, you can download this post as a Wolfram Notebook.

Comments

Join the discussion

!Please enter your comment (at least 5 characters).

!Please enter your name.

!Please enter a valid email address.

1 comment