News, Views & Insights
Mission Mathematica: Making a Photo Booth App in Three Steps
Creating an interactive app could be a complex and painstaking task. Not with Mathematica. Here I will present how I created a photo booth program in three easy steps—mostly during my lunch breaks.
Step One: Architecture
The application will have four main stages. Stage one: We show live webcam images with different image effects applied (possibly multiple pages of them) as a preview, and let the user choose one. Stage two: The chosen image takes up the window, waiting for the user to click a button. Stage three: Count down. Stage four: Capture an image, apply the effect, and add it in a film strip. Repeat.
We also need nice photo effects. We are going to have three categories—three effects for each category. I chose three categories: color effects, transformations, and something else. But of course, you can add more later. We will discuss the details in the next step.
Now, it is time to think about user interface. I followed a popular UI that is pretty common in photo booth applications. The first stage will have a two-by-two grid that shows an original as well as three different effects in real time. Radio buttons at the bottom will provide navigation through the categories. The second stage is rather simple—just a big screen with the click button and the back button underneath it. At the third stage, there will be no interface change except that the counter will be shown instead of buttons. We’ll keep a separate film strip (notebook) for results, in which captured images will be added at the fourth stage.
Step Two: Effects
Let’s spend some time to come up with nice effects. The first category is “color effects”. Here are my choices.
The transformation category requires a bit more effort. The first effect is a mirroring at the center. We crop the right half of an image, create a reflection of the left, then combine them.
Here is a wiggly effect to make you feel dizzy.
The final choice for this category is a “broken mirror” effect, created by our design director, Jeremy Davis. Essentially, we cut an image into vertical pieces and flip them before combining them back together.
Let’s see how they all look.
They are very simple effects, yet very disturbing…
The last category is a mystery category. I will leave it to the readers to come up with their own. Here are my choices:
Erosion and Dilation are basic morphological operations that shrink (erode) or expand (dilate) areas which are not black (background). These give some funny effects that weren’t expected.
Now we are ready to get to the details.
Step Three: Implementation
For capturing and playing webcam images in real time, CurrentImage combined with Dynamic will do the job.
![Dynamic[CurrentImage[]]](https://content.wolfram.com/sites/39/2011/06/DynamicCurrentImage.gif "Dynamic[CurrentImage[]]"){kind=small}
Of course, you can wrap the CurrentImage with any image processing function to get the dynamic effect (or several effects together).
![Dynamic[Map[Function[{f}, f[CurrentImage[]]], {bandw, sepia, psychadelic}]]](https://content.wolfram.com/sites/39/2011/06/INput-14.gif "Dynamic[Map[Function[{f}, f[CurrentImage[]]], {bandw, sepia, psychadelic}]]"){kind=small}
The main program will look roughly like this:
Interactive examples in Mathematica can be written in a more efficient way by localizing Dynamic and using TrackedSymbols to make it update only when a specific variable is changed. However, the program is efficient enough as is, so we will not go all the way to optimize it.
The whole process will be enclosed by DynamicModule and then CreatePalette so that we can generate a standalone palette.
There are a few things that have to be handled before running the process. For instance, filters have to be defined, and the film strip notebook has to be created. These jobs can be defined in the Initialization of the DynamicModule so they will run before the process. Also, when you close the palette, there is some housecleaning that has to be done. The Deinitialization option is useful to define such jobs.
Now let’s look at the details. The two-by-two preview grid at the first stage looks like this:
filters and names are arrays of photo effects and their names. Overlay is a new function in Mathematica 8, which comes in very handy when you want to display multiple expressions at once. Here is an example.
At stage two, we show a webcam image with the chosen effect applied in real time. At stage three, we do the same thing, but we need to count down simultaneously. You can introduce Refresh or Dynamic with UpdateInterval, but as the documentation points out, it is not 100% reliable, especially when the notebook front end is busy with updating webcam images with the fast frame rate.
Is there a more reliable solution? Of course, yes. We introduced the concept of timed evaluations in Mathematica 8. Using this functionality, you can create a scheduled task that can be independently evaluated by specific time interval. For our example, we will create a task that will be evaluated every second to decrease the counter and, as an added bonus, to announce the remaining time.
The task should be created at Initialization and be removed at Deinitialization of the main Dynamic by using RemoveScheduledTask. The task will be triggered and stopped using StartScheduledTask and StopScheduledTask at the third stage.
Finally, at the fourth stage, we make a click sound, write the image into the film strip notebook using NotebookWrite, and go back to the first stage.
I created a notebook that contains all necessary code and resources for the app, including a sound byte for the camera click and some icons for buttons. Unfortunately, we can’t provide it in Computable Document Format (CDF) for the free Wolfram CDF Player, since it is accessing webcams, which is restricted. For the folks who don’t have Mathematica, here is a screen capture of me playing with the program.
So there you have it—mission accomplished! Stay tuned to the Wolfram Blog for more Mission Mathematica installments.
Hello,
I think this is really what users of Home Edition will appreciate. There is no tricky math neither physic. The author chose a very simple problem not to show his skill to solve difficult abstract things but to explain how to use Mathematica to create a clear user interface and a quite complex application. This post should by picked up and shown in the section “What You Can Do with Mathematica Home Edition” where, unfortunately, is nothing similar to encourage more people to experiment with Mathematica. I wish there were more posts like this. A good job.
Very interesting post.
Unfortunately, for the linux versions of Mathematica CurrentImage as well as ImageCapture are not supported giving as an output:
CurrentImage::notsupported: “Image acquisition is not supported on Unix
Hope this to be fixed in future releases.
Nice! Wish I had a camera though. :D Much purely UI-wise to appreciate anyway.
@Miroslav,
Thank you for the suggestion. We will definitely consider it.
@Dimitris,
Yes, quite unfortunately it is unsupported on Linux. And yes, we are considering it for future releases.
… and then please also anti-aliasing on Linux, we’ve been waiting for that since M6, and running M on Linux as guest o/s. Anti-aliasing has been working on Windows since M6, but even in 8.0.1 it doesn’t work on Linux, and M on a guest o/s doesn’t work either.
Is Mathematica primarily aimed at tech hobbysists/enthusiasts? The marketing message seems very confusing because blog posts never seem to focus on real world commercial applications — therefore I’m assuming this is hobby software???
Also given that this sort of photo software is readily available and often bundled with products such as cameras and webcams, devotion of a blog to it in Mathematica also seems to suggest that this is primarily a hobby programming example.
For hobby programming Mathematica looks interesting but I’d like to see Wolfram form some sort of technical group to create real world commercial examples.
thanks
D
Dudley, the following is my own opinion, from someone who is very familiar with M as he has been using it ever since version 2.2 in 1993. The following is not an official statement, I am not a WRI employee (anymore — I used to be).
M is CERTAINLY not “just for hobbyists”, although in some other blogs (outside from this website) it appears as if only hobbyists are posting there. M is used in science and industry, at universities, at small and large corporations, it even runs on the space station. M has a few million users world-wide. Some applications are so powerful that the real problem is people not understanding what is happening, because they only have a traditional IT background, and when they see baffling results they think it’s fake.
“Hobby” or “Recreation” are definitely some applications, but is that a surprise with a system that has such a vast area of applicability?
If you look at the demonstrations or the M library or read MathGroup, you’ll see that most M users don’t use M as “hobby”. It’s mostly about education and applications in science and industry.
I definitely DO think M is underrepresented at the corporations, but that’s their own fault.
As far as this particular blog post is concerned, although it’s definitely not on my “favorites list”, the point of these blogs is to show people how to do neat things yourself with M, in a flexible and code-efficient manner. Just because something exists already in other software, doesn’t make it a “hobby” blog post. You may have to pay money for the other software, it’s “canned” (in the sense you can’t modify it — if a feature wasn’t implemented you don’t have it), and by using it you don’t learn anything. It’s not about giving you this particular feature. It’s about showing you how to get ideas, how to do things, so that you use (part of) these ideas for your own applications. You may learn something about the functional programming style or image processing in M. If you don’t … read the next one. They’re short.
Total agree with Mr. Burns. Although it’s funny to see the webcam direct link capacities on a photo software, I would prefer to see an application of this capacity, for example, on the monitoring or data collecting of a visual phenomenon (ex.: real-time tracing of a moving object), showing also the capability of visually overlapping simulation results with real-time experiment, etc ( I guess there are a lot of more examples, from different fields).
Best regards,
Mooniac pointed out some real facts about M but to really feel the ubiquitous presence of M in high tech and computationally intensive industrial problem solving one has to use M from earlier versions like 2.2 as he mentioned. I on the other hand work in HPC and in the field of mathematical modelling. I was in my 2nd grade in 1993. I and most of my young friends started to use M from at least version 5 or later. We noticed that most information about M in the internet are very outdated and mostly from 90’s. At that time as a pioneering CA system M came to lime light. But now how M is getting used in state of the art mathematical applications? Very little info one will find in the internet. Media presence of M has waned drastically and I guess patronizing the interest of the hobbyist has been one of the reason. Presenting M as high school math apparatus is the reason why “M is underrepresented at the corporations”.
I personally know M has much more power than what appears from most of the Blog posts here. I am sure talented people are in in no shortage at WRI and they can enlighten the vast internet community with example where some real world industrial scale challenges are tackled. No need of complete elaborate solutions but hints and creative approaches highlighting M’s unique power in certain computational fields will suffice. M should prove its worth by solving problems that are novel and that will pave the way for more sustained development in future. Interest of the corporations will grow too. Media will open their eyes once again. The old glory will surely prevail one just need to come up with examples that are pertinent and appealing to 21st century scientific community. This Blog fits the best to show the world what M actually is capable of. For sure creating wonderful GUI based toy apps written with neat and compact code will not revive the charm of M among scientific community. The demonstration website is serving that purpose. This place must be for the best of the best. From the creators of M we expect their best creations which will be interesting from the state of the art scientific and mathematical aspect.
However my comment is a general one considering the flavor of most blog posts here. I understand the point Mooniac made but those “so powerful” applications are not much discussed in today’s media or the over all scientific community. It is time that we take initiative to bring them in the upfront. I thank Dudley for writing his view openly any Mathematica lover will do the same.
Just to clarify, I am not critical of this blog per se, it accomplishes what it sets out to do. The point I was making is that looking through the selection of blog posts there seems to be a consistent “gee whiz” hobbyist/enthusiast theme but very little related to the real world commercial environment, by which I mean as a commercial guy I can read this stuff and think, yeah that’s cool but it bears no relationship to anything I do in my company.
I can see some value in this stuff from an educational standpoint as well. So education uses seem to be clear.
But you would never guess that this software has a commercial use based on how it is presented.
Hi, is there any tutorial about computer vision/augmented reality in mathematica??? Please help.
Very nice,
Nice work, Yu-Sung. I was wondering if you could help me with the following: I would *love* to be able to capture images from more than one webcam connected to a single PC via de CaptureImage[] command. I want to use this to try out Stereo Vision and motion capture with my high school students. Any hope CaptureImage[] will ever allow for that? Does it allow it already?
@Dan,
No, unfortunately, I don’t think that current CaptureImage[] allow connection to two devices.
The default webcam is defined in $DefaultImageDevice, but the only way to change it is to use ImageCapture, which does not take two.
But, I think that there could be a way to use 2 kernels for instance.
I will investigate it a bit and get back to you.