Wolfram Blog
Jon McLoone

How to Make a Webcam Intruder Alarm with Mathematica

November 10, 2010 — Jon McLoone, International Business & Strategic Development

Ever since I wrote the “Doing Spy Stuff with Mathematica” blog post, I have had a feeling that I am being watched. Time to build some office security using Mathematica Home Edition!

First, I am going to make use of an imminent new Mathematica command CurrentImage, which will import a real-time image from a video device. Let’s get some test images using the webcam on my laptop.

Imported real-time test image

And again, but with a couple of seconds to hide from the camera:

Second imported real-time test image

ImageDifference gives us a pixel-by-pixel absolute difference between the two images. My camera isn’t very good, so there is a bit of noise. As a quick way to deal with that, I first blur both images slightly.

Pixel-by-pixel absolute difference between the two images

This image is black where there was no difference between the images. I don’t care how different they are, so I will apply a threshold. I have applied this to the maximum difference out of all the color channels, since we want a change in color to be as important as an overall change in brightness.

Threshold applied to maximum difference out of all the color channels

Now we clump together these differences by filling in gaps of 10 pixels or less.

Differences clumped together

This is all we need to detect an intruder. So let’s put those steps together into a function.

Find changes in the image

We can use this simplified scene-change image in two important ways.

Firstly, we can use it to highlight what has changed in an image. I will use it to darken the rest of the image by 75% and add a white line around the edge of the changed area.

Highlighting what changed between the images

Secondly, we can use it as a measure of the amount that has changed.

Measuring the amount of change

In this case there has been one scene change of over 37,000 pixels. Had the coffee cup been removed or another added, we would see a second smaller component listed as well.

Now I can create a function that tests and acts on the result of the scene changes. If the size of the biggest change is more than 1,000 pixels, then we will call that an intruder event, and we will use the SendMail command to send me email to tell me about it together with the highlighted changed image. I ignore the case of over 70,000 pixels, as I don’t want an alarm when someone turns off the office lights and the entire image changes.

Intruder test function

Next I want to check the current scene against a time lagged store of what the image should look like. We want to update the store regularly, so that lasting changes such as an item being taken from the scene or placed there, or the sun setting, do not produce repeated alerts.

Checking current scene against time lagged store of the image

Finally, we need to call the function repeatedly. I could use an infinite loop, but that would lock my Mathematica completely. I might want to run overnight computations at the same time, or have other events happen periodically, so instead I program a scheduled interrupt (another new capability) that will suspend any running Mathematica program, run this command, and then return to the program.

Scheduled interrupt to run the program

Now let’s use it.

Set the alarm

This gives me 60 seconds to leave the room and then runs the task every 15 seconds, keeping a 4 minute buffer of background images.

Later that night I get the following emails. (OK, not really!) Not bad for a couple dozen lines of code!

Email alert of an intruder event

Email alert of intruder trying to steal from my desk

Posted in: Image Processing
Leave a Comment

18 Comments


Manfred Plagmann

You are on a roll, Jon. One interesting blog after another. This is teasing! I could right now use the functionality you are show casing here. Many thanks.

Posted by Manfred Plagmann    November 10, 2010 at 7:53 pm
Mooniac

Nice application, quite educational. However, if this is supposed to be used as a practical application, I’d caution a few things: a) 15 seconds is a lot of time. Plenty of time to get in, steal or break something, and disappear (or momentarily hide beneath the desk or somewhere else off camera). Most professional motion-detection software uses at least 5 frames per second. The one pointing towards the Pentagon on 911 couldn’t even capture the plane or rocket (whatever theory you believe in) using even more frames per second. b) My motion-detection specifically DOES trigger an alarm if the lights are turned off, because now it WOULD be possible for someone to get in unseen and do whatever they want, as long as they don’t need light for that. c) Also make sure you cover the entry area, not just the desk area, because otherwise the intruder just needs to sneak towards the computer (rob on the ground), pull the power cable, and now your whole system is rendered useless. There’s no point in an intrusion detection system if the intruder can easily turn it off. d) If you run this in complete darkness, make sure you turn off the screen, lest some screen reflections from sending you an email (in the M f/e or in the email program) may be picked up by the camera and trigger a new alert due to pixel changes. You’d get an email every 15 s! Then I’d call that a “low-frequency oscillator”, not an intrusion-detection system.

One could write an interesting “fixed-point study” that uses M to display images based on the images it gets from the camera, when that camera is pointing at the screen (how fast can it be done, color divergences, flip-flops, etc.).

Posted by Mooniac    November 10, 2010 at 10:00 pm
Ross

I have developed a package which adds Python style object-orientation to Mathematica. But I would like to develop automatic garbage collector as well.

About MathOO: http://www.voofie.com/content/169/mathoo-adding-python-style-object-orientation-to-mathematica-with-mathoo-10-beta-launch-alternative-/

And here is the problem I am facing about Temporary variables in Mathematica:

http://www.voofie.com/content/178/adding-garbage-collector-to-mathoo/

Is there any way to workaround the problem? And can Mathematica deal with reference cycles? Thank you very much.

Posted by Ross    November 11, 2010 at 4:47 am
Seth Chandler

This one might be your best yet. Totally cool.

Posted by Seth Chandler    November 11, 2010 at 7:47 am
selvarajan

Wow! If this post isn’t marketing then what is? After reading post after post of image processing , who wouldn’t upgrade?

Posted by selvarajan    November 11, 2010 at 9:38 pm
Daniel Crowley

Couldn’t help but notice that dropping a cloth over the webcam beforehand would not only defeat the webcam from capturing an image of you as an intruder, but given the drastic image change, would not trigger an alert either.

An interesting problem, hm?

Posted by Daniel Crowley    November 12, 2010 at 12:32 am
Łukasz Jankowski

@Daniel Crowley
As a workaround you can use an imminent new Mathematica command NeglectWebcamCloth[]. A systematic solution would be to use a pair of webcams that would monitor each other, besides the door.

Posted by Łukasz Jankowski    November 12, 2010 at 9:54 am
James Bond

” I must equiped with Mathematica Home Edition, instantly.” — Bond

Posted by James Bond    November 12, 2010 at 8:21 pm
Julio de la Yncera

I had a good laugh at the NeglectWebCamCloth[] imminent Mathematica command! :-)

But we could also solve the problem by placing a mirror so that the camera can see itself and therefore able to check itself.

Posted by Julio de la Yncera    November 20, 2010 at 7:48 pm
Mooniac

Despite my positive comment further up, I’d like to mention that there really seems to be a break in the previous “must run on all platforms” philosophy. For CUDA support this is, of course, totally understandable. But not in general. Already the first command, a=CurrentImage[] throws an error

CurrentImage::notsupported: “Image acquisition is not supported on Unix.

I could trace this down to

$DefaultImagingDevice::notsupported: Image acquisition is not supported on Unix.

and

$ImagingDevices::notsupported: Image acquisition is not supported on Unix

Posted by Mooniac    November 23, 2010 at 2:55 pm
Vincent

Is it really so that there is no way to control the settings of the webcam? White balance, contrast brightness and such?

Posted by Vincent    August 19, 2011 at 2:25 am
Jon McLoone

@ Vincent
As long as you are within the dynamic range of your webcam, then you can achieve that by post-processing the image with ImageAdjust. On-camera adjustments will vary between cameras, but it is common that such adjustments are actually post-processing of the original CCD data.

Posted by Jon McLoone    August 23, 2011 at 4:58 am
Dan Reznik

Jon, I really enjoy your posts. i am trying to do a motion capture (attach ping-pong hemispheres — makeshift markers — to a student wearing a black leotard, capture images from multiple webcams, locate the markers on 2d images, do some math, and infer the 3d position of the markers). Do you know if we can make Mathematica 8.0 capture images from more than one webcam simultaneously? What would you suggest?

Posted by Dan Reznik    September 27, 2011 at 12:15 pm
Jon McLoone

@Dan
Thanks.
The methods for switching cameras, is not exposed at the user level except throught the intereractive interface of ImageCapture[].

I suspect that because the image capturing is set up for streaming, that the time taken to swtich between them, even if you could programatically switch, would be too slow to keep up any kind of frame rate. (At the moment). I would probably capture both images to files and then post process them using Import to pick out specific frames at leasure, after the event.

Posted by Jon McLoone    September 30, 2011 at 12:27 pm
Pavel

How can I capture sound (without picture) in Mathematica?
Best regards from Prague
Pavel

Posted by Pavel    November 30, 2011 at 5:57 am
Arihant

I am trying to prototype a mathematica application tha will enable Autonomous Intruder & Anomalous Event Detection, Tracking and Alarm to web users those who want to autonomously and asynchronously monitor their private spaces such as homes, offices as well as outdoor spaces such as backyards, roads and recreation facilities using inexpensive commerical Internet-connected IP-cams and webcams.

I want to employ a proven and tested unsupervised algrothm that will help me accomplish my goal. Any idea which one would be the best? What are the mathematica functions that will help me customize such an algorithm? I am using network license of Mathematica-8.

Posted by Arihant    April 25, 2012 at 7:29 am
Andrestand

Great post. I found your steganography one some weeks ago by googling something, and now that I looked for something like this in order not to program it, I came here.

It would be lovely to also have here a code download link like in the other post, in order not to use some OCR code…

Cheers!

Posted by Andrestand    December 3, 2014 at 7:30 am
Andrestand

ImagesToOCR =
Table[Import[
"http://blog.wolfram.com/data/uploads/2010/11/IntruderAlarm"
ToString[i] “.png”], {i, 2, 3}]~Join~
Table[Import[
"http://blog.wolfram.com/data/uploads/2010/11/IntruderAlarm"
ToString[i] “.png”], {i, 5, 14}]

OCRedCode = Table[TextRecognize[ImagesToOCR[[i]]], {i, 1, 12}];

Posted by Andrestand    December 4, 2014 at 10:15 am


Leave a comment

Loading...

Or continue as a guest (your comment will be held for moderation):