WOLFRAM

News, Views & Insights

Data Analysis and Visualization

Computation & Analysis

Searching Genomes with Mathematica and HadoopLink

Editorial note: This post was written by Paul-Jean Letourneau as a follow-up to his post Mathematica Gets Big Data with HadoopLink. In my previous blog post I described how to write MapReduce algorithms in Mathematica using the HadoopLink package. Now let's go a little deeper and write a more serious MapReduce algorithm. I've blogged in the past about some of the cool genomics features in Wolfram|Alpha. You can even search the human genome for DNA sequences you're interested in. Biologists often need to search for the locations of DNA fragments they find in the lab, in order to know what animal the fragment belongs to, or what chromosome it's from. Let's use HadoopLink to build a genome search engine!
Wolfram Blog Team
Computation & Analysis

Random and Optimal Mathematica Walks on IMDb’s Top Films

Or: How I Learned to Watch the Best Movies in the Best Way I remember when I lived across the street from an art movie theater called Le Club, looking at the movie posters on my way back home was often enough to get me in the ticket line. The director or main actors would ring a bell, or a close friend had recommended the title. Sometimes the poster alone would be appealing enough to lure me in. Even today there are still occasions when I make decisions from limited visual information, like when flipping through movie kiosks, TV guides, or a stack of DVDs written in languages I can't read. So how can Mathematica help? We'll take a look at the top 250 movies rated on IMDb. Based on their posters and genres, how can one create a program that suggests which movies to see? What is the best way to see the most popular movies in sequence?
Matthias Odisio, Software Developer, Algorithms R&D
Announcements & Events

Why Would a Mathematica User Care about R?

The benefits of linking from Mathematica to other languages and tools differ from case to case. But unusually, in the case of the new RLink in Mathematica 9, I think the benefits have very little to do with R, the language. The real benefit, I believe, is in the connection it makes to the R community. When we first added the MathLink libraries for C, there were real benefits in farming out intensive numerical work (though Mathematica performance improvements over the years and development of the compiler have greatly reduced the occasions where that would be worth the effort). Creating an Excel link added an alternative interface paradigm to Mathematica that wasn't available in the Mathematica front end. But in the case of R, it isn't immediately obvious that it does many things that you can't already do in Mathematica or many that it does significantly better. However, with RLink I now have immediate access to the work of the R community through the add-on libraries that they have created to extend R into their field. A great zoo of these free libraries fill out thousands of niches--sometimes popular, sometimes obscure--but lots of them. There are over 4,000 right here and more elsewhere. At a stroke, all of them are made immediately available to the Mathematica environment, interpreted through the R language runtime.
Jon McLoone, Director, Technical Communication & Strategy
Computation & Analysis

Seeing Skin with Mathematica

Detecting skin in images can be quite useful: it is one of the primary steps for various sophisticated systems aimed at detecting people, recognizing gestures, detecting faces, content-based filtering, and more. In spite of this host of applications, when I decided to develop a skin detector, my main motivation lay elsewhere. The research and development department I work in at Wolfram Research just underwent a gentle reorganization. With my colleagues who work on probability and statistics becoming closer neighbors, I felt like developing a small application that would make use of both Mathematica's image processing and statistics features; skin detection just came to my mind. Skin tones and appearances vary, and so do flavors of skin detectors. The detector I wanted to develop is based on probabilistic models of pixel colors. For each pixel of an image given as input, the skin detector provides a probability that the pixel color belongs to a skin region.
Matthias Odisio, Software Developer, Algorithms R&D
Best of Blog

Data Science of the Facebook World

More than a million people have now used our Wolfram|Alpha Personal Analytics for Facebook. And as part of our latest update, in addition to collecting some anonymized statistics, we launched a Data Donor program that allows people to contribute detailed data to us for research purposes. A few weeks ago we decided to start analyzing […]

Stephen Wolfram
Computation & Analysis

Mathematica’s Role in Powering Energy Saving Solutions

Using Mathematica and other Wolfram technologies, Joseph Hirl, founder of Agilis Energy, has developed a new approach to energy analytics that is helping building owners and energy equipment suppliers around the world cut energy consumption and costs. At the core of the company's success is its Mathematica-based dynamic energy analysis application, which gives the full picture of a building's performance, measures the impact of potential operational changes, and quantifies the results. About Mathematica's role in the development of the tool and the Agilis business, Hirl says, "The flexibility of Mathematica is tremendous. Our ability to build and develop this program with a lean staff has allowed us to build out a substantial business." The application, which has now been used at more than 800 sites in at least 12 different industries, begins with data streams, including high-interval smart meter data as well as Mathematica's built-in WeatherData. It then applies sophisticated statistics and dynamic visualization functionality to generate what Hirl calls an "MRI of a building," a dynamic interface with a simulation of the building's energy use and demand and forecasting and benchmarking tools.
Wolfram Blog Team
Education & Academic

Centennial of Markov Chains

On January 23, 1913 of the Julian calendar, Andrey A. Markov presented for the Royal Academy of Sciences in St. Petersburg his analysis of Pushkin's Eugene Onegin. He found that the sequence of consonants and vowels in the text could be well described as a random sequence, where the likely category of a letter depended only on the category of the previous or previous two letters. At the time, the Russian Empire was using the Julian calendar. The 100th anniversary of the celebrated presentation is actually February 5, 2013, in the now used Gregorian calendar. To perform his analysis, Markov invented what are now known as "Markov chains," which can be represented as probabilistic state diagrams where the transitions between states are labeled with the probabilities of their occurrences.
Oleksandr Pavlyk, Senior Kernel Developer, Algorithms R&D
Education & Academic

Volumetric Rendering of Colliding Galaxies

The physics involved in simulating galaxy collisions can be extremely complex. The most accurate simulations take into account not just points representing stars, but also magnetic fields and invisible dark matter, as well as n-body interactions allowing the individual stars to interact with each other. These complex simulations are usually carried out on large-scale supercomputers over long periods of time. One of the more interesting aspects of galaxy collisions is that they can create density variations resulting in all kinds of emergent structure. Density waves can develop that lead to star formation from compressed gas clouds. A couple of years ago, I wrote a Demonstration that provides a simplified solution to galaxy collisions. This Demonstration is designed to run in real time inside a Manipulate, so the problem has been simplified by removing n-body interactions, dark matter, magnetic fields, and so on. Basically, it treats the two galaxies as large point masses with lots of massless test particles orbiting them. The test particles respond only to the two galaxy "centers." In a real galaxy collision, the chances of two stars getting close enough to each other to interact directly is very remote, so it's not too far of a stretch to ignore this effect for a first-order approximation. The more stars that are included in the simulation (by minimizing the star separation parameter), the more intricate the results (and the more computationally intense). In fact, as more stars are added, it becomes easier to see density variations where many test masses cluster together, but it still looks very discrete. Real galaxies, like the Milky Way, can have hundreds of billions of stars. Trying to carry out a point simulation with that many stars becomes a bit taxing on most home systems, and definitely exceeds the time constraints of a real-time dynamic tool like Manipulate. So how can we better visualize these density variations? I decided to try to modify my Demonstration to use one of the new features in Mathematica 9, namely volumetric rendering. This way, we can simulate the galaxy collisions with fewer numbers of points, but render the results as if there were billions of stars, resulting in a more realistic and informative visualization.
Jeff Bryant, Research Programmer, Wolfram|Alpha Scientific Content
Show More