July 22, 2014 — Wolfram Blog
Photography by Tracy Howl and Paul Clarke
Has our newfound massive availability of data improved decisions and lead to better democracy around the world? Most would say, “It’s highly questionable.”
Conrad Wolfram’s TEDx UK Parliament talk poses this question and explains how computation can be key to the answer, bridging the divide between availability and practical accessibility of data, individualized answers, and the democratization of new knowledge generation. This transformation will be critical not only to government efficiency and business effectiveness—but will fundamentally affect education, society, and democracy as a whole.
Wolfram|Alpha and Mathematica 10 demos feature throughout—including a live Wolfram Language generated tweet.
February 21, 2014 — Wolfram Blog Team
Editorial note: This post was written by Paul-Jean Letourneau as a follow-up to his post Mathematica Gets Big Data with HadoopLink.
I’ve blogged in the past about some of the cool genomics features in Wolfram|Alpha. You can even search the human genome for DNA sequences you’re interested in. Biologists often need to search for the locations of DNA fragments they find in the lab, in order to know what animal the fragment belongs to, or what chromosome it’s from. Let’s use HadoopLink to build a genome search engine!
July 15, 2013 — Matthias Odisio, Mathematica Algorithm R&D
Or: How I Learned to Watch the Best Movies in the Best Way
I remember when I lived across the street from an art movie theater called Le Club, looking at the movie posters on my way back home was often enough to get me in the ticket line. The director or main actors would ring a bell, or a close friend had recommended the title. Sometimes the poster alone would be appealing enough to lure me in. Even today there are still occasions when I make decisions from limited visual information, like when flipping through movie kiosks, TV guides, or a stack of DVDs written in languages I can’t read.
So how can Mathematica help? We’ll take a look at the top 250 movies rated on IMDb. Based on their posters and genres, how can one create a program that suggests which movies to see? What is the best way to see the most popular movies in sequence?
May 22, 2013 — Jon McLoone, International Business & Strategic Development
The benefits of linking from Mathematica to other languages and tools differ from case to case. But unusually, in the case of the new RLink in Mathematica 9, I think the benefits have very little to do with R, the language. The real benefit, I believe, is in the connection it makes to the R community.
When we first added the MathLink libraries for C, there were real benefits in farming out intensive numerical work (though Mathematica performance improvements over the years and development of the compiler have greatly reduced the occasions where that would be worth the effort). Creating an Excel link added an alternative interface paradigm to Mathematica that wasn’t available in the Mathematica front end. But in the case of R, it isn’t immediately obvious that it does many things that you can’t already do in Mathematica or many that it does significantly better.
However, with RLink I now have immediate access to the work of the R community through the add-on libraries that they have created to extend R into their field. A great zoo of these free libraries fill out thousands of niches–sometimes popular, sometimes obscure–but lots of them. There are over 4,000 right here and more elsewhere. At a stroke, all of them are made immediately available to the Mathematica environment, interpreted through the R language runtime.
May 9, 2013 — Matthias Odisio, Mathematica Algorithm R&D
Detecting skin in images can be quite useful: it is one of the primary steps for various sophisticated systems aimed at detecting people, recognizing gestures, detecting faces, content-based filtering, and more. In spite of this host of applications, when I decided to develop a skin detector, my main motivation lay elsewhere. The research and development department I work in at Wolfram Research just underwent a gentle reorganization. With my colleagues who work on probability and statistics becoming closer neighbors, I felt like developing a small application that would make use of both Mathematica‘s image processing and statistics features; skin detection just came to my mind.
Skin tones and appearances vary, and so do flavors of skin detectors. The detector I wanted to develop is based on probabilistic models of pixel colors. For each pixel of an image given as input, the skin detector provides a probability that the pixel color belongs to a skin region.
April 24, 2013 — Stephen Wolfram
More than a million people have now used our Wolfram|Alpha Personal Analytics for Facebook. And as part of our latest update, in addition to collecting some anonymized statistics, we launched a Data Donor program that allows people to contribute detailed data to us for research purposes.
A few weeks ago we decided to start analyzing all this data. And I have to say that if nothing else it’s been a terrific example of the power of Mathematica and the Wolfram Language for doing data science. (It’ll also be good fodder for the Data Science course I’m starting to create.)
We’d always planned to use the data we collect to enhance our Personal Analytics system. But I couldn’t resist also trying to do some basic science with it.
I’ve always been interested in people and the trajectories of their lives. But I’ve never been able to combine that with my interest in science. Until now. And it’s been quite a thrill over the past few weeks to see the results we’ve been able to get. Sometimes confirming impressions I’ve had; sometimes showing things I never would have guessed. And all along reminding me of phenomena I’ve studied scientifically in A New Kind of Science.
So what does the data look like? Here are the social networks of a few Data Donors—with clusters of friends given different colors. (Anyone can find their own network using Wolfram|Alpha—or the
SocialMediaData function in Mathematica.)
March 5, 2013 — Wolfram Blog Team
Using Mathematica and other Wolfram technologies, Joseph Hirl, founder of Agilis Energy, has developed a new approach to energy analytics that is helping building owners and energy equipment suppliers around the world cut energy consumption and costs.
At the core of the company’s success is its Mathematica-based dynamic energy analysis application, which gives the full picture of a building’s performance, measures the impact of potential operational changes, and quantifies the results. About Mathematica‘s role in the development of the tool and the Agilis business, Hirl says, “The flexibility of Mathematica is tremendous. Our ability to build and develop this program with a lean staff has allowed us to build out a substantial business.”
The application, which has now been used at more than 800 sites in at least 12 different industries, begins with data streams, including high-interval smart meter data as well as Mathematica‘s built-in WeatherData. It then applies sophisticated statistics and dynamic visualization functionality to generate what Hirl calls an “MRI of a building,” a dynamic interface with a simulation of the building’s energy use and demand and forecasting and benchmarking tools.
February 4, 2013 — Oleksandr Pavlyk, Kernel Technology
On January 23, 1913 of the Julian calendar, Andrey A. Markov presented for the Royal Academy of Sciences in St. Petersburg his analysis of Pushkin’s Eugene Onegin. He found that the sequence of consonants and vowels in the text could be well described as a random sequence, where the likely category of a letter depended only on the category of the previous or previous two letters.
At the time, the Russian Empire was using the Julian calendar. The 100th anniversary of the celebrated presentation is actually February 5, 2013, in the now used Gregorian calendar.
To perform his analysis, Markov invented what are now known as “Markov chains,” which can be represented as probabilistic state diagrams where the transitions between states are labeled with the probabilities of their occurrences.
January 8, 2013 — Jeffrey Bryant, Scientific Information Group
The physics involved in simulating galaxy collisions can be extremely complex. The most accurate simulations take into account not just points representing stars, but also magnetic fields and invisible dark matter, as well as n-body interactions allowing the individual stars to interact with each other. These complex simulations are usually carried out on large-scale supercomputers over long periods of time. One of the more interesting aspects of galaxy collisions is that they can create density variations resulting in all kinds of emergent structure. Density waves can develop that lead to star formation from compressed gas clouds.
A couple of years ago, I wrote a Demonstration that provides a simplified solution to galaxy collisions. This Demonstration is designed to run in real time inside a Manipulate, so the problem has been simplified by removing n-body interactions, dark matter, magnetic fields, and so on. Basically, it treats the two galaxies as large point masses with lots of massless test particles orbiting them. The test particles respond only to the two galaxy “centers.” In a real galaxy collision, the chances of two stars getting close enough to each other to interact directly is very remote, so it’s not too far of a stretch to ignore this effect for a first-order approximation. The more stars that are included in the simulation (by minimizing the star separation parameter), the more intricate the results (and the more computationally intense). In fact, as more stars are added, it becomes easier to see density variations where many test masses cluster together, but it still looks very discrete. Real galaxies, like the Milky Way, can have hundreds of billions of stars. Trying to carry out a point simulation with that many stars becomes a bit taxing on most home systems, and definitely exceeds the time constraints of a real-time dynamic tool like Manipulate. So how can we better visualize these density variations? I decided to try to modify my Demonstration to use one of the new features in Mathematica 9, namely volumetric rendering. This way, we can simulate the galaxy collisions with fewer numbers of points, but render the results as if there were billions of stars, resulting in a more realistic and informative visualization.