Wolfram Blog
Michael Pilat

Computable Data—Already Updated in Mathematica

May 8, 2007 — Michael Pilat, Senior Software Engineer, Algorithms R&D

Well, Mathematica 6 has only been out a week, and I’m happy to announce that we’ve already released an update! What’s better, you probably already have it if you use Mathematica 6!

How is that possible?

After exploring the What’s New website, you might have noticed the page for Load-on-Demand Curated Data, which says our “efficient load-on-demand mechanism makes hundreds of gigabytes of carefully curated and continually updated data immediately available inside Mathematica for use in computations.”

We’ve done a lot of work to aggregate data in a variety of disciplines, from chemistry to graph theory, geography to linguistics. This data is collected from a broad range of sources and processed both automatically and by knowledgeable experts here at Wolfram Research, with the goal of providing data that is consistent, computable and accurate.

How is this data delivered to you? At first, you might guess it’s shipped along with the many other features of the product. But the box doesn’t have a dozen CDs in it and there’s only a single download from our online store. That’s because Mathematica itself automatically downloads the data it needs, when it needs it.

Data about financial markets, for example, changes constantly, and accordingly we designed the function FinancialData to download live data in real-time. Even things you might consider constant can change too: new elements are discovered, or the members of the UN Security Council change. Thus, functions like ElementData and CountryData download data and cache it locally, so that your computations are always as fast and accurate as possible.

We can update these data collections completely independent of Mathematica releases, and you can benefit from the updates immediately.

Here’s an inside look at how that happens…

Our data experts are constantly researching and collecting data. Once updates or additions to a data set have been made, the data is processed by custom software tools (written in Mathematica!) into the download components that the data function would request from our data servers.

These components are then subjected to both automated quality-assurance testing and manual verifications. After the data has been approved, we push the new updates onto our servers, and they are available for download.

Finally, Mathematica will notice that updates are available, and the next time you use a function with updates, you’ll see a temporary blue box like this in your notebook while the update is downloaded and applied:

Updating data from Wolfram Research data server (27%)

The data is downloaded automatically and immediately available. You don’t even have to restart your kernel!

Comments are closed.