Wolfram Blog http://blog.wolfram.com News, views, and ideas from the front lines at Wolfram Research. Tue, 30 Aug 2016 16:52:31 +0000 en hourly 1 http://wordpress.org/?v=3.2.1 New Wolfram Language Books http://blog.wolfram.com/2016/08/26/new-wolfram-language-books/ http://blog.wolfram.com/2016/08/26/new-wolfram-language-books/#comments Fri, 26 Aug 2016 15:38:33 +0000 Zach Littrell http://blog.internal.wolfram.com/?p=32028 We are constantly surprised by what fascinating applications and topics Wolfram Language experts are writing about, and we’re happy to again share with you some of these amazing authors’ works. With topics ranging from learning to use the Wolfram Language on a Raspberry Pi to a groundbreaking book with a novel approach to calculations, you are bound to find a publication perfect for your interests.

Getting Started with Wolfram Language and Mathematica for Raspberry Pi, Essentials of Programming in Mathematica, Geospatial Algebraic Computations, Theory and Applications

Getting Started with Wolfram Language and Mathematica for Raspberry Pi, Kindle Edition

If you’re interested in the Raspberry Pi and how the Wolfram Language can empower the device, then you ought to check out this ebook by Agus Kurniawan. The author takes you through the essentials of coding with the Wolfram Language in the Raspberry Pi environment. Pretty soon you’ll be ready to try out computational mathematics, GPIO programming and serial communication with Kurniawan’s step-by-step approach.

Essentials of Programming in Mathematica

Whether you are already familiar with programming or completely new to it, Essentials of Programming in Mathematica provides an excellent example-driven introduction for both self-study and a course in programming. Paul Wellin, an established authority on Mathematica and the Wolfram Language, covers the language from first principles to applications in natural language processing, bioinformatics, graphs and networks, signal analysis, geometry, computer science and much more. With tips and insight from a Wolfram Language veteran and more than 350 exercises, this volume is invaluable for both the novice and advanced Wolfram Language user.

Geospatial Algebraic Computations, Theory and Applications, Third Edition

Advances in geospatial instrumentation and technology such as laser scanning have resulted in tons of data—and this huge amount of data requires robust mathematical solutions. Joseph Awange and Béla Paláncz have written this enhanced third edition to respond to these new advancements by including robust parameter estimation, multi-objective optimization, symbolic regression and nonlinear homotopy. The authors cover these disciplines with both theoretical explorations and numerous applications. The included electronic supplement contains these theoretical and practical topics with corresponding Mathematica code to support the computations.

Boundary Integral Equation Methods and Numerical Solutions: Thin Plates on an Elastic Foundation, Micromechanics with Mathematica, Tendências Tecnológicas em Computação e Informática (Portuguese), The End of Error: Unum Computing

Boundary Integral Equation Methods and Numerical Solutions: Thin Plates on an Elastic Foundation

For graduate students and researchers, authors Christian Constanda, Dale Doty and William Hamill present a general, efficient and elegant method for solving the Dirichlet, Neumann and Robin boundary value problems for the extensional deformation of a thin plate on an elastic foundation. Utilizing Mathematica’s computational and graphics capabilities, the authors discuss both analytical and highly accurate numerical solutions for these sort of problems, and both describe the methodology and derive properties with full mathematical rigor.

Micromechanics with Mathematica

Seiichi Nomura demonstrates the simplicity and effectiveness of Mathematica as the solution to practical problems in composite materials, requiring no prior programming background. Using Mathematica’s computer algebra system to facilitate mathematical analysis, Nomura makes it practical to learn micromechanical approaches to the behavior of bodies with voids, inclusions and defects. With lots of exercises and their solutions on the companion website, students will be taken from the essentials, such as kinematics and stress, to applications involving Eshelby’s method, infinite and finite matrix media, thermal stresses and much more.

Tendências Tecnológicas em Computação e Informática (Portuguese)

For Portuguese students and researchers interested in technological trends in computation and informatics, this book is a real treat. The authors—Leandro Augusto Da Silva, Valéria Farinazzo Martins and João Soares De Oliviera Neto—gathered studies from both research and the commercial sector to examine the topics that mark current technological development. Read about how challenges in contemporary society encourage new theories and their applications in software like Mathematica. Topics include the semantic web, biometry, neural networks, satellite networks in logistics, parallel computing, geoprocessing and computation in forensics.

The End of Error: Unum Computing

Written with Mathematica by John L. Gustafson, one of the foremost experts in high-performance computing and the inventor of Gustafson’s law, The End of Error: Unum Computing explains a new approach to computer arithmetic: the universal number (unum). The book discusses this new number type, which encompasses all IEEE floating-point formats, obtains more accurate answers, uses fewer bits and solves problems that have vexed engineers and scientists for decades. With rich illustrations and friendly explanations, it takes no more than high-school math to learn about Gustafson’s novel and groundbreaking unum.

Want to find even more Wolfram technologies books? Visit Wolfram Books to discover books ranging across both topics and languages.

]]>
http://blog.wolfram.com/2016/08/26/new-wolfram-language-books/feed/ 0
Webinar Training Series: Wolfram|Alpha in the Economics Classroom http://blog.wolfram.com/2016/08/22/webinar-training-series-wolframalpha-in-the-economics-classroom/ http://blog.wolfram.com/2016/08/22/webinar-training-series-wolframalpha-in-the-economics-classroom/#comments Mon, 22 Aug 2016 19:05:56 +0000 Ishwarya Vardhani http://blog.internal.wolfram.com/?p=32124 Are you a teacher who’s been asked “Why am I learning this?”, “How is this going to help me in real life?” and other variations of this question by your students? I know that I faced this when I was teaching, and it can be tough to provide a satisfactory response. However, being able to address this issue is critical in the classroom. We believe that Wolfram|Alpha provides one way to do so.

The Wolfram Knowledgebase, our ever-growing repository of curated computable data, gives you instant access to trillions of data elements across thousands of domains. With Wolfram|Alpha, you can query these data points using natural language (plain English) right in your classroom.

By using real-world data, students have the opportunity to direct their learning toward areas that they care about. In the economics classroom, you can discuss GDP using data about real countries, data that is current and citable. Explore Wolfram|Alpha’s trove of socioeconomic data that will open multiple areas of inquiry in the classroom. A wonderful side effect that I’ve found with using a tool like Alpha is that it also teaches you to pose queries intelligently. Being able to carefully construct a problem is an integral step in the process of thinking critically.

Join us for a special training event on August 24 to learn more about using Wolfram|Alpha in the classroom. This session in the Wolfram|Alpha for Educators: Webinar Training Series will focus on the economics classroom. Previous sessions in this series focused on calculus and physics classrooms, and you can watch our past event recordings online.

If you would like to attend this event, you can register here. Registration is free, and no prior programming experience or knowledge of Wolfram technologies is necessary. We will also have an interactive Q&A chat session where you can participate in discussions with other attendees. I hope to see you there.

]]>
http://blog.wolfram.com/2016/08/22/webinar-training-series-wolframalpha-in-the-economics-classroom/feed/ 0
Training Webinars: New in the Wolfram Language and Mathematica 11 http://blog.wolfram.com/2016/08/17/training-webinars-new-in-the-wolfram-language-and-mathematica-11/ http://blog.wolfram.com/2016/08/17/training-webinars-new-in-the-wolfram-language-and-mathematica-11/#comments Wed, 17 Aug 2016 17:48:52 +0000 Zach Littrell http://blog.internal.wolfram.com/?p=32061 3D printing. Audio. Machine learning. Neural networks. There are 555 completely new functions, major new areas of functionality and a vast deepening of core capabilities in Version 11 of the Wolfram Language and Mathematica. Continuing a three-decade tradition of aggressive innovation, Version 11 is filled to the brim with cutting-edge technology, and we’re excited to share with you how to put all these new features to use.

Join us for a special two-part webinar event, New in the Wolfram Language and Mathematica Version 11, on August 23, 2016, from 2–3:30pm EDT (6–7:30pm GMT) and August 30, 2016, from 2–4pm EDT (6–8pm GMT). Take the opportunity to explore the new features in the Wolfram Language and Mathematica with experts at Wolfram Research, then engage in interactive Q&A with the developers after the presentations.

One example of what's new in Version 11: You can now generate 3D-printable models algorithmically

On Day 1, our Wolfram Language experts will give a comprehensive overview and presentations on four major new areas of functionality—3D printing, audio, improved machine learning and neural networks. Learn how to shape and print 3D models locally or in the cloud. Synthesize, process and analyze audio for a variety of applications. Identify over 10,000 objects and classify and extract features from all sorts of data. And define, train and apply neural networks in a variety of ways, with built-in support for GPU and out-of-core training.

On Day 2, explore the breadth and depth of Version 11

Then on Day 2, we’ll dive into the wide breadth and depth of Version 11. Discover how to compute geometry, geography, statistics, language and visualizations in new, fascinating ways. And with the tight integration between your computer, the Wolfram Language and the Wolfram Cloud, you can now build dynamic web interfaces, access tons of new curated data and connect remote systems with channel-based communication. Even the user interface has been upgraded, with new ways to interact with a Wolfram Language notebook, code captions, enhanced autocomplete and multilingual spellcheck.

To join us at the free virtual events on August 23, 2016, from 2–3:30pm EDT (6–7:30pm GMT) and August 30, 2016, from 2–4pm EDT (6–8pm GMT), please register here.

]]>
http://blog.wolfram.com/2016/08/17/training-webinars-new-in-the-wolfram-language-and-mathematica-11/feed/ 0
Finding Pokémon GO’s Shortest Tour to Compute ’em All! http://blog.wolfram.com/2016/08/12/finding-pokemon-gos-shortest-tour-to-compute-em-all/ http://blog.wolfram.com/2016/08/12/finding-pokemon-gos-shortest-tour-to-compute-em-all/#comments Fri, 12 Aug 2016 16:33:24 +0000 Bernat Espigulé-Pons http://blog.internal.wolfram.com/?p=32070 There’s a Computed Pokémon nearby! Here is a Poké Spikey. This will help you catch ’em all! In this blog post I will share with you several data insights about the viral social media phenomenon that is Pokémon GO. First I will get you familiarized with the original 151 Pokémon that have now invaded our real world, and then I’ll show you how to find the shortest tour to visit your nearby gyms.

A Poké Spikey—you can catch 'em all with the Wolfram Language!

I first came across Pokémon in 1999 when my twin brother and I got Pokémon Blue and Red versions, respectively, along with the Green and Yellow Game Boy Color handhelds shown below. To catch ’em all, we learned to team up. We used a Game Link Cable to trade (and later clone) Pokémon, so we could explore the Kanto region twice as fast. Nostalgia, no doubt, inspired millions to play the recently released Pokémon GO game.

Our first introduction to Pokémon

Pokémon GO has inherited the Pokémon from the Red and Blue versions. But the game dynamics are completely different. You can no longer “catch ’em” all from the sofa. You have to get some exercise in order to help Professor Willow study all these creatures that are now living in our augmented reality. There are three ways to get new specimens: catch ’em, evolve ’em or hatch their eggs using incubators, which requires you to walk a certain distance in kilometers (to honor the Meter Man). Once you catch ’em all, your Pokédex should look like this:

All 151 Pokémon currently available in Pokémon GO—click to see the (very long) full image

These are only Generation 1 Pokémon. But future releases of Pokémon GO will come with additional types of Pokémon, and the trading of Pokémon will be enabled. To make sense of all the different types of Pokémon currently available in the game, I’ve made the following graph (edges are connections of Pokémon with their type and their evolved Pokémon):

A graph of Pokémon with ther types and evolved forms—click for huge full-size image

Dots in blue are Pokémon from newer generations (not yet available in the augmented reality game) that evolve into Generation 1 Pokémon (currently available). I suspect that a possible way to get them in the future will be by means of breeding their evolved forms. Here is how to sneak a peek at these cuties:

A preview glimpse of newer-generation Pokémon

You can do the same for the forthcoming grown-ups. An extreme example is Eevee, which will eventually evolve into eight different types of Pokémon (currently it only evolves into Vaporeon, Jolteon, or Flareon).

Eevee evolutions

Oh! There is still one Pokémon evolving from Eevee missing, Sylveon:

Sylveon

A cool thing about having Pokémon data built into the Wolfram Language is that you can quickly get some insights into these fictional creatures. For example, here is how I’ve quantified all their different weights and heights using a ListLogLogPlot:

Pokémon height vs. weight

To inject these creatures into the real world, Pokémon GO uses the user-generated data from another augmented reality game called Ingress, also developed by Niantic. They have taken Ingress portals, in public spaces such as libraries, sculptures, churches, strange buildings, shops or places Ingress players found interesting, and they have dressed them up as Pokémon Gyms or PokéStops, depending on the portal’s popularity. PokéStops provide experience (XP points) and accessories (Pokéballs, Potions, …) to players who are checking them out. They also serve to attract wild Pokémon when a lure module is installed on them. If you live in a rural area with few PokéStops, you will be better off going to an urban area to play the game. (Most wild Pokémon don’t live in the wild.) For example, here is how insanely dense, in terms of Pokémon GO, a city like Paris can be: Applidium successfully extracted all the coordinates contained in a Pokémon GO app response from their office located in the middle of Paris. I downloaded and imported their KML file into a notebook to build the following custom GeoGraphics:

Extracted Pokémon GO coordinates in Paris

Black birds represent the wild Pokémon that spawned in the area when the fake player requested a response from the Pokémon GO servers. These Pokémon tend to cluster in parks and gardens surrounded by blue GeoMarkers, which are Ingress portals (points of interest) that have been converted into PokéStops and Gyms.

Pokémon Gyms are often placed in well-known public spaces that have their own geolocated entries in Wikipedia. Using our built-in WikipediaData function, I obtained the GeoPosition of locations in Paris with Wikipedia entries that are probably Gyms in the game (following the logic on how Gyms’ locations have been assigned in the Pokémon GO Game), and then I computed the FindShortestTour to visit ’em all:

Shortest path to all probable Pokémon Gyms in Paris, courtesy of the Wolfram Language

Brilliant! With this tour I could hatch three eggs and take control over 65 gyms! I must say that I haven’t tried this strategy yet. But I plan to put it to work in Barcelona. In another post, I talked about how my Runkeeper activities evolved over the span of a year. This was before Pokémon GO was released. These activities and the activities of millions of other players, including Olympic medalists, have dramatically changed.

Me hatching 1000 Pokémon GO eggs!

There is so much data in the Pokémon universe that can be computed with the Wolfram Language. For example, Pokémon have six stats that form the basis for how they perform in battle: hit points, attack, defense, special attack, special defense and speed. As with other numerical values in Wolfram|Alpha, you can refer to them by name in equations or query them with conditions or with ordinals. I will stop my exploration here, but I can’t wait to hear your adventures using computations in the realm of Pokémon.

]]>
http://blog.wolfram.com/2016/08/12/finding-pokemon-gos-shortest-tour-to-compute-em-all/feed/ 1
Today We Launch Version 11! http://blog.wolfram.com/2016/08/08/today-we-launch-version-11/ http://blog.wolfram.com/2016/08/08/today-we-launch-version-11/#comments Mon, 08 Aug 2016 16:28:32 +0000 Stephen Wolfram http://blog.internal.wolfram.com/?p=32026 I’m thrilled today to announce the release of a major new version of Mathematica and the Wolfram Language: Version 11, available immediately for both desktop and cloud. Hundreds of us have been energetically working on building this for the past two years—and in fact I’ve personally put several thousand hours into it. I’m very excited about what’s in it; it’s a major step forward, with a lot of both breadth and depth—and with remarkably central relevance to many of today’s most prominent technology areas.

Featured areas in Version 11 of Mathematica and the Wolfram Language

It’s been more than 28 years since Version 1 came out—and nearly 30 years since I started its development. And all that time I’ve been continuing to pursue a bold vision—and to build a taller and taller stack of technology. With most software, after a few years and a few versions, not a lot of important new stuff ever gets added. But with Mathematica and the Wolfram Language it’s been a completely different story: for three decades we’ve been taking major steps forward at every version, progressively conquering vast numbers of new areas.

It’s been an amazing intellectual journey for me and all of us. From the very beginning we had a strong set of fundamental principles and a strong underlying design—and for three decades we’ve been able to just keep building more and more on these foundations, creating what is by now an unprecedentedly vast system that has nevertheless maintained its unity, elegance and, frankly, modernity. In the early years we concentrated particularly on abstract areas such as mathematics. But over time we’ve dramatically expanded, taking ever larger steps and covering ever more kinds of computation and knowledge.

Each new version represents both a lot of new ideas and a lot of hard work. But more than that, it represents ever greater leverage achieved with our technology. Because one of our key principles is automation, and at every version we’re building on all the automation we’ve achieved before—in effect, we’ve got larger and larger building blocks that we’re able to use to go further and further more and more quickly. And of course what makes this possible is all that effort that I and others have put in over the years maintaining a coherent design for the whole system—so all those building blocks from every different area fit perfectly together.

With traditional approaches to software development, it would have taken a great many years to create what we’ve added in Version 11. And the fact that we can deliver Version 11 now is a direct reflection of the effectiveness of our technology, our principles and our methodology. And as I look at Version 11, it’s very satisfying to see how far we’ve come not only in what’s in the system, but also in how effectively we can develop it. Not to mention that all these directions we’ve been pursuing for so many years as part of the logical development of our system have now turned out to be exactly what’s needed for many of today’s most active areas of technology development.

For many years we called our core system Mathematica. But as we added new directions in knowledge and deployment, and expanded far beyond things related in any way to “math”, we decided to introduce the concept of the Wolfram Language to represent the core of everything we’re doing. And the Wolfram Language now defines the operation not only of Mathematica, but also of Wolfram Development Platform and Wolfram Programming Lab, as well as other products and platforms. And because all our software engineering is unified, today we’re able to release Version 11 of all our Wolfram-Language-based systems, both desktop and cloud.

A sample of products based on the Wolfram Language

OK, so what’s the big new thing in Version 11? Well, it’s not one big thing; it’s many big things. To give a sense of scale, there are 555 completely new functions that we’re adding in Version 11—representing a huge amount of new functionality (by comparison, Version 1 had a total of 551 functions altogether). And actually that function count is even an underrepresentation—because it doesn’t include the vast deepening of many existing functions.

The way we manage development, we’ve always got a portfolio of projects going on, from fairly small ones, to ones that may take five years or more. And indeed Version 11 contains the results of several five-year projects. We’re always keen to deliver the results of our development as quickly as possible to users, so we’ve actually had several intermediate releases since Version 10—and effectively Version 11 represents the combination of many completely new developments together with ones that we’ve already previewed in 10.1, 10.2, 10.3 and 10.4. (Many functions that were tagged as “Experimental” in 10.x releases are now in full production form in Version 11.0.)

The First Things You Notice…

When you first launch Version 11 on your desktop the first thing you’ll notice is that notebooks have a new look, with crisper fonts and tighter design. When you type code there are lots of new autocompletions that appear (it’s getting harder and harder to type the wrong thing), and when you type text there’s a new real-time spellchecker, that we’ll be continually updating to make sure it has the latest words included.

If your computer system is set to any of a dozen languages other than English, you’ll also immediately see something else: every function is automatically annotated with a “code caption” in the language you’ve set:

Code captions in Wolfram Language code, here in Japanese

When you actually run code, you’ll notice that messages look different too—and, very helpfully for debugging, they let you immediately see what chain of functions was being called when the message was produced.

3D Printing

There are lots of big, meaty new areas in Version 11. But let’s jump right into one of them: 3D printing. I made my first 3D printout (which didn’t last long before disintegrating) back in 2002. And we’ve had the ability to export to STL for years. But what’s new and exciting in Version 11 is that we’ve built a complete pipeline that goes from creating 3D geometry to having it printed on your 3D printer (or through a printing service).

Version 11 provides a complete pipeline for 3D printing

Often in the past I’ve wanted to take a 3D plot and just make a 3D print of it. And occasionally I’ve been lucky, and it’s been easy to do. But most of the time it’s a fiddly, complicated process. Because graphics that display on the screen don’t necessarily correspond to geometry that can actually be printed on a 3D printer. And it turns out to be a difficult problem of 3D computational geometry to conveniently set up or repair the geometry so it really works on a 3D printer. (Oh, and if you get it wrong, you could have plastic randomly squirting out of your printer.)

In Version 11 it’s finally realistic to take any 3D plot, and just 3D print it. Or you can get the structure of a molecule or the elevation around a mountain, and similarly just 3D print it. Over the years I’ve personally made many 3D printouts. But each one has been its own little adventure. But now, thanks to Version 11, 3D printouts of everything are easy. And now that I think about it, maybe I need a 3D printout of the growth of the Wolfram Language by area for my desk…

Machine Learning & Neural Networks

In a sense, Mathematica and the Wolfram Language have always been doing AI. Over the years we’ve certainly been pioneers in solving lots of “AI-ish” problems—from mathematical solving to automated aesthetics to natural language understanding. But back in Version 10 we also made a great step forward in machine learning—developing extremely automated core functions (Classify and Predict) for learning by example.

I have to say that I wasn’t sure how well these functions would do in practice. But actually it’s been amazing to see how well they work—and it’s been very satisfying to see so many of our users being able to incorporate machine learning into their work, just using the automation we’ve built, and without having to consult any machine learning experts.

In Version 11 we’ve made many steps forward in machine learning. We’ve now got clean ways not just to do classification and prediction, but also to do feature extraction, dimension reduction, clustering and so on. And we’ve also done a lot of training ourselves to deliver pre-trained machine-learning functions. Machine-learning training is an interesting new kind of development. At its core, it’s a curation process. It’s just that instead of, say, collecting data on movies, you’re collecting as many images as possible of different kinds of animals.

Built into Version 11 are now functions like ImageIdentify that identify over 10,000 different kinds of objects. And through the whole design of the system, it’s easy to take the features that have been learned, and immediately use those to train new image classifiers vastly more efficiently than before.

ImageIdentify lets you easily and efficiently train new image classifiers

We’ve done a lot to automate today’s most common machine learning tasks. But it’s become clear in the past few years that an amazing number of new areas can now be tackled by modern machine learning methods, and particularly by using neural networks. It’s really an amazing episode in the history of science: the field of neural networks, that I’ve followed for almost 40 years, has gone from seeming basically hopeless to being one of the hottest fields around, with major new discoveries being made almost every week.

But, OK, if you want to get involved, what should you do? Yes, you can cobble things together with a range of low-level libraries. But in building Version 11 we set ourselves the goal of creating a streamlined symbolic way to set up and train neural networks—in which as much of what has to be done as possible has been automated. It’s all very new, but in Version 11 we’ve now got functions like NetGraph and NetChain, together with all sorts of “neural net special functions”, like DotPlusLayer and ConvolutionLayer. And with these functions it’s easy to take the latest networks and quickly get them set up in the Wolfram Language (recurrent networks didn’t quite make it into Version 11.0, but they’re coming soon).

Neural network examples using new Version 11 functionality

Of course, what makes this all really work well is the integration into the rest of the Wolfram Language. The neural network is just a Graph object like any other graph. And inputs like images or text can immediately and automatically be processed using standard Wolfram Language capabilities into forms appropriate for neural network computation.

From their name, “neural networks” sound like they’re related to brains. But actually they’re perfectly general computational structures: they just correspond to complex combinations of simple functions. They’re not unrelated to the simple programs that I’ve spent so long studying—though they have the special feature that they’re set up to be easy to train from examples.

We’ve had traditional statistical data fitting and interpolation forever. But what’s new with neural networks is a vastly richer space of possible computational structures to fit data to, or to train. It’s been remarkable just in the past couple of years to see a sequence of fields revolutionized by this—and there will be more to come.

I’m hoping that we can accelerate this with Version 11. Because we’ve managed to make “neural net programming” really just another programming paradigm integrated with all the others in the Wolfram Language. Yes, it’s very efficient and can deal with huge training sets. But ultimately probably the most powerful thing is that it immediately fits in with everything else the Wolfram Language does. And even in Version 11 we’re already using it in many of our internal algorithms in areas like image, signal and text processing. It’s still early days in the history of “neural net programming”—but I’m excited for the Wolfram Language to play a central part in what’s coming.

Audio

OK, let’s turn to another new area of Version 11: audio. Our goal is to be able to handle any kind of data directly in the Wolfram Language. We’ve already got graphics and images and geometry and networks and formulas and lots else, all consistently represented as first-class symbolic structures in the language. And as of Version 11, we’ve now always got another type of first-class data: audio.

Version 11 introduces fully integrated support for audio processing

Audio is mostly complicated because it’s big. But in Version 11 we’ve got everything set up so it’s seamless to handle, say, an hour of audio directly in the Wolfram Language. Behind the scenes there’s all sorts of engineering that’s caching and streaming and so on. But it’s all automated—and in the language it’s just a simple Audio object. And that Audio object is immediately amenable to all the sophisticated signal processing and analysis that’s available in the Wolfram Language.

Bones, Foods, and the Universe…

The Wolfram Language is a knowledge-based language. Which means it’s got lots of knowledge—about both computation and the world—built into it. And these days the Wolfram Language covers thousands of domains of real-world knowledge—from countries to movies to companies to planets. There’s new data flowing into the central Wolfram Knowledgebase in the cloud all the time, and we’re carefully curating data on new things that exist in the world (who knew, for example, that there were new administrative divisions recently created in Austria?). Lots of this data is visible in Wolfram|Alpha (as well as intelligent assistants powered by it). But it’s in the Wolfram Language that the data really comes alive for full computation—and where all the effort we put into ensuring its alignment and consistency becomes evident.

We’re always working to expand the domains of knowledge covered by the Wolfram Language. And in Version 11 several domains that we’ve been working on for many years are finally now ready. One that’s been particularly difficult is anatomy data. But in Version 11 we’ve now got detailed 3D models of all the significant structures in the human body. So you can see how those complicated bones in the foot fit together. And you can do computations on them. Or 3D print them. And you can understand the network of arteries around the heart. I must say that as I’ve explored this, I’m more amazed than ever at the level of morphological complexity that exists in the human body. But as of Version 11, it’s now a domain where we can actually do computations. And there are perhaps-unexpected new functions like AnatomyPlot3D to support it. (There’s certainly more to be done, by the way: for example, our anatomy data is only for a single “average adult male”, and the joints can’t move, etc.)

Built right in to Version 11: Detailed 3D models of all the significant structures in the human body

A completely different domain of data now handled in the Wolfram Language is food. There’s a lot that’s complicated in this domain. First, there are issues of ontology. What is an apple? Well, there’s a generic apple, and there are also many specific types of apples. Then there are issues of defining amounts of things. A cup of strawberries. Three apples. A quarter pounder. It’s taken many years of work, but we’ve now got a very robust symbolic way to represent food—from which we can immediately compute nutritional properties and lots of other things.

Version 11 offers extensive technical data on nutritional and other properties of thousands of foods

Another area that’s also been long coming is historical country data. We’ve had very complete data on countries in modern times (typically from 1960 or 1970 on). But what about earlier history? What about Prussia? What about the Roman Empire? Well, in Version 11 we’ve finally got at least approximate border information for all serious country-like entities, throughout recorded history. So one can do computations about the rise and fall of empires right from within the Wolfram Language.

Easily find—and do computation with—the maximum geographical extent of the Roman Empire and much more

Talking of history, a small but very useful addition in Version 11 is historical word frequency data. Just ask WordFrequencyData for a time series, and you’ll be able to see how much people talked about “war”—or “turnips”—at different times in history. Almost every plot is a history lesson.

WordFrequencyData in Version 11

Another convenient function in Version 11 is WikipediaData, which immediately gives any Wikipedia entry (or various kinds of data it contains). There’s also WolframLanguageData, which gives computable data on the Wolfram Language itself—like the examples in its documentation, links between functions, and so on.

In many domains one’s mostly just dealing with static data (“what is the density of gold?”; “what was the population of London in 1959?”). But there are other domains where one’s not really interested in static data so much as data-backed computation. There are several new examples of this in Version 11. Like human mortality data (“what is the probability of dying between age X and Y?”), standard ocean data (“what is the pressure at a depth X?”), radioactive stopping power and human growth data—as well as data on the whole universe according to standard cosmological models.

Also new in Version 11 are WeatherForecastData and MathematicalFunctionData. Oh, as well as data on Pokémon and lots of other useful things.

The Wolfram Language has caught 'em all

Computing with Real-World Entities

One of the very powerful features of the Wolfram Language is its ability to compute directly with real-world entities. To the Wolfram Language, the US, or Russia, or a type of lizard are all just entities that can be manipulated as symbolic constructs using the overall symbolic paradigm of the language. Entities don’t directly have values; they’re just symbolic objects. But their properties can have values: [[USA]]["Population"] is 322 million.

But let’s say we don’t just want to take some entity (like the US) and find values of its properties. Let’s say instead we want to find what entities have certain properties with particular values. Like let’s say we want to find the 5 largest countries in the world by population. Well, in Version 11 there’s a new way to do this. Instead of specifying a particular explicit entity, we instead specify a computation that implicitly defines a class of entities. And so for example we can get a list of the 5 largest countries by population like this:

EntityList[Entity["Country", "Population" -> TakeLargest[5]]]

TakeLargest[5] is an operator form of a new function in Version 11 that gets the largest elements in a list. Implicit entities end up making a lot of use of operator forms—much like queries in Dataset. And in a sense they’re also making deep use of the symbolic character of the Wolfram Language—because they’re treating the functions that define them just like data.

The whole mechanism of entities and properties and implicit entities works for all the different types of entities that exist in the Wolfram Language. But as of Version 11, it’s not limited to built-in types of entities. There’s a new construct called EntityStore that lets you define your own types of entities, and specify their properties and values and so on—and then seamlessly use them in any computation.

Just as Dataset is a powerful hierarchical generalization of typical database concepts, so EntityStore is a kind of symbolic generalization of a typical relational database. And if you set up a sophisticated entity store, you can just use CloudDeploy to immediately deploy it to the cloud, so you can use it whenever you want.

Geo Everything

One aspect of “knowing about the real world” is knowing about geography. But the Wolfram Language doesn’t just have access to detailed geographic data (not only for Earth, but also for the Moon, Mars, even Pluto); it can also compute with this data. It’s got a huge collection of geo projections, all immediately computable, and all set up to support very careful and detailed geodesy. Remember spherical trigonometry? Well, the Wolfram Language doesn’t just assume the Earth is a sphere, but correctly computes distances and areas and so on, using the actual shape of the Earth.

Just a few of the many geo projections in the Wolfram Language

When it comes to making maps, the Wolfram Language now has access not only to the street map of the world, but also to things like historical country borders, as well as at least low-resolution satellite imagery. And given the street map, there’s an important new class of computations that can be done: travel directions (and travel times) along streets from anywhere to anywhere.

Travel directions, time, and distance—to anywhere

Don’t Forget Calculus…

Version 11 has lots and lots of new capabilities across all areas of the Wolfram Language. But it’s also got lots of new capabilities in traditional Mathematica areas—like calculus. And back in earlier versions, what we’ve just added for calculus in Version 11 is big enough that it would have undoubtedly been the headline new feature of the version.

One example is differential eigensystems: being able to solve eigenvalue versions of both ODEs and PDEs. There’s a huge stack of algorithmic technology necessary to make this possible—and in fact we’ve been building towards it for more than 25 years. And what’s really important is that it’s general: it’s not something where one has to carefully set up some particular problem using elaborate knowledge of numerical analysis. It’s something where one just specifies the equations and their boundary conditions—and then the system automatically figures out how to solve them.

Calculus examples using new Version 11 functionality

Back around 1976 I wrote a Fortran program to solve an eigenvalue version of the 1D Schrödinger equation for a particle physics problem I was studying. In 1981 I wrote C programs to do the same thing for some equations in relativistic quantum mechanics. I’ve been patiently waiting for the day when I can just type in these problems, and immediately get answers. And now with Version 11 it’s here.

Of course, what’s in Version 11 is much more powerful and more general. I was dealing with simple boundary conditions. But in Version 11 one can use the whole Wolfram Language geometry system—and all the data we have—to set up boundary conditions. So it’s easy to find the eigenmodes of a “drum” of any shape—like the shape of the US.

Differential eigensystems computations, fully integrated with the power of the Wolfram Language

For something like that, there’s no choice but to do the computation numerically. Still, Version 11 will do differential eigensystem computations symbolically when it can. And Version 11 also adds some major new capabilities for general symbolic differential equations. In particular, we’ve had a large R&D project that’s now gotten us to the point where we can compute a symbolic solution to pretty much any symbolic PDE that would appear in any kind of textbook or the like.

Back in 1979 when I created a precursor to Mathematica I made a list of things I hoped we’d eventually be able to do. One of the things on that list was to solve integral equations. Well, I’m excited to be able to say that 37 years later, we’ve finally got the algorithmic technology stack to make this possible—and Version 11 introduces symbolic solutions to many classes of integro-differential equations.

There’s more in calculus too. Like Green’s functions for general equations in general domains. And, long awaited (at least by me): Mellin transforms. (They’ve been a favorite of mine ever since they were central to a 1977 particle physics paper of mine.)

It’s not classic calculus fare, but in Version 11 we’ve also added a lot of strength in what one might consider “modern calculus”—the aspects of calculus needed to support areas like machine learning. We’ve got more efficient and robust minimization, and we’ve also got sophisticated Bayesian minimization, suitable for things like unsupervised machine learning.

Education

Things like partial differential equations are sophisticated math, that happen to be very important in lots of practical applications in physics and engineering and so on. But what about more basic kinds of math, of the kind that’s for example relevant to high-school education? Well, for a long time Mathematica has covered that very thoroughly. But as our algorithmic technology stack has grown, there are a few new things that become possible, even for more elementary math.

One example new in Version 11 is full automatic handling of discontinuities, asymptotes, and so on in plots of functions. So now, for example, Tan[x] is plotted in the perfect high-school way, not joining –∞ and +∞. For Tan[x] that’s pretty simple to achieve. But there’s some seriously sophisticated algorithmic technology inside to handle this for more complicated functions.

Full automatic handling of discontinuities, asymptotes, and more

And, by the way, another huge new thing in Version 11 is MathematicalFunctionData—computable access to 100,000 properties and relations about mathematical functions—in a sense encapsulating centuries of mathematical research and making it immediately available for computation.

We’ve been doing a lot recently using the Wolfram Language as a way to teach computational thinking at all levels. And among many other things, we’ve wanted to make sure that any computation that comes up—say in math—in elementary school education is really elementary to do in the Wolfram Language. And so we’ve got little functions like NumberExpand, which takes 123 and writes it as {100, 20, 3}. And we’ve also got RomanNumeral and so on.

And, partly as a tribute to the legacy of Logo, we’ve introduced AnglePath—a kind of industrial-scale version of “turtle graphics”, that happens to be useful not just for elementary education, but for serious simulations, say of different types of random walks.

AnglePath, new in Version 11

Making Everything Fit Together

One of the central goals of the Wolfram Language is to have everything seamlessly work together. And in Version 11 there are some powerful new examples of this going on.

Time series, for example, now work directly with arithmetic. So you can take two air pressure time series, and just subtract them. Of course, this would be easy if all the time points in the series lined up. But in Version 11 they don’t have to: the Wolfram Language automatically handles arbitrarily irregular time series.

Another example concerns units. In Version 11, statistical distributions now work seamlessly with units. So a normal distribution can have not just a variance of 2.5, but a variance of 2.5 meters. And all computations and unit conversions are handled completely automatically.

Geometry and geometric regions have also been seamlessly integrated into more parts of the system. Solvers that used to just take variables can now be given arbitrary regions to operate over. Another connection is between images and regions: ImageMesh now takes any image and constructs a geometric mesh from it. So, for example, you can do serious computational geometry with your favorite cat picture if you want.

One final example: random objects. RandomInteger and RandomReal are old functions. Version 8 introduced RandomVariate for picking random objects from arbitrary symbolic probability distributions. Then in Version 9 came RandomFunction, for generating functions from random processes. But now in Version 11 there’s more randomness. There’s RandomPoint, which picks a random point in any geometric region. And there’s also RandomEntity that picks a random entity, as well as RandomWord—that’s useful for natural language processing research, as well as being a nice way to test your vocabulary in lots of languages… And finally, in Version 11 there’s a whole major new mathematical area of randomness: random matrices—implemented with all the depth and completeness that we’ve made a hallmark of Mathematica and the Wolfram Language.

Visualization

One of the long-term achievements of Mathematica and the Wolfram Language has been that they’ve made visualization a routine part of everyday work. Our goal has always been to make it as automatic as possible to visualize as much as possible. And Version 11 now makes a whole collection of new things automatic to visualize.

There are very flexible word clouds that let one visualize text and collections of strings. There are timeline plots for visualizing events in time. There are audio plots that immediately visualize short and long pieces of audio. There are dendrograms that use machine learning methods to show hierarchical clustering of images, text, or any other kind of data. There are geo histograms to show geographic density. There’s a TextStructure function that diagrams the grammar of English sentences. And there are anatomy plots, to show features in the human body (making use of symbolic specifications, since there aren’t any explicit coordinates).

A few of the many visualization possibilities with the Wolfram Language

What other kinds of things are there to visualize? Well, one thing I’ve ended up visualizing a lot (especially in my efforts in basic science) are the rules for simple programs like cellular automata. And in Version 11 we’ve added RulePlot for automatically visualizing rules in many different styles.

RulePlot automatically visualizes rules for Turing machines, cellular automata, and more

Another longstanding visualization challenge has been how to automatically visualize 3D distributions of data. The issue tends to be that it’s hard to “see into” the 3D volume. But in Version 11 we’ve got a bunch of functions that solve this in different ways, often by making slices at positions that are defined by our geometry system.

Automatically visualize 3D distributions of data

In the quest for automation in visualization, another big area is labeling. And in Version 11 we’ve added Callout to make it possible to specify callouts for points, lines and regions (we’ve already got legends and tooltips and so on). There’s a trivial way to do callouts: just always put them (say) on the left. But that’d be really bad in practice, because lots of callouts could end up colliding. And what Version 11 does instead is something much more sophisticated, involving algorithmically laying out callouts to optimally achieve aesthetic and communication goals.

Automatic, elegant placement of callout labels

From Strings to Text…

Mathematica and the Wolfram Language have always been able to handle character strings. And in Version 10 there was a huge step forward with the introduction of Interpreter—in which we took the natural language understanding breakthroughs we made for Wolfram|Alpha, and applied them to interpreting strings in hundreds of domains. Well, in Version 11 we’re taking another big step—providing a variety of functions for large-scale natural language processing and text manipulation.

There are functions like TextWords and TextSentences for breaking text into words and sentences. (It takes fancy machine learning to do a good job, and not, for example, to be confused by things like the periods in “St. John’s St.”) Then there are functions like TextCases, which lets one automatically pick out different natural-language classes, like countries or dates, or, for that matter, nouns or verbs.

It’s pretty interesting being able to treat words as data. WordList gives lists of different kinds of words. WordDefinition gives definitions.

Then there are multilingual capabilities. Alphabet gives pretty much every kind of alphabet; Transliterate transliterates between writing systems. And WordTranslation gives translations of words into about 200 languages. Great raw material for all sorts of linguistics investigations.

Multilingual functionality in Version 11

The Most Modern Systems Programming

The Wolfram Language is arguably the highest-level language that’s ever been created. But in Version 11 we’ve added a bunch of capabilities for “reaching all the way down” to the lowest level of computer systems. First of all, there’s ByteArray that can store and manipulate raw sequences of bytes. Then there are functions that deal with raw networks, like PingTime and SocketConnect.

There’s a new framework for publish-subscribe “channels”. You can create a channel, then either the Wolfram Language or an external system can send it data, and you can set up a “listener” that will do something in your Wolfram Language session whenever the data arrives. There’s a lot that can be built with this setup, whether it’s connecting to external services and devices, handling notifications and third-party authentication—or creating your very own chat system.

Something else new in Version 11 is built-in cryptography. It’s a very clean symbolic framework that lets you set up pretty much whatever protocol you want, using public or private key systems.

What about interacting with the web? The symbolic character of the Wolfram Language is again very powerful here. Because for example it lets one have HTTPRequest and HTTPResponse as symbolic structures. And it also lets one have functions like URLSubmit, with symbolically defined handler functions for callbacks from asynchronous URL submission. There’s even now a CookieFunction, for symbolic handling of cookies.

Yes, one can do systems programming in pretty much any language—or even for example in a shell. But what I’ve found is that doing it in the Wolfram Language is incredibly more powerful. Let’s say you’re exploring the performance of a computer system. Well, first of all, everything you’re doing is kept nicely in a notebook, where you can add comments, etc. Then—very importantly—everything you do can be immediately visualized. Or you can apply machine learning, or whatever. Want to study network performance? Use PingTime to generate a list of ping times; then immediately make a histogram, correlate with other data, or whatever.

Something else we’re adding in Version 11 is FileSystemMap: being able to treat a file system like a collection of nested lists (or associations) and then mapping any function over it. So for example you can take a directory full of images, and use FileSystemMap to apply image processing to all of them.

Oh, and another thing: Version 11 also includes, though it’s still tagged as experimental, a full industrial-strength system for searching text documents, both locally and in the cloud.

Building Things on the Web

An incredibly powerful feature of the Wolfram Language is that it runs not only on the desktop but also in the cloud. And in Version 11 there are lots of new capabilities that use the cloud, for example to create things on the web.

Let’s start with the fairly straightforward stuff. CloudDeploy[FormFunction[...]] lets you immediately create a form-based app on the web. But now it’s easy to make the form even more sophisticated. There are lots of new types of “smart fields” that automatically use natural language understanding to interpret your input. There are new constructs, like RepeatingElement and CompoundElement, that automatically set up fields to get input for lists and associations. And there’s a whole new Programmable Linguistic Interface that lets you define your own grammar to extend the natural language understanding that’s already built into the Wolfram Language.

The forms you specify symbolically in the Wolfram Language can be quite sophisticated—with multiple pages and lots of interdependencies and formatting. But ultimately they’re still just forms where you set up your input, then submit it. Version 11 introduces the new AskFunction framework which lets you set up more complex interactions—like back-and-forth dialogs in which you “interview” the user to get data. In the Wolfram Language, the whole process is specified by a symbolic structure—which CloudDeploy then makes immediately active on the web.

It’s a goal of the Wolfram Language to make it easy to build complex things on the web. In Version 11, we’ve added FormPage to let you build a “recycling form” (like on wolframalpha.com), and GalleryView to let you take a list of assets in the Wolfram Language, and immediately deploy them as a “gallery” on the web (like in demonstrations.wolfram.com).

GalleryView in Version 11: Immediately deploy assets as a web gallery

If you want to operate at a lower level, there are lots of new functions, like URLDispatcher and GenerateHTTPResponse, that let you determine exactly how web requests will be handled by things you set up in the cloud.

Also new in Version 11 are functions like CloudPublish and CloudShare that let you control access to things you put in the cloud from the Wolfram Language. A small but I think important new feature is SourceLink, which lets you automatically link from, say, a graphic that you deploy in the cloud back to the notebook (also in the cloud) in which it was created. I think this will be a great tool for “data-backed publication”—in which every picture you see in a paper, for example, links back to what created it. (Inside our company, I’m also insisting that automated reports we generate—from the Wolfram Language of course—include source links, so I can always get the raw data and analyze it myself or whatever.)

Data Into the Cloud

Already there experimentally in Version 10—but now fully supported in Version 11—is the whole Wolfram Data Drop mechanism, which lets you accumulate data from anywhere into the Wolfram Cloud. I have to say I think I underestimated the breadth of usefulness of the Wolfram Data Drop. I thought it would be used primarily to store data from sensors and the like. And, yes, there are lots of applications along these lines. But what I’ve found is that the Data Drop is incredibly useful purely inside the Wolfram Language. Let’s say you’ve got a web form that’s running in the Wolfram Language. You might process each request—but then throw the result into a databin in the Wolfram Data Drop so you can analyze them all together.

Wolfram Data Drop is basically set up to accumulate time series of data. In Version 11 another way to store data in the cloud is CloudExpression. You can put any Wolfram Language expression into a cloud expression, and it’ll be persistently stored there, with each part being extracted or set by all the usual operations (like Part and AppendTo) that one could use on a symbol in a Wolfram Language session. CloudExpression is a great way to store structured data where one’s continually modifying parts, but one wants all the data to be persistent in the cloud.

Things you store in the cloud are immediately persistent. In Version 11, there’s also LocalObject—which is the local analog of CloudObject—and provides persistent local storage on your machine. LocalCache is a seamless way of ensuring that things you’re using are cached in the local object store.

Connecting Everywhere

In the Wolfram Language we curate lots of data that we include directly in the knowledgebase—but we also curate ways to access more data, such as external APIs. Version 11 includes many new connections, for example to Flickr, Reddit, MailChimp, SurveyMonkey, SeatGeek, ArXiv and more.

Version 11 adds many new connections to external APIs

The Wolfram Language is also an extremely powerful way to deploy your own APIs. And in Version 11 there’s an expanding set of authentication mechanisms that are supported for APIs—for example PermissionsKey for giving appids. CloudLoggingData also now gives what can be extremely detailed data about how any API or other cloud object you have is being accessed.

An API that you call on the web basically gets data passed to it through the URL that it’s given. In Version 11 we have a new kind of API-like construct that operates not through the web and URLs, but through email. MailReceiverFunction is like APIFunction, except when it’s deployed it defines an email address, and then any mail that’s sent to that email address gets fed to the code in the mail receiver function. MailReceiverFunction lets you get very detailed in separating out different parts of a mail message and its headers—and then lets you apply any Wolfram Language function you want—so you can do arbitrarily sophisticated automation in processing email. And particularly for someone like me who gets a huge amount of email from humans as well as automated systems, this is a pretty nice thing.

WolframScript

You can access the Wolfram Language through a notebook, on the desktop or in the cloud. You can access it through a scheduled task in the cloud, or through an API or a mail receiver function. It’s always been possible to run the Wolfram Language from a command line too, but in Version 11 there’s a powerful new way to do that, using WolframScript.

The idea of WolframScript is to provide a very simple but flexible interface to the Wolfram Language. WolframScript lets you run on a local Wolfram Engine on your computer—or just by saying -cloud it lets you run in the cloud. It lets you run code from a file or directly from the command line. And it lets you get back results in any format—including text or images or sounds or PDF or CDF or whatever. And in the usual Unix way, it lets you use #!wolframscript to create a script that can be called standalone and will run with WolframScript.

There’s more too. You can set up WolframScript to operate like a Wolfram Language FormFunction—pulling in arguments of whatever types you specify (and doing interpretation when needed). And you can also use WolframScript to call an API you’ve already defined in the cloud.

In our own company, there are lots of places where we’re using the Wolfram Language as part of some large and often distributed system. WolframScript provides a very clean way to just “throw in a Wolfram Language component” anywhere you want.

WolframScript provides a simple, flexible interface to the Wolfram Language

The Core Language

I’ve talked about all sorts of things that broaden and deepen the algorithmic capabilities of Mathematica and the Wolfram Language. But what about the structure of the core Wolfram Language itself? Of course, we’re committed to always maintaining compatibility (and I’m happy to say that all our attention to design on an ongoing basis tends to make this rather easy). But we also want to progressively strengthen and polish the language.

In natural languages, one process of evolution tends to be the construction of new words from common idioms. And we’re doing essentially the same thing in the Wolfram Language. We’ve made an active effort to study what “lumps of repeated computational work” appear most frequently across lots of Wolfram Language code. Then—assuming we can come up with a good name for a particular lump of computational work—we add it as a new function.

In the early days of Mathematica I used to take the point of view that if there were functions that let you do something using an idiom, then that was fine. But what I realized is that if an idiom is compressed into a single function whose name communicates clearly what it’s for, then one gets code that’s easier to read. And coupled with the convenience of not have to reconstruct an idiom many times, this justifies having a new function.

For the last several years, two initiatives we’d had within our company are Incremental Language Development (ILD), and Language Consistency & Completeness (LCC). The idea of ILD is to do things like introduce functions that are equivalent to common idioms. The idea of LCC is to do things like make sure that anything—like pattern matching, or units, or symbolic URLs—is supported wherever it makes sense across the system.

So, for example, a typical ILD addition in Version 11 is the function MinMax that returns min and max in a list (it’s amazing how many fiddly applications of Map that saves). A typical LCC addition is support for pattern matching in associations.

Between ILD and LCC there are lots of additions to the core language in Version 11. Functions like Cases have been extended to SequenceCases—looking for sequences in a list instead of individual elements. There’s also now SequenceFoldList, which is like FoldList, except it can “look back” to a sequence of elements of any length. In a similar vein, there’s FoldPairList, which generalizes FoldList by allowing the result “returned” at each step to be different from the result that’s passed on in the folding process. (This might sound abstract, and at some level it is—but this is a very useful operation whenever one wants to maintain a separate internal state while sequentially ingesting data.)

Another new construct that might at first sound weird is Nothing. Despite its name, Nothing does something very useful: whenever it appears in a list, it’s immediately removed. Which means, for example, that to get rid of something in a list, you just have to replace it by Nothing.

There are lots of little conveniences we’ve added in Version 11. First, for example, now has a second argument, that says what to give if there’s no first element—and avoids having to include an If for that case. There’s also a nice general mechanism for things like this: UpTo. You can say Take[list,UpTo[4]] to get up to 4 elements in list, or however many elements there happen to be. UpTo is supported in lots of places—and it simplifies lots of code.

Another little convenience is Echo. When you’re trying to tell what’s going on inside a piece of code, you sometimes want to print some intermediate result. Echo is a function that prints, then returns what it printed—so you can sprinkle it in your code without changing what the code does.

It’s hard to believe there are useful basic list operations still to add, but Version 11 has a few. Subdivide is like Range except it subdivides the range into equal parts. TakeLargest and related functions generalize Max etc. to give not just the largest, but the n largest elements in a list.

There’s a function Groupings, which I’d been thinking about for years but only now figured out a good design for—and which effectively generates all possible trees formed with certain binary or other combiners (“what numbers can you get by combining a list of 1s in all possible ways with Plus and Times?”).

There are nips and tucks to the language in all sorts of places. Like in Table, for example, where you can say Table[x,n] rather than needing to say Table[x,{n}]. And in general there are lots and lots of things that make the core of Version 11 of the Wolfram Language just a little smoother, nicer and more elegant to use.

And There’s Much More

This has been a long blog post. But I’m not even close to having covered everything that’s new in Version 11. There’s more information on the web. Check out the featured new areas, or the information for existing users or the summary of new features. (See also the list of new features specifically from Version 10.4 to 11.0.)

But most of all, get into using Version 11! If you want quick (and free) exposure to it, try it in the Wolfram Open Cloud. Or just start using Mathematica 11 or Version 11 of any other products based on the Wolfram Language.

I’ve been using test versions of Version 11 for some time now—and to me Version 10 already looks and feels very “old fashioned”, lacking those nice new interface features and all the new functionality and little conveniences. I’m really pleased with the way Version 11 has turned out. It’s yet another big step in what’s already been a 30-year journey of Mathematica and the Wolfram Language. And I’m excited to see all the wonderful things that all sorts of people around the world will now be able to do for first time with Mathematica 11 and the other Version 11 Wolfram Language products.

]]>
http://blog.wolfram.com/2016/08/08/today-we-launch-version-11/feed/ 39
Celebrate National Coloring Book Day with Wolfram (and Four Crayons) http://blog.wolfram.com/2016/08/02/celebrate-national-coloring-book-day-with-wolframand-four-crayons/ http://blog.wolfram.com/2016/08/02/celebrate-national-coloring-book-day-with-wolframand-four-crayons/#comments Tue, 02 Aug 2016 15:14:51 +0000 Zach Littrell http://blog.internal.wolfram.com/?p=31937 Happy National Coloring Book Day! When my coworkers suggested that I write a blog post celebrating this colorful occasion, I was, frankly, tickled pink by the idea. Coloring is a fun, therapeutic activity for anyone of any age who can color inside the lines—or occasionally just a little outside, if they’re more like me. And as the newest member of the Wolfram Blog team, I wanted to see in what fun ways I could add a little color to the Wolfram Blog.

While looking through Wolfram|Alpha’s massive collection of popular curves, from Pokémon to ALF to Stephen Wolfram, I realized that all of the images built into the Wolfram Knowledgebase would be great for coloring. So, I figured, why not make my own Wolfram coloring book in Mathematica? Carpe colores!

Each of the popular curves in the Knowledgebase can be accessed as an Entity in the Wolfram Language and comes with a wide variety of properties, including their parametric equations. But there’s no need to plot them yourself—they also conveniently come with an "Image" property already included:

Stephen Wolfram curve from Wolfram|Alpha

And with a few lines of code, the Wolfram Language can get rid of the axes and thicken the lines to make them just right for a coloring book page:

Dr. Zoidberg coloring page

I encourage you to take this function, plop it in a notebook, create some coloring book pages, print them out, and spoil the coloring connoisseur in your life—especially if that person is yourself.

But that’s not enough for me: I want to make an interactive interface so I can color these popular curves directly in Mathematica. Ideally what I would like is to click any region of an image, and every pixel within that region gets colored, saving me from the stress of coloring within the lines. In other words, once I have loaded a popular curve, I need to segment the image into individual regions that can be colored. With the MorphologicalComponents function, I can do just that:

Using the MorphologicalComponents function to color specific regions

To capture the mouse location for each mouse click, I embed my image in a ClickPane. I can then identify which region is clicked by checking the mouse location with MorphologicalComponents and replacing its current color with whatever the currently selected color is. There are tons of different methods I could have used to implement this sort of “flood fill” algorithm in the Wolfram Language, just like how there is more than one way to color the same picture.

I wrapped up this functionality in a simple GUI to select from the available popular curves, and I had a ball trying out my little program and coloring the library of curves my way. The notebook with my code is included in this post, so feel free to try it out!

GUI available with popular curves

It’s a fast-paced world nowadays, though. I understand if you’re too busy to arduously fill out a coloring book page. That doesn’t mean you have to miss out on the fun! You can always just ask Wolfram|Alpha for a popular curve, randomly colored. Sure, the coloring may look anywhere between unconventional and absurd, but it’s fun to see how Wolfram|Alpha ends up coloring your favorite Futurama character or historical figure:

Wolfram|Alpha randomly colors Pikachu-like curve

While I’m ruminating on computational coloring, it would be a real shame not to mention one of the most important coloring problems of all: the four-color theorem, one of the first computer-assisted proofs and a surprisingly difficult problem to solve. Appel and Haken proved in 1977 that any planar map can be colored so that no two regions sharing a border (bigger than a point) have the same color, using only four colors. In other words, you can color a geographic map so each country is a different color than its neighbors using just red, green, blue, and yellow, like the map of Europe below. That box of 100 different crayons may be nifty, but four is really all you need:

Map of Europe using only four colors

A fun challenge you can try is finding a four-coloring for a coloring book page. But to be true to the computational roots of the four-color problem, you can always be clever and use the Wolfram Language to find a four-coloring for you. To accomplish this, I adapted a method that can also be used to four-color a geographic map in the Wolfram Language. Given four colors—let’s say purple, blue, pink, and green—they can be mapped to pairs of Booleans, like the following:

Four colors

I can enforce that two regions, A and B, don’t have the same color by creating a Boolean equation with Xor that checks that their respective Boolean pairs aren’t the exact same, using the convention that the color for A is represented by (x[A],y[A]):

Ensuring two regions have different colors using Xor

And I can do a simple test to see if regions A and B are neighbors by using the Wolfram Language’s built-in morphological tools. With Dilation, I expand region A up, down, left, and right a couple of pixels and see if there’s any overlap with B. The Wolfram Language can then find a four-coloring solution for all equations for each pair of neighbors with FindInstance. The complete code for four-coloring is in the included notebook, so I encourage you to try it out and find four-colorings for your favorite curves!

And besides saving you money on crayons, finding four-colorings has practical applications in real-world scheduling problems, where you’re assigning people or things to time slots instead of colors to regions. So feel free to brag while coloring today that you’re really just exploring difficult mathematical problems in a relaxing, colorful manner.

Four-coloring Abe Lincoln

There are many more different ways to chromatically express yourself in the Wolfram Language that I didn’t even touch on. With advanced image processing functions, machine learning tools, and data about famous artwork in the Wolfram Knowledgebase, the sky’s the limit. I hope you have a relaxing Coloring Book Day, and feel free to share any ideas or questions you have about coloring with the Wolfram Language!

Download this post as a Computable Document Format (CDF) file. New to CDF? Get your copy for free with this one-time download.

]]>
http://blog.wolfram.com/2016/08/02/celebrate-national-coloring-book-day-with-wolframand-four-crayons/feed/ 0
Introducing Wolfram|Alpha Add-ons for Google Drive http://blog.wolfram.com/2016/07/28/introducing-wolframalpha-add-ons-for-google-drive/ http://blog.wolfram.com/2016/07/28/introducing-wolframalpha-add-ons-for-google-drive/#comments Thu, 28 Jul 2016 16:47:25 +0000 Stephanie Oh http://blog.internal.wolfram.com/?p=31911 Wolfram|Alpha answers a ton of computational and factual questions every day—through our website, mobile apps, APIs, and from within the Wolfram Language itself. Now we would like to introduce a new way to harness the power of computation with the Wolfram|Alpha Add-ons for Google Drive. These free add-ons for Google Docs and Google Sheets enable you to bring up a Wolfram|Alpha sidebar next to your file or get Wolfram|Alpha results instantly, all without interrupting your workflow. To install these add-ons in your Google Drive, click the buttons at the top of our support page.

Once installed, you’ll have a few different options for how to use your new add-ons. First, you can open a Wolfram|Alpha sidebar inside Docs or Sheets by going to the Wolfram|Alpha for Docs or Wolfram|Alpha for Sheets Add-on menu (from within your Docs or Sheets file) and choosing Open Wolfram|Alpha Sidebar. This opens a sidebar that lets you type in your queries and send them to Wolfram|Alpha. When you click the orange equal sign button Equal sign, result pods from your query will show up in the sidebar. You can then copy these results as images or plain text to use in your document. Or use the sidebar to look up the molecular structure of caffeine, visualize the Poisson distribution, find the processor speed for the Atari 400, get nutritional information for chocolate chip cookies, and all sorts of other things. It’s the full power of Wolfram|Alpha, right inside Google Drive.

Opening a Wolfram|Alpha sidebar in Google Docs
Opening a Wolfram|Alpha sidebar in Google Sheets

The Wolfram|Alpha sidebar is invaluable when doing general research. But if you have a specific question you need answered, there’s an even simpler option for getting Wolfram|Alpha results. In either a Docs or Sheets file, just highlight some text and go to the Wolfram|Alpha for Docs or Wolfram|Alpha for Sheets Add-on menu, choose Compute Selection with Wolfram|Alpha, and Wolfram|Alpha will replace the selection with its result. For example, if you want to know the mass of a proton, just type “mass of the proton,” highlight it, compute the selection, and the mass of the proton will appear, units and all! What’s more, if you select multiple cells in Sheets, you can compute them all at the same time with a single click.

Send queries directly to Wolfram|Alpha from within Google Drive

Whether you’re gearing up to tackle next semester’s history class—or just looking to make your daily grind a little easier—go ahead and try out the Wolfram|Alpha Add-ons for Google Drive today. Simplify your workflow, compute answers on the fly, and bring the power of Wolfram|Alpha to your spreadsheets and documents.

]]>
http://blog.wolfram.com/2016/07/28/introducing-wolframalpha-add-ons-for-google-drive/feed/ 4
Finding the Most Unhygienic Food in the UK http://blog.wolfram.com/2016/07/21/finding-the-most-unhygienic-food-in-the-uk/ http://blog.wolfram.com/2016/07/21/finding-the-most-unhygienic-food-in-the-uk/#comments Thu, 21 Jul 2016 14:29:32 +0000 Jon McLoone http://blog.internal.wolfram.com/?p=31824 The UK, like many other countries, runs a food hygiene inspection system that tries to ensure that establishments with poor hygiene standards improve or are shut down. As is often the case, the data collected for operational reasons can provide a rich source of insight when viewed as a whole.

Questions like “Where in the UK has the poorest food hygiene?”, “What kinds of places are the most unhygienic?”, and “What kinds of food are the most unhygienic?” spring to mind. I thought I would apply Mathematica and a little basic data science and provide the answers.

The collected data, over half a million records, is updated daily and is openly available from an API, but this API seems to be targeted at performing individual lookups, so I found it more efficient to import the 414 files from this site instead.

All data in this blog post was extracted on July 15, 2016. You can find the Wolfram Language code for importing and other utilities in the CDF at the foot of this post.

Oxford

As a warmup, I started with somewhere I knew, so here is the Dataset representing my local city of Oxford.

Importing Oxford food rating data

Oxford food rating dataset

There are 1,285 places to buy food in Oxford, and the rating scheme grades them from 0 (“Urgent improvement necessary”) through 5 (“Very good”).

We can throw the ratings onto a map of Oxford and see, as I would expect, concentrations of establishments around the tourist center and along major arterial roads.

Mapping ratings for places to buy food on Oxford map

Map of places to buy food in Oxford

We can see that the vast majority are rated 4 or 5 (in green). We should only be concerned about the 0, 1, and 2 ratings (“Urgent improvement necessary”, “Major improvement necessary”, and “Improvement necessary”), so let’s look at just those.

Mapping places to buy food rated with 0, 1, and 2

Mapping places to buy food rated with 0, 1, and 2

There are obvious clusters in the center (where all the tourists go) and along Cowley Road (leading to Temple Cowley), which is where a lot of students live. But these also have lots of good establishments. So to normalize for that, we must find the average rating for a location. Since no two establishments are in exactly the same place, I need to create a function that collects all the data within a certain distance of a geo position and finds the average rating.

Creating a function to collect all data within a certain distance of a geo position

We can now run that function over the entire map grid to create a moving average value of hygiene. I have used 0.4 miles for the averaging disk, which is large enough to collect quite a few establishments at a time but small enough to avoid blurring the whole city together.

Running previous function over the map grid to create a moving average value of hygiene

Running previous function over the map grid to create a moving average value of hygiene

My initial intuition proved right. Cowley Road and the area between the city center and the station are areas of poor average hygiene, but there is also a hotspot in the southwest that I can’t explain. The best average hygiene is in the north, Walton Manor to Summertown, which are the expensive parts of Oxford and the Headington area.

Which councils are failing to protect us?

I am happy that the data is plausible and I have understood it, but there is another issue we must consider before going for our answers: data quality. While the Food Hygiene Rating Scheme is controlled by the national Food Standards Agency, it is operated by over 400 different local authorities. Are they all doing a consistent job? One of the promised benefits of open data is that we can hold our governments accountable—so let’s do that. This is the kind of analysis that I hope central government is doing too.

We can easily look at who is on top of the workload by counting the fraction of businesses that are not yet rated.

Unrated establishments

Unrated establishments

So if you eat out in North Norfolk, you might be nervous to discover that nearly 25% of establishments have never been inspected.

Suspicious in another way is that around a third of the authorities have inspected every business. That would be great if it were true, but since new businesses must open regularly, you would expect to find a few that are awaiting inspection, so this may just indicate that these authorities don’t record (or perhaps even know about) new establishments until they are inspected.

Eateries awaiting inspection

Time since rating

We can also see how often the average establishment is inspected. The best authorities inspect establishments at least once per year.

How often the average establishment is inspected

But alarmingly, Croydon has an average time since inspection of over 3.5 years. A lot can change in that time.

Average time since last rating from longest to shortest

I can’t see an easy way to measure if the different authorities are applying the rules in a consistent way when they do inspect, so I am just going to have to trust that the values are equivalent.

Regional differences

So back to our original questions. First I am going to throw out all data that does not have a numerical rating. Unfortunately, this excludes Scotland, which runs a different scheme that provides only a pass-or-fail-type conclusion.

Removing data with no numerical rating

We still have plenty of data to work with…

Amount of data that can used

The good news is that most establishments are “Good” or “Very good.”

Histogram of rating value

The average rating value across the country is 4.37.

Average rating across the country

Here is a quick map of all the 0-rated establishments in the country.

Map of 0-rated estabilshments in the country

Map of 0-rated establishments in the country

The easiest way to group the data is by the local authority that collected it, since that is stored in every record. By that measure, Newham in London is the worst, with an average rating of 3.4.

Grouping data by the local authority that collected it

And the best is Torridge in Devon at 4.86.

Grouping data by the local authority that collected it, starting with the best rating

But we can use the "PostCode" key to be much more precise. A full UK postcode is shared by around 15 properties. That is too fine grained, as we will find a lot of postcodes with only one restaurant. We need a collection to infer anything about a neighborhood, so I will use only the first part of the postcode, and throw out all postcodes that do not contain at least 10 establishments.

Using PostCode key

Finally, I hooked up a postcode API to translate back from the partial postcode to a location name.

The result puts E13, in East London, at the bottom of the list, with adjacent postcodes E12, E7, E8, and E15 also on the list. Indeed, nearly all of the worst postcodes are parts of London, apart from a few Birmingham postcodes.

Translating partial postcode to a location name

Topping the best hygiene-rated postcodes is Craigavon in Northern Ireland, with a perfect score.

Best hygiene-rated postcodes

Regional trends

Can we infer some long-distance trends? For the whole country, we have lots of data and are not looking for very small features. There is a much faster method than the one I used on Oxford. Essentially, by aggregating over square regions rather than circular, I can round each geo position once, rather than having to test it repeatedly for membership of the region. I round all the locations to the nearest 20 miles and then aggregate all the points that now share the same location. I then repeat the process, shifting the box centers by 5 miles to create a moving average square. The Wolfram Language code is attached in a CDF at the bottom of the blog. Here is the result.

UK contour map

So there is an unhygienic center in London (as we already saw) that spreads toward Birmingham (going around north Oxfordshire) before turning east at Manchester until it reaches Hull. There is another notable low area in South Wales around, but not centered on, Cardiff. Generally, rural areas appear to be more hygienic, particularly North Devon, North Wales, and East Cumbria.

What kind of establishments are least hygienic?

Enough regional anthropology. Let’s consider what kind of food is safe. The analysis of the "BusinessType" key is reassuringly predictable. Fast food is the worst; schools and hospitals are the best.

Average rating by business type

We can drill deeper by inferring something about the food from the business name. Here is a function to measure the average hygiene rating for all establishments containing a particular word.

Function to measure the average hygiene rating for all establishments containing a particular word

To reduce the search and ensure enough data for conclusions, I will pick out a list of all words that appear in at least 100 different business names.

Reducing to words that appear in at least 100 different business names

And now for each word, we calculate the average rating for businesses using that word in their names.

Average rating for businesses using that word in their names

Amusingly, “lucky” appears on the list of the worst word associations. The worst is “halal.” With the exception of Dixy (which appears to mostly be linked to a chain), they are words associated with small, independent businesses.

Words and their average ratings

We can see it more easily, though less precisely, as a WordCloud of the 80 worst-rated words.

80 worst-rated words WordCloud

The words associated with the best ratings are mostly large chains, who presumably can put more effort into good management processes. At the top of the list is the Japanese-inspired restaurant chain Wagamama, followed by upmarket supermarket chain Waitrose. There are also some school- and hospital-related words.

Words with best ratings

WordCloud for 80 best-rated words

Of course, none of this necessarily has anything to do with how good the food tastes, and it is unproven whether there is any link between satisfying the food inspectors and making safe food.

If you really care about food hygiene, then the best advice is probably just to never be rude to the waiter until after you have gotten your food!

Download this post as a Computable Document Format (CDF) file. New to CDF? Get your copy for free with this one-time download.

]]>
http://blog.wolfram.com/2016/07/21/finding-the-most-unhygienic-food-in-the-uk/feed/ 9
Behind Wolfram|Alpha’s Mathematical Induction-Based Proof Generator http://blog.wolfram.com/2016/07/14/behind-wolframalphas-mathematical-induction-based-proof-generator/ http://blog.wolfram.com/2016/07/14/behind-wolframalphas-mathematical-induction-based-proof-generator/#comments Thu, 14 Jul 2016 18:30:30 +0000 Connor Flood http://blog.internal.wolfram.com/?p=31741 An idea, some initiative, and great resources allowed me to design and create the world’s first online syntax-free proof generator using induction, which recently went live on Wolfram|Alpha.

Induction-based proof generator on Wolfram|Alpha

Motivation

It is no secret that Wolfram|Alpha is a well-used tool for math students, especially those taking first-year college and university math courses. In fact, one of the most common responses I receive from students when I explain my role with Wolfram|Alpha is, “OMG, that site saved me during calculus.” This is usually a reference to the differential equation solver, or derivative and integral features of Wolfram|Alpha.

A majority of math topics that are taught at the high-school and first-year level are computation based, similar to the features listed above. This means questions on tests and exams are often in the following form:

  • Solve for x
  • Find the derivative of f(x)
  • Determine the roots of the following equation

Computation questions typically involve performing a series of calculations (i.e. steps) in order to obtain a final result. Topics that can be solved computationally can often be done so using repetitive approaches. Once a student has mastered the limited steps and rules associated with one of these topics, they should be able to handle just about any problem that is thrown at them on an assignment or an exam related to that topic. This is because computation-based questions are generally consistent in the approaches used to solve them. Similar to learning how to calculate derivatives of functions, once a student has gained experience with all the necessary “derivative rules,” answering these questions is a matter of repeatedly applying those rules.

The problem, though, is what about topics that are not computation based? More specifically, how do students study and practice questions for math topics that don’t have a limited set of rules or approaches? I encountered this problem as a student in a first-year discrete math course. The topic being taught was proofs by principle of mathematical induction, which I will give a brief background on. This topic was completely different than anything I had previously been taught in math, for a couple reasons:

  • Proofs are not computation questions. The same set of rules can’t be applied to every question.
  • Proofs don’t have a single final answer.

It’s weird to think of a math problem that doesn’t have a nice, simple, clean answer at the bottom of the page. If a student was solving an equation on a test, they might use five lines to show their steps, and then at the very end they would likely have the final answer written in the form “x = 21.” Mathematical proofs, however, don’t work that way. In fact, the “answer” actually is the steps. The goal of a proof is to logically show why something is true.

Consider if you were playing the game Clue (a.k.a. Cluedo), a murder mystery board game. The goal of the game is to determine who the murderer is, the weapon used, and the location of the crime. Someone would therefore win by stating something along the lines of, “Colonel Mustard with the revolver in the dining room.” Now imagine instead of trying to figure out this information, you were told who the murderer was, the weapon used, and the location of the crime at the very beginning of the game. The goal now is then not to determine the conclusion (since you were already told it) but instead figure out why this is true. Based on your own cards and conclusions made from information being revealed throughout the game, you should eventually be able to prove using logic why the information you were told at the beginning of the game must be true. Therefore, you might win the game by stating something along the lines of, “Since I have all of the weapon cards besides the revolver, and Joe has all the location cards except the dining room, and I know neither of you have Colonel Mustard based on your cards played, I have proved the proposed murderer to be true.”

I am not at all suggesting that the above example would be a fun alteration to Clue; in fact, it would likely kill the fun of the game (bad joke). However, it demonstrates the type of question/answer format that proofs represent. Below is a sample induction proof question a first-year student might see on an exam:

Prove using mathematical induction that 8^n – 3^n is divisible by 5, for n > 0.

The assertion made, that 8^n – 3^n is divisible by 5 when n is greater than 0, is completely true (assuming n is an integer). However, the question is asking the student to logically show why the statement is correct. Just like in that alternate Clue example, you were told the information at the beginning of the game and then asked to prove why it’s true. Writing these proofs is not trivial for most students, and is often viewed as one of the more difficult math concepts to master.

Due to the complexity of these questions, students often rely on online resources for further help (e.g. Wolfram|Alpha). If students had a differential equation they were solving and wanted to verify their answer, they can easily input it into Wolfram|Alpha and be given the final answer as well as the step-by-step solution. This approach can also be taken for derivatives, integrals, simplifying expressions, etc.

… but what about proofs?

We’ve established that proofs are not considered to be computation questions. As a result, to my understanding, no calculator or online tool has been able to generate solutions for proof questions… until now.

I vividly remember in my first year wanting to verify my solutions for proofs on an assignment. I had written out what I thought were the correct solutions to the proof questions. But how could I check if my answers were right? Surprisingly, there were only two ways to verify the correctness of my proofs:

  • Find the exact same proof as an example online (which was very rare)
  • Compare my solutions with someone else in the class, and assume if we both had the same solution that we must both be right (bad assumption)

So why is it so easy to find a “derivative calculator” online, but not a “proof calculator”?

The answer is mainly due to the fact that proofs have generally not been considered computable. Since the same set of rules can’t be applied to cover 100% of proofs, a computer has difficulty creating the logical steps of which the proof is composed. My solution to this problem was to build a library of pattern-matched proofs (as explained further below). This approach demonstrated the existence of computation playing a role in constructing proofs, and led to the creation of a working “proof generator.”

Prototype

Over a summer I began building a prototype application, which was able to perform a limited number of proofs using the principle of mathematical induction (PMI). PMI is a common tactic for constructing proofs. The logic for proving a query true via PMI is often compared to proving that a line of dominoes will all fall down in succession. First, you want to prove that the first domino will fall over if you push it. Second, you want to prove that if any given domino falls over, the one following it will also fall over. By proving these two facts, you are able to show that a line of dominoes will fall over if the first one is pushed.

This domino example well describes the logic behind induction proofs, which are composed of:

  • Base case (first domino): Prove the statement is true for the base case (the smallest n value), i.e. if you were proving something was true for n ≥ 1, the base case is showing the statement is true for n = 1.
  • Inductive step (consecutive dominoes): This is the more challenging step. In the inductive step, you assume the statement is true for some value (i.e. k), and then try to show this is also true for k + 1.

If both of these steps are done correctly, then the proof is complete.

Before development, I had discussed building this application with a professor at my undergraduate university (Algoma University). He compared this challenging idea to building a recipe book that contains every recipe in the world. You can keep adding as many recipes as you can get your hands on, but it is essentially impossible to include every recipe ever made. Similarly, I can keep adding more coverage for proofs to be generated, but I will never be able to include every proof that a user will ask for.

While this was a rather notable limitation, I thought about it in a slightly different sense. Instead I considered that if someone new to cooking wanted a cookbook, it might be a fairly large cookbook. There’s a wide range of recipes that they would want to try and make. At the same time, they would not likely be relying on a recipe for Gordon Ramsay’s Beef Wellington to be included in the book. The level of difficulty for preparing that dish is likely too high for someone who is brand-new to cooking. As for someone who is new to learning proofs (first-year university level), they likely won’t need access to a resource that generates very advanced proofs. As long as their resource can just cover queries used at the first-year level, then in their eyes the resource has terrific coverage. This was an important distinction made in the planning stage.

During the development of the prototype, I quickly realized there was a rather large challenge right out of the gate. Many of the proofs I wanted the application to cover involved simplifying expressions, which can be a very complex program on its own to write. It didn’t make sense to reinvent the wheel, and this was when I first discovered Wolfram|Alpha’s APIs. Using its APIs, I could simply make calls directly to Wolfram|Alpha (i.e. simplify an expression), and return the result back to my application. I had a few specific questions about the usage of the APIs, and so I decided to just contact Wolfram|Alpha directly with my questions.

The helpful person at the other end answered my questions quickly, and then the conversation took a direction I didn’t expect. After I explained the little side project I was building, I was asked to demonstrate the application to a few different members at Wolfram|Alpha (after signing a nondisclosure agreement, of course). At this point the application was fairly limited in terms of the number of proofs it could handle, but it performed well as a prototype or proof of concept. The demonstration went well, but I was once again surprised at the direction the meeting took. It was just a couple weeks after the demonstration that I was offered—and then accepted—a position with Wolfram|Alpha as a Math Content Developer.

How the app works

I moved to the Wolfram|Alpha HQ in Champaign, Illinois, the following summer. There, I was able to work on several projects related to new math features being added to the site. However, my main project was to reimplement my induction proof program to become part of Wolfram|Alpha.

The goal of the project was that a student would be able to input any induction proof question they had in a first-year course. For that to become a reality, I scoured the internet and several textbooks to search for all of the induction proof problems I could find… there were a lot.

To be clear, this project was not a database of all induction questions I could find. For the program to work consistently for all proofs (even ones it hadn’t seen before), I needed to first generalize the types of induction proofs that students are being taught. The decision was made early on that the program would handle three main types of queries for proofs:

  • Summation/product equalities, e.g. prove 1 + 2 + 3 +… + n = n(n + 1)/2, for n > 0
  • Expression divisibility, e.g. prove 8^n – 3^n is divisible by 5, for n > 0
  • Expression inequalities, e.g. prove 2^n > n^3, for n > 9

Using these three main types of proof queries, I then further broke down each of these query types into subsets of more specific types of proofs. Below is an example of what I mean by that.

Let’s say we want to write a proof for the following:

Prove 8^n – 3^n is divisible by 5, for n > 0

Instead of “hard-coding” the full solution of this proof into the application, I want to generalize it as much as I can so that it can possibly cover even more proofs. Therefore, I don’t add very specific proofs to the application, but instead add pattern-matched proofs.

In the expression 8^n – 3^n, I really don’t care all that much about the 8 and the 3. The reason I don’t care is because even if those numbers change, it’s not going to change the structure of the proof being generated. That means if a user instead asked to prove that 10^n – 3^n is divisible by 7, the exact same steps would be followed for the 8^n – 3^n example, just with different values. In order to take advantage of this, I would therefore write a proof solution for this type of query:

Prove a^nb^n is divisible by d, for n > x

By pattern matching the user’s input, if it is detected that their query is in this form (i.e. follows this pattern), then their proof will be generated according to this structure. The variables (a, b, n, d, x) are then extracted in order to provide the values for the generated proof.

This concept is then applied to many different structures for proof queries. The pattern-matching approach seemed efficient in quickly covering most of the example proof questions I came across online and in textbooks. This approach is what allows the application to generate proofs for queries it hasn’t seen before, so long as it matches the pattern of one of the structures the app is aware of (of which there are many).

However, specific pattern matching is really only used for the expression divisibility and expression inequalities query types. For the summation/product equalities, the application is theoretically able to handle 100% of these queries. There is a specific mathematical approach for summation/product equalities that can be followed, regardless of the right-hand expression of the equality or the contents of the summation or product. This approach works both for proofs and disproofs of summation equalities. This is a nice feature, since it ensures complete coverage of induction proofs for this query type, so long as Wolfram|Alpha doesn’t time out due to large input.

For expression divisibility, most generated proofs are based solely off pattern matching the input. There is, however, a somewhat general-case algorithm used as a last-ditch effort, in case the given expression was not successfully pattern matched. The algorithm performs a certain manipulation on the expression in an attempt to put the expression into a specific form. This form is simply the addition of two terms, where one term is the inductive hypothesis and the other term contains a coefficient that has the given divisor as a factor. If this manipulation is successful, then a proof can be generated. Otherwise, it will fail.

As for expression inequalities, all the generated proofs are pattern matched since I am unaware of any general-case algorithm to apply toward expression inequality proofs. It makes more sense to pattern match the difference between the two expressions. Not only is this beneficial for inequality manipulation but this is also a crucial step in case any terms need to be eliminated. For example, here is a possible query:

Prove 2n + 7 + sin(n) < (n +7)^2 + sin(n), for n > 0

The proof generated here would ideally be pattern matched for the following:

Prove a n + c < (n + b)^d, for n > g

The problem is that the pattern-matched expression doesn’t include a sin(n) on each side of the inequality. However, if the difference between each side of the expression (e.g. left-hand side – right-hand side) is taken, the new query would just be 2n + 7 – (n+7)^2, which would be pattern matched against a n + c – (n+b)^d, which matches correctly.

It is also important to note that some of these variables can sometimes be optional (i.e. equal to 0 or 1), such as the constants, coefficients, and exponents.

Once testing for the application began, I noticed there were two types of queries that were consistently generating incorrect proofs. While the goal is to include coverage for as many proofs as possible, it is much, much, much worse to generate a proof that is mathematically incorrect as opposed to not generating a proof at all. As soon as the application starts producing invalid content, the quality and reliability of the application goes down the drain.

The first type of query that generated invalid proofs can be demonstrated with the following example:

Prove 4n + 7 > 2^n, for n ≥ 0

In this example, the base case (n = 0) correctly passes (since 7 > 1). However, the inductive step should fail because the actual claim being made is false (i.e. when n = 5, since 27 ≱ 32). But for whatever reason, the app attempted to generate an inductive step to make the proof valid. This resulted in faulty mathematics, and needed to be eliminated. Proofs using induction are terrific for verifying claims, but they are not so great for disproving claims. The decision was made then to not generate proofs (or “disproofs”) for expression inequality queries where the base case is valid but the given query is false.

The other type of query that generated invalid proofs can be explained using this example:

Prove 4n < 2^n, for n ≥ 0

This is actually a fairly tricky example. The graph below illustrates the comparison of these expressions:

Graph illustrating the comparison of the expressions

The base case of this example is n = 0, which results in 4(0) < 2(0), which simplifies to 0 < 1, which is true. Since the base case passes, the inductive step can proceed. Like the previous example, this query also generated an invalid inductive step. The problem is that many of the manipulations that are done in the inductive step may be assuming a certain range of k. For example, in the above proof, one of the steps relies on the fact 3 < 2^k, for k ≥ 2. However, the range of k values between the base case and k ≥ 2, where n = k, has not been proven to satisfy the given claim of
“Prove 4n < 2^n”. Therefore, the base case must now also include n = 1 and n = 2 in the base case coverage. Note that a “modified proof” would need to suggest that n > 4 since the case that n = 4 is not true for 4n < 2^n.

If a manipulation is made in the inductive step that requires n ≥ 2 but the base case is n = 0, then we haven’t proven the query to be true for n = 1 or n = 2. This is very similar to the false paradox “all horses are the same color.” The solution to this problem was not to get rid of the proof altogether. Instead, the base case step would test multiple values of n if any assumption was to be made in the inductive step. Therefore, the query 4n < 2^n would fail in the base case because it is not true for
n = 1.

The final challenge with this project involved the natural language processing (NLP). This allows a user to ask a question in plain English and have Wolfram|Alpha understand what they’re looking for. While I had no applicable experience with NLP before, the existing parsing/scanning framework on Wolfram|Alpha made it fairly easy to integrate the NLP component with the application. The main challenge was trying to determine all the possible ways a user can ask Wolfram|Alpha to prove a query using induction. This continues to be an ongoing development, as different phrasings for queries are still being added.

Using the app

Here’s a screen shot of the application on Wolfram|Alpha:

Application on Wolfram|Alpha

By clicking on the “Step-by-step solution” button (a Wolfram|Alpha Pro feature), you will be able to see the generated proof. All users, however, will be able to see the above content. Additional information has been added to the application in order to provide important details to the user. A feature added to this result page is that a different divisor is suggested when the proof is invalid.

Below is a screen shot of the generated proof with the “modified divisor”:

Generated proof with the modified divisor

For the multiple-base case example discussed, two proofs were generated for this result—one that shows the initial proof (invalid), and then a second proof that recommends a different base case in order to make the proof valid:

Two proofs generated for the multiple-base case example

One final, cool feature is that an invalid summation/product equality proof will be corrected with a new recommended expression:

Invalid summation/product equality proof will be corrected with a new recommended expression

Aftermath

The application went live on Wolfram|Alpha on October 15, 2015 (I’ll never forget it). This was approximately fifteen months after I began designing the prototype program. This date was also six days before I presented the application at the Wolfram Technology Conference on my twenty-first birthday. The success and feedback from this project has been more than I ever could have imagined.

By taking an idea and some initiative, I landed a position with the world’s largest computational knowledge engine (where I’ve been an employee for more than a year now). This initiative also played a key role in being accepted as a graduate student in the University of Waterloo’s symbolic computation department.

While the goal of the project was to generate induction proofs, this whole process was a terrific proof for the rewards that can result from imagination, originality, and hard work.

]]>
http://blog.wolfram.com/2016/07/14/behind-wolframalphas-mathematical-induction-based-proof-generator/feed/ 17
Wolfram Language Books around the World http://blog.wolfram.com/2016/07/06/wolfram-language-books-around-the-world/ http://blog.wolfram.com/2016/07/06/wolfram-language-books-around-the-world/#comments Wed, 06 Jul 2016 14:27:34 +0000 Zach Littrell http://blog.internal.wolfram.com/?p=31710 The population of Wolfram Language speakers around the globe has only grown since the language’s inception almost thirty years ago, and we always enjoy discovering users and authors who share their passion for Wolfram technologies in their own languages. So in this post, we are highlighting foreign-language books around the world that utilize Wolfram technologies, from a mathematical toolbox in Japanese to an introduction on bioinformatics from Germany.

Basic Mathematica Primer; Mathematica Basic Training Course; Mathematica-Based Digital Physics

Basic Mathematica Primer (Korean)
Our first title is from a group of authors fluent in both Korean and the Wolfram Language. Introducing Korean-language speakers to the powerful fundamentals of Mathematica, the authors—Seong-Deog Yang, Jang hoon Lee, Jiwon Hwang, Chae Yeong Lee, and Myeong Hwan An—combine rich source code and friendly instructions for novices to the Wolfram Language, systematically covering all the basic knowledge needed to explore mathematics with Wolfram technology.

Mathematica Basic Training Course (Chinese)
The Wolfram Language is widely used in China, with its tools for symbolic computation, graphics, high-precision calculations, and more. Hanlong Li, Ying Sui, Shuxian Miu, and Ting Han explore applications in higher mathematics, linear algebra, differential equations, probability, computing methods, operations research, and mathematical modeling with a large selection of examples, including advanced features in Mathematica 10. The authors even offer examples of writing and calling your own software packages to develop programs with the Wolfram Language.

Mathematica-Based Digital Physics (Chinese)
Focusing on the domain of digital physics, Junqin Jiang combines Mathematica’s array of tools with research methods and programming design ideas to help the reader master physics concepts. With interactive examples covering quantum mechanics, electric and magnetic fields, and also numerical solutions and trajectories for dynamics and differential equations, Mathematica-Based Digital Physics promises to make learning the basic numerical methods for solving physics problems much easier than it would be with traditional textbooks.

Toolbox for Mathematics, Mathematica Basic; Income Support Benefits Law; Algorithmen von Hammurapi bis Gödel: Mit Beispielen aus den Computeralgebrasystemen Mathematica und Maxima, 4 Auflage; Methoden der Bioinformatik: Eine Einführung zur Anwendung in Biologie und Medizin, 2 Auflage

Toolbox for Mathematics, Mathematica Basic (Japanese)
Etsuo Miyaoka has written a must-have book for not just Japanese Mathematica beginners but also for experienced users still exploring the features of Mathematica 10 and beyond. The first half is aimed at high-school- and early-college-level mathematics, and compares common mathematical expressions with Wolfram Language commands. The discussion then turns in the latter half to the mathematical expressions found in business and 3D schemes, and how Mathematica continues to be the definitive computing system to rely on.

Income Support Benefits Law (Japanese)
One of the domains that the Wolfram Language continues to stand out in worldwide is finance and economics. In this augmented edition, Konosuke Kimura uses Mathematica to create not only the programs but also the accompanying graphics and visualizations for creating linear graduated tax tables, computing income gaps, and calculating income tax revenue and annual expenditures for income support benefits.

Algorithmen von Hammurapi bis Gödel: Mit Beispielen aus den Computeralgebrasystemen Mathematica und Maxima, 4 Auflage (German)
Europe is also full of mathematicians, computer scientists, and other specialists that speak fluent Wolfram. Jochen Ziegenbalg, Oliver Ziegenbalg, and Bernd Ziegenbalg explore the motivation, development, and procedures of algorithms throughout history with meaningful and concrete examples. Using modern tools such as Mathematica’s computer algebra system, students, teachers, and even hobbyists can learn to appreciate the historical and humanistic dimensions of the algorithms all around them.

Methoden der Bioinformatik: Eine Einführung zur Anwendung in Biologie und Medizin, 2 Auflage (German)
Marc-Thorsten Hütt and Manuel Dehnert provide an intriguing and comprehensible introduction to both the mathematical foundations and concrete methods of bioinformatics. Covering questions in bioinformatics and systems biology, the authors give step-by-step implementations of important algorithms in Mathematica, exploring the concepts at the programming level as well. And this second edition extends on its predecessor by addressing more-current topics such as next-generation sequencing, GWAS data, and protein interaction networks.

These titles are just a few of the global family of books that include the Wolfram Language. You can find more Wolfram technologies books, both for English and other languages, by visiting the Wolfram Books site.

]]>
http://blog.wolfram.com/2016/07/06/wolfram-language-books-around-the-world/feed/ 0