I’ve been interested in education for a long time, and when someone suggests that the software system I’ve been working on for 20 years is bad for education, I take it personally.

So I was upset when a New Scientist magazine article “Physics Tool Makes Students Miss the Point,” reported on a study by Thomas Bing and Edward Redish, “Symbolic Manipulators Affect Mathematical Mindsets,” strongly implying that the study concluded that replacing paper-and-pencil calculations with Mathematica was educationally unsound.

And I was greatly relieved to find that the study itself says no such thing. Bing and Redish don’t recommend banishing Mathematica; they welcome it in their classrooms and point out many positive things about it, along with one relatively minor pitfall they suggest ways to work around.

What mindset led the reporter to jump to such a reactionary conclusion? Why use such an inflammatory headline in connection with level-headed research that showed, when you get right down to it, virtually the opposite of what the New Scientist headline says?

The question of what technology to use in the classroom comes up all the time, and the resulting debate often generates more heat than light. People feel strongly about the subject because at its heart it is a question about what it means to be human.

On one side are technologists who feel that any technology, particularly theirs, is a good thing that teachers should drop everything to adopt. These are the people who said television was going to end war by bringing knowledge and enlightenment to all.

On the other side are Luddites, anti-technology throwbacks who define correct education as whatever they got when they were in school. These are the people who were against ballpoint pens because they ruined students’ ability to write properly with a quill.

See what I mean by more heat than light? This is the context in which the New Scientist note should be seen: It is pandering to one side of this ideological divide. How do we leave these stereotypes behind and arrive at a more nuanced position on how we should spend classroom time?

Schools are not just about teaching practical skills, they are about our hope for the future. They must help our children think with depth and clarity, see the world with open eyes and ultimately be our worthy successors in the great enterprise we call civilization.

This creates a natural preservationist streak in education: We who built this place were educated in a certain way, with certain tools and methods, and if you propose to change anything you’d better start by explaining what’s so bad about the way we were raised.

The “don’t fix what ain’t broke” line of argument here is fundamentally wrong. There are a lot of things that have been and are now being taught in schools that are not like music and art: They are not part of what it means to be human. They are just bits of transitory knowledge or practical skills that happen to be useful in certain times and places and not in others. They can be let go. The trick, of course, is telling the difference.

Fortunately Bing and Redish have studied this question in one particular case, Mathematica in the advanced physics classroom, so let’s look at what their study actually says.

The problem they address is an old one, well known even in the slide-rule era—that students are tempted to approach science problems primarily as computational challenges, to start “plugging and chugging” before they have fully thought through the scientific concepts and mathematical representations involved.

The authors find that Mathematica, by making mathematical computation much faster and more efficient, has a tendency to encourage students to go there even sooner.

The authors do not say that Mathematica harms the students’ understanding of the mathematics they are doing. Those who use Mathematica, they believe, get a better “math sense” than those who don’t, and the authors even express admiration for the sophistication with which the students understand the mathematics they are doing, an ability that has come about in part because of their extensive use of Mathematica.

The students just might start using that enhanced mathematical ability a bit too soon, before they have fully digested the physics of the problem.

Well, I suppose that is a danger. While I’m sure this has never happened to you, dear reader, I have to admit that I have occasionally found myself an hour into a problem only to realize that I’ve been spending my time down a blind alley, when what I really needed was to step back and think about the problem a bit more. But as the authors say, going down blind alleys is far from unique to Mathematica usage.

Bing and Redish, in a wonderful turn of phrase, describe Mathematica as an “epistemologically potent tool”, by which they mean a tool that changes your relationship to knowledge.

Absolutely! Anyone who has used Mathematica for any length of time knows that it forever changes your concept of what kinds of knowledge you can interact with fluidly, or create easily.

Just as Google has caused people to have a persistent feeling that pretty much anything can be looked up in a few seconds, habitual Mathematica use causes people to have a persistent feeling that just about anything can be computed, probably before breakfast.

Both are of course only partly true, but what matters is the change in attitude: If you assume you’ll be able to find any fact you might need, you soon start reflexively googling things, perhaps even before clearly understanding the question you’re trying to answer. And if you assume you’ll be able to quickly do any math you need done, you start relying on math more and perhaps doing computations that, upon further reflection, didn’t really need to be done.

It’s hard to see this as a particularly sinister effect; it’s just one variable in the complex interaction between student and problem. It changes the dynamic somewhat, as any powerful tool will, and indeed should. And contrary to the impression you might get from New Scientist, the authors don’t say the solution is to throw out Mathematica. They just sensibly warn that you can’t simply throw it into the mix; you have to adjust your teaching to it, watch for danger signs of over-reliance and teach students new paradigms for working with it. Who would argue with that?

Now, let’s follow the advice of the study and take a step back to look at this pedagogical problem from a broader perspective.

What Mathematica is replacing in this context is manual algebraic computation—manipulating formulae, doing integrals, etc. by hand with pencil and paper.

But that’s a technology too: Pencil-and-paper algebra is a highly developed and specific set of algorithms worked out since the 1400s to allow for efficient hand computation. Its use influences students too, pushing them in one direction or another just as much as a tool like Mathematica does.

One thing about hand calculation is that it’s very time-consuming and error-prone, and as a result people go out of their way to avoid it. I think it’s not unfair to say that centuries of reliance on hand calculation pushed science a little too far in the direction of analysis and pre-simplification of problems, purely to avoid the hassle of doing pages and pages of algebra. Many branches of science are experiencing something of a renaissance through the fact that problems long considered far too difficult can now be tackled through the use of computer-assisted reasoning. This is a result of Mathematica and programs like it removing the skew introduced by the limitations and peculiarities of an earlier technology, the pencil.

These limitations strongly affect homework too—not just how it’s done but also how it’s designed.

In one of the two main examples cited in the study, the students waste time carefully and cleverly using Mathematica to solve a difficult integral that, it turns out, they didn’t need to solve in the first place because it cancels out. Why does it cancel out? Because if it didn’t, this wouldn’t be a homework problem; it would be way too hard to solve by hand.

A student who didn’t have Mathematica but did have rudimentary test-taking skills would have realized immediately that the problem couldn’t be that hard, and would have started looking for ways around the integral.

You can call that a higher level of thinking, and blame Mathematica for leading the students astray, but I think this misses the point. In any sort of real-world scientific problem, chances are you actually would need to solve the integral. Are we really serving our students well by shielding them from this reality and encouraging them to look for tricks and shortcuts they know must be there simply by virtue of the fact that they’re working on a homework problem and not a real-world problem?

Sure, sometimes you want a simple, diagrammatic sketch of a problem with simplifications that lead to clarity. But it’s very limiting if you’re forced to make all problems simple enough to do by hand. Mathematica gives you, among many other things, the option of exposing students to a much wider range of closer-to-real-world problems. And that is a good thing, no question about it.

Think about it this way: Imagine that use of Mathematica were well-established, routine and expected in all higher-level physics courses, and that teaching methods were thoroughly adapted to its presence. Now imagine that someone did a study investigating the effects, positive and negative, of removing it and requiring students to do the algebra and calculus stages of the problems by hand.

The negative consequences would be legion! Effects would range from hours wasted on sign errors (carefully propagated through pages of detailed algebra) to an almost pathological aversion to speculatively exploring multiple alternate approaches, a key skill in any science. Dropped into a Mathematica-rich environment, the “new technology” of hand computation would look like a pretty poor substitute.

OK, you say I’m not being fair here, but that’s exactly my point: It’s not fair the other way around either. You can’t undertake a meaningful evaluation of Mathematica by dropping it into a hand-calculation-based context and then listing only the negative effects. (And in fact only the New Scientist’s note does this; the study authors themselves find more positive than negative things to say.)

I’m complaining here about what seems like an unfair reflex reaction against use of a particular new technology in the classroom. But negative reactions to classroom technology are not always wrong, in fact they are quite often justified. A lot of companies are trying to sell high-tech snake oil (i.e. crap) to schools, and educators need to study the choices carefully.

How does one decide whether a new technology is worth introducing into the classroom? I like the criteria suggested by the study: Is the software an “epistemologically potent tool”? Does it give students a fundamentally new way of looking at the world, some widened horizon, broadened perspective, or deepened capacity for insight?

This is where a lot of “educational software” falls flat on its face. Typical page-turners or insipid drill, quiz, now-go-play-a-video-game programs simply don’t have anything to add to the world. They are re-hashings and reformulations of the same old same old, with a lot of time-wasting graphics thrown in. (I’m not going to go into a whole rant here; suffice to say that I don’t like these things, and study after study has concluded that they do not work.)

But that’s not Mathematica.

Mathematica is a powerful, deep and life-changing creation. It fundamentally alters your relationship to computation. It leads you to think differently. As the authors of this study conclude, that is not a bad thing. It is a powerful thing—and like any powerful thing it needs to be treated with respect, and sometimes with caution.

If you bring Mathematica into your classroom it will change your students, it will change you, it will change your teaching. And if you aren’t aware of that and don’t respond to it, yes, there is a danger that at first some of those changes will be for the worse. But even taking the most negative possible interpretation of Bing and Redish’s study, the dangers are mild, easily avoided and strongly outweighed by the advantages.

And if you don’t believe me I encourage you to read the original study, not the misleading summary.