The New World of Notebook Publishing
October 24, 2019 — Stephen Wolfram
Wolfram Notebooks on the Web
We’ve been working towards it for many years, but now it’s finally here: an incredibly smooth workflow for publishing Wolfram Notebooks to the web—that makes possible a new level of interactive publishing and computation-enabled communication.
You create a Wolfram Notebook—using all the power of the Wolfram Language and the Wolfram Notebook system—on the desktop or in the cloud. Then you just press a button to publish it to the Wolfram Cloud—and immediately anyone anywhere can both read and interact with it on the web.
It’s an unprecedentedly easy way to get rich, interactive, computational content onto the web. And—together with the power of the Wolfram Language as a computational language—it promises to usher in a new era of computational communication, and to be a crucial driver for the development of “computational X” fields.
When a Wolfram Notebook is published to the cloud, it’s immediately something people can read and interact with. But it’s more than that. Because if you press the Make Your Own Copy button, you’ll get your own copy of the notebook, which you can not only read and interact with, but also edit and do computation in, right on the web. And what this means is that the notebook becomes not just something you look at, but something you can immediately use and build on.
And, by the way, we’ve set it up so that anyone can make their own copy of a published notebook, and start using it; all they need is a (free) Cloud Basic account. And people with Cloud Basic accounts can even publish their own notebooks in the cloud, though if they want to store them long term they’ll have to upgrade their account. (Through the Wolfram Foundation, we’re also developing a permanent curated Notebook Archive for public-interest notebooks.)
There are lots of other important workflows too. On a computer, you can immediately download notebooks to your desktop, and run them there natively using the latest version of the Wolfram Player that we’ve made freely available for many years. You can also run notebooks natively on iOS devices using the Wolfram Player app. And the Wolfram Cloud app (on iOS or Android) gives you a streamlined way to make your own copy of a notebook to work with in the cloud.
You can publish a Wolfram Notebook to the cloud, and you can use it as a complete, rich webpage. But you can also embed the notebook inside an existing webpage, providing anything from a single (perhaps dynamically updated) graphic to a full interactive interface or embedded document.
And, by the way, the exact same technology that enables Wolfram Notebooks in the cloud also allows you to immediately set up Wolfram Language APIs or form interfaces, for use either directly on the web, or through client libraries in languages like Python and Java.
The Story of Notebooks
We invented notebooks in 1988 as the main interface for Mathematica Version 1.0, and over the past three decades, many millions of Wolfram Notebooks have been made. Some record ongoing work, some are exercises, and some contain discoveries small and large. Some are expositions, presentations or online books and papers. Some are interactive demonstrations. And with the emergence of the Wolfram Language as a full-scale computational language, more and more now serve as rich computational essays, communicating with unprecedented effectiveness in a mixture of human language and computational language.
Over the years, we’ve progressively polished the notebook experience with a long series of user interface innovations, adapted and optimized for successive generations of desktop systems. But what’s allowed us now to do full-scale notebook publishing on the web is that—after many years of work—we’ve managed to get a polished version of Wolfram Notebooks that run in the cloud, much as they do on desktop.
Create a notebook on the desktop or in the cloud, complete with all its code, hierarchical sections, interactive elements, large graphics, etc. When it’s published as a cloud notebook people will be able to visit it just like they would visit any webpage, except that it’ll automatically “come alive” and allow all sorts of interaction.
Some of that interaction will happen locally inside the web browser; some of it will automatically access servers in the cloud. But in the end—reflecting our whole hyperarchitecture approach—Wolfram Notebooks will run seamlessly across desktop, cloud and mobile. Create your content once, and let people not only read it anywhere, but also interact with it, as well as modify and compute with it.
What’s in a Notebook
When you first go to a Wolfram Notebook in the cloud it might look like an ordinary webpage. But the fact that it’s an active, computational document means there are lots of things you can immediately do with it. If you see a graphic, you’ll immediately be able to resize it. If it’s 3D, you’ll be able to rotate it too. Notebooks are typically organized in a hierarchy of cells, and you can immediately open and close groups of cells to navigate the hierarchy.
There can also be dynamic interactive content. In the Wolfram Language, functions like Manipulate automatically set up interactive user interfaces in notebooks, with sliders and so on—and these are automatically active in a published cloud notebook. Other content can be dynamic too: using functions like Dynamic you can for example dynamically pull data in real time from the Wolfram Knowledgebase or the Wolfram Data Drop or—if the user allows it—from their computer’s camera or microphone.
When you write a computational essay you typically want people to read your Wolfram Language code, because it’s part of how you’re communicating your content. But in a Wolfram Notebook you can also use Iconize to just show an iconized version of details of your code (like, say, options for how to display graphics):
Normally when you do a computation in a Wolfram Notebook, there’ll be a succession of In[ ] and Out[ ] cells. But you can always double-click the Out[ ] cell to close the In[ ] cell, so people at first just see the output, and not the computational language code that made it.
One of the great things about the Wolfram Language is how integrated and self-contained it is. And that means that it’s realistic to pick up even fragments of code from anywhere in a notebook, and expect to have it work it elsewhere. In a published notebook, just click a piece of code and it’ll get copied so you can paste it into a notebook you’re creating, on the cloud or the desktop.
A great source of “ready-made” interactive content for Wolfram Notebooks is the 12,000+ interactive Demonstrations in the Wolfram Demonstrations Project. Press Copy to Clipboard and you can paste the Demonstration (together with closed cells containing its code) into any notebook.
Once you’ve assembled the notebook you want, you can publish it. On the desktop, go to File > Publish to Cloud. In the cloud, just press Publish. You can either specify the name for the published notebook—or you can let the system automatically pick a UUID name. But you can take any notebook—even a large one—and very quickly have a published version in the cloud.
It didn’t take long after we invented notebooks back in 1988 for me to start thinking about using them to enable a new kind of computational publishing, with things like computational journals and computational books. And, indeed, even very early on, there started to be impressive examples of what could be done.
But with computation tied to the desktop, there was always a limit to what could be done. Even before the web, we invented systems for distributing notebooks as desktop files. Later, when web browsers existed, we built plugins to access desktop computation capabilities from within browsers. And already in the mid-1990s we built mechanisms for generating content through web servers from within webpages. But it was only with modern web technology and with the whole framework of the Wolfram Cloud that the kind of streamlined notebook publishing that we’re releasing today has become possible.
But given what we now have, I think there’s finally an opportunity to transform things like scientific and technical publishing—and to make them truly take advantage of the computational paradigm. Yes, there can be rich interactive diagrams, that anyone can use on the web. And, yes, things can be dynamically updated, for example based on real-time data from the Wolfram Knowledgebase or elsewhere.
But important as these things are, I think they ultimately pale in comparison to what Wolfram Notebooks can do for the usability and reproducibility of knowledge. Because a Wolfram Notebook doesn’t just give you something to read or even interact with; it can also give you everything you need to actually use—or reproduce—what it says.
Either directly within the notebook, or in the Wolfram Data Repository, or elsewhere in the cloud, there can for example be underlying data—say from observations or experiments. Then there can be code in the notebook that computes graphics or other outputs that can be derived from this data. And, yes, that code could be there just to be there—and could be hidden away in some kind of unreadable computational footnote.
But there’s something much more powerful that’s now uniquely possible with the Wolfram Language as it exists today: it’s possible to use the language not just to provide code for a computer to run, but also to express things in computational language in a way that not just computers, but also humans, can readily understand. Technical papers often use mathematical notation to succinctly express mathematical ideas. What we’ve been working toward all these years with the Wolfram Language is to provide a full-scale computational language that can also express computational ideas.
So let’s say you’ve got a technical paper that’s presented as a Wolfram Notebook, with lots of its content in the Wolfram Language. What can you do with it? You can certainly run the computational language code to make sure it produces what the paper says. But more important, you can take pieces of that computational language code and build on it, using it yourself in your own notebook, running it for different cases, modifying it, etc.
Of course, the fact that this can actually work in practice is incredibly nontrivial, and relies on a huge amount of unique development that we’ve done. Because first and foremost, it requires a coherently designed, full-scale symbolic computational language—because that’s the only way it’s realistic to be able to take even small fragments of code and have them work on their own, or in different situations. But there’s more too: it’s also critical that code that works now goes on working in the future, and with the design discipline we’ve had in the Wolfram Language we have an impressive history of compatibility spanning more than 30 years.
Back in the 1970s when I started writing technical papers, they typically began as handwritten documents. Later they were typed on a typewriter. Then when a journal was going to publish them, they would get copyedited and typeset, before being printed. It was a laborious—and inevitably somewhat expensive—process.
By the 1980s, personal computers with word processors and typesetting systems were becoming common—and pretty soon journals could expect “camera-ready” electronic versions of papers. (As it happens, in 1986 I started what may have been the first journal to routinely accept such things.)
And as the technology improved, the quality of what an author could readily make and what a publisher could produce in a fully typeset journal gradually converged, leaving the primary role of the journal being around branding and selectivity, and for many people calling its value into question.
But for computational journals it’s a new story. Because if a paper has computational language code in it, there’s the immediate question of whether the code actually runs, and runs correctly. It’s a little like the old process of copyediting a paper so it could be typeset. There’s real human work—and understanding—that’s needed to make sure the code runs correctly. The good news is that one can use methods from software quality assurance, now enhanced by things like modern machine learning. But there’s still real work to be done—and as a result there’s real value to be added by the process of “official publication” in a computational journal, and there’s a real reason to actually have a computational journal as an organized, potentially commercial, thing.
We’ve been doing review and curation of submissions to the Wolfram Demonstrations Project for a dozen years now. And, yes, it takes work. But the result is that we can be confident that the Demonstrations we publish actually run, and will go on doing so. For the Wolfram Data Repository we also have a review process, there to ensure that data is computable at an appropriate level.
One day there’ll surely be “first-run” computational journals, where new results are routinely reported through computational essays. But even before that, we can expect ancillary computational journals, that provide genuine “computation-backed” and “data-backed” publication. There just hasn’t been the technology to make this work properly in the past. Now, with the Wolfram Language, and the new streamlined web publishing of Wolfram Notebooks, everything that’s needed finally exists.
Changing the Way I Work
It’s always a sign that something is important when it immediately changes the way one works. And that’s certainly something that’s happened for me with notebook publishing.
I might give a talk where I build up a notebook, say doing a live experiment or a demonstration. And then at the end of the talk, I’ll do something new: I’ll publish the notebook to the cloud (either by pressing the button or using CloudPublish). Then I’ll make a QR code of the notebook URL (say using BarcodeImage), and show it big on the screen. People in the audience can then hold up their phones to read the QR code—and then just click the URL, and immediately be able to start using my notebook in the Wolfram Cloud on their phones.
I can tell that notebook publishing is getting me to write more, because now I have a convenient way to distribute what I write. I’ll often do computational explorations of one thing or another. And in the past, I’d just store the notebooks I made in my filesystem (and, yes, over 30+ years I’ve built up a huge number). But now it’s incredibly fast to add some text to turn the notebooks into computational essays—that I can immediately publish to the cloud, so anyone can access them.
Sometimes I’ll put a link to the published notebook in a post like this; sometimes I’ll do something like tweet it. But the point is that I now have a very streamlined way to give people direct access to computational work I do, in a form that they can immediately interact with, and build on.
From a technical development point of view, the path to where we are today has been a long and complex one, involving many significant achievements in software engineering. But the result is something conceptually clear and simple, though extremely powerful—that I think is going to enable a major new level of computation-informed communication: a new world of notebook publishing.
More about Wolfram Notebooks:
To comment, please visit the copy of this post at Stephen Wolfram’s Writings »