Two years ago we released Version 12.0 of the Wolfram Language. Here are the updates in connectivity since then, including the latest features in 13.0. The contents of this post are compiled from Stephen Wolfram’s Release Announcements for 12.1, 12.2, 12.3 and 13.0.

Paclet System

Paclets for All (March 2020)

&#10005 Length[PacletFind[]]

In my Wolfram Language system right now, I have 467 paclets installed. What is a paclet? It’s a modular package of code and other resources that gets installed into a Wolfram Language system to deliver pretty much any kind of functionality.

We first invented paclets in 2006, and we’ve been increasingly using them to do incremental distribution of a great many pieces of Wolfram Language functionality. Paclets are versioned, and can be set to automatically update themselves. Up until now, paclets have basically been something that (at least officially) only we create and distribute, from our central paclet server.

But as of Version 12.1, we’re opening up our paclet system so anyone can use it, and we’re making it a fully documented and supported part of the Wolfram Language. Ultimately, a paclet is a file structure that contains assets or resources of various kinds, together with a special PacletInfo.wl file that defines how the paclet should integrate itself into a Wolfram Language system.

A paclet can set up code to execute at startup time. It can set up symbols whose definitions will be autoloaded. It can install documentation. It can put items into menus. And in general it can set up assets to be used in almost any part of the fairly complex structure that is a deployed Wolfram Language system.

Typically a paclet is distributed as a single archive file, and there are many ways someone can get such a paclet file. We maintain a central paclet server that’s used by the Wolfram Language system to get automatic downloads. But in the near future, we’re also going to have a full paclet repository through which users will be able to distribute paclets. (We’re also going to make it possible for Wolfram Enterprise Private Clouds to have their own paclet repositories.)

I might mention how I see the Wolfram Paclet Repository relating to our Wolfram Function Repository. The Function Repository is built to extend the functionality of the Wolfram Language one function at a time, always maintaining the overall structure and consistency of the language. The Paclet Repository will let people distribute complete environments that may serve some particular purpose, but may not maintain the structure and consistency of the overall language. Paclets will be extremely useful, and we intend them to be as freely used as possible. So whereas the Wolfram Function Repository has a curation process to ensure a certain level of design consistency, we plan to make the Wolfram Paclet Repository basically completely open.

The goal is to allow a rich ecosystem of user-contributed paclets to develop. The Paclet Repository will serve as a smooth distribution channel for Wolfram Language material. (And by the way, I think it will be quite common for functions in the Function Repository to actually be based on code that’s distributed through the Paclet Repository—with the Function Repository serving as a streamlined and structured presentation mechanism for the functions.)

In Version 12.1 there are a variety of functions for creating and managing paclets. With the Wolfram Language, PacletObject is the symbolic representation of a paclet. Here’s a trivial example of what a paclet might be like:

&#10005 PacletObject[<|"Name" -> "TrivialPaclet", "Version" -> "1.0", "Extensions" -> {{"Kernel", "Context" -> "TrivialPackage`"}}|>]

This can then be packaged into a .paclet file using CreatePacletArchive—and this file can then be distributed just as a file, or can be put on a paclet server. Once someone has the file, it’s just a question of using PacletInstall, and the paclet will come to life, inserting the necessary hooks into a Wolfram Language system so that its contents are appropriately used or exposed.

Introducing Tools for the Creation of Paclets (December 2021)

The Function Repository is all about creating single functions that add functionality. But what if you want to create a whole new world of functionality, with many interlinked functions? And perhaps one that also involves not just functions, but for example changes to elements of your user interface too. For many years we’ve internally built many parts of the Wolfram Language system using a technology we call paclets—that effectively deliver bundles of functionality that can get automatically installed on any given user’s system.

In Version 12.1 we opened up the paclet system, providing specific functions like PacletFind and PacletInstall for using paclets. But creating paclets was still something of a black art. In Version 13.0 we’re now releasing a first round of tools to create paclets, and to allow you to deploy them for distribution as files or through the Wolfram Cloud.

The paclet tools are themselves (needless to say) distributed in a paclet that is now included by default in every Wolfram Language installation. For now, the tools are in a separate package that you have to load:

&#10005

To begin creating a paclet, you define a “paclet folder” that will contain all the files that make up your paclet:

&#10005

This sets up the basic outline structure of your paclet, which you can then add files to:

As an alternative, you could specify some components in your paclet right when you first create the paclet:

&#10005

There are all sorts of elements that can exist in paclets, and in future versions there’ll be progressively more tooling to make it easier to create them. In Version 13.0, however, a major piece of tooling that is being delivered is Documentation Tools, which provides tools for creating the same kind of documentation that we have for built-in system functions.

You can access these tools directly from the main system Palettes menu:

Now you can create as notebooks in your paclet function reference pages, guide pages, tech notes and other documentation elements. Once you’ve got these, you can build them into finished documentation using PacletDocumentationBuild. Then you can have them as notebook files, standalone HTML files, or deployed material in the cloud.

Coming soon will be additional tools for paclet creation, as well as a public Paclet Repository for user-contributed paclets. An important feature to support the Paclet Repository—and that can already be used with privately deployed paclets—is the new function PacletSymbol.

For the Function Repository you can use ResourceFunction["name"] to access any function in the repository. PacletSymbol is an analog of this for paclets. One way to use a paclet is to explicitly load all its assets. But PacletSymbol allows you to “deep call” into a paclet to access a single function or symbol. Just like with ResourceFunction, behind the scenes all sorts of loading of assets will still happen, but in your code you can just use PacletSymbol without any visible initialization. And, by the way, an emerging pattern is to “back” a collection of interdependent Function Repository functions with a paclet, accessing the individual functions from the code in the Function Repository using PacletSymbol.

Introducing Context Aliases (December 2021)

When you use a name, like x, for something, there’s always a question of “which x?” From the very beginning in Version 1.0 there’s always been the notion of a context for every symbol. By default you create symbols in the Global context, so the full name for the x you make is Global`x.

When you create packages, you typically want to set them up so the names they introduce don’t interfere with other names you’re using. And to achieve this, it’s typical to have packages define their own contexts. You can then always refer to a symbol within a package by its full name, say Package`Subpackage`x etc.

But when you just ask for x, what do you get? That’s defined by the setting for $Context and $ContextPath.

But sometimes you want an intermediate case. Rather than having just x represent Package`x as it would if Package` were on the context path $ContextPath, you want to be able to refer to x “in its package”, but without typing (or having to see) the potential long name of its package.

In Version 13.0 we’re introducing the notion of context aliases to let you do this. The basic idea is extremely simple. When you do Needs["Context`"] to load the package defining a particular context, you can add a “context alias”, by doing Needs["Context`"->"alias`"]. And the result of this will be that you can refer to any symbol in that context as alias`name. If you don’t specify a context alias, Needs will add the context you ask for to $ContextPath so its symbols will be available in “just x” form. But if you’re working with many different contexts that could have symbols with overlapping names, it’s a better idea to use context aliases for each context. If you define short aliases there won’t be much more typing, but you’ll be sure to always refer to the correct symbol.

This loads a package corresponding to the context ComputerArithmetic`, and by default adds that context to $ContextPath:

&#10005

Now symbols with ComputerArithmetic can be used without saying anything about their context:

&#10005

This loads the package defining a context alias for it:

&#10005

Now you can refer to its symbols using the alias:

&#10005

The global symbol $ContextAliases specifies all the aliases that you currently have in use:

&#10005

By the way, just like our convention about symbols having names that start with uppercase letters, it’s been a common general convention to have context names for packages also start with uppercase letters. Now that we have context aliases as well, we’re suggesting the convention of using lowercase letters for these.

Databases & File Import/Export

Just Type SQL (December 2020)

In Version 12.0 we introduced powerful functionality for querying relational databases symbolically within the Wolfram Language. Here’s how we connect to a database:

&#10005 db = DatabaseReference[ FindFile["ExampleData/ecommerce-database.sqlite"]]

Here’s how we connect the database so that its tables can be treated just like entity types from the built-in Wolfram Knowledgebase:

&#10005 EntityRegister[EntityStore[RelationalDatabase[db]]]

Now we can for example ask for a list of entities of a given type:

&#10005 EntityList["offices"]

What’s new in 12.2 is that we can conveniently go “under” this layer, to directly execute SQL queries against the underlying database, getting the complete database table as a Dataset expression:

&#10005 ExternalEvaluate[db, "SELECT * FROM offices"]

These queries can not only read from the database, but also write to it. And to make things even more convenient, we can effectively treat SQL just like any other “external language” in a notebook.

First we have to register our database, to say what we want our SQL to be run against:

&#10005 RegisterExternalEvaluator["SQL", db]

And now we can just type SQL as input—and get back Wolfram Language output, directly in the notebook:

Importing PDF (December 2020)

Want to analyze a document that’s in PDF? We’ve been able to extract basic content from PDF files for well over a decade. But PDF is a highly complex (and evolving) format, and many documents “in the wild” have complicated structures. In Version 12.2, however, we’ve dramatically expanded our PDF import capabilities, so that it becomes realistic to, for example, take a random paper from arXiv, and import it:

&#10005 Import["https://arxiv.org/pdf/2011.12174.pdf"]

By default, what you’ll get is a high-resolution image for each page (in this particular case, all 100 pages).

If you want the text, you can import that with "Plaintext":

&#10005 Import["https://arxiv.org/pdf/2011.12174.pdf", "Plaintext"]

Now you can immediately make a word cloud of the words in the paper:

&#10005 WordCloud[%]

This picks out all the images from the paper, and makes a collage of them:

&#10005 ImageCollage[Import["https://arxiv.org/pdf/2011.12174.pdf", "Images"]]

You can get the URLs from each page:

&#10005 Import["https://arxiv.org/pdf/2011.12174.pdf", "URLs"]

Now pick off the last two, and get images of those webpages:

&#10005 WebImage /@ Take[Flatten[Values[%]], -2]

Depending on how they’re produced, PDFs can have all sorts of structure. "ContentsGraph" gives a graph representing the overall structure detected for a document:

&#10005 Import["https://arxiv.org/pdf/2011.12174.pdf", "ContentsGraph"]

And, yes, it really is a graph:

&#10005 Graph[EdgeList[%]]

For PDFs that are fillable forms, there’s more structure to import. Here I grabbed a random unfilled government form from the web. Import gives an association whose keys are the names of the fields—and if the form had been filled in, it would have given their values too, so you could immediately do analysis on them:

&#10005 Import["https://www.fws.gov/forms/3-200-41.pdf", "FormFieldRules"]

Untangling Email, PDFs and More (December 2021)

What do email threads really look like? I’ve wondered this for ages. And now in Version 13.0 we have an easy way to import MBOX files and see the threading structure of email. Here’s an example from an internal mailing list of ours:

&#10005

Now we can do standard graph operations on this:

&#10005

An important new feature of Version 12.2 was the ability to faithfully import PDFs in a variety of forms—including page images and plain text. In Version 13.0 we’re adding the capability to import PDFs as vector graphics.

Here’s an example of pages imported as images:

&#10005

Now here’s a page imported as vector graphics:

&#10005

And now, to prove it’s vector graphics, we can actually go in and modify it, right down to the strokes used in each glyph:

Now that we have Video in Wolfram Language, we want to be able to import as many videos as possible. We already support a very complete list of video containers and codecs. In Version 13.0 we’re also adding the ability to import .FLV (Flash) videos, giving you the opportunity to convert them to modern formats.

External Services & Operations

External Connectivity (March 2020)

We want the Wolfram Language to provide a consistent computational representation of as much as possible. And that means that in addition to things like the molecules we just discussed, we want our language to be able to represent—and seamlessly interact with—all the other kinds of computational systems that exist in the world, whether they be programs, languages, databases or whatever. The list of kinds of things we can deal with is very long—but in Version 12.1 we’ve made some significant additions.

The Wolfram Language has been able to call programs in other languages through what’s now WSTP since 1989, but in recent years we’ve been working to make it ever easier and more automatic to do this. And for example in Version 11.2 we introduced ExternalEvaluate, which provides a high-level way to directly evaluate code in external languages, and, whenever possible, to get results back in a symbolic form that can be seamlessly used in the Wolfram Language.

In Version 12.1 we’ve added Julia, Ruby and R to our collection of external languages. There are all sorts of practical issues, of course. We have to make sure that an appropriate installation exists on a user’s computer, and that the data types used in programs can be meaningfully converted to Wolfram Language.

But in the end it’s very convenient. In a notebook, just type > at the beginning of a line, select your language, and enter the code, and evaluate:

&#10005 > [1,2,3+3]

But this doesn’t only work interactively. It’s also very convenient programmatically. For example, you can create a function in the external language, that is then represented symbolically in the Wolfram Language as an ExternalFunction object, and which, when called, runs code in the external language:

&#10005 > def square(x) x * x end

&#10005 % /@ {45, 135, 678, 34}

For each different language, however, one’s dealing with a whole new world of structures. But since we have built-in support for ZeroMQ (as well as having a connection to Jupyter available), we at least have the plumbing to deal with a very wide range of languages.

But particularly for languages like Python where we have full built-in connectivity, one of the significant things that becomes possible is to have functions that work just like standard Wolfram Language functions but are fully or partly implemented in a completely different language. Of course, to have this work seamlessly requires quite a bit of system support, for automated installation, sandboxing, etc. And for example, one of the things that’s coming is the ability to call functions containing Python code even in the Wolfram Cloud.

In addition to external languages, another area of expansion is external storage systems of all kinds. We’ve already got extensive support for the Bitcoin and Ethereum blockchains; in Version 12.1 we added support for the ARK blockchain. In addition, we introduced support for two external file storage systems: IPFS and Dropbox.

This is all it takes to put a Spikey into the globally accessible IPFS file system:

&#10005 ExternalStoragePut[CloudGet["https://wolfr.am/L9rTMCn6"], ExternalStorageBase -> "IPFS"]

Here’s the content identifier:

&#10005 %["CID"]

And now we can get our Spikey back:

&#10005 ExternalStorageGet["QmcNotbm3RZLv7caaPasU8LiHjNCCuuEemrCgbLAsN49TZ"]

You can do the same kind of thing with Dropbox, and after authentication (either through a browser, or through our new SystemCredential mechanism, discussed below) you can put expressions or upload files, and they’ll immediately show up in your Dropbox filesystem.

Given the framework that’s been introduced in 12.1, we’re now in a position to add connections to other external file storage systems, and those will be coming in future versions.

In addition to plain files, we also have a sophisticated framework for dealing with relational databases. Much of this was introduced in Version 12.0, but there are some additions in 12.1. For example, you can now also connect directly to Oracle databases. In addition, there are new functions for representing relational set operations: UnionedEntityClass, IntersectedEntityClass, ComplementedEntityClass.

And, of course, these work not only on external databases but also on our own built-in knowledgebase:

&#10005 SortedEntityClass[UnionedEntityClass[ \!\(\* NamespaceBox["LinguisticAssistant", DynamicModuleBox[{Typeset`query$$ = "EU", Typeset`boxes$$ = TemplateBox[{"\"European Union\"", RowBox[{"EntityClass", "[", RowBox[{"\"Country\"", ",", "\"EuropeanUnion\""}], "]"}], "\"EntityClass[\\\"Country\\\", \\\"EuropeanUnion\\\"]\"", "\"countries\""}, "EntityClass"], Typeset`allassumptions$$ = {{ "type" -> "Clash", "word" -> "EU", "template" -> "Assuming \"${word}\" is ${desc1}. Use as \ ${desc2} instead", "count" -> "2", "Values" -> {{ "name" -> "CountryClass", "desc" -> "a class of countries", "input" -> "*C.EU-_*CountryClass-"}, { "name" -> "Unit", "desc" -> "a unit", "input" -> "*C.EU-_*Unit-"}}}}, Typeset`assumptions$$ = {}, Typeset`open$$ = {1, 2}, Typeset`querystate$$ = { "Online" -> True, "Allowed" -> True, "mparse.jsp" -> 0.363881`6.012504373043238, "Messages" -> {}}}, DynamicBox[ToBoxes[ AlphaIntegration`LinguisticAssistantBoxes["", 4, Automatic, Dynamic[Typeset`query$$], Dynamic[Typeset`boxes$$], Dynamic[Typeset`allassumptions$$], Dynamic[Typeset`assumptions$$], Dynamic[Typeset`open$$], Dynamic[Typeset`querystate$$]], StandardForm], ImageSizeCache->{232., {7., 15.}}, TrackedSymbols:>{ Typeset`query$$, Typeset`boxes$$, Typeset`allassumptions$$, Typeset`assumptions$$, Typeset`open$$, Typeset`querystate$$}], DynamicModuleValues:>{}, UndoTrackedVariables:>{Typeset`open$$}], BaseStyle->{"Deploy"}, DeleteWithContents->True, Editable->False, SelectWithContents->True]\), EntityClass["AdministrativeDivision", "AllUSStatesPlusDC"]], "Population" -> "Descending"][{"Name", "Population"}] // Dataset

We’ve been very active over the years in supporting as many file formats as possible. In Version 12.1 we’ve added the popular new HEIF image format. We’ve also updated our DICOM importer, so you can take those CT scans and MRIs and immediately start analyzing them with Image3D and our 3D image processing.

Like, this is part of our director of R&D’s knee:

&#10005 knee = Import["knee_mr/DICOMDIR", {"Image3D", 1, 1, 1}]; Image3D[knee, ColorFunction -> (Blend[{{0., RGBColor[0.05635, 0.081, 0.07687, 0.]}, {0.0777045, RGBColor[0.702347, 0.222888, 0.0171385, 0.0230167]}, {0.3, RGBColor[1., 0.6036, 0., 0.303215]}, {0.66, RGBColor[1., 0.9658, 0.4926, 0.661561]}, {1., RGBColor[1., 0.6436, 0.03622, 1.]}}, #]& )] // ImageAdjust // Blur[#, 3] &

In Version 11.3, we added MailServerConnect for connecting directly to mail servers. In 12.1, we’ve added a layer of caching, as well as a variety of new capabilities, that make the Wolfram Language uniquely powerful for programmatic mail processing. In addition, in Version 12.1 we’ve upgraded our capabilities for importing mail messages from EML and MBOX, in particular adding more controls for attachments.

Yet another new feature of 12.1 is stronger support for ZIP and TAR files, both their creation through CreateArchive, and their extraction through ExtractArchive—so you can now routinely handle tens of thousands of files, and gigabytes of data.

Whenever one is connecting to external sites or services, there are often issues of authentication. We’d had some nice symbolic ways to represent authentication for some time, like SecuredAuthenticationKey (that stores OAuth credentials). But for example in cases where you’ve got to give a username and password, there’s always been the issue of where you store those. In the end, you want to give them as part of the value for an Authentication option. But you don’t want to have them lying around in plaintext.

And in Version 12.1 there’s a nice solution to this: SystemCredential. SystemCredential ties into your system keychain—the encrypted storage that’s provided by your operating system, and secured by the login on your computer.

It’s as easy as this to store things in your system keychain:

&#10005 SystemCredential["secret"] = "it's a secret"

&#10005 SystemCredential["secret"]

RemoteEvaluate: Compute Someplace Else… (December 2020)

Along the lines of “use Wolfram Language everywhere” another new function in Version 12.2 is RemoteEvaluate. We’ve got CloudEvaluate which does a computation in the Wolfram Cloud, or an Enterprise Private Cloud. We’ve got ParallelEvaluate which does computations on a predefined collection of parallel subkernels. And in Version 12.2 we’ve got RemoteBatchSubmit which submits batch computations to cloud computation providers.

RemoteEvaluate is a general, lightweight “evaluate now” function that lets you do a computation on any specified remote machine that has an accessible Wolfram Engine. You can connect to the remote machine using ssh or wstp (or http with a Wolfram Cloud endpoint).

&#10005 RemoteEvaluate["ssh://byblis67.wolfram.com", Labeled[Framed[$MachineName], Now]]

Sometimes you’ll want to use RemoteEvaluate to do things like system administration across a range of machines. Sometimes you might want to collect or send data to remote devices. For example, you might have a network of Raspberry Pi computers which all have Wolfram Engine—and then you can use RemoteEvaluate to do something like retrieve data from these machines. By the way, you can also use ParallelEvaluate from within RemoteEvaluate, so you’re having a remote machine be the master for a collection of parallel subkernels.

Sometimes you’ll want RemoteEvaluate to start a fresh instance of Wolfram Engine whenever you do an evaluation. But with WSTPServer you can also have it use a persistent Wolfram Language session. RemoteEvaluate and WSTPServer are the beginning of a general symbolic framework for representing running Wolfram Engine processes. Version 12.2 already has RemoteKernelObject and $DefaultRemoteKernel which provide symbolic ways to represent remote Wolfram Language instances.

Big Computations? Send Them to a Cloud Provider! (December 2020)

It comes up quite frequently for me—especially given our Physics Project. I’ve got a big computation I’d like to do, but I don’t want to (or can’t) do it on my computer. And instead what I’d like to do is run it as a batch job in the cloud.

This has been possible in principle for as long as cloud computation providers have been around. But it’s been very involved and difficult. Well, now, in Version 12.2 it’s finally easy. Given any piece of Wolfram Language code, you can just use RemoteBatchSubmit to send it to be run as a batch job in the cloud.

There’s a little bit of setup required on the batch computation provider side. First, you have to have an account with an appropriate provider—and initially we’re supporting AWS Batch and Charity Engine. Then you have to configure things with that provider (and we’ve got workflows that describe how to do that). But as soon as that’s done, you’ll get a remote batch submission environment that’s basically all you need to start submitting batch jobs:

&#10005 env = RemoteBatchSubmissionEnvironment[ "AWSBatch", <|"JobQueue" -> "arn:aws:batch:us-east-1:123456789012:job-queue/MyQueue", "JobDefinition" -> "arn:aws:batch:us-east-1:123456789012:job-definition/MyDefinition:\ 1", "IOBucket" -> "my-job-bucket"|>]

OK, so what would be involved, say, in submitting a neural net training? Here’s how I would run it locally on my machine (and, yes, this is a very simple example):

&#10005 NetTrain[NetModel["LeNet"], "MNIST"]

And here’s the minimal way I would send it to run on AWS Batch:

&#10005 job = RemoteBatchSubmit[env, NetTrain[NetModel["LeNet"], "MNIST"]]

I get back an object that represents my remote batch job—that I can query to find out what’s happened with my job. At first it’ll just tell me that my job is “runnable”:

&#10005 job["JobStatus"]

Later on, it’ll say that it’s “starting”, then “running”, then (if all goes well) “succeeded”. And once the job is finished, you can get back the result like this:

&#10005 job["EvaluationResult"]

There’s lots of detail you can retrieve about what actually happened. Like here’s the beginning of the raw job log:

&#10005 job["JobLog"]

But the real point of running your computations remotely in a cloud is that they can potentially be bigger and crunchier than the ones you can run on your own machines. Here’s how we could run the same computation as above, but now requesting the use of a GPU:

&#10005 RemoteBatchSubmit[env, NetTrain[NetModel["LeNet"], "MNIST", TargetDevice -> "GPU"], RemoteProviderSettings -> <|"GPUCount" -> 1|>]

RemoteBatchSubmit can also handle parallel computations. If you request a multicore machine, you can immediately run ParallelMap etc. across its cores. But you can go even further with RemoteBatchMapSubmit—which automatically distributes your computation across a whole collection of separate machines in the cloud.

Here’s an example:

&#10005 job = RemoteBatchMapSubmit[env, ImageIdentify, WebImageSearch["happy", 100]]

While it’s running, we can get a dynamic display of the status of each part of the job:

&#10005 job["DynamicStatusVisualization"]

About 5 minutes later, the job is finished:

&#10005 job["JobStatus"]

And here are our results:

&#10005 ReverseSort[Counts[job["EvaluationResults"]]]

RemoteBatchSubmit and RemoteBatchMapSubmit give you high-level access to cloud compute services for general batch computation. But in Version 12.2 there is also a direct lower-level interface available, for example for AWS.

Connect to AWS:

&#10005 aws = ServiceConnect["AWS"]

Once you’ve authenticated, you can see all the services that are available:

&#10005 aws["Services"]

This gives a handle to the Amazon Translate service:

&#10005 aws["GetService", "Name" -> "Translate"]

Now you can use this to call the service:

&#10005 %["TranslateText", "Text" -> "今日は良い一日だった", "SourceLanguageCode" -> "auto", "TargetLanguageCode" -> "en" ]

Of course, you can always do language translation directly through the Wolfram Language too:

&#10005 TextTranslation["今日は良い一日だった"]

Shell, Java, …: New Built-in External Connections (May 2021)

Over the past several versions we’ve added support for direct interactions with a sequence of external languages, both programmatically through functions like ExternalEvaluate, and as parts of notebooks. Version 12.3 adds support for several additional languages.

First, there’s the shell. Back in Version 1.0, there was the notion of “shell escapes”: type ! at the beginning of a line, and everything after it would be sent to your operating system shell. A third of a century later, it’s a bit more polished and sophisticated, though it’s the same basic idea.

Type > in a notebook, and select Shell, then type your shell command:

&#10005 date

The stdout from the shell will be echoed as it’s generated, and then what comes back will be a symbolic object—from which it’s possible to extract things like exit code, or stdout:

&#10005 %["StandardOutput"]

In earlier versions, we added capabilities for languages like Python, Julia, R, etc., as well as SQL. In this version, we’re also adding support for Octave (yes, the function names are not great):

&#10005 inv(1+eye(4))

But the important point here is that data structures have been connected so that an Octave array comes back as an appropriate expression, in this case a list of lists (containing approximate numbers, because that’s all Octave handles).

By the way, although external language cells in notebooks are nice, you definitely don’t have to use them, and you can use ExternalEvaluate—or ExternalFunction—to do things purely programmatically.

We’ve had tight integration with Java in Wolfram Language through J/Link for more than 20 years. But in Version 12.3 we’ve set things up so that instead of using J/Link’s sophisticated symbolic interface to Java, you can just enter Java code directly in ExternalEvaluate and external language cells:

&#10005 System.getProperty("java.version")

Basic Java data structures are returned as standard Wolfram Language expressions:

&#10005 new int [20]

Java objects are represented symbolically through J/Link:

&#10005 new Object()

Everything interacts seamlessly with J/Link. And for example, you can create Java objects directly using J/Link—that you can subsequently use with Java you enter in an external language cell:

&#10005 Needs["JLink`"]; fmt = JavaNew["java.text.DecimalFormat", "#.0000"]

If you define a Java function it gets represented symbolically as an ExternalFunction object:

&#10005 import java.text.DecimalFormat; String[] formatArray(double[] d, DecimalFormat fmt) { String[] result = new String[d.length]; for (int i = 0; i < d.length; i++) result[i] = fmt.format(d[i]); return result; }

This particular function takes a list of numbers, and a Java object—of the kind we created above with J/Link:

&#10005 %[{1, 2, 3}, fmt]

(Yes, this particular operation is extremely easy to do directly in Wolfram Language.)