Recent versions of Mathematica introduced useful data functions related to linguistics. In Version 7, Mathematica‘s integrated string manipulation and visualization functions provide a powerful platform for computational linguistics. Both DictionaryLookup and WordData give programmatic access to curated data that’s ready for computation.

DictionaryLookup has been extended to cover more languages, while WordData has information on word meaning, structure, and usage, as well as cognitive and grammatical relationships between words. Let’s look at a range of examples, starting with some interesting and amusing word facts.

DictionaryLookup currently contains word lists for 27 different languages:

The longest word in all its 27 languages is the Finnish term for a fancy four-color press:

And we can plot the distribution of English words starting with each letter:

These are, for example, all of DictionaryLookups‘s palindromes that are both English and French words:

Remember that some words in WordData are considered too obscure for DictionaryLookup, and that unlike DictionaryLookup, WordData is English-only.

By using both WordData and DictionaryLookup we can find some hidden word facts, such as the English words with the most syllables:

or find that the word “break” has 76 definitions in WordData and DictionaryLookup and that the word “set” has 44 definitions in WordData:

This is the distribution of the number of words in WordData with a given number of meanings:

and these are the words in WordData that have more than 50 different possible meanings:

These words occur as the maximum number of different parts of speech:

We can find 692 antonym pairs in English where both words have the same length:

Here is a random selection out of the 692:

We can learn that “underground” and “underfund” are the English words in DictionaryLookup that begin and end with the letters “und”:

and that there are only five English words in there that end in “-dous”:

and that these are its only words in English that begin and end with the same four letters:

With a slight change we can do the same for any of the other 26 languages in DictionaryLookup. In Spanish there are, like in English, nine words that begin and end with the same four letters:

“Therein” contains twelve words without rearranging any of its letters:

The same word can form 24 words if we allow letters to be skipped but not reordered:

Only the word “neither” can be formed by rearranging “therein”:

and there are four English words that can be formed using only vowels:

There are several more in the French language, though:

For query methods, we can use string expressions, regular expressions, or a combination of both. For example, this is a way to get the longest English word with a repeated vowel by using a regular expression and a pure function:

The only DictionaryLookup English words with three consecutive double letters are those containing “bookkeep,” found using a string expression:

and there are six English words in it with four different consecutive vowels:

We can ask whether the word “queue” is pronounced the same way when the last four letters are removed (this says it is):

There are 862 sets of words that have the same pronunciation. A total of 1796 English words are pronounced exactly the same as at least one other word:

and at least 68 sets with more than two words with the very same pronunciation:

Ed Pegg Jr has written a Rhyme Finder Demonstration using Mathematica‘s WordData:

We can keep on asking DictionaryLookup questions. These are the only words in it with a double consecutive “u” in English:

and there are at least 20 English words with no vowels:

with “rhythms” being the longest:

We can take advantage of Mathematica‘s visualization capabilities and plot a vowel distribution table of those languages in DictionaryLookup that use the Latin alphabet, and find that among these languages, vowel distribution is very similar—in most, “a” is more common than “e”, “e” more common than “i”, and so on, with Latin having the most equal distribution:

By using ExampleData we can find that word statistics vary not only by language, but by whether the words are chosen from a dictionary or from written text.

We can verify Zipf’s law as applied to word and letter frequencies.

And by using WordData, we can find the synonyms of a word, the synonyms of those synonyms, et cetera. Plotting the connections between all those synonyms produces a 3D semantic network. Here are all the words related to “computer” out to four degrees of separation.

It is amazing how many experiments one can imagine and easily perform by using these built-in data functions. But even more exciting are the types and numbers of problems that can be addressed in fields like computational linguistics and natural language processing with Mathematica.

Download this notebook