A couple of weeks ago I shared a package for controlling the Raspberry Pi version of Minecraft from Mathematica (either on the Pi or from another computer). You can control the Minecraft API from lots of languages, but the Wolfram Language is very well aligned to this task—both because the rich, literate, multiparadigm style of the language makes it great for learning coding, and because its high-level data and computation features let you get exciting results very quickly.

Today, I wanted to share four fun Minecraft project ideas that I had, together with simple code for achieving them. There are also some ideas for taking the projects further.

If you haven’t already installed the MinecraftLink package, follow the instructions here.

Rendering a Photo in Minecraft

The Minecraft world is made up of blocks of different colors and textures. If we arrange these appropriately, we can use the colors to create grainy pictures. I want to automate this process of converting pictures to Minecraft blocks.

The first step is to start a new world in Minecraft on the Raspberry Pi, then load the MinecraftLink package:

&#10005 <<MinecraftLink`

If you are using Mathematica on a different computer than the Pi (as I am), you need to connect the two together using the IP address or the name of the Raspberry Pi (your address will be different than mine, shown here). If you are running the code on the Pi, you can skip this step:

&#10005 MinecraftConnect["10.10.163.22"]

The MinecraftLink package automatically installs some MinecraftBlock Entity data from the Wolfram Data Repository.

Some of those entities include images that I can analyze to figure out the block’s average color. First, I need to select all the entities that have images available. But I found out the hard way that we have to remove a few of those blocks for various reasons: the transparency of blocks like glass and cobweb just look bad, and some blocks must be removed because of the game physics of Minecraft. Soft blocks like sand fall off the picture, fire only exists on top of certain blocks and water spreads all over the picture, so those are all removed from the list.

&#10005 EntityList[Entity["MinecraftBlock", "Image" -> ImageQ]]

&#10005 available=Complement[EntityList[Entity["MinecraftBlock", "Image" -> ImageQ]],{Entity["MinecraftBlock", "Glass"],Entity["MinecraftBlock", "Leaves"],Entity["MinecraftBlock", "Cobweb"],Entity["MinecraftBlock", "Sand"],Entity["MinecraftBlock", "Gravel"],Entity["MinecraftBlock", "Snow"],Entity["MinecraftBlock", "Fire"],Entity["MinecraftBlock", "WaterStationary"]}]

Here are the images that we have:

&#10005 Magnify[{#["Image"],#}&/@available,0.6]

Most blocks (subject to lighting) are the same on all faces, but a few have different textures on their side faces than their top faces. I plan to look at all blocks from the side, so I want to figure out what the blocks’ average side-face color is. To do this, I created the following mask for the position of the side-face pixels of the gold block:

&#10005 mask = Erosion[ DominantColors[CloudGet["https://wolfr.am/xJ2pPzQS"], 4, "CoverageImage"][[2]], 2]

Because all the images have the same shape and viewpoint, I can apply that mask to every block to pick out their front-face pixels:

&#10005 mask RemoveAlphaChannel[Entity["MinecraftBlock","WoodBirch"]["Image"]]

To make sure I am using a like-for-like measurement, I remove the transparency layer (AlphaChannel) and put them all into the same color space. Then I just ask for the average pixel value and convert that back to an average color (working in HSB color gives more perceptually correct averaging of colors):

&#10005 averageColor[block_]:=Hue[ImageMeasurements[ColorConvert[RemoveAlphaChannel[block["Image"],LightBlue],"HSB"],"Mean",Masking->mask]]

&#10005 colors=Map[averageColor,available]

We can now look at our available palette:

&#10005 ChromaticityPlot[colors]

You can see from this plot of colors in the visible color space that we have rather poor coverage of high-saturation colors, and something of a gap around the green/cyan border, but this is what we have to work with.

Now we need a function that takes a color and picks out the block name that is nearest in color. The Wolfram Language already knows about perceptual color distance, so Nearest handles this directly:

&#10005 getName[color_]:=First[Nearest[MapThread[Rule,{colors,available}],color]];

For example, the block that is closest to pure red is wool orange:

&#10005 getName[Red]

Now we need a function that will take a picture and drop its resolution to make it more “blocky” and simplify the image to use only the colors that are available to us:

&#10005 toBlockColors[img_,size_]:=ColorQuantize[ImageResize[img,size],colors];

Let’s now apply that to a well-known picture:

&#10005 toBlockColors[CloudGet["https://wolfr.am/xJ2fHJWp"], 50]

Now we just have to count through the pixels of that image, find the name of the block with the nearest color to the pixel and place it in the corresponding place in the Minecraft world:

&#10005 putPicture[{x0_,y0_,z0_},img_]:= Block[{dims=ImageDimensions[img]}, Do[ MinecraftSetBlock[{dims[[1]]-x+x0,y+y0,z0},getName[RGBColor[ImageValue[img,{x,y}]]]], {x,dims[[1]]},{y,dims[[2]]}]];

Find a big open space…

And run the program on a simple image:

&#10005 putPicture[{30, 0, 0}, toBlockColors[CloudGet["https://wolfr.am/xJ2fHJWp"], 50]]

You can use Import to bring images into the system, but the Wolfram Language provides lots of images as part of its Entity system. For example, you can fetch famous works of art:

&#10005 Entity["Artwork", "AmericanGothic::GrantWood"]["Image"]

Here is a detail from American Gothic (Grant Wood’s sister) in blocks:

&#10005 putPicture[{30,0,0},toBlockColors[ImageTake[Entity["Artwork", "AmericanGothic::GrantWood"]["Image"],{25,75},{10,50}],50]]

You can even, at an incredibly low frame rate, make an outdoor movie theater by streaming frames from your webcam onto the wall. Here is me working on this blog post!

&#10005 While[True,putPicture[{30,0,0},toBlockColors[CurrentImage[],50]]]

An interesting challenge would be to reverse this process to generate a map of the Minecraft world. You would need to scan the surface block type at every position in the Minecraft world and use the color map created to color a single pixel of the output map.

Recreating the Real World in Minecraft

This project sounds quite hard, but thanks to the built-in data in the Wolfram Language, it is actually very simple.

Let’s suppose I want to create my home country of the United Kingdom in Minecraft. All I need to do is place a grid of blocks at heights that correspond to heights of the land in the United Kingdom. We can get that data from the Wolfram Language with GeoElevationData:

&#10005 ListPlot3D[Reverse/@GeoElevationData[Entity["Country", "UnitedKingdom"]],PlotRange->{-10000,20000},Mesh->False]

You will see that the data includes underwater values, so we will need to handle those differently to make the shape recognizable. Also, we don’t need anywhere near as much resolution (GeoElevationData can go to a resolution of a few meters in some places). We need something more like this:

&#10005 ListPlot3D[Reverse/@GeoElevationData[Entity["Country", "UnitedKingdom"],GeoZoomLevel->3],PlotRange->{0,5000},Mesh->False]

Now let’s make that into blocks. Let’s assume I will choose the minimum and maximum heights of our output. For any given position, I need to create a column of blocks. If the height is positive, this should be solid blocks up to the height, and air above. If the height is negative, then it is solid up to the point, water above that until we reach a given sea level value and then air above that.

&#10005 createMapColumn[{x_,y_,z_},seaLevel_,min_,max_]:=(MinecraftSetBlock[{{x,min,z},{x,y,z}},"Dirt"];If[y>=seaLevel,MinecraftSetBlock[{{x,y,z},{x,max,z}},Entity["MinecraftBlock", "Air"]], MinecraftSetBlock[{{x,y,z},{x,seaLevel-1,z}},Entity["MinecraftBlock", "WaterStationary"]];MinecraftSetBlock[{{x,seaLevel,z},{x,max,z}},"Air"]]);

Now we just need to create a column for each position in our elevation data.

All the work is in transforming the numbers. The reversing and transposing is to get the coordinates to line up with the compass correctly, QuantityMagnitude gets rid of units, and the rest is vertical scaling:

&#10005 MinecraftElevationPlot[data0_,{x0_,seaLevel_,z0_}, maxHeight_:5]:= Block[{data=QuantityMagnitude[Reverse[Map[Reverse,Transpose[data0]]]],scale, min,dims}, dims=Dimensions[data]; scale= maxHeight/Max[Flatten[data]]; min= Round[scale*Min[Flatten[data]]]; Do[createMapColumn[{Round[x0+i],Floor[scale data[[i,j]]+seaLevel],z0+j},Round[seaLevel], seaLevel+min,Round[maxHeight+seaLevel]],{i,dims[[1]]},{j,dims[[2]]}]]

Before we start, we can use the following code to clear a large, flat area to work on and put the camera high in the air above the action:

&#10005 MinecraftSetBlock[{{-40,-10,-40},{40,0,40}},"Grass"]; MinecraftSetBlock[{{-40,0,-40},{40,50,40}},"Air"];

&#10005 MinecraftSetCamera["Fixed"]; MinecraftSetCamera[{0,25,0}]

And now we can place the map:

&#10005 MinecraftElevationPlot[GeoElevationData[Entity["Country", "UnitedKingdom"],GeoZoomLevel->2],{-15,0,-15},5]

You can just see that the land is higher in mountainous Scotland. You can see that better with the camera in the usual position, but the coastline becomes harder to see.

Alternatively, here is the view of the north ridge of Mount Everest, as seen from the summit:

&#10005 MinecraftSetCamera["Normal"] MinecraftElevationPlot[GeoElevationData[GeoDisk[Entity["Mountain", "MountEverest"],3 mi],GeoZoomLevel->9],{-15,-18,-15},30]

A nicer version of this might switch materials at different heights to give you snowcapped mountains, or sandy beaches. I will leave that for you to add.

Rendering a CT Scan in Minecraft

If you are unlucky enough to need an MRI or CT scan, then you might end up with 3D image data such as this built-in example:

&#10005 Show[ExampleData[{"TestImage3D","CThead"}],BoxRatios->1]

This might seem complex, but it is actually a simpler problem than the photo renderer we did earlier, because color isn’t very meaningful in the CT world. Our simplest approach is just to drop the scan’s resolution, and convert it into either air or blocks.

&#10005 Binarize[ImageResize[ExampleData[{"TestImage3D","CThead"}],{80,80,80}]]

We can easily find the coordinates of all the solid voxels, which we can use to place blocks in our world:

&#10005 Position[ImageData[%],1]

We can wrap all of that into a single function, and add in an initial position in the Minecraft world. I added the small pause because if you run this code from a desktop computer, it will flood the Minecraft server with more requests than it can handle, and Minecraft will drop some of the blocks:

&#10005 fixCoordinates[{a_,b_,c_}]:={c,-a,b} (*Mapping coordinate systems*)

&#10005 minecraftImage3D[img_Image3D,pos_List,block_,threshold_:Automatic]:=( MinecraftSetBlock[{pos,pos+ImageDimensions[img]},"Air"];Map[(Pause[0.01];MinecraftSetBlock[pos+#,block])&,fixCoordinates/@Position[ImageData[Binarize[img,threshold]],1]];)

And here it is in action with the head:

&#10005 minecraftImage3D[ ImageResize[ExampleData[{"TestImage3D","CThead"}],{40,40,40}],{0,40,0},"GoldBlock"]

But one thing to understand with 3D images is that there is information “inside” the image at every level, so if we change the threshold for binarizing then we can pick out just the denser bone material and make a skull:

&#10005 Binarize[ImageResize[ExampleData[{"TestImage3D","CThead"}],{80,80,80}],0.4]

&#10005 minecraftImage3D[ ImageResize[ExampleData[{"TestImage3D","CThead"}],{40,40,40}],{0,40,0},"GoldBlock",0.4]

An interesting extension would be to establish three levels of density and use the glass block type to put a transparent skin on the skull. I will leave that for you to do. You can find DICOM images on the web that can be imported into the Wolfram Language with Import, but beware—some of those can be quite large files.

Automatic Pyramid Building

The final project is about creating new game behavior. My idea is to create a special block combination that triggers an automatic action. Specifically, when you place a gold block on top of a glowstone block, a large pyramid will be built for you.

The first step is to scan the surface blocks around a specific point for gold and return a list of surface gold block positions found:

&#10005 scanForGold[{x0_,y0_,z0_}]:=Block[{goldPos={}, height = MinecraftGetHeight[{x,z}]}, Do[Pause[0.1];If[MinecraftGetBlock[{x,height-1,z}]===Entity["MinecraftBlock","GoldBlock"],AppendTo[goldPos,{x,height-1,z}]],{x,x0-1,x0+1},{z,z0-1,z0+1}]; goldPos];

Next, we look under each of the gold blocks that we found and see if any have glowstone under them:

&#10005 checkGoldForGlowstone[goldPos_]:=FirstCase[goldPos,{x_,y_,z_}/;MinecraftGetBlock[{x,y-1,z}]===Entity["MinecraftBlock","GlowstoneBlock"]]

Now we need a function that performs the resulting actions. It posts a message, removes the two special blocks and sets the pyramid:

&#10005 pyramidActions[found_]:=(MinecraftChat["Building Pyramid"]; MinecraftSetBlock[{found,found-{0,1,0}},"Air"]; MinecraftSetBlock[found-{0,1,0},"GoldBlock",Pyramid[],RasterSize->12]);

We can now put all of that together into one function that scans around the current player and runs the actions on the first matching location. The PreemptProtect is a bit subtle. Because I am going to run this as a background task, I need to make sure that I don’t perform two actions at once, as the messages going back and forth to the Minecraft server may get muddled:

&#10005 pyramidCheck[]:=PreemptProtect[Block[{found=checkGoldForGlowstone[scanForGold[MinecraftGetTile[]]]},If[Not[MissingQ[found]],pyramidActions[found]]]]

All that is left is to run this code repeatedly every five seconds:

&#10005 task=SessionSubmit[ScheduledTask[pyramidCheck[],5]]

I place the blocks like this…

… walk up within one block of the special column and wait for a couple of seconds, until this happens…

To stop the task, you can evaluate…

&#10005 TaskRemove[task]

Well, that’s the end of my short series on Minecraft coding in the Wolfram Language. There are lots of fun knowledge domains and computation areas that I could have injected into Minecraft. I had thought of using reading surface blocks and constructing a 3D mini-map of the world using the Wolfram Language’s data visualization, or creating a solar system of orbiting planets using the differential equation solver. I also considered creating a terrain generator using 3D cellular automata or fractals. But I do have a day job to do, so I will leave those for others to try. Do post your own project ideas or code on Wolfram Community.

---

Download this post as a Wolfram Notebook.