On Unity 5 game engine, heightmaps, Photoshop, ImageMagick, Linux, etc.

If one is using the Unity 5 computer game engine to create a simulacrum of space there is a bit of a puzzle when creating the Terrain upon which one’s character avatar may wish to perambulate.

The Terrain can be created by Importing a special type of black-and-white picture Map as viewed from above the ground, whereupon the brightnesss of the gray corresponds to the Altitude of a piece of terrain.

The format that one will come across when googling this idea is referred to as the “RAW” standard. But, in truth, Raw is not standard at all – there are a multitude of Raw formats. In fact that is what makes one Raw – there is no identifying information attached to the data to determine it’s contents nor it’s format. Raw indeed.

But through some futzing around one can discover some helpful details. When the Unity 5 engine says that it wishes to have a “RAW” format image as input for a Heightmap, it is actually referring to the RAW format as output by the Photoshop computer image manipulation program, published by the Adobe corporation of California. The reader may be surprised to note that the Adobe company does not, in fact, have any business related to the manufacture or construction of dwellings of mud. But they are kind enough to describe their RAW format here on their website.

https://helpx.adobe.com/photoshop/using/file-formats.html

RAW in this case simply means a sequence of pixels. Raw pixels. No header, no footer, no description, no nothing.

How does that work? Let’s say you have an image of 3 pixels by 3 pixels.

white white white
black black black
white black white

xx

That is 9 pixels total.

How does that get translated to numbers?

Well we could do 0 for black, 1 for white

1 1 1
0 0 0
1 0 1

That’s cool. That’s actualy… 1 bit per pixel. That’s the type of thing you might see on an e-ink display, where there is only Black or White, no gray in between.

But of course, in Unity game engine Heightmaps we are probably wanting to have variations in height. So we go grayscale.

white white white
black black black
white gray gray

xx

Great! How would that look in numbers? Well…. we could say that now White is 2, Black is 0, and Gray is 1 – putting Gray halfway between Black and White.

2 2 2
0 0 0
2 1 1

Great! Now we have 2-bit color. Why? 00 = binary ‘0’, 01 = binary ‘1’, 10 = binary ‘2’, 11 = binary ‘3’. We only need two bits to represent all our shades of gray here. In binary the numbers look like this:

10  10  10
00  00  00
10  01  01

Now…. what would that look like in a RAW file?

It would simply be a sequence of numbers. One row after another, just listed all in order, left to right top to bottom. In decimal:

2 2 2 0 0 0 2 1 1

In binary:

10 10 10 00 00 00 10 01 01

Ok. But surely our terrain has more than three levels? Yes. Indeed.

In fact, the Unity engine will need 8 bits for a heightmap. That is 256 different shades of color, black white and 254 shades in between. Nice. But how can we test this?

Here is a little C program.

#include <stdio.h>
#include <stdint.h>

main() {
        FILE *f = fopen("image.raw","w+");
        uint8_t data;
        for (int i=0;i<128;i++) {
                for (int j=0;j<128;j++) {
                        data = i + j;
                        printf("%i ",data);
                        fwrite(&data,1,1,f);
                }
                printf("\n");
        }
}


What does it do?

It makes a file called ‘image.raw’, which is just a 128 by 128 pixel, 8-bit per pixel, Photoshop-style RAW image. You can compile it and run it on any machine with a decent C compiler, although doing so is a bit out of scope of this document, but “Hello World C Windows” or “Hello World C OSX” should be good starting searches to do that.

But what are the values of those pixels the C program is writing to the file? Ahh… this uses a nice simple formula, at pixel 0,0 the value will be 0+0, or 0. At pixel 10,5 it will be 10+5, or 15. At pixel 127,127 it will be 127+127, or 254. All in all there will be 128 x 128 pixels (16 384), ranging in value from 0 to 254, roughly in a nice slope.  The program also prints out the numbers that it’s writing to the file so you can kind of get a feel for what it’s doing. After running there should be a ‘image.raw’ file of 16384 bytes:

don@serebryanya:/tmp$ ls -l image.raw
-rw-rw-r-- 1 don don 16384 May 13 21:08 image.raw

OK. How to import to Unity 5? Start Unity. Start a new project. Start a new scene. “Create” a “Terrain”. Now click on the Terrain. Now over on the “Inspector” side, there is a little ‘sprocket’. Click that.Now down at the very bottom, scroll down, there is an ‘Import RAW’. Click it. Now choose the ‘image.raw’ file generated.

You should get this little window. Note it has auto-detected the width and height and depth at least. (If it can’t… often it will just show ‘1’ for the width and height). Byte Order won’t matter, since each pixel is one byte (more on this later in this article!):

import8bit

Sure enough, import it into Unity 5 engine using the RAW format and you get this:

flatty1

It’s a bit tough to see here but this is a flat plane, that is at a bit of an angle from the ‘grid plane’ of the world. But it is a nice, basic, working Import of a Photo-shop style .RAW heightmap that did not use Photoshop at all. Just a simple C program to spit out bytes.

For a better look we can alter the C file a bit to give it more of a ‘curvy’ shape, using the good old “Square” of the x coordinate. (0,0 => 0,   5,0 => 25   10,0 => 100, etc… but scaled so they fit under the value of 250) and try again:

#include <stdio.h>
#include <stdint.h>

main() {
        FILE *f = fopen("image.raw","w+");
        uint8_t data;
        for (int i=0;i<128;i++) {
                for (int j=0;j<128;j++) {
                        data = (float(i*i) / float(128*128))*250;
                        printf("%i ",data);
                        fwrite(&data,1,1,f);
                }
                printf("\n");
        }
}

curvy1

There. That’s a nice curvy shape heightmap imported from byte data generated by a simple C program.

That is the beauty and tragedy of RAW. It’s got no header and no formatting info like a PNG file or JPG file so it’s not quite clear what might be in it – and the confusing name as been mixed around with various other image data patterns from cameras and so on.

To add to this, doing the standard “ImageMagick Convert” in Linux will not work in the obvious manner. For example – take a PNG file, use convert to change it to a ‘raw’ suffix file…. you will have the following result:

don@serebryanya:/tmp$ convert x.png x.raw
don@serebryanya:/tmp$ file x.png 
x.png: PNG image data, 128 x 128, 8-bit/color RGBA, non-interlaced
don@serebryanya:/tmp$ file x.raw 
x.raw: PNG image data, 128 x 128, 8-bit grayscale, non-interlaced
don@serebryanya:/tmp$ ls -l x.png
-rw-rw-r-- 1 don don 1842 May 13 17:40 x.png
don@serebryanya:/tmp$ ls -l x.raw
-rw-rw-r-- 1 don don 1192 May 13 21:40 x.raw
don@serebryanya:/tmp$ hexdump -C x.raw | head -5
00000000  89 50 4e 47 0d 0a 1a 0a  00 00 00 0d 49 48 44 52  |.PNG........IHDR|
00000010  00 00 00 80 00 00 00 80  08 00 00 00 00 e6 55 3e  |..............U>|
00000020  17 00 00 00 04 67 41 4d  41 00 00 b1 8f 0b fc 61  |.....gAMA......a|
00000030  05 00 00 00 01 73 52 47  42 00 ae ce 1c e9 00 00  |.....sRGB.......|
00000040  00 20 63 48 52 4d 00 00  7a 26 00 00 80 84 00 00  |. cHRM..z&......|

See? That’s not a RAW file. That’s a PNG file. ImageMagick’s “.raw” type is not anything like the Photoshop style “RAW”. A bit of a conundrum.

However we are not completely without hope. What you can do is web search for “Headerless Imagemagick”. Those are the magic phrases to use. And you find this:

http://www.imagemagick.org/discourse-server/viewtopic.php?t=22755

You can use the ‘.gray’ suffix and get a Photoshop-ish raw file.

don@serebryanya:/tmp$ convert x.png x.gray
don@serebryanya:/tmp$ ls -l x.gray
-rw-rw-r-- 1 don don 16384 May 13 21:47 x.gray
don@serebryanya:/tmp$ hexdump -C x.gray | head -3
00000000  37 37 37 37 37 37 37 37  37 37 37 37 37 37 37 37  |7777777777777777|
00000020  37 37 37 37 37 37 37 37  37 00 00 00 00 00 00 00  |777777777.......|

Much better! No header… it’s the proper size (128 x 128 bytes)

But what about this 256 level thing? Surely we want mountains with more than 256 levels of elevation? That seems rather blocky…. and even Minecraft can do more than 256 right?

Unity can do 16-bit heightmaps as well. Two bytes! That is all the numbers from 0 to 65536. That’s quite a bit more than 256 different levels of elevation. 65536 shades of gray, for 65536 levels of elevation! Not bad. If we were doing Mount Everest, who is about 30,000 feet, we’d be able to describe the mountain down to roughly half-a-foot increments. Not too shabby.    (Metric…? uhm… lets say 64k levels for roughly an 8k meters mountain, so roughly 1/8th of a meter per level! )

But that is where the Byte Order thing comes in. Perhaps you know, and you can skip to the end here…. but in case you don’t…. let’s go. I like to remember this, as it still amazes me after all these years. You see computers don’t tend to store 16-bit numbers, they tend to store bytes, which are 8 bit numbers. And there are two ways of writing a 16 bit number with two bytes. Say you have two bytes – byte A and byte B.  Together they make a 16 bit number. Which should come first when the machine builds the 16 bit number in it’s electronic brain?

Byte A - 00000111
Byte B - 10100001

00000111   10100001      < byte A is on the left

10100001   00000111      < byte A is on the right

How does the computer know which one to pick first out of the file, when it needs to build a 16 bit number? If it picks the wrong one, you get the wrong 16 bit number! This is called Endianness or Byte Order or other things. It’s not hard to find more info, here are some nice examples from Wikipedia, Apple, and Adobe Forums:

http://en.wikipedia.org/wiki/Endianness

https://developer.apple.com/library/mac/documentation/CoreFoundation/Conceptual/CFMemoryMgmt/Concepts/ByteOrdering.html

https://forums.adobe.com/message/2609795

The “Mac” vs “PC” is kind of confusing, because byte order is more linked to the hardware processor chip, like the Intel or AMD CPU you are using, not the operating system necessarily. PowerPC was the chips Macs used to use… and PowerPC was ‘big endian’. But nowdays, in the mid 2010s, Mac and PC are both Intel/AMD-style chips… which are little endian. So its all kind of a confusing muddle. To me anyways.

But the basic important thing for the Unity game engine, and importing Heightmaps, is that Heightmap from Photoshop-style RAW images with 16-bit pixels can lists their bytes in a different order and sometimes this can confuse Unity.

What happens with our C program in 16 bits mode? Let’s try it.

#include <stdio.h>
#include <stdint.h>

main() {
        FILE *f = fopen("image.raw","w+");
        uint16_t data;
        for (int i=0;i<128;i++) {
                for (int j=0;j<128;j++) {
                        data = (float(i*i) / float(128*128))*60000;
                        printf("%i ",data);
                        fwrite(&data,2,1,f);
                }
                printf("\n");
        }
}



It’s still the same nice little curvy shsape. The changes are that ‘data’ is now 16 bits. And… the scaling factor is to 60,000 not 250 like before. Remember that 16 bits maxes out at 65536 so this should be pretty good as a test.

But now… when we import into Unity, we need to care about the Byte Order…

Because when we use Mac, we get this:

mac

But if we use Windows byte order we get this:

windows

What is going on?

Lets look at a hexdump of some of the data

00005f00  1a 81 1a 81 1a 81 1a 81  1a 81 1a 81 1a 81 1a 81

We have two bytes, lets call them A and B. A is 1a hex, or 26 decimal. Byte B is 81 hex or 129 decimal. When we read them in different order, we get a different 16 bit number. Here’s a little Python computer language snippet that shows this:

>> 0x1a
26
>>> 0x001a
26
>>> 0x0081
129
>>> 0x8100
33024
>>> 0x1a00
6656
>>> 0x811a
33050
>>> 0x1a81
6785

See…. if you read B before A, from the .raw image file, you wind up with a Grayscale value of 33060. But if you read A before B you get a grayscale value of 6785. That’s quite a big difference. If you think of it in terms of the Elevation for your Heightmap Terrain, that’s roughly a 27,000 unit difference. On our old Mt Everest example from above, we are off by a bit less than half the mountain! ! !

So what can we do then?

It’s kind of simple really…we just need to try both types of byte-order on the Unity import window. If Mac doesn’t work, try Windows. If Windows doesn’t work, try Mac. There are only two options!

But… what about things ont he web about Interleaving? And numerous other problems with heightmap images? What about people without Photoshop? What about Linux users? How to make photoshoppy heightmaps then? Do we have to write C programs? No!

ImageMagick can probably do it.

ImageMagick can even do it in 16 bits. Here is an example. (and check out http://www.imagemagick.org for info on downloading and installing.)

Let’s say we had a standard Grayscale Heightmap-ish PNG file, like, I dunno…. this Crater on the planet Mars from the HiRISE project of the Mars Reconaissance Orbiter mission? (see my previous blog post if you wish, here ).

x

(Credit to JPL/NASA/University of Arizona/US Geological Survey, see http://www.uahirise.org/dtm/ for the HiRISE camera digital terrain models!)

Well..  first we Crop the image to a nice Square shape, which is what the Unity game engine likes. Also do it in  an image editor – any old image editor that allows cropping and easy resizing/saving of image files. (I used Kolourpaint). Now… we do a little ImageMagick Magic….. with the -depth 16 command to make it 16 bits…. and…

don@serebryanya:/tmp$ file x.png 
x.png: PNG image data, 512 x 512, 8-bit/color RGBA, non-interlaced
don@serebryanya:/tmp$ convert x.png -depth 16 x.gray
don@serebryanya:/tmp$ mv x.gray x.raw


(Now, the image input is only 8 bits, but it could be 16 or 32 or 24 or whatever. In theory, convert will guarantee us a 16 bit output file. Theoretically)

Now…. we can import our newly generated 16 bit raw image into Unity…. and change our Byte Order from “Mac” to “Windows…”

winnn

marsss

Tada!!!!!

Ok… so the height is a bit exaggerated. Probably not too many hills on mars with a 80% grade. However with a bit of scaling and detective work it could be made even more accurate to the real thing. I think it looks cool IMHO.

So. To sum up.

1. Unity 5 Game Engine Heightmaps use the “Photoshopy” RAW image format, which is not a standard at all.

2. You can convert PNG, JPEG, etc images to 16 bit grayscale in a Photoshop style RAW heightmap format using the ImageMagick conversion program named “convert”. You need to put the suffix ‘.gray’ on your output file and the ‘-depth 16’ flag on the command.

3. You may have to play with the Byte Order on the Unity import window…. and…

4. There are many other issues involved of course in making a good usable terrain. Other sites have much better articles on the other details, like making sure your heightmap width and height are less than your terrain width and height!

Hopefully this is a somewhat decent basic introduction to the basic ideas, though, of Unity Game Engine’s Heightmap import format.

Good luck and thanks for reading.

http://github.com/donbright

Advertisements

About donbright

https://patreon.com/donbright https://github.com/donbright http://dwitter.net/u/donbright
This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to On Unity 5 game engine, heightmaps, Photoshop, ImageMagick, Linux, etc.

  1. Thanks for this tutorial. Extremely helpful and informative.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s