Dissecting Graphics For The Average Gamer (Part 2)

Camila's shoulder also shows the disadvantages of Gouraud shading.

3 Dimensional Flatness

You all know that a 3D virtual world is primarily composed of polygons. While each successive generation of consoles rendered more polygons on screen, there’s still a limit for every system. Let say there’s a ten meter road with hundreds of pebbles strewn about. Although they’re small, the amount of pebbles really add up. With the ever increasing need for graphical detail, those extra 500,000 polygons just won’t do. The polygons should be used on more important details than some small pebbles. All they could do in the 64/32-bit days was create a pebble texture. It’s still just a flat piece of paper. Nowadays, we have bump mapping. Artists can now create extra details without the expense of using extra polygons, and look far better than just a plain flat texture. You’re adding detail far beyond what the system’s capable of.

Bump mapping is a very general term that can actually refer to a number of different techniques. Let me go over a few popular effects and give you the basics of what they do. Again, we’re only scratching the surface of what they are, because that’s all I know. Some of them might be too hardware intensive for the Wii, but if developer managed to actually get it running on the Wii, some good old fashion “Oooooooh’s” and “Ahhhhhhh’s” are in ordered.

Bump Mapping – Like I said before, bump mapping is also a general term, but when I say bump mapping, I am referring to this technique. Bump mapping uses a grey scale height map (an image that stores the surface elevation values for 3D graphics) to calculate the brightness and color of each pixel, and how it responds to lights. A texture is place over it to give whatever it’s rendering to appear more realistic. When you compare a simple textured surface with that of a bump mapped surface, the results are much richer.

This is a height map without textures on it.

The GameCube is quite capable of bump mapping, and so was the Xbox. The PS2, not so much (more on this later). The smaller RAM/VRAM on the Cube might have contributed to the lack of bump mapping, but the Wii received quite a boost in RAM. Seems like it just comes down to effort and willingness of the developer.

Click on the images to enlarge and compare the snow in 3 different games. 2 with bump and 1 without.

Normal Mapping – This is probably the most popular right now out of all these techniques. Normal mapping is an enhanced form of bump mapping. It’s commonly used to greatly enhance the appearance of a low polygon model to look like a high polygon model. Unlike bump mapping where it uses a grey scale height map to store 3D data, a normal map is an RGB texture that can be created from various sources. I think the most common is ripped from a high resolution high detailed model, but artists can also render them from scratch, or rip them from height maps. Each texel from the normal map is used to compute lighting rather than the original vertex. The normal map will greatly increase the detail, complexity, and lighting of the original polygonal model. Whereas bump map applies pertusion on the normal (the polygon that you see), the normal map replaces the normal to give the mesh a higher polygon look hence the name normal mapping. Sometimes it’s not about creating bumps, but smooth surfaces to hide the sharp edges of the polygons.

This is a normal map without textures.

Here’s an easy way to visualize how normal mapping works: Imagine a really scrawny guy wearing a latex suit to make him look more muscular. But instead of latex, the suit’s muscle details are actually optical illusions created by light.

Both pictures are the same wall but from different angles. The left image is a frontal view. Notice the bumpiness on the bricks (the right side brick wall). The right image is from another angle, but the bricks are  flat.

Normal mapping can do everything bump mapping can do, and more. I can’t emphasis this enough, but normal mapping isn’t just used to create bumps. You can smooth out a polygonal model and completely hide any sharp edges. Because CG artists are a bunch of douche bags, they like to cause further confusion by calling this bump mapping, when normal mapping isn’t just use to create bumps.

Now for the mind-blowing part. Every console from Dreamcast and on are capable of normal mapping.

To back it up, I have the word of Simon F from the Beyond3d forums. He’s worked in the PowerVR group of Imagination Technologies (nee Videologic) for over 16 years and in other 3D related positions at other companies prior to that. I figured I’d get someone in the know to quote to prevent crazed fan boy outrage.

“Dreamcast had a normal map texture format and could do dot products with an incoming light direction to modify the shading of surfaces (you could even change the opacity if you wanted to).

The normal map vectors, however, were not stored in Cartesian coordinates but in a polar-ish form. At the time I was worried that, if we used Cartesian coordinates, the cost of renormalisation of the vectors in the texture (e.g. due to bilinear filtering) and of the per-vertex light vectors would be too high. I shouldn’t have worried since 1) re-normalisation can be done with relatively little hardware and 2) when other hardware came along that did normal mapping with Cartesian vectors I don’t think it did either.”

Link the Dreamcast patent here.

Did you get all that? Me neither.

The reason I quoted Simon is to dispel any myths about the power required for normal mapping. It isn’t the be all end all of 3D graphics technology, and it isn’t even “next-gen”. I figured it would be better to hear it from someone with real experience in the 3D graphics industry. Just because it wasn’t used doesn’t mean it’s not capable of it.

The first Xbox GPU had hardware support for normal maps. This made it easier for the console to use it. The Wii and PS2 most likely had to use the CPU to do software based normal mapping, and this eats up power from the CPU.

Environment Mapped Bump Mapping (EMBM) – In my opinion, EMBM is by far the most interesting when discussing a weaker hardware like the Wii. EMBM is just bump mapping with an extra coat of light. It can simulate multiple light sources shining on an object; apply environment mapping onto a bump map; fake specular lighting; and — my personal favorite — animate realistic looking water.

Before we go any further, let me clear up environment mapping first. Environment mapping is used to create reflections on polygonal surfaces. While it is possible to perfectly mirror something, it can also simply reflect light. This part is very important, since I made a mistake thinking environment map has to mirror the image, and not just reflect lighting. Many of these effects look very similar, and can be easily confused with one another. I thought DK: Jungle Beat was doing Phong reflections, but it was just environment mapping. Specular lighting can also be faked with EMBM, and vise-versa.

Now then, let’s get back to some of the effects possible thanks to EMBM. As I said before, it can simulate multiple light sources, and it can create bumps. Let’s turn our attention to the screen shot below.

This looks like EMBM, but I can't say for certain since you can do the same thing with normal maps and vice versa.

Look at the lighting on the alien gun. You see how there are different color lights reflecting off of it? That alone created a couple of illusions. It’s telling you there are multiple light sources within the scene, the relative position of the light from the gun, and the surface properties of the gun. Using that information, your mind automatically processes the image into something hard, metallic, and bumpy. Now look at the alien bug creature, and you’ll see the lighting is tracing the contour of the shell. It’s telling you which direction the light is coming from, and the surface properties on the shell. The little bumpy detail is created by the reflection and refraction of light. It creates an optical illusion based on how light bounces off of a hard reflective bumpy surface. What you don’t see is that the light moves along the surface of the creature/gun as it moves. If the light remains perfectly still, it would destroy the illusion. The funny thing is that there may or may not be a dynamic light shining on it. It’s faking so many elements with simple manipulation of lights and color.

If you need more convincing, check out Super Mario Galaxy. That game is the measuring stick for Wii graphics. EMBM is used to create various effects, some subtle, some more eye catching. Just look at the rolling boulders with the red gems. If you pay close attention to them, you’ll see a brief reflection of lighting moving across the surface of the rocky parts. It moves by very quickly, but help add more depth and detail to the boulder. If you think the boulder moves too fast to see it, why not look at the water?

Water is one of my personal favorite graphical effects. It’s inconceivable to think that water could be recreated using bump mapping, but that’s exactly how it’s done. With a combination of moving bump maps and a reflective environment maps, it is able to convincingly simulate the fluid motion of waves with just 4 vertices. What you’re seeing are various bumps moving across the surface while distorting the reflected image. In Mario Galaxy’s case, it’s reflecting light off of the surface of the water. You’ll also notice that you won’t see the reflected light until you’re at a certain angle/distance away from the water. This is a combination of a few more effects that I’ll address later, but it creates a very convincing look and the fluid dynamics of water.

The Gamecube/Wii has built in hardware support for EMBM. Sadly, we rarely got to see the beautiful reflective water that it’s capable of, or used on many surfaces to enhance the look. At least right now on the Wii, there are more games with EMBM rendering for water (especially from 3rd parties) compared to the Gamecube. There’s really no excuse not to have realistic water in the Gamecube, though.

See how the lighting is used to bring out the bumpy details? Seeing it in motion will better explain it.

Parallax Mapping – As we continue to move down the list, the effects get more and more advanced. We’ve finally moved onto something the Wii hasn’t seen before: Parallax Mapping.

While bump mapping adds pertusions, parallax mapping also adds depressions. The groves in brick walls will have extra depth thanks to motion parallax. What it does is distort the texture coordinates based on viewpoint to achieve motion parallax effects. As your viewpoint move, the higher pixels in the height map will move faster, while the lower pixels barely move at all creating motion parallax. The bumps themselves will also pop out more compared to other techniques. This gives the illusion of depth on a flat surface, whereas all the previous effects above only adds geometric detail. Even cooler is the fake bumps can conceal the view of other bumps like real geometry. The only down side is at grazing angles, the textures start to move up incorrectly. It’s very hard to notice, and must be done at grazing angles. There is another downside, but that’s where we have the next effect…

No Parallax Mapping

Parallax Mapping

Parallax mapping also has another use. With its ability to add depth to bumps, parallax maps can be used to create things like 3D bullet holes. Since it’s still very expensive to have the game actually alter the geometry, parallax mapping can give the illusion of a 3D bullet hole without extra polygons.

Parallax Occlusion Mapping – Probably the most advanced bump map on the list. Currently (2009), the only game I know that uses POM is Crysis. Not even the PS3/360 can do it — yet. Whoever is able to get this running in a game on the Wii is a witch and should be burned at the stake.

POM removes the texture problem at steep angle, and also provides occlusion (the absence of light). It is able to properly self-shadow and adds perspective correction to textures. The more precise lighting will solve the swimming problem mentioned before. I really didn’t get to see this in action very much. Crysis isn’t exactly the most friendly game to run on a weaker PC. Things are getting to the point where it’s harder and harder to break it down easily. At the very least, you should understand it can create accurate self-shadows on these fake 3D surfaces. It’s pretty tough to tell them apart from objects rendered with polygons.

This video demonstrates normal, parallax, and parallax occlusion mapping.

Displacement Mapping – I don’t know any game that supports it, or will support it. The reason I went beyond what the Wii’s theoretically limit is because of the crazy displacement mapping patent that Nintendo filed years ago. I never bothered to read it thoroughly because I hate reading patents. It’s such a pain. Whatever that’s there has nothing to do with the Wii. If Nintendo found a way to do displacement mapping their own way, good for them, but that doesn’t mean jack squat for the Wii. Even the PS3/360 can’t render them in realtime.

Displacement mapping uses a height map that actually alters the surface of the polygon. It literally deforms the vertex without adding new polygons to the mesh. It’s like taking clay and molding it into something. You can do a simple demo right now to understand displacement mapping. Take two fingers to pinch your cheeks (the one on your face!!!) and pull it. You’ve just displacement mapped your own face. Your face didn’t actually grow anything, it was just deformed due to the pulling. Your fingers acted like the texture used to displace the vertex on the mesh. The GPU will actually treat the displacement map like real geometry — providing accurate occlusion and self-shadowing. A height map is needed to calculate the new position of the vertex. I think extra hardware like a vertex shader or even the more advanced tessellator is required for this. If not, a very powerful CPU is needed. I can’t name a single game that uses displacement mapping, so just forget about the patent. Oh, and you can let go of your face now.

That about covers the more popular bump mapping effects for now. There are some minor effects I didn’t cover due to lack of knowledge, or they‘re just the same effect with a different name. If you ever see anything that you don’t understand, try googling it, and you might find that it’s something you already know; but with a different name.

Getting all the names correct might not be as important as being able to tell them apart. Remember when everybody thought Wii couldn’t handle normal mapping? Suppose they’re right, but what some might not know is that EMBM is an equally good substitute. EMBM could easily have been used in place of normal mapping, and people wouldn’t be able to tell the difference. Normal mapping is only more complex due to the lack of hardware support, whereas EMBM is built into the Wii GPU. I could be very wrong about the alien bug creature example from before. When I first started writing this article, The Conduit hasn’t advanced as far as it has today. Right now, I’m doubting whether the example is actually EMBM, but EMBM will look just as good. Unless you actually programmed the game, it’s very hard to tell the difference. I guess due to the hype, normal mapping is perceived to be superior by gamers.

Lights, camera, shadows!

Remember when I compared shading to shading hand-drawn art with your pencil? Imagine lighting as shining a flashlight onto your artwork. Shading (not shader) is a lighting model, while lighting is used to brighten a scene. Where there’s light, there’s shadow. Lighting speaks for itself, but being able to see and appreciate shadowing is tricky. Not only are these effects subtle, but they’re also hardware intensive. Don’t underestimate the power needed to render this black silhouette. It’s also much harder to help you visualize the differences like with the previous bump map section due to its complexity. You either need a high degree of mathematics or just good understanding of how light works in the real world. I don’t understand shadows in 3D graphics very well yet to go too in-depth about them, but I didn’t want to completely avoid talking about it since light and shadows are two sides of the same coin.

Pre-baked Lighting – Also called Light Maps, is lighting and shadows calculated before hand and painted onto the texture. Pre-baked lighting isn’t something rendered in realtime by the GPU. That means they have no affect on other objects; just on the surface it’s painted on. Let say there’s a fireplace, the surrounding walls will glow like it’s being lit by the fire, but if your character walks up to the fire, the light won’t affect your character. If that fire is extinguished, the wall will continue to glow. It’s not like every game will use interactive light, so it won’t be a problem. When used properly, it can improve the look of a game greatly.

Bloom Lighting – One of the most abused effects in recent years. Bloom produces intense light and an optical phenomenon know as “airy disc.” To go in-depth about airy disc would deviate too much into the field of lenses optics, but it’s not hard to see. Airy disc casts a soft blur around the object while the intense light also bleeds into fringe of other objects. If you still don’t understand, just take a look at the picture below.

Besides the bright lights, notice how there’s a fuzzy blur around Samus? That’s the result of bloom.

High Dynamic Range Rendering (HDR) – HDR is like a higher level of bloom. It produces very realistic lighting while preserving a wide range of colors and contrast. According to Nvidia: bright things can be really bright, dark things can be really dark, and details can be seen in both. The thought of seeing more details in dark areas sounds confusing, and I admit I still don’t truly “get” it. Luckily, it’s very easy to see and compare with the images below.

This image I found on wikipedia demonstrates HDR nicely. The brightness is obvious, but also take a close look at the darker areas and compare them. They look darker, and yet you can still see the details better than the non-HDR image.

HDR also computes reflections. When an intense light hits a surfaces, some of that light is also reflected. These reflections are barely noticeable with the older lighting techniques. With HDR, the intense light will also produce an intense reflection. Please don’t confuse this with the reflections I talked about in EMBM. The reflections you see in EMBM doesn’t necessarily mean there’s actually a light shining on the surface; it’s an illusion created with shiny textures. Light from HDR really is light hitting and reflecting off the surface.

The other thing HDR does is add eye adaptation. It’s a simulation of how the pupil copes with changing lighting conditions. There’s a delay in how well you see as you move from a bright place into a dark interior and vise-versa. While you won’t be able to see into a dark interior area from the outside, but your vision will slowly adapt as you go in. Similarly, when you look back out into a bright area, the brightness outside looks greatly intensified. You can barely see anything outside in the bright light, but your dark interior surroundings are quite visible. This is not a feature exclusive to HDR, but I did learn about this from reading about HDR.

I remember HDR being big back in the day. All video cards takes a huge performance hit with it enabled. Both the PS3 and 360 are quite capable of it, but it can be faked on the older systems. It’s been recently confirmed that the Wii is capable of HDR, but it remains to be seen whether they’re faking it or not. If HDR isn’t used carefully, it could end up making games look uglier as a result.

Pre-baked Shadows – Technically, pre-baked lighting already handles shadows as well, but I want to go a little deeper than just a black colors. Used properly, pre-baked shadows can look even better than realtime shadows. These shadows are pre-painted, extra effects like soft edges can be added to increase the realism without taking a hit in performance. If an object casting a shadow is destroyed, the shadow could still be seen. Serious Sam II uses both pre-baked lighting and shadows even on destructible objects. It’s still a good looking game, mind you, but it’s unsettling to see the tree’s shadow on the ground even after I’ve vaporized it.

Look at the shadow on the ground and the tree casting the shadow. The tree is gone from the second pic, but the shadow remains.

Dynamic Shadows – As I said before, my knowledge in shadowing is extremely limited. I still felt it needs to be addresses even with what little I know, because light and shadow are inseparable. When I use the term dynamic shadow, I’m not even sure if it’s a proper term. Perhaps stencil shadows might be more accurate? I’m talking about shadows that actually mimics your character’s movements or rotate depending on the time the position of the sun. Much better than the lousy circle shadows you see in some games. To get a shadow, a model is needed to generate the shadow. By model, I mean a real 3D polygonal model. Imagine a scene out doors with stencil shadows used everywhere. The tree, the leaves, the cliffs, the characters, a fence, and god knows whatever the hell the game needs. That’s a staggering amount of polygons just for some black silhouettes. Luckily, you can cheat by making the shadow model less detailed, but that’s still a lot. PC games like Doom 3 uses very accurate models to generate their shadows. This is also why pre-baked shadows can still have their place since they’re just painted on.

Self-shadowing – The name is self-explanatory. Outside of dynamic shadows, most other shadowing effects are extremely hard to spot the difference. Note the word “self” since that’s exactly what the shadow affects, and nothing else. Any shadows cast onto other objects is a result of other shadows. It really doesn’t get any easier than that. What you might not know is how GPU intensive this is. Even though the Dreamcast is capable of it, I see video card benchmarking games with self shadows turned off. This is probably due to the older systems using less detail geometry putting less stress on the GPU. Like a lot of other shadows, a model is needs to generate the shadows. The more characters and more complex the model, the tougher it is on the GPU. The strengths and weaknesses are the same as stencil shadows. It’s not an eye popping graphical technique, but it’s more processor intensive than you might think.

Notice the hand casts a shadow on its leg? That’s a self shadow.

There is also a “soft” self-shadow that is slightly different. The edges of the shadows have a soft blur that slowly fades as oppose to a solid black color. This is one of the reasons why today’s modern games have horrible shadow edges because they use soft shadows instead of a solid black shadow. Doom 3 doesn’t use soft shadows of any kind, so in some ways, they actually look better. Making the shadows soft once again increases the stress on the GPU. The word “soft” can be applied to anything, it refers to softening the edges.

I know this isn’t realtime, but the soft blur around the shadows with the Doom 3 example from above.

If only more attention was paid to the model detail, lighting, and shadowing, Wii games wouldn’t look so dated. Resident Evil 4 is the perfect example. The human models are quite detailed, and the lighting helps the atmosphere. Now what if someone scotch taped another Gamecubed together and used the extra ram to create more pre-baked shadows from all those trees, and used more bump maps on the rocky surfaces? It wouldn’t look anywhere near Gears of War, but no one would confuse it for a Dreamcast game either.

Now what if there’s still a little power left to render more shaders? What else can you add to enhance the look? There’s a lot more that can be done. They may not be used as wildly as some of the effects mentioned above, but they will add quite a bit to the graphical flare.

Tune next week or whenever the hell I’m done for part 3. No, we’re not even half way through yet.

< Part 1