Stencil Buffer


In a previous article we saw how to use the scissors test in order to clip drawing within a rectangular are of the screen. This is a very simple technique but with very limited use too. The situation that arises in real world games is "windows" of an arbitrary geometry. Let me give you some examples to make things a little more clear. The first is when drawing a mirror image, we need to clip to the shape of the mirror. When drawing what we can see through the window of a ship, the round shape of which cannot be described with a viewport or clipped with scissors. I am sure you can find a lot more situations we need a more advanced and flexible technique.

The answer to our problem is called "Stencil Buffer". It takes its name from the stencil tool used in drawing. The image shows what this tool is like. The material is removed in the areas marked withy black making it hollow. If we put it on a piece of paper and apply some paint we will draw the selected shape. Exactly the same can be done in OpenGL.

First we create the stencil (buffer in computer memory) and then we use it to clip our drawing within the desired area.

How it works

First the stencil buffer is cleared to black (filled with 0). Then we draw our shape and instruct OpenGL to assign 1 wherever we draw any polygons. When we finish with the creation of the stencil buffer we switch to using it. We tell OpenGL that from that point on when we draw on pixels with the value of 1 it is valid and when we draw on 0s it is invalid. This way on a pixel by pixel check we draw only within the desired area. Let's go step by step and see how it is done.

First we must clear the stencil buffer along with the depth buffer. Then we set the projection and view transformations along with any visual effects like fog, skybox and shaders. And then we proceed with the creation of the stencil buffer.

	glClear(GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);	// clear screen and stencil buffer
	// build the stencil buffer
	glColorMask(0,0,0,0);				// set color mask
	glEnable(GL_STENCIL_TEST);				// enable stencil buffer for "marking" the floor
	// prepare stencil buffer for setting up
	glStencilFunc(GL_ALWAYS, 1, 1);			// always passes, 1 bit plane, 1 as mask
	glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);		// we set the stencil buffer to 1 where we draw any polygon
	glDisable(GL_DEPTH_TEST);				// disable depth testing

Then we switch to normal draqwing. In the example we use the floor as a mirror. This means we must draw the mirrored world upside down.

	glEnable(GL_DEPTH_TEST);				// disable depth testing
	glColorMask(1,1,1,1);				// set color mask
	// set the stencil buffer to clipping mode
	glStencilFunc(GL_EQUAL, 1, 1);			// we draw only where the stencil is 1
	glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);		// don't change the stencil buffer
	// draw the world in inversed y (mirror)
	glScalef(1.0f, -1.0f, 1.0f);			// mirror y axis
	g_light.apply();					// apply the light in the current projection

	// draw a reflection of the horizon 
	g_skybox.render(g_cam.vLocation.x, g_cam.vLocation.y, g_cam.vLocation.z);

	// draw the reflection of the spheres
	glColor4f(1.0f, 0.0f, 0.0f, 0.5f);

Now that we have finished with mirror effect we disable the stencil buffer and p[roceed with normal drawing of the world, after we bring the world up again (remember the vertical inversion for mirroring)

	// we have finished with mirroring, stop stencil testing
	glDisable(GL_STENCIL_TEST);				// we don't need the stencil buffer any more
	glEnable(GL_BLEND);				// enable blending to fake transparency
	glBlendFunc(GL_SRC_ALPHA, GL_ONE);			// set blending mode to mix based on src alpha
	// bring the y axis up again and proceed
	glScalef(1.0f, -1.0f, 1.0f);
	// reapply the light

	// draw the floor
	glColor4f(.0f, .0f, 1.0f, 1.f);
	glDisable(GL_BLEND);				// disable blending

	// draw normal objects
	glColor4f(1.0f, 0.0f, 0.0f, 1.f);

It is not that hard to do it after all. It really would have been more interesting if the mirror was not rectangular but elliptical. Nevertheless its shape after the projection is not rectangular and it shows that we defined a "random" shape for clipping area.

You can download the complete example from here.


41 Kyprou street
Peraia, GR 57019
tel.: (+30) 6944721027


Copyright © GusOnGames 2011-2019

Increase your website traffic with