In a previous article we saw how to use the scissors test in order to clip drawing within a rectangular are of the screen. This is a very simple technique but with very limited use too. The situation that arises in real world games is "windows" of an arbitrary geometry. Let me give you some examples to make things a little more clear. The first is when drawing a mirror image, we need to clip to the shape of the mirror. When drawing what we can see through the window of a ship, the round shape of which cannot be described with a viewport or clipped with scissors. I am sure you can find a lot more situations we need a more advanced and flexible technique.
The answer to our problem is called "Stencil Buffer". It takes its name from the stencil tool used in drawing. The image shows what this tool is like. The material is removed in the areas marked withy black making it hollow. If we put it on a piece of paper and apply some paint we will draw the selected shape. Exactly the same can be done in OpenGL.
First we create the stencil (buffer in computer memory) and then we use it to clip our drawing within the desired area.
How it works
First the stencil buffer is cleared to black (filled with 0). Then we draw our shape and instruct OpenGL to assign 1 wherever we draw any polygons. When we finish with the creation of the stencil buffer we switch to using it. We tell OpenGL that from that point on when we draw on pixels with the value of 1 it is valid and when we draw on 0s it is invalid. This way on a pixel by pixel check we draw only within the desired area. Let's go step by step and see how it is done.
First we must clear the stencil buffer along with the depth buffer. Then we set the projection and view transformations along with any visual effects like fog, skybox and shaders. And then we proceed with the creation of the stencil buffer.
glClear(GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); // clear screen and stencil buffer ... // build the stencil buffer glColorMask(0,0,0,0); // set color mask glEnable(GL_STENCIL_TEST); // enable stencil buffer for "marking" the floor // prepare stencil buffer for setting up glStencilFunc(GL_ALWAYS, 1, 1); // always passes, 1 bit plane, 1 as mask glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE); // we set the stencil buffer to 1 where we draw any polygon glDisable(GL_DEPTH_TEST); // disable depth testing draw_surface();
Then we switch to normal draqwing. In the example we use the floor as a mirror. This means we must draw the mirrored world upside down.
glEnable(GL_DEPTH_TEST); // disable depth testing glColorMask(1,1,1,1); // set color mask // set the stencil buffer to clipping mode glStencilFunc(GL_EQUAL, 1, 1); // we draw only where the stencil is 1 glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP); // don't change the stencil buffer // draw the world in inversed y (mirror) glScalef(1.0f, -1.0f, 1.0f); // mirror y axis g_light.apply(); // apply the light in the current projection // draw a reflection of the horizon g_skybox.render(g_cam.vLocation.x, g_cam.vLocation.y, g_cam.vLocation.z); // draw the reflection of the spheres glColor4f(1.0f, 0.0f, 0.0f, 0.5f); g_w.render();
Now that we have finished with mirror effect we disable the stencil buffer and p[roceed with normal drawing of the world, after we bring the world up again (remember the vertical inversion for mirroring)
// we have finished with mirroring, stop stencil testing glDisable(GL_STENCIL_TEST); // we don't need the stencil buffer any more glEnable(GL_BLEND); // enable blending to fake transparency glBlendFunc(GL_SRC_ALPHA, GL_ONE); // set blending mode to mix based on src alpha // bring the y axis up again and proceed glScalef(1.0f, -1.0f, 1.0f); // reapply the light g_light.apply(); // draw the floor glColor4f(.0f, .0f, 1.0f, 1.f); draw_surface(); glDisable(GL_BLEND); // disable blending // draw normal objects glColor4f(1.0f, 0.0f, 0.0f, 1.f); g_w.render();
It is not that hard to do it after all. It really would have been more interesting if the mirror was not rectangular but elliptical. Nevertheless its shape after the projection is not rectangular and it shows that we defined a "random" shape for clipping area.