Hello again everyone. In my earlier post I put up some examples of what edge detection with a Sobel operator can do to a Quake level and some models and now I will discuss how to implement it. There are many different tutorials that can be found online which are confusing and contain a lot of mathematical detail. So I thought that for a change of pace I would take a more pragmatic angle to the subject. I implemented this post processing technique using DirectX 11 but I am sure that one could be able to do something similar using OpenGL.
Setting Up For Post Processing
One of the first questions that I ran into when implementing this was "where in the pipeline do I put post processing?" and this turns out to be a very valid question. In a straight forward rendering pipeline you create a render target view that is using a bitmap which is the buffer for the swap chain. Below is the code for what I am talking about:
ID3D11Texture2D *pBackBuffer = NULL;
//grab the pointer to the 2d texture from the swap chain
m_pSwapChain->GetBuffer(0, __uuidof(ID3D11Texture2D), (void**)pBackBuffer);
//Create a new render target view from that texture
m_pDevice->CreateRenderTargetView(pBackBuffer, NULL, m_pScreenRenderTargetView);
Then every frame we clear that render target, render the world and then present. But now we need to hi jack this a bit. What we need to do is to be able to render like normal, read from that texture, render the result of the Sobel edge detection to another texture and then present the finished product. To do this we create an additional render target view and keep it in the system, this will involve creating a new texture which can both be a shader resource and a render target view, below is the texture description for the new render target view texture:
D3D11_TEXTURE2D_DESC textureDesc =
{
windowWidth, windowHeight,
1, 1,
DXGI_FORMAT_R8G8B8A8_UNORM,
{ 1, 0 },
D3D11_USAGE_DEFAULT,
D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET,
0, 0
};
At the start of the frame set this new render target view, render the world to that and now you have a 2D picture of the world, we will call this the "first render pass". At this point you would normally render this picture but we made this texture a shader resource for a reason! Now set to the original render target view, set the shaders to your sobel shader and set the result of the first render pass as a shader resource, render a quad which fills the screen in clip space (xy position is from -1 to 1 and the z position should be a really small number, try 0.005) and finally present.
Performing Sobel Edge Detection
Now that we have post processing in place we can create the shaders. All the work is really done in the pixel shader as all the vertex shader needs to do is copy over the positions and texture coordinates because the texture that you received from your first pass is already in clip space (a.k.a. screen space). In the pixel shader you first need to sample from the first render pass texture all of the texture coordinates (TC) around the "base" TC to get their color value. I do this by creating a sort of matrix of 9 color values with the middle value being the color value of the "base" TC, we will call this the "pixel matrix". You will also need two additional matrices, an x and y convolution mask which estimates the change in color in the x and y direction, go HERE to see these matricies. Now once you have this find the sum of all the products of a matrix component by component multiplication of the pixel matrix by the x convolution mask and the y convolution mask, keep these values separate. Finally subtract the color value of the first render pass texture sampled with the base TC by the saturate of the sum of the dot product of the x sum and the y sum and return this value as your new pixel color. Experiment with these values to create some really cool looking effects!
If you have any questions please e-mail me or leave a comment.
-Dom
