Fall 2009 CS 513 Final Project

The goal of this project was to take a photograph and place a model into that photo and match the lighting.

For this project, two cheap light probes were built for less than $10. Shiny and diffuse Christmas balls were used to probe the reflective and diffuse components of the scene. They were attached to dowel rods using duct tape and placed into the tree.

A consumer grade 10-megapixel ultracompact camera was placed on a tripod and used to take pictures of the spheres:




This resulted in the following sphere maps:





These sphere maps were mapped onto the model to produce an initial image:



Later, an image of a fingerprint was added to decrease the specularity of the model where the fingerprint left oil residue and caused dirt to accumulate.


Antialiasing was performed using the accumulation buffer. Slightly different view / projection matrices were used for each pass and the results averaged.

Ambient occlusion was approximated using only 26 lights. Shadow maps were computed for each pass and baked into a texture. Ideally at least 128 lights would be used.


Seams are always a problem with texture baking. Earlier in the semester, I wrote some software to find the seams in an OBJ model and transport texture data across seams to minimize their impact. The software isn't perfect, but it helps greatly.

The ambient occlusion is used to diminish the amount of light that is reflected by the model by decreasing the amount of available light in the shader. The end result has a pewter look to it:






This project was done on a 17" MacBook Pro running Snow Leopard. To build and download:

wget http://riskybacon.com/classes/cs513/final/cs513-final-jbowles.tar.bz2
tar jxf cs513-final-jbowles.tar.bz2
cd cs513-final-jbowles/final
make
./final

Coming back to ambient occlusion

My advisor (that's Dr. Joe Kniss) won't let me give up :)

He was right to give me crap, a trained monkey could have found the bug: to blend together the shadow maps, you need the correct texture coordinate to index into the previous shadow map. I had the wrong texture coordinate. Specifically, I was using the texture coordinate to index into the depth map. That was just stupid. Here's two blended shadow maps from lights at (1,0,0) and (-1,0,0):

And with 6 lights, one on each axis:


With 26 lights:


I need more lights. Here's the map on the model

Using the ambient occlusion to change how much light ends up on Frank:

Adding some rust / blemishes

It's hard to see in the previews, but some rust has been added to the model. The rust is a bit blurry, so it doesn't look very rusty. The rust diminishes the specularity at those points and adds some rust colored spots:

Frank with Fingerprints

I have to give up on ambient occlusion for now. I'm sure that I'm doing something stupid, but I'm running out of time to finish this project and can't seem to figure out the problem. So I moved on to adding fingerprints to Frank. The fingerprints change the amount of specular reflection.



With a rust texture blended in:

Ambient Occlusion test

Here's my first try at texture baking some ambient occlusion:

This isn't what I'd expected. In the torso portion of the map (to the left and under the feet) I'd expect some darker areas. I'm building this by taking a lot of depth images / shadow mapped images from all around the model and blending them together. Maybe my sphere of lights isn't where I think it is.

Blurry Frank

In an attempt to get some depth of field into the image, I ended up with the model being out of focus:


I'll need to play with that some more. This was done in a similar fashion to the antialiasing by jittering the projection matrix.

Antialiasing with Accumulation Buffer

I've flipped the sphere map texture coordinates which makes the scene a little more believable and added antialiasing via the accumulation buffer:

9 samples were taken, which means each frame was drawn 9 times. This really slows things down.