Extracting frames from a Panasonic MTS file

MTS files are just an mpeg4 file. The command line utility ffmpeg will let you extract the frames:


Building:

svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg
cd ffmpeg
./configure
make && make install

Example usage with an HD camera

ffmpeg -i 00000.MTS -s hd1080 -f image2 waa-%03d.png

This will rip all the frames into waa-%03d.png.

7 spheres

Here's my image of 7 spheres. I'm completely underwhelmed.




Indices of refraction: center: 2.0, the others range from 2.1 to 2.6. The outside index of refraction is 1.0. This scene has almost as much as a vacuum as my brain. I'm just not getting this.

This image is horrible and I'm about ready to scrap my entire raytracer. You can see just how horrible it is by clicking on it to view the 1024x1024 version. There's 1000 samples/pixel and it took 1072 seconds to render.

Hatch marks

What's up with those hatch marks? Is my sphere just too dark? Should I pick light grey instead of almost black? I dunno.

I've picked three variables to change and see when the hatch marks show up:

1. Are the refracted rays rendered?
2. Is the diffuse component rendered?
3. Are the reflected rays rendered?

Image	Refract	Diffuse	Reflection	Hatching
	no	no	no	no
	yes	no	no	yes
	no	yes	no	no
	yes	yes	no	yes
	no	no	yes	no
	yes	no	yes	yes
	no	yes	yes	no
	yes	yes	yes	yes

Only when refraction is enabled do we see the hatch marks. This narrows it down to the following:

1. Specular highlights
2. Direction of refracted ray
3. Fresnel term
4. Tone mapping
5. And probably about 10 other things

Things that did not remove the hatch marks:
Removing tone mapping
Removing specular highlights

Removing the specular highlight does not remove the hatch marks. I'm not going to bother posting a pic. It looks like the sphere with everything enabled, just no specular highlight.

Changing from floats to doubles

In an attempt to remove some of the banding I'm seeing, I've changed from floats to doubles:

Index of refraction 1.33, double precision floating point:


Index of refraction 1.33, single precision floating point:


Difference image of the two:


The banding is gone, but now I'm seeing some ellipse shaped dark spots that I didn't expect to see.

The next three images use an index of refraction of 2.69.

Double precision:


Single precision:


Difference:

Fixed Phong shading

Here are the results with the fixed Phong shading:
Index of refraction 1.33, camera farther:

Index of refraction 1.33, camera closer:

Index of refraction 2.69, camera farther:

Index of refraction 2.69, camera closer:

I don't like the amount of noise in these images. There are 512 samples per pixel. A relatively lame sample selection process is being used: lay them all out on a regular grid and use a random number generator to jitter the samples within their grid boundaries. This doesn't seem to be any better than simply choosing 512 pixels laid out on a regular grid.

Carefully checking math

Turns out that I was a bit sloppy with my Snell's law implementation. Here are some new images that look a bit more correct:

Index of refraction: 1.33, camera farther

Index of refraction: 1.33, camera closer

Index of refraction: 2.69, camera farther

Index of refraction: 2.69, camera closer


That blue square is a messed up specular reflection. Looks like I might be a bit sloppy with my Blinn-Phong model, too.

Clamping reflection power

Here are images using the same code as before, but with the reflection power clamped to 1.0:

Index of refraction 1.0 to 1.0:

Index of refraction 1.0 to 1.33:

Index of refraction transition from 1.33 to 1.00


What's up with that light blue square? Is that a crappily rendered specular highlight?

Ok, back to reflections. These images look much better to me, but I'm not sure if they are correct. I expect a large reflection in the 1.33 image as it is sitting over an infinite plane.