Mini-HW 13: Light-Field Photography

(Task 1): Draw a rainbow.

Hopefully this gave you a chance to break out your crayons from grade school or something and really take a moment to relax. We're in some stressful times and it's really important to practice self care. If you do feel stressed, come talk to any of the course staff. We're here for you :)

Side-note from the TAs: you are halfway through this course, and look how far you've come! You've learned about the rasterization pipeline and common ways of representing geometry, and now are getting into one of the most difficult yet rewarding parts of the course: path tracing. You guys rock!

Now lets get into the actual assignment. There is no math involved in this homework, you just need to answer in words.

In the Radiometry lecture, we briefly discussed the idea of a light field camera. Light field cameras work by adding a micro-lens array between the main lens and the sensor, dividing the image up into a bunch of smaller, overlapping images. Light field cameras were inspired by the bead-like eyes of many insects that can see multiple images at once, and give us many useful properties to work with when capturing the world.

Having a micro-lens array allows us to parameterize the direction of light between the main and micro-lens as a 2-plane parameterization. Light enters the camera at some 2D point (u,v) and hits the sensor at another 2D point (s,t). These 4 degrees of freedom are enough for us to parametrize a line so long as we fix the planes. Thus, we not only know the intensity of light on our sensor, but also where it came from.

(Task 2): Your 4-year old sibbling sees you studying the Radiometry slides and asks what a light field camera is. He tells you he knows how a basic camera works (rays enter the main lens into a grid of cells within the camera that record color intensities). How would you tell him how a light field camera works? Remember, this is a 4-year old. Being able to talk about technical concepts to people outside of the graphics domain is a very valuable skill! (Hint: you only need to write a sentence or two. There is a very clever solution that requires only talking about basic cameras without talking about any internal camera components. Still stuck? check out this link)

(Task 3): How would you do nearest-neighbor sampling from a 2-plane parameterization of a light field? When would finding the closest ray be particularly useful? (Hint: there are multiple answers to the second part of this question.)

(Task 4): What are some cool effects you can do with light field photographs offline? That is, given a light field image, what effects can you run on them solely based on re-integrating the senor pixels? (Hint: look at page 10 of this paper and the title of this paper)

(Task 5): From a high level, how would you extend A3 to add light field photography? We're nice on grading this question, we just want to see how you think.