Notes on Transient Imaging
In fall 2014, I took a seminar course in computer graphics, where we reviewed an emerging technology of capturing the propagation of light pulses. This work was sparked in recent years by the work of Velten, Raskar and Bawendi from MIT in their 2011 paper titled “Picosecond camera for time-of-flight imaging”. In response to this work, a team at the University of British Columbia (UBC) developed a much cheaper alternative (with certain limitations) to the hardware setup originally proposed by Velten et al. In their 2013 SIGGRAPH paper titled “Low-budget Transient Imaging using Photonic Mixer Devices”, Heide et al. developed a way to modify and use a relatively cheap time-of-flight (ToF) camera built by PMD technologies (called CamBoard nano, which is sold out, and possibly replaced by their CamBoard pico model) along with inexpensive lasers to capture light propagation. Although their setup is more limited than the one presented by Velten et al., it has certainly showed that transient imaging research can be more accessible to researchers with a smaller hardware budget.
The team from MIT posted a few videos of their results on YouTube. Here is one of them featuring a pulse of light going through a coke bottle:
The team from UBC also posted a video of their results on Vimeo:
Low-budget Transient Imaging using Photonic Mixer Devices from Felix Heide on Vimeo.
In 2014, paper by Lin et al., titled “Fourier analysis on transient imaging by multifrequency time-of-flight camera”, appeared in the Computer Vision and Pattern Recognition (CVPR) conference proceedings improving on the method proposed by Heide et al. for retrieving a transient image from the data captured by their time-of-flight camera. I found their discovery very elegant and decided to study it to see if I can find any further improvements. Over the course of the seminar, I compiled a set of notes giving a detailed description of the mathematical tools proposed by Lin et al. while analyzing possible areas of improvement (although I couldn’t make any new contributions of my own). So since nothing came of my dabbling in this field, I decided to release these notes that may perhaps be of use to someone freshly pursuing this area of research. I recommend reading the original paper by Lin et al. and perhaps even Heide et al., before diving into these notes.
Notes on Fourier analysis on ToF transient imaging:
