On the Use of Machine Learning for Solving Computational Imaging Problems

Sponsored by “The Flint Fund Series on Quantum Devices and Nanostructures”

 

 

Massachusetts Institute of Technology, Mechanical Engineering Department

Singapore-MIT Alliance for Research and Technology (SMART) Centre

 

Computational Imaging (CI) systems consist of two parts: the physical part where light propagates through free space or optical elements such as lenses, prisms, etc. finally forming a raw intensity image on the digital camera; and the computational part, where algorithms try to restore the image quality or extract other type of information from the raw image data. A broad spectrum of computational imaging approaches exist: in one extreme, computer vision, the physical part typically comprises standard imaging optics; at the other extreme, in lens-less imaging the burden of forming images or extracting other types of information from the optical field falls entirely on the computation.

 

In this seminar I will discuss the emerging trend in deploying Machine Learning (ML) algorithms, and deep neural networks (DNNs) in particular, to cooperate with CI systems for image formation—a task distinct from image interpretation and pattern recognition where most other ML researchers operate. I will review the basic mathematical and physical underpinnings of CI and ML, emphasizing their common reliance on optimization and regularization. Subsequently, I will review recent and ongoing research on the use of ML for four topical CI problems: (1) super-resolution, i.e. recovering fine detail in undersampled or blurry images; (2) lensless imaging, especially phase retrieval at visible and x-ray wavelengths; (3) image formation at extremely low photon counts; and (4) imaging through scatter. I will also discuss some of the challenges facing this rapidly growing field of research, as well as opportunities for new contributions and directions.

 

George Barbastathis received the Diploma in Electrical and Computer Engineering in 1993 from the National Technical University of Athens (Πολυτεχνείο) and the MSc and PhD degrees in Electrical Engineering in 1994 and 1997, respectively, from the California Institute of Technology (Caltech.) After post-doctoral work at the University of Illinois at Urbana-Champaign, he joined the faculty at MIT in 1999, where he is now Professor of Mechanical Engineering. He has worked or held visiting appointments at Harvard University, the Singapore-MIT Alliance for Research and Technology (SMART) Centre, the National University of Singapore, and the University of Michigan – Shanghai Jiao Tong University Joint Institute (密西根交大學院) in Shanghai, People’s Republic of China. His research interests are in machine learning and optimization for computational imaging and inverse problems; and optical system design, including artificial optical materials and interfaces. He is member of the Society for Photo Instrumentation Engineering (SPIE), the Institute of Electrical and Electronics Engineering (IEEE), and the American Society of Mechanical Engineers (ASME). In 2010 he was elected Fellow of the Optical Society of America (OSA) and in 2015 he was a recipient of China’s Top Foreign Scholar (“One Thousand Scholar”) Award.