Deep Learning for Computational Imaging

This textbook offers an introduction to deep learning for solving inverse problems. It introduces deep neural networks and deep neural network based signal and image reconstruction techniques. It discusses robustness aspects, how to evaluate and test different methods, and data-centric aspects.

Computational techniques for image reconstruction problems enable imaging technologies including high-resolution microscopy, astronomy and seismology, computed tomography, and magnetic resonance imaging. Until recently, methods for solving such inverse problems were derived by experts without any learning. Now, the best performing image reconstruction methods are based on deep learning. This textbook gives the first comprehensive introduction to deep learning based image reconstruction methods. This book first introduces important inverse problems in imaging, including denoising and reconstructing an image from few and noisy measurements, and explains what makes those problems hard and interesting. Then, the book briefly discusses traditional optimization and sparsity based reconstruction methods, as well as optimization techniques as a basis for training and deriving deep neural networks for image reconstruction. The main part of the book is about how to solve image reconstruction problems with deep learning techniques: The book first disuses supervised deep learning approaches that map a measurement to an image as well as network architectures for imaging including convolutional neural networks and transformers. Then, reconstruction approaches based on generative models such as variational autoencoders and diffusion models are discussed, and how un-trained neural networks and implicit neural representations enable signal and image reconstruction. The book ends with a discussion on the robustness of deep learning based reconstruction as well as a discussion on the important topic of evaluating models and datasets, which are a critical ingredient of deep learning based imaging.