Variational Image Processing

Current state-of-the-art methods in image and data processing almost entirely rely on a variational approach: The optimal output is found by finding a minimizer of an energy function or "model". Most interesting tasks are ill-posed: the minimizer either does not exist, is not unique, or is affected by noise in an uncontrollable way.

In this research area, we focus on modelling application-specific prior information to resolve the ill-posedness and to fully exploit domain-specific knowledge. Precisely formulating such prior knowledge in the form of an energy function makes it possible to systematically develop image processing methods that are more accurate and require less data.

The resulting problems are often non-differentiable and high- or infinite-dimensional, which requires the development of new mathematical tools and numerical optimization methods. We focus in particular on problems with combinatorial aspects, where a number of discrete decisions has to be made, such as the image segmentation problem.

Application areas include processing and analysis of images, videos, as well as general two- and higher-dimensional data, such as directional, tensor-, or height data in medicine, biology, and earth sciences.

Keywords

  • Variational and energy minimization methods in data processing
  • Regularization strategies and modelling prior knowledge
  • Convex relaxation methods for combinatorial problems
  • Sparsity and adaptive regularization
  • Large-scale non-smooth optimization
  • Optimal transport
  • Processing of non-scalar data - directions, tensors, manifolds

Current topics

Convex relaxations of non-convex problems

Jan Lellmann

For general non-convex functions, finding a global minimizer is a hard problem. In this project, we focus on replacing non-convex problems by convex - but larger - problems whose solution gives a good approximate solution of the original problem.

Collaborators: E. Bae, D. Cremers, E. Laude, T. Möllenhoff, M. Möller, C. Schnörr, E. Strekalovskiy, X.-C. Tai

Sparsity-based and non-local regularization

Jan Lellmann

Sparsity-based priors allow to robustly restore and reconstruct data that consist of "few simple" objects, and can be very well approximated by non-smooth energy functions. We focus in particular higher-order total variation, higher-order non-local regularization, and applications of optimal transport.

Collaborators: F. Lenzen, D. Lorenz, K. Papafitsoros, C. Schnörr, C. Schönlieb, D. Spector, T. Valkonen

Earth science and remote sensing

Jan Lellmann

Earth science and remote sensing problems are often complicated by the size of the data sets, as well as missing data. We develop methods for individual tree classification from multi-sensor data, as well as methods for surface interpolation from sparse measurement samples.

Collaborators: X. Cai, J. Lee, J.-M. Morel, S. Parisotto, C.-B. Schönlieb

Applications

  • Image restoration - denoising, deblurring, inpainting
  • Image segmentation
  • Earth sciences - Reconstruction and interpolation of digital elevation maps, analysis of LiDAR- and hyperspectral data
  • Medical image analysis - Segmentation and analysis of MRI data of thalamic nuclei 

Contact

Software

Selected Publications

  1. Sublabel–Accurate Relaxation of Nonconvex Energies. T. Möllenhoff, E. Laude, M. Möller, J. Lellmann, D. Cremers. Best Paper Honorable Mention at CVPR 2016. [pdf]
  2. Analysis and Application of a non-local Hessian. J. Lellmann, K. Papafitsoros, C. Schönlieb, D. Spector, SIAM J. Imaging Sci., 8(4), 2161–2202, 2015. [pdf]
  3. Imaging with Kantorovich-Rubinstein Discrepancy. J. Lellmann, D. A. Lorenz, C. Schönlieb, T. Valkonen, SIAM J. Imaging Sci., 7(4), 2833–2859, 2014. [pdf]
  4. Solving QVIs for Image Denoising with Adaptive Constraint Sets. F. Lenzen, J. Lellmann, F. Becker, and C. Schnörr, SIAM Journal on Imaging Science, 2014. [pdf]
  5. Anisotropic Third-Order Regularization for Sparse Digital Elevation Models. J. Lellmann, J.-M. Morel, and C. Schönlieb, Scale Space Var. Meth. Comp. Vis. (SSVM), 2013. [pdf]
  6. Discrete and Continuous Models for Partitioning Problems. J. Lellmann, B. Lellmann, F. Widmann, and C. Schnörr, International Journal of Computer Vision, 2013. [pdf]
  7. Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem. J. Lellmann, F. Lenzen, and C. Schnörr, Journal of Mathematical Imaging and Vision, 2012. [pdf]
  8. Continuous Multiclass Labeling Approaches and Algorithms. J. Lellmann and C. Schnörr, SIAM Journal on Imaging Sciences, 2011. [pdf]

Acknowledgments