Low-Rank Matrix Recovery and Completion via Convex Optimization

http://perception.csl.uiuc.edu/matrix-rank/applications.html

	Background Modeling Video frames with a static background constitute a set of coherent images. It is often desirable to detect any activity in the foreground for surveillnace purposes. Foreground activity can be modeled as sparse errors corrupting a set of highly correlated images. We use the Robust PCA algorithm to separate the foreground pixels from the background.
	Removing Shadows and Specularities from Face Images It is known that well-aligned face images of a person under varying illumination lie very close to a low-dimensional linear subspace. However, in practice, these images deviate from this model due to self-shadowing and specularities. Thus, we have a set of coherent images corrupted by essentially sparse errors. The Robust PCA is a powerful tool to get rid of such errors and retrieve cleaner images potentially better suited for face recognition applications.
↓	Batch Image Alignment In recent years, the availability of large amounts of visual data online has renewed interest in large, unconstrained datasets. These images pose three major challenges to most existing computer vision algorithms: significant illumination variation, partial occlusion, and poor or no alignment. Misalignment makes it impossible to compare two or more images for recognition or classification. We introduce a new algorithm, named RASL, for robustly aligning linearly correlated images (or signals), despite large occlusions and corruptions. Our solution builds on recent advances in rank minimization and formulates the batch alignment problem as the solution of a sequence of convex programs. For more information, please click here.
	Transform Invariant Representation of Textures Matching points or regions between two images is a fundamental problem in computer vision. The problem is especially challenging because textures can appear very different from different viewpoints. We propose a new algorithm, named TILT, that produces a transformation-invariant low-rank representation of textures on planar surfaces. Our solution uses matrix rank as a measure of symmetry present in the texture and builds on recent advances in rank minimization to find the invariant representation. For more information, please click here.
	Robust Photometric Stereo Photometric stereo refers to the problem of estimating the normal map of a scene given multiple images under different illuminations. Typically, the object of interest is assumed to obey a Lambertian reflectance model. However, real images are often corrupted by non-Lambertian effects such as shadows and specularities. We propose a powerful and efficient technique to recover the normal map accurately despite such errors in the data. For more information, please click here.
	Camera Calibration Camera calibration is a classic problem in computer vision. Almost all of calibration methods rely on extraction of certain local features, and then assemble them to establish correspondences, calculate vanishing points, infer lines or conic curves for calibration. It is well-known that in practice it is difficult to accurately extract features in all images due to the presence of noise, occlusion, image blur, and change of illumination and viewpoint. We propose a simple, accurate, and flexible method to calibrate the intrinsic parameters of a camera together with (possibly significant) lens distortion. This new method can work under a wide range of practical scenarios: using multiple images of a known pattern, multiple images of an unknown pattern, single or multiple image(s) of multiple patterns, etc. Moreover, this new method does not rely on extracting any low-level features such as corners or edges. For more details, please refer to our paper.
	Unwrapping Generalized Cylindrical Surfaces Cylindrical surfaces are useful to model to model the geometry of some curved building facades, or even deformed book pages. Recovering the 3D shape of a cylindrical object from a single perspective image is a challenging task. However, the problem is better posed if the texture on the object's surface has some inherent structure. We consider the special case when the texture has low-rank (see Transform Invariant Representation of Textures above) when unwrapped onto a plane. Most existing methods largely rely upon low-level feature extraction and matching for this purpose. We propose a new method to holistically and robustly tackle this problem using the image pixels directly. The proposed method extends TILT to deal with low-rank textures on generalized cylindrical surfaces in 3D space, and thus is naturally robust to occlusions and other corruptions. For more details, please refer to our paper.
	Holistic 3D Reconstruction of Urban Structures from Low-Rank Textures We introduce a new approach to reconstructing accurate camera geometry and 3D models for urban structures in a holistic fashion, i.e., without relying on extraction or matching of traditional local features such as points and edges. Instead, we use semi-global or global features based on transform invariant low-rank textures, which are ubiquitous in urban scenes. Modern high-dimensional optimization techniques enable us to accurately and robustly recover precise and consistent camera calibration and scene geometry from single or multiple images of the scene. We demonstrate how to construct 3D models of large-scale buildings from sequences of multiple large-baseline uncalibrated images that conventional SFM systems do not apply. For more details, please refer to our paper.