Springer Science and Business Media LLC

Tensors are nowadays a common source of geometric information. In this paper, we propose to endow the tensor space with an affine-invariant Riemannian metric. We demonstrate that it leads to strong theoretical properties: the cone of positive definite symmetric matrices is replaced by a regular and complete manifold without boundaries (null eigenvalues are at the infinity), the geodesic between two tensors and the mean of a set of tensors are uniquely defined, etc. We have previously shown that the Riemannian metric provides a powerful framework for generalizing statistics to manifolds. In this paper, we show that it is also possible to generalize to tensor fields many important geometric data processing algorithms such as interpolation, filtering, diffusion and restoration of missing data. For instance, most interpolation and Gaussian filtering schemes can be tackled efficiently through a weighted mean computation. Linear and anisotropic diffusion schemes can be adapted to our Riemannian framework, through partial differential evolution equations, provided that the metric of the tensor space is taken into account. For that purpose, we provide intrinsic numerical schemes to compute the gradient and Laplace-Beltrami operators. Finally, to enforce the fidelity to the data (either sparsely distributed tensors or complete tensors fields) we propose least-squares criteria based on our invariant Riemannian distance which are particularly simple and efficient to solve.

We developed a 3D archive system for Japanese traditional performing arts. The system generates sequences of 3D actor models of the performances from multi-view video by using a graph-cuts algorithm and stores them with CG background models and related information. The system can show a scene from any viewpoint as follows; the 3D actor model is integrated with the background model and the integrated model is projected to a viewpoint that the user indicates with a viewpoint controller. A challenge of generating the actor models is how to reconstruct thin or slender parts. Japanese traditional costumes for performances include slender parts such as long sleeves, fans and strings that may be manipulated during the performance. The graph-cuts algorithm is a powerful 3D reconstruction tool but it tends to cut off those parts because it uses an energy-minimization process. Hence, the search for a way to reconstruct such parts is important for us to preserve these arts for future generations. We therefore devised an adaptive erosion method that works on the visual hull and applied it to the graph-cuts algorithm to extract interior nodes in the thin parts and to prevent the thin parts from being cut off. Another tendency of the reconstruction method using the graph-cuts algorithm is over-shrinkage of the reconstructed models. This arises because the energy can also be reduced by cutting inside the true surface. To avoid this tendency, we applied a silhouette-rim constraint defined by the number of the silhouette-rims passing through each node. By applying the adaptive erosion process and the silhouette-rim constraint, we succeeded in constructing a virtual performance with costumes including thin parts. This paper presents the results of the 3D reconstruction using the proposed method and some outputs of the 3D archive system.

There are two main strategies for solving correspondence problems in computer vision: sparse local feature based approaches and dense global energy based methods. While sparse feature based methods are often used for estimating the fundamental matrix by matching a small set of sophistically optimised interest points, dense energy based methods mark the state of the art in optical flow computation. The goal of our paper is to show that this separation into different application domains is unnecessary and can be bridged in a natural way. As a first contribution we present a new application of dense optical flow for estimating the fundamental matrix. Comparing our results with those obtained by feature based techniques we identify cases in which dense methods have advantages over sparse approaches. Motivated by these promising results we propose, as a second contribution, a new variational model that recovers the fundamental matrix and the optical flow simultaneously as the minimisers of a single energy functional. In experiments we show that our coupled approach is able to further improve the estimates of both the fundamental matrix and the optical flow. Our results prove that dense variational methods can be a serious alternative even in classical application domains of sparse feature based approaches.

This study investigates variational image segmentation with an original data term, referred to as statistical overlap prior, which measures the conformity of overlap between the nonparametric distributions of image data within the segmentation regions to a learned statistical description. This leads to image segmentation and distribution tracking algorithms that relax the assumption of minimal overlap and, as such, are more widely applicable than existing algorithms. We propose to minimize active curve functionals containing the proposed overlap prior, compute the corresponding Euler-Lagrange curve evolution equations, and give an interpretation of how the overlap prior controls such evolution. We model the overlap, measured via the Bhattacharyya coefficient, with a Gaussian prior whose parameters are estimated from a set of relevant training images. Quantitative and comparative performance evaluations of the proposed algorithms over several experiments demonstrate the positive effects of the overlap prior in regard to segmentation accuracy and convergence speed.

Photometric camera calibration is often required in physics-based computer vision. There have been a number of studies to estimate camera response functions (gamma function), and vignetting effect from images. However less attention has been paid to camera spectral sensitivities and white balance settings. This is unfortunate, since those two properties significantly affect image colors. Motivated by this, a method to estimate camera spectral sensitivities and white balance setting jointly from images with sky regions is introduced. The basic idea is to use the sky regions to infer the sky spectra. Given sky images as the input and assuming the sun direction with respect to the camera viewing direction can be extracted, the proposed method estimates the turbidity of the sky by fitting the image intensities to a sky model. Subsequently, it calculates the sky spectra from the estimated turbidity. Having the sky $$RGB$$ values and their corresponding spectra, the method estimates the camera spectral sensitivities together with the white balance setting. Precomputed basis functions of camera spectral sensitivities are used in the method for robust estimation. The whole method is novel and practical since, unlike existing methods, it uses sky images without additional hardware, assuming the geolocation of the captured sky is known. Experimental results using various real images show the effectiveness of the method.

This paper presents a matching method for 3D shapes, which comprises a new technique for surface sampling and two algorithms for matching 3D shapes based on point-based statistical shape descriptors. Our sampling technique is based on critical points of the eigenfunctions related to the smaller eigenvalues of the Laplace-Beltrami operator. These critical points are invariant to isometries and are used as anchor points of a sampling technique, which extends the farthest point sampling by using statistical criteria for controlling the density and number of reference points. Once a set of reference points has been computed, for each of them we construct a point-based statistical descriptor (PSSD, for short) of the input surface. This descriptor incorporates an approximation of the geodesic shape distribution and other geometric information describing the surface at that point. Then, the dissimilarity between two surfaces is computed by comparing the corresponding sets of PSSDs with bipartite graph matching or measuring the L 1-distance between the reordered feature vectors of a proximity graph. Here, the reordering is given by the Fiedler vector of a Laplacian matrix associated to the proximity graph. Our tests have shown that both approaches are suitable for online retrieval of deformed objects and our sampling strategy improves the retrieval performances of isometry-invariant matching methods. Finally, the approach based on the Fiedler vector is faster than using the bipartite graph matching and it has a similar retrieval effectiveness.

The development of a real-time and robust RGB-T tracker is an extremely challenging task because the tracked object may suffer from shared and specific challenges in RGB and thermal (T) modalities. In this work, we observe that the implicit attribute information can boost the model discriminability, and propose a novel attribute-driven representation network to improve the RGB-T tracking performance. First, according to appearance change in RGB-T tracking scenarios, we divide the major and special challenges into four typical attributes: extreme illumination, occlusion, motion blur, and thermal crossover. Second, we design an attribute-driven residual branch for each heterogeneous attribute to mine the attribute-specific property and therefore build a powerful residual representation for object modeling. Furthermore, we aggregate these representations in channel and pixel levels by using the proposed attribute ensemble network (AENet) to adaptively fit the attribute-agnostic tracking process. The AENet can effectively make aware of appearance change while suppressing the distractors. Finally, we conduct numerous experiments on three RGB-T tracking benchmarks to compare the proposed trackers with other state-of-the-art methods. Experimental results show that our tracker achieves very competitive results with a real-time tracking speed. Code will be available at https://github.com/zhang-pengyu/ADRNet.

This paper is about automatically reconstructing the full 3D surface of an object observed in motion by a single static camera. Based on the two paradigms, structure from motion and linear intensity subspaces, we introduce the geotensity constraint that governs the relationship between four or more images of a moving object. We show that it is possible in theory to solve for 3D Lambertian surface structure for the case of a single point light source and propose that a solution exists for an arbitrary number point light sources. The surface may or may not be textured. We then give an example of automatic surface reconstruction of a face under a point light source using arbitrary unknown object motion and a single fixed camera.