Journal of Mathematical Imaging and Vision

The residual neural network (ResNet) is a popular deep network architecture which has the ability to obtain high-accuracy results on several image processing problems. In order to analyze the behavior and structure of ResNet, recent work has been on establishing connections between ResNets and continuous-time optimal control problems. In this work, we show that the post-activation ResNet is related to an optimal control problem with differential inclusions and provide continuous-time stability results for the differential inclusion associated with ResNet. Motivated by the stability conditions, we show that alterations of either the architecture or the optimization problem can generate variants of ResNet which improves the theoretical stability bounds. In addition, we establish stability bounds for the full (discrete) network associated with two variants of ResNet, in particular, bounds on the growth of the features and a measure of the sensitivity of the features with respect to perturbations. These results also help to show the relationship between the depth, regularization, and stability of the feature space. Computational experiments on the proposed variants show that the accuracy of ResNet is preserved and that the accuracy seems to be monotone with respect to the depth and various corruptions.

Graph-based representations have been proved powerful in computer vision. The challenge that arises with large amounts of graph data is that of computationally burdensome edit distance computation. Graph kernels can be used to formulate efficient algorithms to deal with high dimensional data, and have been proved an elegant way to overcome this computational bottleneck. In this paper, we investigate whether the Jensen-Shannon divergence can be used as a means of establishing a graph kernel. The Jensen-Shannon kernel is nonextensive information theoretic kernel, and is defined using the entropy and mutual information computed from probability distributions over the structures being compared. To establish a Jensen-Shannon graph kernel, we explore two different approaches. The first of these is based on the von Neumann entropy associated with a graph. The second approach uses the Shannon entropy associated with the probability state vector for a steady state random walk on a graph. We compare the two resulting graph kernels for the problem of graph clustering. We use kernel principle components analysis (kPCA) to embed graphs into a feature space. Experimental results reveal that the method gives good classification results on graphs extracted both from an object recognition database and from an application in bioinformation.

Processing of multidimensional image data which were acquired by a linear imaging system of unknown point-spread function (PSF) is an important problem whose solution usually requires image restoration based on blind deconvolution (BD). Since BD is an ill-posed and often impossible task, we propose an alternative approach that enables to skip the restoration. We introduce a new class of image descriptors which are invariant to convolution of the original image with arbitrary centrosymmetric PSF. The invariants are based on image moments and can be defined in the spectral domain as well as in the spatial domain. The paper presents theoretical results as well as numerical examples and practical applications.

We introduce a new framework based on Riemann-Finsler geometry for the analysis of 3D images with spherical codomain, more precisely, for which each voxel contains a set of directional measurements represented as samples on the unit sphere (antipodal points identified). The application we consider here is in medical imaging, notably in High Angular Resolution Diffusion Imaging (HARDI), but the methods are general and can be applied also in other contexts, such as material science, et cetera, whenever direction dependent quantities are relevant. Finding neural axons in human brain white matter is of significant importance in understanding human neurophysiology, and the possibility to extract them from a HARDI image has a potentially major impact on clinical practice, such as in neuronavigation, deep brain stimulation, et cetera. In this paper we introduce a novel fiber tracking method which is a generalization of the streamline tracking used extensively in Diffusion Tensor Imaging (DTI). This method is capable of finding intersecting fibers in voxels with complex diffusion profiles, and does not involve solving extrema of these profiles. We also introduce a single tensor representation for the orientation distribution function (ODF) to model the probability that a vector corresponds to a tangent of a fiber. The single tensor representation is chosen because it allows a natural choice of Finsler norm as well as regularization via the Laplace-Beltrami operator. In addition we define a new connectivity measure for HARDI-curves to filter the most prominent fiber candidates. We show some very promising results on both synthetic and real data.

In this work, doubly stochastic Poisson (Cox) processes and convolutional neural net (CNN) classifiers are used to estimate the number of instances of an object in an image. Poisson processes are well suited to model events that occur randomly in space, such as the location of objects in an image or the enumeration of objects in a scene. The proposed algorithm selects a subset of bounding boxes in the image domain, then queries them for the presence of the object of interest by running a pre-trained CNN classifier. The resulting observations are then aggregated, and a posterior distribution over the intensity of a Cox process is computed. This intensity function is summed up, providing an estimator of the number of instances of the object over the entire image. Despite the flexibility and versatility of Cox processes, their application to large datasets is limited as their computational complexity and storage requirements do not easily scale with image size, typically requiring $$O(n^3)$$ computation time and $$O(n^2)$$ storage, where n is the number of observations. To mitigate this problem, we employ the Kronecker algebra, which takes advantage of direct product structures. As the likelihood is non-Gaussian, the Laplace approximation is used for inference, employing the conjugate gradient and Newton’s method. Our approach has then close to linear performance, requiring only $$O(n^{3/2})$$ computation time and O(n) memory. Results are presented on simulated data and on images from the publicly available MS COCO dataset. We compare our counting results with the state-of-the-art detection method, Faster RCNN, and demonstrate superior performance.

We present a novel representation of shape for closed contours in ℝ2 or for compact surfaces in ℝ3 explicitly designed to possess a linear structure. This greatly simplifies linear operations such as averaging, principal component analysis or differentiation in the space of shapes when compared to more common embedding choices such as the signed distance representation linked to the nonlinear Eikonal equation. The specific choice of implicit linear representation explored in this article is the class of harmonic functions over an annulus containing the contour. The idea is to represent the contour as closely as possible by the zero level set of a harmonic function, thereby linking our representation to the linear Laplace equation. We note that this is a local represenation within the space of closed curves as such harmonic functions can generally be defined only over a neighborhood of the embedded curve. We also make no claim that this is the only choice or even the optimal choice within the class of possible linear implicit representations. Instead, our intent is to show how linear analysis of shape is greatly simplified (and sensible) when such a linear representation is employed in hopes to inspire new ideas and additional research into this type of linear implicit representations for curves. We conclude by showing an application for which our particular choice of harmonic representation is ideally suited.

It is well known that the P3P problem could have 1, 2, 3 and at most 4 positive solutions under different configurations among its three control points and the position of the optical center. Since in any real applications, the knowledge on the exact number of possible solutions is a prerequisite for selecting the right one among all the possible solutions, and the study on the phenomenon of multiple solutions in the P3P problem has been an active topic since its very inception. In this work, we provide some new geometric interpretations on the multi-solution phenomenon in the P3P problem, and our main results include: (1) the necessary and sufficient condition for the P3P problem to have a pair of side-sharing solutions is the two optical centers of the solutions both lie on one of the three vertical planes to the base plane of control points; (2) the necessary and sufficient condition for the P3P problem to have a pair of point-sharing solutions is the two optical centers of the solutions both lie on one of the three so-called skewed danger cylinders;(3) if the P3P problem has other solutions in addition to a pair of side-sharing (point-sharing) solutions, these remaining solutions must be a point-sharing (side-sharing ) pair. In a sense, the side-sharing pair and the point-sharing pair are companion pairs; (4) there indeed exist such P3P problems that have four completely distinct solutions, i.e., the solutions sharing neither a side nor a point, closing a long guessing issue in the literature. In sum, our results provide some new insights into the nature of the multi-solution phenomenon in the P3P problem, and in addition to their academic value, they could also be used as some theoretical guidance for practitioners in real applications to avoid occurrence of multiple solutions by properly arranging the control points.

This paper outlines a new geometric parameterization of 2D curves where parameterization is in terms of geometric invariants and parameters that determine intrinsic coordinate systems. This new approach handles two fundamental problems: single-computation alignment, and recognition of 2D shapes under Euclidean or affine transformations. The approach is model-based: every shape is first fitted by a quartic represented by a fourth degree 2D polynomial. Based on the decomposition of this equation into three covariant conics, we are able, in both the Euclidean and the affine cases, to define a unique intrinsic coordinate system for non-singular bounded quartics that incorporates usable alignment information contained in the polynomial representation, a complete set of geometric invariants, and thus an associated canonical form for a quartic. This representation permits shape recognition based on 11 Euclidean invariants, or 8 affine invariants. This is illustrated in experiments with real data sets.