Modeling and Information Processing

SAM - Structure et mouvement
Resetting images and optical flow

The topic of estimating apparent motion between images (to summarize we will talk globally about "optical flow", but the problem takes multiple forms, depending on the context) in a sequence has been the subject of a great deal of work since the pioneering articles of Horn and Schunck [1] and Lucas and Kanade [2]. These efforts are due to the multiple uses of the estimated field of displacement, for example for the coding of video sequences, for evaluating the distance to an obstacle (in the context of the "see and avoid" function of autonomous systems), for odometry, for detecting and tracking targets, for measuring velocity fields by imaging in fluid mechanics, etc.

The work done at the DTIM since 2002 is aimed at developing fast and efficient optical flow estimating algorithms based on an image interpolation model, for example, by separable B-splines. After developing a "robust Horn and Schunck" type algorithm in 2003 [3, 4], the work concentrated on a fast algorithm based on window resetting in the line of the work by Lucas and Kanade [2].

Figure 1: Resetting a mask of a simulation image (from the processing applied to an IGN image). From top left to bottom right: (1) image, (2) quadratic resetting criteria with bilinear interpolation, (3) resetting criteria with cubic B-spline interpolation, (4) resetting criteria with exact interpolation. The black lines on masks (2) and (3) indicate the paths of resetting estimators associated with each type of interpolation.

The objective is also to evaluate the performance of these algorithms in a context of realistic use, taking into account, in particular, the image aliasing. Performance evaluation work has also been done on image resetting in the context of work concerned with super-resolution (see evaluating resolution). This year, we have done empirical work on resetting performance in order to start with the experimental validation of previous theoretical work. This study shows an influence of image aliasing on the performance of the estimate of displacement. This well known aspect is nevertheless little discussed in the literature. We show that, in this respect, the choice of method of image interpolation (indispensable in a context of sub-pixel resetting) is very important, as shown in figure 1, in which the cubic B-spline interpolation allowed us to smooth the resetting criteria and improve the movement estimator's performance.

There are multiple prospects for this work. On the one hand, we intend to continue with our work on the accuracy of calculation of the displacement field in the context of collaborative activities with other departments (for example: Composite Structures and Materials- DMSC) concerning the estimation of distortions of samples of composite materials by imaging (work planned in 2005). Generally speaking, the topic of estimating displacement fields by imaging interests many departments in various scientific branches of Onera and we are currently considering a multidisciplinary project on this subject.

On the other hand, this work has been done in conjunction with studies of the topic of super-resolution, a technique that demands sub-pixel displacement fields, all within a fast, or even real time, time frame: collaborative work is currently under way on this last point with the Axis department of the IEF (A. Dupret, L. Lacassagne).

Finally, the work on the precision of resetting, in particular at sub-pixel level, is also of interest to the 3D topic in the context of stereovision with a very low base line.

Publications

[1] K. B. Horn and B. Schunck, Determining optical flow, Artificial Intelligence, vol. 17, p. 185-204, 1981.

[2] B. Lucas and T. Kanade,  An iterative image registration technique with an application to stereo vision, IJCAI'81, p. 674-679, 1981.

[3] G. Le Besnerais, F. Champagnat et Gilles Rochefort, Robust optical flow estimation using B-spline Image Models, ISSPA'03, Paris, juillet 2003.

[4] G. Le Besnerais et F. Champagnat, B-spline image model for energy minimization-based optical flow estimation, soumis à IEEE Trans. On Image Processing, octobre 2004.

Last Update: 11 October 2006 - © ONERA 2009 - Terms of use