WO2010020624A1

WO2010020624A1 - Method of metric telemetry

Info

Publication number: WO2010020624A1
Application number: PCT/EP2009/060649
Authority: WO
Inventors: Denis Marraud
Original assignee: European Aeronautic Defence And Space Company - Eads France
Priority date: 2008-08-20
Filing date: 2009-08-18
Publication date: 2010-02-25
Also published as: FR2935188A1; FR2935188B1

Abstract

The invention relates to a method and a device for metric telemetry on objects viewed by cameras, video surveillance for example, with a view to their identification. The method according to the invention comprises the following steps: - defining a first list containing points of a first image taken on a zone observed by at least one camera and a second list containing a second image of the same points captured on a reference plane, - estimating a transformation between said lists of points, - translating by means of the estimated transformation measurements carried out on the first image into metric dimensions, - storing said first image as reference image and said transformation as reference transformation, deducing from the estimated transformation a set of parameters intended to translate metric height measurements, said set of parameters comprising, in particular, the image roll, the picture-taking distance, the angle of incidence or the angular resolution of said camera.

Description

METRIC TELEMETRY METHOD

TECHNICAL AREA

The invention lies in the field of searching for information in a video archive and more specifically relates to a metric telemetry method and device on objects seen by cameras, video surveillance for example, in order to their identification.

The invention also relates to a computer program stored on a recording medium and adapted, when executed on a computer, to implement the method according to the invention.

STATE OF THE PRIOR ART

The very large number of cameras in some CCTV networks require the use of automated processing to draw the operator's attention to events of interest. To limit the number of false alarms, it is essential to be able to characterize the detected objects in particular in terms of dimensions and trajectories in a terrestrial reference system.

In most current systems, cameras are neither precisely calibrated nor georeferenced. Only their position and the approximate footprint are sometimes used to indicate to an operator the camera closest to their area of interest. Also, for the most advanced systems, a complete calibration of the camera is required. This calibration uses a cumbersome procedure to the base of the charts to be moved in front of the camera, a procedure that must also be repeated regularly to compensate for calibration drift or extrinsic parameter variations as is the case for PTZ cameras

(Pan Tilt Zoom). Such a configuration procedure is difficult to envisage in a remote monitoring system comprising for several thousand cameras spread over a large area such as a city or an airport for example.

An object of the invention is to allow an operator to automatically configure several cameras.

DISCLOSURE OF THE INVENTION The object of the invention is achieved by means of a metric telemetry method based on a simple and fast procedure of geo-referencing images in a common reference.

The process according to the invention comprises the following steps:

define a first list containing points of a first image taken on an area observed by at least one camera and a second list containing a second image of these same points captured on a reference plane,

- estimate a transformation between said lists of points,

translating the measurements made on the first image into metric dimensions by means of the estimated transformation. storing said first image as a reference image and said transformation as a reference transformation.

In order to refine the estimated transformation, the method according to the invention provides the operator with a visualization tool enabling him to judge the quality of the alignment between the reference image and the image of the camera transformed with the medium. of the estimated transformation by superposition of the aligned images or by visualization of the residues.

Once the initial transformation has been validated, the image used for the calibration is stored as a reference and used continuously to update the current transformation in case of drift of the camera parameters or in case of changes of the extrinsic parameters (zoom, rotation line aiming, ...).

The updating of said reference transformation comprises the following steps: estimating the current transformation between a current image captured by the camera and the memorized reference image,

isolating descriptors that are invariant to the changes in scale of the current camera image and the reference camera image, combining the estimated current transformation and the reference transformation stored according to said descriptors in the event of drifting of the parameters of the camera or changes to extrinsic parameters such as zoom or line of sight rotation. Advantageously, said update of the current transformation is performed automatically.

The invention is applicable in all intelligent video surveillance systems whose function is to raise alarms on complex events requiring a low rate of false alarm and thus a fine characterization of detected objects, as well as on automatic annotation systems. CCTV streaming for indexing and investigation purposes.

The method provides geo-referenced measurements and trajectories that can be used for the recognition of detected objects.

In a first embodiment of the method according to the invention, the second image is an ortho-image of the observed area.

In a second alternative embodiment of the method according to the invention, the second image is a CAD model of the area observed. The method according to the invention further comprises a step of deriving from the estimated transformation a set of parameters intended to translate measurements made on the first image into metric dimensions (length, width, height), said set of parameters comprising, in particular and without limitation, the image roll, the shooting distance, the angle of incidence or the angular resolution of the camera.

The method according to the invention is implemented by a metric telemetry device comprising: means for defining a first list containing points of a first image taken on an area observed by at least one camera and a second list containing a second image of these same points captured on a reference plane,

means for estimating a transformation between said lists of points,

means for translating the measurements made on the first image into metric dimensions by means of the estimated transformation.

means for storing said first image as a reference image and said transformation as a reference transformation.

Preferably, the device according to the invention further comprises means for updating said reference transformation comprising:

means for estimating the current transformation between a current image captured by the camera and the stored reference image; means for isolating descriptors that are invariant to the changes in scale of the current camera image and the camera image; reference,

means for combining the estimated current transformation and the stored reference transformation as a function of said descriptors in the event of drifts of the camera parameters or changes in the extrinsic parameters such as the zoom or the line of sight rotation.

The method according to the invention is implemented in said device by a computer program stored on a recording medium and adapted, when executed on a computer, for:

estimating a transformation between said lists of points; translating, by means of the estimated transformation of the measurements made on the first image into lengths and metric heights.

storing said first image as a reference image and said transformation as a reference transformation.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features and advantages of the invention will emerge from the description which follows, taken by way of non-limiting example, with reference to the appended figures in which:

FIG. 1 is a diagram schematically illustrating the initialization phase of the method according to the invention,

FIG. 2 is a diagram schematically illustrating the update phase of the reference transformation according to the invention,

FIG. 3 is a diagram schematically illustrating an automatic telemetry phase using the current transformation estimated according to the method of the invention, FIGS. 4A and 4B schematically illustrate an iterative estimate of the image roll by application of the method according to the invention.

DETAILED PRESENTATION OF PARTICULAR EMBODIMENTS

Figure 1 illustrates the initial use of peer points between the camera image and the reference image to estimate the reference transformation.

Step 2 consists of capturing points of a first image 4 captured by a camera, and these same points selected manually by an operator, or automatically by exploiting and matching descriptors that are invariant to the changes in shooting from a camera. reference plane 6 on an observed area.

Step 10 consists of calculating a reference transformation from the points entered by the camera and points selected from the reference plane 6. At step 12, the image captured by the camera and the image obtained at from the reference plane 6 are displayed and compared to define the reference transformation 14 and the reference image 16.

The reference image 16 and the reference transformation 14 are then stored.

FIG. 2 schematically illustrates the update phase of the transformation obtained in step 10. This phase is entirely automatic and is based on the pairings of descriptors invariant to the changes of scale extracted from a current image 20 and the reference image 16 obtained during this initialization phase.

Step 22 consists of a registration of the current image 20 and the reference image 16 to verify the parameters of the reference transformation 14.

If the parameters obtained do not correspond to those obtained in step 10, a new transformation is calculated, in step 24, between the current image 20 and the reference image 16.

According to a characteristic of the invention, to avoid a drift of the reference image 20 over time, the reference transformation 14 is updated by composition of the estimated transformation between the current image 20 and the image of the image. reference 16 and said reference transformation 14. The reference image 20 is then updated in step 26 to take into account the extension of the field of view in the case of PTZ (for Pan Tilt Zoom) cameras, while keeping unchanged the initial field of view of the reference image 20.

FIG. 3 illustrates the automatic telemetry method using the estimated current transformation 28 and updated in step 24. In this example, a mobile object detection step 30 provides a detection mask used to extract the measurements in the picture.

From the current transformation, the shooting parameters can be calculated. These are then used in step 32 to transform measurements made from the current image in metric measurements.

In step 34, an automatic estimation of the parameters of the current transformation is performed from the transformation between the current image 20 and the following metric reference data:

- the roll image

- the angle of incidence of the shooting - the angular resolution

- shooting distance

These parameters are estimated thanks to geometric considerations on the lengths of segments judiciously chosen in the image plane and transformed on the ground by the current transformation.

Note that the transformation of two horizontal segments S1 and S2 in the current image, of the same length on either side of the center of the image, makes it possible to estimate the roll iteratively by exploiting the fact that for a zero roll, the ground lengths of these two projected segments should be identical. Conversely, in the presence of image roll, and if the line of sight is not vertical, the projected lengths will be different. In this case, it is sufficient to iteratively correct the roll until the equality criterion of the projected lengths is actually verified.

Figure 4A schematically illustrates an image captured by a camera and Figure 4B illustrates the same ground image viewed from above. Has two horizontal segments S1 and S2 of the same length In FIG. 4A correspond to two ground segments S'1 and S '2 of different lengths (FIG. 4B) due to the image roll and the perspective effect due to a non-vertical angle of incidence. The same two segments S _c I and S _C 2 corrected image roll and transformed in the reference image project in two segments of the same length due to the elimination of the perspective effect since the two projected segments are then at the same distance from the point of view.

With the assumption that the pixels of the image captured by the camera has a square shape, calculating the transformations, with zero roll, of three segments of the same length in the image, ie a first horizontal segment and two vertical segments, having all three for origin the center image, allows to find, in particular, the angle of incidence, the angular resolution and the shooting distance.

Indeed, if L | designates the projected length of the vertical segment in the image below the nearest image center of the sensor, Ls the projected length of the vertical segment above the image center farthest from the sensor so that Ls> Li, and L _H the length projected horizontal segment, then: The angle of incidence θ ₀ is given by:

The shooting distance between the optical center and the ground point targeted by the center image is given by:

The angular resolution is given by:

p

where 1 denotes the length of the three segments in the image plane.

Claims

1. Metric telemetry method characterized in that it comprises the following steps: - define a first list containing points of a first image taken on an area observed by at least one camera and a second list containing a second image of these same points entered on a reference plane, - estimate a transformation between the said points lists,

translating, by means of the estimated transformation of the measurements made on the first image into metric dimensions, storing said first image as a reference image and said transformation as a reference transformation,

deriving from the estimated transformation a set of parameters intended to translate measurements made on the first image into lengths and metric heights, said parameter set comprising, in particular, the image roll, the shooting distance, the angle of incidence or angular resolution of said camera.

The method of claim 1 further comprising a step of updating said reference transformation consisting of:

estimating the current transformation between a current image captured by the camera and the memorized reference image, isolating descriptors that are invariant to the changes in scale of the current camera image and the reference camera image,

- Combining the estimated current transformation and the stored reference transformation according to said descriptors in case of drifting camera parameters or changes in extrinsic parameters such as zoom or line of sight rotation.

3. The method of claim 1 wherein said second image is an ortho-image of the observed area.

4. The method of claim 2 wherein said second image is a CAD model of the observed area.

5. The method of claim 1 wherein the update of the current transformation is performed automatically.

6. Metric telemetry device characterized in that it comprises: - means for defining a first list containing points of a first image taken on an area observed by at least one camera and a second list containing a second image of these same points entered on a reference plane, - means for estimating a transformation between said lists of points, means for translating, by means of the estimated transformation, measurements made on the first image into lengths and metric heights,

means for storing said first image as a reference image and said transformation as a reference transformation, means for deriving from the estimated transformation a set of parameters intended to translate measurements made on the first image into lengths and metric heights, said a parameter set including, in particular, the image roll, the shooting distance, the angle of incidence or the angular resolution of the camera.

The device of claim 6 further comprising means for updating said reference transformation comprising:

means for estimating the current transformation between a current image captured by the camera and the memorized reference image,

means for isolating descriptors that are invariant to the changes in scale of the current camera image and the reference camera image, means for combining the estimated current transformation and the stored reference transformation according to said descriptors in case of drifts camera settings or extrinsic parameter changes such as zoom or line of sight rotation.

8. Computer program stored on a recording medium and intended, when executed on a computer to implement the method according to one of claims 1 to 6.