US 5675380 A Abstract The device includes systems for picking up (1), acquiring (2) and digitizing (3) a distorted source image (SI), a system for processing digital images (4, 47) for constructing a distortion-corrected target image (TI) corresponding to the source image (SI), this system including a first sub-assembly (4) for predetermining, on the basis of the data of a distorted TEST source image (SG°), an optical center (OC
^{n}) and a polynomial function (F_{n}) for correcting radial distortions around this optical center, and a second sub-assembly (47) which applies the polynomial correction function to each address of the pixel of the target image (TI) for furnishing the address of a point in the distorted source image (SI) in which an intensity data is present which is applied to the initial address in the target image. The method includes the computation, in a TEST source image (SG°), of an optical center (OC^{n}) at the center of optical distortions, and of a polynomial function (F^{n}) for correcting distortions for constructing a TEST target image (TI).Claims(11) 1. A device for forming an image, comprising:
a camera system; a system for acquisition of data of a source image distorted by the camera system; a system for digitizing said source image including a first image memory for storing intensity data of each pixel marked by an address in a bidimensional matrix; and a system for processing the digital image for constructing a distortion-corrected target image corresponding to the source image, in which the image processing system includes: a first sub-assembly for predetermining, on the basis of image data of a distorted TEST source image, an optical center and a polynomial function for correcting radial distortions around said optical center; and a second sub-assembly including a memory for storing the optical center and the polynomial function determined by the first sub-assembly, a computing block for applying the polynomial function to each pixel address of the target image, and for supplying an address of a pixel in the distorted source image in which an intensity data is present which is to be applied to the initial address in the target image, and a second image memory for storing data of the target image. 2. A device for forming images as claimed in claim 1, in which:
the first sub-assembly includes blocks for treating the source image TEST, said source image TEST being formed as a grating, referred to as source grating, for constructing a target image TEST, referred to as theoretical grating, said blocks comprising: a block for extracting reference points at the points of intersection of the bars of the source grating; a block for estimating the first approximate theoretical grating, computing the address of a center and the step size of said first theoretical grating; a block for estimating a distortion-corrected theoretical grating, computing by way of iteration the address of a center and the step size of said corrected theoretical grating; a block for computing a correction polynomial of the radial distortions, providing a transformation rule for passing a point of the corrected theoretical grating at the iteration n to a reference point of the distorted source grating, which transformation rule operates at the iteration n; a block for computing a patterning error due to the iteration n transformation rule; and a block for computing a modified address of the optical center which minimizes the patterning error. 3. A method of correcting geometrical optical distortions produced in an image by a camera system, said method comprising the preliminary steps of:
a) acquiring data and digitizing a TEST source image, including storing intensity data of each pixel marked by an address in a bidimensional matrix; b) estimating an optical center located, at best, at the optical distortion center of the TEST source image and the ratio of said optical center computed in a digital image to be constructed, referenced TEST target image representing the corrected TEST source image of the optical distortions; and c) estimating a polynomial function for causing the address of a corresponding point, denoted reference point, in the distorted TEST source image to correspond to the address of a pixel in the corrected TEST target image, based on the hypothesis that the geometrical optical distortions in the TEST source image are radial around the distortion center, so that said corresponding points are aligned, at best, with the estimated optical center. 4. A method as claimed in claim 3, comprising, in the preliminary steps, the iterative steps of:
d) estimating the best polynomial function capable of minimizing, at an iteration of the order of Nopt, the patterning error realized at the localization of the pixels of the TEST target image constructed at a preceding iteration in accordance with the hypothesis of radial distortions around an optical center estimated at said preceding iteration; and e) estimating, at an iteration of the order of Nlast, an optimized modified optical center and re-estimating under these conditions a new polynomial function capable of further minimizing the patterning error realized at the localization of the pixels of the reconstructed target image in accordance with the hypothesis of radial distortions around said estimated optimized modified optical center. 5. A method as claimed in claim 4, in which the preliminary steps are performed once, said method comprising, at the start of the preliminary steps, the steps of:
a _{o}) acquiring the data of a source image distorted by the camera system, digitizing and storing the intensity data of each pixel; andf') correcting the optical distortions by applying the best estimated polynomial function to a digital target image to be constructed, so as to supply, on the basis of each address in said target image, a corresponding address in the distorted digital source image in which an intensity function is found which is applied to said address in the target image to be constructed. 6. A method as claimed in claim 3, in which the preliminary steps are performed once, said method comprising, at the start of the preliminary steps, the steps of:
a _{0}) acquiring the data of a source image distorted by the camera system, and digitizing and storing intensity data of each pixel; andf) correcting the optical distortions by applying the estimated polynomial function to a digital target image to be constructed, so as to supply, on the basis of each address of this target image, a corresponding address in the distorted digital source image in which an intensity data is found which is applied to said address in the target image to be constructed. 7. A method as claimed in claim 3, in which:
in the preliminary step a) of acquiring the data and digitizing, the TEST source image is the digitized image of a rectangular mesh whose bars are parallel to the rows and columns of pixels, to the most approximate optical distortions referred to as source grating; in the step b), said optical center is estimated in an iterative manner on the basis of a zero iteration comprising the construction of a first distortion-corrected TEST target image, referred to as first target grating, by means of the sub-steps of: extracting one point per zone of intersection of the bars of the source grating, these extracted points being denoted as reference points of the source grating; estimating a reference point which is nearest to the center of distortions of the source grating; transferring this reference point into the first target grating for constituting a first center of the grating; estimating a first grating pitch for said target grating; and estimating a first approximate optical center for said first target grating coinciding with said transferred reference point and coinciding with the center of the grating. 8. A method as claimed in claim 7, in which in step d), the iterative estimation of the best polynomial function comprises the estimation, in a zero-order iteration, of a first polynomial function comprising the sub-steps of:
constructing points of the first target grating on the basis of the center of the grating and the grating step to correspond to the reference points of intersection of the bars of the source grating; causing the grating points of the first target grating and the reference points of the source grating to correspond by localizing the center of the grating coinciding with the reference point which is nearest to the center of distortion; forming the pairs constituted by the grating points and the corresponding reference points step by step from the center to the edges of the target grating; estimating the pairs of radii formed by segments joining the optical center of the first target grating and each point of the pairs of target grating points and the reference point; and computing the first polynomial function as the one which best connects the pairs of radii in conformity with the hypothesis in accordance with which the distortions are radial around the optical center. 9. A method as claimed in claim 8, in which in step d), the iterative estimation of the best polynomial function comprises the estimation, in an n>0 order iteration, of a polynomial function comprising the sub-steps of:
constructing a target grating at the order of n, having an optical center and grating points defined by a center of the grating and a grating pitch; causing the grating points of the target grating to correspond to the reference points of the source grating by localizing the center of the target grating by means of its coordinates determined in the preceding iteration; forming pairs of grating points and reference points step by step from the center towards the edges of the target image; estimating the pairs of radii formed by segments joining the known optical center and each point of the pairs; and computing the polynomial function as the one which best connects the pairs of radii in conformity with the hypothesis in accordance with which the geometrical distortions are radial around the optical center. 10. A method as claimed in claim 9, in which the construction of a target grating at the current iteration (n) comprises the sub-steps of:
localizing an optical center which is re-updated; localizing the grating points and corresponding reference points constituting pairs formed at the preceding iteration; estimating the geometrical distances on the abscissa of the grating points of the radii passing through the re-updated optical center and through the corresponding reference points of the pairs formed at this preceding iteration; estimating a criterion, referred to as radial criterion, which expresses the radial hypothesis that the best corrected target grating corresponds to the minimization of the geometrical distances; and minimizing the radial criterion expressed as a function of the coordinates of the center and of the components of the grating pitch at the iteration, supplying the center coordinates and the components of the target grating pitch at the iteration. 11. A method as claimed in claim 10, in which the step b) of estimating the optical center is realized in an iteration loop in which, at each current iteration, the localization of the optical center is modified and then reintroduced in the construction of the current target grating for determining a new polynomial function, and estimating the corresponding patterning error the best localization of the center being that which results in the smallest patterning error corresponding to the best polynomial function.
Description 1. Field of the Invention The invention relates to a device for forming an image, comprising: a camera system, a system for acquisition of the data of a source image distorted by the camera system, a system for digitizing said source image including a first image memory for storing the intensity data of each pixel marked by an address in a bi-dimensional matrix, and a system for processing the digital image for constructing a distortion-corrected target image corresponding to the source image. The invention also relates to a method of correcting geometrical optical distortions produced by such a camera device in an image. The invention is used for correcting digitized images in the field of X-ray systems or in video systems, or in digital image systems. Significant geometrical distortions may be introduced by the objective of a camera in the image produced by this camera, particularly if this objective is of the wide angle type. These geometrical distortions are most frequently of the barrel type or of the pincushion type. These geometrical distortions appear even if the objective of the camera has a very good quality. 2. Description of the Related Art A device for compensating optical imperfections in a television camera is already known in the state of the art from UK Patent Application GB 2,256,989, corresponding to U.S. Pat. No. 5,276,519. This device comprises a camera registering the image formed by the optical system and means for digitizing this image, which include storage of the intensity data of each current pixel. This device also comprises an error corrector for compensating the geometrical, registering and chromatic errors and for also compensating the optical imperfections of the lens system. This error corrector is controlled by a correction control unit. This correction control unit receives exterior information components via an interface, these components being suitable for programming the unit so that it can apply control signals to the error corrector which take the parameters of the camera system into account. Under these conditions, the error corrector is capable of correcting the pixel data. In the error corrector, a cartographic memory receives these control signals from the correction control unit in dependence upon the parameters of the camera system. This cartographic memory is tabulated as a function of these parameters for defining the cartography which is necessary for correcting the faults of the optical system with a given lens type and under given camera conditions. The output of the cartographic memory is applied to an interpolator used for enhancing the definition of the output image. The cartographic memory is calibrated by applying a test pattern to the pick-up camera having a regular square grating design and by manually adjusting the stored parameters so that a corrected grating is obtained at the output of the device. This operation may be realized by displaying, for example the output signal of the device on a screen provided with a superimposed graticule. The device known from U.S. Pat. No. 5,276,519 does not correct the imperfections of the camera system in an automatic manner. It is necessary to supply it with data about the focal length of the lens, the camera distance and the zoom rate used. Moreover, a cartographic memory should contain tables for each lens type and each camera condition, these tables including information relating to the new addresses to be assigned to the pixels of the target image for replacing the original addresses of these pixels in the source image, that is to say, in the image directly originating from the camera system and being beset with imperfections. The target image may thus be corrected from the distortions of the source image. No automatic tabulation means for the cartographic memory or automatic means for computing the correction functions to be applied to the pixels of the source image for obtaining the pixels of the target image are described in the document cited hereinbefore. Only means for manual calibration of the cartographic memory have been described, including the superposition of the distorted target image of a grating on a reference graticule and the manual correction of the input parameters of the camera system. It is an object of the present invention to provide a device and a method for calculating correction functions of the optical distortions of a camera system in order to obtain pixel data with which a corrected target image can be constructed. It is another object of the present invention to provide such a device and a method for automatically computing such correction functions without having to take the parameters of the camera system into account. These objects are achieved by means of a device as described in the opening paragraph, in which the system for processing the image includes: a first sub-assembly for predetermining, on the basis of image data of a distorted TEST source image, an optical center and a polynomial function for correcting radial distortions around said optical center, and a second sub-assembly including a memory for storing the optical center and the polynomial function determined by the first sub-assembly, a computing block applying the polynomial correction function to each pixel address of the target image for supplying the address of a pixel in the distorted source image in which an intensity data is present which is to be applied to the initial address in the target image, and a second image memory for storing data of the reconstructed target image. A method of correcting optical distortions produced in an image by a camera system comprises the preliminary steps of: a) acquiring data and digitizing a TEST source image, including the storage of the intensity data of each pixel marked by an address in a bi-dimensional matrix, b) estimating an optical center located at best at the optical distortion center of the TEST source image and the ratio of said optical center computed in a digital image to be constructed, referenced TEST target image representing the corrected TEST source image of the optical distortions, and c) estimating a polynomial function for causing the address of a corresponding point denoted reference point in the distorted source image to correspond to the address of a pixel in the corrected TEST target image, based on the hypothesis that the geometrical optical distortions in the source image are radial around the distortion center, so that said corresponding points are aligned at best with the estimated optical center. In the preliminary steps, such a method may also comprise the iterative steps of: d) estimating the best polynomial function F e) estimating, at an iteration of the order of Nlast, an optimized modified optical center OC These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. In the drawings: FIG. 1A shows a device for forming images including a camera system and two sub-assemblies of a system for processing digital images, one for determining and the other for applying the geometrical correction rules for optical distortions of the camera system; FIGS. 1B and 1C illustrate the calibration step of acquiring a test pattern by the camera system; FIG. 2A shows a block diagram of the first sub-assembly of the image processing system for performing a method of determining the correction rules and determining the distortion center for geometrically correcting the optical distortions produced by the camera system, and FIG. 2B shows a block diagram of the second sub-assembly of the image processing system; FIGS. 3A to 3G show source and target images in the different steps of the method; FIG. 4A shows an intersection in the source grating and illustrates the non-linear filtering operation for extracting the reference points; FIG. 4B shows an intersection in the source grating for determining the reference point R FIG. 4C shows a horizontal sub-assembly HS FIG. 4D shows a central sub-assembly S FIG. 4E illustrates the formation of rectangular rings having an increasing size for forming pairs P FIGS. 5A and 5B show source images which are pincushion and barrel-distorted, respectively; FIG. 6A illustrates the operation of causing a grating point TR FIG. 7A illustrates the determination of the errors Δ FIG. 7B illustrates the determination of the pairs of radii X FIG. 7C represents the points of the pairs P FIG. 8A illustrates the determination of a polynomial F FIG. 8B illustrates the determination of the first polynomial F° at the zero iteration in the iterative method and FIG. 8C illustrates the determination of an improved polynomial F FIG. 8D shows the points of the pairs P I. DEVICE (FIGS. 1A, 2A, 2B) A camera system may generally include an optical lens system of the wide angle type or a zoom lens system which produce geometrical optical distortions in the image formed by this camera. These distortions appear even when the optical system has a very good quality. With reference to FIG. 1A, a device for forming an image comprises a camera system 1 consisting of an optical lens system which supplies a distorted image of a scene. This optical lens system 1 is attached to a camera 2 which converts the optical image into electric signals by way of, for example, a CCD. These electric signals are applied to a digitization system 3 which is capable of storing the digitized image data, i.e., the intensity data relating to each address of the pixels of the image, in a bi-dimensional matrix, in a first picture memory. These digitized image data relate to the distorted image. This device for forming images also comprises a system for processing the digital signal, which system comprises a sub-assembly 47 processing the digitized image data stored in the picture memory of the digitization system 3 for supplying data of a corresponding digital image reconstructed with a correction of distortions, which data are finally stored in a second picture memory. The digital image data at the output of the digitization system 3, relating to an image distorted by the camera system, will hereinafter be referred to as source image SI, and the digital image data supplied by the sub-assembly 47 of the image processing system relating to the distortion-corrected reconstructed image, will be referred to as target image TI. The sub-assembly 47 of the image processing system comprises, on a chip-card or in a look-up table, rules for correcting the distortions for constructing the corrected target image TI. These rules may be determined by means of a method comprising preliminary steps. One of the objects of the invention is to provide such a method for determining these correction rules in an automatic manner. The correction rules are determined once for all operations in a first sub-assembly 4 of the processing system. Subsequently, they are automatically applied by means of the sub-assembly 47 of the image processing system referred to as second sub-assembly 47. With the switch INT shown in FIG. 1A, the first sub-assembly 4 of the processing system can be switched off when these rules have been determined. The first sub-assembly 4 includes, as shown in FIG. 2A, a block 41 for extracting reference points (R As shown in FIG. 2B, the second sub-assembly 47 includes a memory 48 connected to the output of the first sub-assembly 4 for storing the optical center and the polynomial function determined by the first sub-assembly 4. A computing block 49 then applies the polynomial correction function to each pixel address of the target image (TI) for supplying the address of a pixel in the distorted source image (SI), received from the digitization system and first picture memory 3, in which an intensity data is present which is to be applied to the initial address in the target image. A second image memory 50 is then provided for storing data of the target image. The method of determining the correction rules performed in the first sub-assembly 4 of the image processing system will subsequently be described. This first sub-assembly 4 may also be an extension of the digitization system 3, or it may be integrated in this digitization system 3. In the device for forming the image according to the invention, as shown in FIGS. 1A, 2A and 2B: (a) in the first sub-assembly 4 of the processing system, computing steps are carried out for determining once for all operations: rules for correcting the geometrical distortions produced by the camera system 1, and precise localization of the center of the geometrical distortions which is also the optical center of the lens system of the camera in the majority of cases; and (b) in the second sub-assembly 47 of the processing system, computing steps are carried out for constructing a distortion-corrected target image TI by means of said rules and the knowledge of the center of distortion for each distorted source image SI. This will be continued with a description of: a method of determining these rules for correcting the geometrical distortions and for concomitantly determining the precise localization of the distortion center, carried out in the first sub-assembly 4, and a method of constructing the target image TI by using this knowledge, carried out in the second sub-assembly 47. II. METHOD OF DETERMINING THE CORRECTION RULES AND THE DISTORTION CENTER (FIG. 1, Block 4; FIG. 2A, Blocks 41 to 46) A method of concomitantly determining these correction rules and the precise coordinates of the distortion center will be described hereinafter with reference to FIG. 2A which shows the different steps by way of a block diagram. This method does not require any preliminary measurements, such as the measurement of the focal length, or the measurement of the camera distance, or the real measurement of the test pattern pitch. This method is based on the hypothesis that the geometrical distortions generated by the camera system are radial, that is to say, in the target image constructed by the second sub-assembly 47 of the processing system, a corrected point should be present on a radius determined by the distorted point and by the distortion center. For this reason, this method requires, in addition to determining the correction rules, a concomitant precise determination of the distortion center. Nevertheless, this method does not require any preliminary precise real measurements. In this method, the aim is not a precise measurement by means of, for example a sensor, of the localization of the optical center of the camera, because only the distortion center of the source image is interesting due to the fact that the distortions are radial around the distortion center. With reference to FIG. 2A, this method, performed in the first sub-assembly 4, comprises at least the following steps: II. A. ACQUISITION OF A PATTERN This step comprises the following sub-steps: II.A1. Construction of a Pattern (FIG. 1A) A test pattern M is realized. To this end, the design of a grating on a plane, rigid support is preferably realized. The meshes of this grating may be square-shaped or rectangular. In one example, this pattern may be a design on a white base support of 1 m×1.50 m representing horizontal and vertical black bars forming squares as shown at M in FIG. 1A. II.A2. Calibration of the Camera (FIG. 1B, FIG. 1C, FIG. 3A) The camera is placed to face the plane of the pattern M for acquisition of a net image of this pattern. A calibration is realized before pick-up. This calibration consists of: rendering the plane support of the pattern M perpendicular to the optical axis X'x of the pick-up camera, as illustrated in FIG. 1B; rendering the bars of the grating parallel to the rows and columns of the CCD element of the camera 2, as illustrated in FIG. 1C; and placing the plane support of the grating in such a way that the whole image plane of the camera is covered by the grating squares, as illustrated in FIG. 3A. The calibration conditions thus essentially consist of verifying three conditions of orthogonality, two of which are verified when the optical axis X'X of the camera is perpendicular to the plane of the pattern and the third is verified when the bars of the pattern grating are parallel to the orthogonal rows and columns of the CCD element of the camera. The calibration operation may be performed by simply using squares. This calibration is not coercive because it appears that the method according to the invention can withstand possible imperfections of this calibration. II.A3. Acquisition of an Image of the Grating, Serving as a Pattern (FIG. 1A and FIG. 3A), Referred to as Source Grating By means of the camera 2 provided with its optical lens system 1, focusing and pickup is realized of the pattern grating M. The digitization system 3 provides a digitized image in the form of a grating referred to as source grating SG The distortions may be pincushion-shaped as in FIG. 5A or barrel-shaped as in FIGS. 5B and 3A. II.B. EXTRACTION OF THE REFERENCE POINTS R An image of the distorted source grating is digitized and stored in the digitization and storage means 3 shown in FIG. 1. With reference to FIG. 3A, the points of intersection of the vertical and horizontal bars of the distorted source grating SG As far as the notations are concerned: for the reference points R for the images of the source grating, for example, SG With reference to FIG. 2A, the series of operations constituting the steps of the method according to the invention comprises a first step performed in the block 41 consisting of the extraction of reference points R filtering of the source image for increasing the intensity of the zones of intersection, thresholding the filtered image intensities; and labelling the thresholded points for forming zones and extracting the barycenter of each zone. II.B1. Raising the Level of Intensity of the Zones of Intersection by Means of Filtering (FIG. 3b; FIG. 2A, Block 41) In the case taken as an example, in which the bars of the test pattern grating are represented in black on a white or bright background, the zones of intersection of the vertical and horizontal bars are black as shown in FIG. 3A. In this sub-step, the digitized image of the distorted pattern SG This operation may be carried out by means of a first method, known to those skilled in the art, consisting of a linear filtering operation by means of linear filters corresponding to correlation masks having the form of a cross. Instead of a test pattern in the form of a grating, it is also possible to choose a test pattern of dots. Experience has proved that camera registration of a dotted test pattern is less satisfactory than that in the form of a grating because of glare effects. The registration of a test pattern in the form of a grating has the additional advantage that more useful information can be supplied for satisfactorily performing the method according to the invention. Here, a method is proposed which uses a non-linear filtering operation which is more efficient than the known linear filtering operation. The advantages of this non-linear filtering operation with respect to a conventional linear filter are that: it produces no glare, it can withstand the distortion at the intersection, and it is easy to carry out. In the step of extracting the reference points, performed on the digitized source image SG An intersection of a horizontal grating bar and a vertical grating bar is shown diagrammatically in FIG. 4A. The edges of the horizontal bar and the vertical bar are shown by way of distorted lines in the distorted source image. In this step, a filter is used at each point of the source image SG the center C of the filter, four cardinal points S, N, E, W, and four diagonal points SE, SW, NE, NW. The 8 points surrounding the center C of the filter are entirely defined by the distances d1 and d2, where d1 is the distance measured in pixels between the center C and the cardinal points (d1=C-S, C-N, C-E, C-W), and d2 is the distance measured in pixels between the center C and the diagonal points (d2=C-SE, C-SW, C-NE, C-NW). The distance d1 is chosen in such a way that the cardinal points are situated within the design of the bars of the grating, where the center C is situated in the zone of intersection. This result is obtained by choosing d1 to be of the order of 1 to 5 pixels; generally, of the order of half the thickness of a bar of the grating measured in pixels in the distorted image. The distance d2 is chosen in such a way that the corresponding diagonal points are situated at the bottom of the image, that is, outside the regions of the grating bars. This effect is obtained by choosing d2, measured in pixels, to be of the order of half the pitch of the distorted grating in the digitized image. For example, for a grating having a pitch of the order of 50 pixels, d2 may be 20 pixels. By choosing reasonable distances d1 and d2 which those skilled in the art can determine by means of several routine tests without any precision being required, this filtering operation may reveal that, when the center of the filter is within a zone of intersection, the cardinal points S, N, E, W are within the regions of the bar design of the grating and the diagonal points are effectively situated at the bottom, even in the regions of the distorted image where the strongest distortions are found. In the example of the grating formed from black bars on a white background before filtering in the image SG By way of the non-linear filtering operation according to the invention, a measurement is realized which is expressed by the following measurement of the FILT criterion: FILT=Min(NW,NE,SE,SW)-Max(C,N,S,E,W). In this criterion, Min(NW,NE,SW,SE) means that the minimum intensity relating to the diagonal points is evaluated, and Max(C,N,S,E,W) means that the maximum intensity relating to the cardinal points including the center is evaluated. When the filter is correctly centered at an intersection, each diagonal point normally has a large intensity, so the minimum intensity evaluated for the intensity of these points is even larger. On the other hand, each of the five points, the center and the cardinal points normally has a smaller intensity, so the maximum of the intensity evaluated for the intensities of these points is even lower. The result is that there is a large difference between this Min and this Max. The non-linear filter realizes the evaluation of this FILT criterion at each point of the distorted image of FIG. 3A. The result of computing the FILT criterion constitutes the output of the non-linear filter according to the invention. The detected regions of intersection are those where the measurement of this criterion is largest. The obtained image resulting from this filtering operation is a new source grating SG II.B2 Thresholding the Increased Intensities of the Points (FIG. 2A, Block 41; FIG. 3C) After the sub-step of increasing the intensity of the points of the zones of intersection, a sub-step of thresholding the intensity of these points is realized. Here, a method will be proposed for performing said thresholding sub-step in order to detect said reference points. In accordance with this method, said thresholding sub-step is performed in the enhanced image SG Those skilled in the art, who have acquired the image of the distorted source grating SG counting of the number N1 of the intersections in the image, counting, by taking the digitization into account, of the approximate number N2 of pixels contained in one zone of intersection: each zone of intersection has four sides and its surface is given by the product of thicknesses in pixels of the horizontal and vertical bars of the grating in the distorted image, as is shown, for example, in FIG. 4A, and computing the searched number Nb equal to the product of the number of intersections N1 in the distorted image by the approximate number of pixels N2 in a zone of intersection, this number Nb constituting the threshold Nb=N1×N2. With the aid of this threshold, the pixels pertaining to the zones of intersection are extracted. This extraction consists of retaining a number equal to the computed number Nb among the pixels having the highest intensities after the filtering operation in the enhanced image SG To this end, a histogram is realized by accounting, for each intensity level, for the number of pixels having this intensity. Starting from the highest intensity level, the threshold operation consists of summing the number of pixels per decreasing level of intensity and of terminating the summation when the number Nb threshold is reached. The pixels having the highest intensity are thus retained and their intensity and address data are stored. At the start of this operation, the image realized from the test grating is only formed of bright zones containing a certain number of pixels which have a higher or equal intensity as compared with the intensity corresponding to the threshold Nb in the histogram, while the background is uniformly dark and deprived of bar details which would be seen after filtering of the image SG II.B3. Labelling the Thresholded Points and Computing the Barycenter of the Zones of Intersection (FIG. 3C; FIG. 2A, Block 41) The pixels extracted during the previous thresholding sub-step are regrouped by means of a labelling method. The zones of intersection, each containing several regrouped thresholded pixels, are identified and then their barycenter is computed. For computing the barycenter, each zone of intersection is derived to a single point defined by a pair of coordinates which are not necessarily integer numbers because they are the result of a computation of the barycenter of a zone comprising several pixels. With reference to FIG. 3C, the barycenters thus extracted constitute points which will hereinafter be referred to as reference points R An address in the source image SG In FIG. 3C, the barycenters of extracted reference points R II.C. ESTIMATION OF A FIRST THEORETICAL GRATING (FIG. 2A, Block 42) The inventive method now comprises steps for constructing a target image from a grating, referred to as theoretical grating TG At the start of these steps, the method according to the invention provides transform functions permitting the construction of this corrected theoretical grating TG According to the invention, the corrected theoretical grating TG To arrive at a precise construction of this improved theoretical grating TG estimation of a point referred to as center of the target grating, denoted GC°, localized at an intersection of a horizontal and a vertical bar of this first theoretical grating. It is referenced by means of its pixel coordinates, gcx determination of the position of an optical center OC estimation of a pitch GM causing the points TR In FIG. 3C and further Figures illustrating the present description, the reference points R 11.C1. Estimation of a Center GC This first theoretical grating is entirely defined as soon as a center GC In this step, the estimation of a center GC By applying the hypothesis of radial distortions, it appears that the real center of distortion of the source grating SG The estimation of a center GC an estimation of a starting point of reference R the transfer of this starting point R II.C1 a. Estimation of a Reference Point R In a general manner, any function capable of furnishing the estimation of a reference point denoted R With reference to FIG. 4B, a reference point R a horizontal sub-assembly denoted HS a vertical sub-assembly denoted VS Among all the reference points of the source grating SG For determining the reference points R According to the invention, an alignment criterion with respect to a horizontal band and a vertical band of lengths equal to the horizontal and vertical components of the approximate pitch GM It follows from the radial hypothesis that the nearer the tested reference point R With reference to the diagram of FIG. 4B, the following alignment criterion is proposed:
AL(ka)=Σ (0,5 gmx in which rx The proposed criterion AL(ka) is computed for each assembly S the larger the measurement represented by this criterion AL(ka), the more points exist which relate to this assembly S this criterion is thus a naturally satisfactory description of a slightly distorted region. In this computation of the alignment criterion AL(ka), the reference points of the assembly S II.C1 b. Transfer of the Retained Reference Point R The reference points R An optical center OC I.C2. Estimation of a Pitch GM The approximate value GM a first filtering operation for determining a growth value of a pitch GM°° of the first theoretical grating, and a second filtering operation for determining a more precise value of a pitch GM II.C2 a. First Filtering Operation for Determining a Growth Value of a Pitch GM°° for the First Theoretical Grating (FIG. 4C) In the previous sub-step, a center of the grating GC With reference to FIG. 4B, an assembly of reference points S a horizontal sub-assembly denoted HS a vertical sub-assembly denoted VS With reference to FIG. 4C, the first gross computation of the two components gmx°° and gmy°° of a pitch for the first theoretical grating is performed by: computing the component gmx°° along the Ωx axis of the fixed reference frame as the result of the filtering operation by means of a median filter applied to the distances HD computing the component gmy°° along the Ωy axis of the fixed reference frame as the result of the filtering operation by means of a median filter applied to the distances between the reference points of the vertical sub-assembly HS To this end, the reference points are arranged in the horizontal sub-assembly HS The median filters have the advantage that they furnish results which are resistant to localization errors of the points processed. The result remains particularly correct in the horizontal or vertical band of processed points even when a point is missing because it has not been extracted or when an extracted point exists by mistake. The median filters give better results than the simple computations of average values. The measurements effected in this first phase of the filtering operation constitute a strong pair gmx°°, gmy°° for a gross value of the pitch in the construction of the first theoretical grating. However, this gross value does not have the sufficient precision for constructing the first theoretical grating. For this reason, a second filtering operation is performed for providing a value of the grating pitch which is sufficiently precise to start this construction. II.C2 b. Second Filtering Operation for Determining a More Precise Value of a Pitch GM With reference to FIG. 4D, a sub-assembly S A median filter is used again which now acts on the values of the distances between the reference points thus defined. In this second filtering operation, 6 horizontal intervals HS It will be noted that in each of these two filtering operations, reference points used for passing the median filters may be erroneous or missing. However, due to the fact that these first and second filtering operations are based on the selection of median values of the intervals considered, the results are not notably affected. At the end of this sub-step, performed in two operations, the result is: a strong measurement, and a sufficiently precise measurement of a pitch GM°=(gmx°, gmy°) for the construction of the first theoretical grating. These data are stored. In the method according to the invention, the strength of the image starting data for constructing the first theoretical grating, namely: the center of the grating GC the optical center OC the pitch of the grating GM°, are very important because the invention envisages to provide an automatic method of determining the correction functions. II.C3. Estimation of First Pairs P With the knowledge of: coordinates gcx°, gcy components gmx°, gmy in the fixed reference frame Ωx, Ωy, the first theoretical grating is constructed by placing, in this fixed reference frame, the intersections of its bars denoted TR In this step, reference points, as shown in FIG. 3C, are used for the source image SG This correspondence is written in the form of:
P in which f and g are functions of correspondence between the reference points R For example, R In the region of small distortion near the center of the grating GC°, the operation of realizing correspondence consists of selecting the pairs P As one moves away from the center, this manner of operation can no longer be used because the points of the pairs will be increasingly further away from each other and towards the edges of the source image SG With the object of solving this problem, one starts from the center of the grating GC In accordance with this method, for constructing pairs P Thus, at the same time, the source grating SG the first ring is composed of the center of the grating GC the second rectangular ring is composed of its 8 direct neighbors of GC°, and the third rectangular ring is composed of its 16 second neighbors of GC°. In FIG. 4E, the points TR The more the ring is remote from the center of the grating GC°, the more pairs P With reference to FIG. 6A, in a current rectangular ring α and after a reference point R
V This vector V With reference to FIG. 6B, in a subsequent ring β which is further remote from the center of the grating GC°, a reference point is searched in the source image SG A new error vector V A particular proximity criterion has thus been defined according to which, on the basis of an error vector between the points of a pair in a ring, the pairs are formed in the next ring which is further remote from the center and the new error vector is measured which has become larger in this subsequent ring. And based on the error vector measured in the preceding ring, and also step by step, the method is repeated until the size of the rings is equal to the size of the image. II-D CONSTRUCTION OF A THEORETICAL GRATING BY MEANS OF AN ITERATION PROCESS BASED ON THE HYPOTHESIS OF RADIAL DISTORTIONS (FIG. 2A, Blocks 43 to 46) Based on the characteristics of the first theoretical grating TG°, the method according to the invention consists of constructing the best possible theoretical grating now, that is to say, constructing a theoretical grating TG To solve this problem, the method according to the invention is iterative. At each iteration, this method comprises the steps of: computing a polynomial F computing the patterning error E modification of the optical center OC estimation of a new theoretical grating TG In the previous steps, a zero power index (0) is assigned to all the notations relating to the first estimations of the center GC°, the pitch GM With reference to FIG. 2A, this iterative method forms a loop. Supposing that the construction elements of a first theoretical grating, such as TG a polynomial F° for the correction of radial distortions is first computed in the block 44, a patterning error E° is subsequently computed in the block 45, the position of the approximate optical center, which was OC having modified the optical center, which is now denoted OC at this instant, the iteration method is continued, in the block 44, by computing the correction polynomial F The iteration n will now be performed at the output of the block 46 of the loop after the position of the optical center has been modified, thus becoming OC This knowledge of the new optical center OC In the same block 43, the new points TR II.D1. Precise Estimation of the Grating Center GC With reference to FIG. 2A, in the block 43, at iteration n, one starts from the fact that the optical center OC One also starts from the pairs of points P
P where the power indexes indicate the iteration number n-1 at which these values have been determined. One starts also from the fact that: the center of the grating GC the pitch of the theoretical grating GM At the iteration n: an improved grating center GC and an improved grating step GM To this end, the basic hypothesis is used, which is referred to as radial hypothesis, according to which, if the optical center OC This indicates that the points TR FIG. 7A shows, from the block 46, the optical center OC In this step of the method, at the iteration n, this geometrical distance Δ More precisely, a distance:
Δ is defined which is the geometrical distance, stored at the iteration n-1, between a point TR In the fixed reference frame Ωx, Ωy, this can be written as:
TR In this formula, the grating points TR In this formula, the parameters λ For example, for realizing FIG. 7A, it has been supposed that the grating point is at λ The center GC This radial criterion can be written as:
Φ In order that this criterion Φ Any method of minimization leading to a non-singular solution is satisfactory and may be used. According to the invention, a method is proposed which includes the following steps: start with a reasonable value of the pitch of the grating GM minimization of the radial criterion Φ minimization of the radial criterion Φ repetition of these operations as many times as is necessary until a required precision has been achieved. II-D2. Estimation at the Iteration N of Pairs of Reference Points and Corresponding Theoretical Grating Points (FIG. 2A, Block 43) This operation is performed by carrying out exactly the same procedure as for the first theoretical grating, except that the values of the center and the pitch GC At the iteration n, in the block 43 of FIG. 2A, there is now in the fixed reference frame Ωx, Ωy: an update of the coordinates of the center of the grating GC an update of the components of the pitch of the grating GM the position of the optical center OC These characteristics allow the construction of a new updated theoretical grating TG The method of causing the reference points R This new operation of providing correspondence between the grating points of the theoretical grating TG II-E COMPUTATION OF A RADIAL CORRECTION POLYNOMIAL (FIG. 2A, Block 44) Starting from the pairs P a first radius K a second radius Y With reference to FIG. 7B, these radii are expressed by:
X
Y in which (TR These pairs of radii are computed for each pair P A polynomial function F With reference to FIG. 8A, radii X With reference to FIG. 8B, a graph shows the axes of coordinates X and Y graduated in a number of pixels, with six pairs 0P With reference to FIG. 8C, a graph shows the axes of coordinates X and Y graduated in a number of pixels, with six other pairs P With reference to FIGS. 8A to 8C, it is found that the function F With reference to FIG. 8A, at the iteration n, the error E
E It is important to note that the polynomial F a point TR to a reference point R This polynomial F II-F. COMPUTATION OF THE PATTERNING ERROR (FIG. 2A, Block 45) With reference to FIG. 2A, in the chain constituting the iteration n, and at the step symbolized by the block 45, the best radial correction polynomial F A patterning error E This minimal error E II-G. MODIFICATION OF THE OPTICAL CENTER OF THE THEORETICAL GRATING (FIG. 2A, Block 46) The patterning error E In the iteration chain shown in FIG. 2A, a loop is found which is concretely shown as the block 46 whose input receives the data of the block 45 for computing the error energy and whose output supplies to the block 43 a modified optical center OC It will be recalled that, according to the invention, the minimization of the energy of the error E To this end, at each iteration from, for example, 0 to 10, the position of the optical center OC
OC is modified for cooperating in the minimization of the patterning error E In a first method, a large search zone is defined around the initial optical center denoted OC A second method, which is less costly as far as the computation time is concerned, is based on the computation of gradients. In two successive iterations, a first localization and then a second localization are tested in a given direction, starting from the first optical center OC°. According as these first tests contribute to a decrease or an increase of the patterning error E The number Nlast of the iteration, at which the position of this best optical center is formed for minimizing the patterning error, is stored. II-H. OUTPUT OF THE ITERATION CHAIN Under these conditions, the best update conditions at the output of the block 45, shown in FIG. 2A, correspond to an iteration in which the conditions found at Nopt and the conditions found at Nlast are made to cooperate. This iteration will be the last iteration: it is numbered Nopt(Nlast). The best corresponding polynomial function for minimizing the patterning error is then written as:
G=F and its rectangular components are two polynomials written as: ##EQU1## These components are polynomials having two coordinates because each must be applied to a point of the pair P With reference to FIG. 7C and FIG. 3D, it appears that after the iteration operations, which have led to the determination of conditions corresponding to the last iteration Nopt(Nlast), the optical center now has a position:
OC and the grating points TR With reference to FIG. 8D, the points of the pairs P An ideal theoretical grating corresponding to these conditions is shown in FIGS. 3F and 3G, TG Once the polynomial function G, or the components G The knowledge of the functions G, or G III. CORRECTION OF THE OPTICAL DISTORTIONS IN AN IMAGE (FIG. 1, Sub-assembly 47; and FIG. 2A, Sub-assembly 47, and FIG. 2B) As stated hereinbefore, the components G In the target image TI to be constructed, the coordinates of each pixel X The value of the intensity at the monochromatic light level, or grey level, which must be attributed to the current pixel X
X
Y As the point found at the address X Patent Citations
Non-Patent Citations
Referenced by
Classifications
Legal Events
Rotate |