US 8032327 B2 Abstract A method for obtaining three-dimensional surface points of an object in an object coordinate system having two groups of steps. The method includes providing a set of target positioning features on the object. In a first group of steps, acquiring 2D first images of the object, extracting 2D positioning features; calculating a first set of calculated 3D positioning features; computing first transformation parameters, cumulating the first set of transformed 3D positioning features to provide and augment the set of reference 3D positioning features. In a second group of steps, providing a projected pattern on a surface of the object; acquiring 2D second images of the object, extracting 2D surface points and second sets of 2D positioning features; calculating a set of 3D surface points; calculating a second set of calculated 3D positioning features; computing second transformation parameters, transforming the 3D surface points into transformed 3D surface points.
Claims(5) 1. A method for obtaining three-dimensional surface points of an object in an object coordinate system, comprising:
providing a set of target positioning features on said object, each of said target positioning features being provided at a fixed position on said object, said object coordinate system being defined using said target positioning features, said target positioning features being provided by one of a set of external fixed projectors projecting said features and fixedly secured features;
in a first group of steps:
acquiring at least a pair of 2D first images of said object by at least a pair of cameras, at least a first portion of said set of target positioning features being apparent on said pair of first images, each said pair of said 2D first images being acquired from a corresponding first pair of viewpoints referenced in a sensing device coordinate system;
using an electronic chip in communication with said at least said pair of cameras for implementing the steps of:
extracting, from said 2D first images, at least two first sets of 2D positioning features from a reflection of said target positioning features of said first portion on said surface;
calculating a first set of calculated 3D positioning features in said sensing device coordinate system using said first sets of 2D positioning features;
computing first transformation parameters for characterizing a current spatial relationship between said sensing device coordinate system and said object coordinate system, by matching corresponding features between said first set of calculated 3D positioning features in said sensing device coordinate system and a set of reference 3D positioning features in said object coordinate system, said reference 3D positioning features being cumulated from previous observations;
transforming said first set of calculated 3D positioning features into a first set of transformed 3D positioning features in said object coordinate system using said first transformation parameters;
cumulating said first set of transformed 3D positioning features to provide and augment said set of reference 3D positioning features;
in a second group of steps:
providing a projected pattern on a surface of said object using a pattern projector;
acquiring at least a pair of 2D second images of said object by said cameras, said projected pattern and at least a second portion of said set of target positioning features being apparent on said pair of second images, each said pair of said 2D second images being acquired from a corresponding second pair of viewpoints referenced in said sensing device coordinate system;
wherein said first portion of said set of target positioning features and said second portion of said set of target positioning features being one of the same, partly the same or different;
using said electronic chip for implementing the steps of:
extracting, from said 2D second images, at least one set of 2D surface points from a reflection of said projected pattern on said surface, and at least two second sets of 2D positioning features from a reflection of said target positioning features of said second portion on said surface;
calculating a set of 3D surface points in said sensing device coordinate system using said set of 2D surface points;
calculating a second set of calculated 3D positioning features in said sensing device coordinate system using said second sets of 2D positioning features;
computing second transformation parameters for characterizing a current spatial relationship between said sensing device coordinate system and said object coordinate system, by matching corresponding features between said second set of calculated 3D positioning features in said sensing device coordinate system and said set of reference 3D positioning features in said object coordinate system;
transforming said set of 3D surface points into a set of transformed 3D surface points in said object coordinate system using said second transformation parameters.
2. The method as claimed in
transforming said second set of calculated 3D positioning features into a second set of transformed 3D positioning features in said object coordinate system using said second transformation parameters;
cumulating said second set of transformed 3D positioning features to provide and augment said set of reference 3D positioning features.
3. The method as claimed in
4. The method as claimed in
5. The method as claimed in
Description The present application is a continuation of U.S. patent application Ser. No. 11/817,300 filed Aug. 28, 2007 by Applicant, now U.S. Pat. No. 7,912,673, which is a national phase entry of PCT patent application no. PCT/CA06/00370 filed on Mar. 13, 2006, which in turns claims priority benefit on U.S. provisional patent application No. 60/660,471 filed Mar. 11, 2005, the specifications of which are hereby incorporated by reference. The present invention generally relates to the field of three-dimensional scanning of an object's surface geometry, and, more particularly, to a portable three-dimensional scanning apparatus for hand-held operations. Three-dimensional scanning and digitization of the surface geometry of objects is now commonly used in many industries and services and their applications are numerous. A few examples of such applications are: inspection and measurement of shape conformity in industrial production systems, digitization of clay models for industrial design and styling applications, reverse engineering of existing parts with complex geometry, interactive visualization of objects in multimedia applications, three-dimensional documentation of artwork and artifacts, human body scanning for better orthesis adaptation or biometry. There remains a need to improve the scanning devices used for 3D scanning of an object. According to one broad aspect, there is provided a method for obtaining three-dimensional surface points of an object in an object coordinate system having two groups of steps. The method first comprises providing a set of target positioning features on the object, each of the target positioning features being provided at a fixed position on the object, the object coordinate system being defined using the target positioning features, the target positioning features being provided by one of a set of external fixed projectors projecting the features and fixedly secured features. In one embodiment, the method comprises, in a first group of steps acquiring at least a pair of 2D first images of the object by at least a pair of cameras, at least a first portion of the set of target positioning features being apparent on the pair of first images, each pair of the 2D first images being acquired from a corresponding first pair of viewpoints referenced in a sensing device coordinate system; using an electronic chip in communication with the at least the pair of cameras for implementing the steps of: extracting, from the 2D first images, at least two first sets of 2D positioning features from a reflection of the target positioning features of the first portion on the surface; calculating a first set of calculated 3D positioning features in the sensing device coordinate system using the first sets of 2D positioning features; computing first transformation parameters for characterizing a current spatial relationship between the sensing device coordinate system and the object coordinate system, by matching corresponding features between the first set of calculated 3D positioning features in the sensing device coordinate system and a set of reference 3D positioning features in the object coordinate system, the reference 3D positioning features being cumulated from previous observations; transforming the first set of calculated 3D positioning features into a first set of transformed 3D positioning features in the object coordinate system using the first transformation parameters; cumulating the first set of transformed 3D positioning features to provide and augment the set of reference 3D positioning features. In one embodiment, the method comprises, in a second group of steps, providing a projected pattern on a surface of the object using a pattern projector; acquiring at least a pair of 2D second images of the object by the cameras, the projected pattern and at least a second portion of the set of target positioning features being apparent on the pair of second images, each pair of the 2D second images being acquired from a corresponding second pair of viewpoints referenced in the sensing device coordinate system; wherein the first portion of the set of target positioning features and the second portion of the set of target positioning features being one of the same, partly the same and different; using the electronic chip for implementing the steps of: extracting, from the 2D second images, at least one set of 2D surface points from a reflection of the projected pattern on the surface, and at least two second sets of 2D positioning features from a reflection of the target positioning features of the second portion on the surface; calculating a set of 3D surface points in the sensing device coordinate system using the set of 2D surface points; calculating a second set of calculated 3D positioning features in the sensing device coordinate system using the second sets of 2D positioning features; computing second transformation parameters for characterizing a current spatial relationship between the sensing device coordinate system and the object coordinate system, by matching corresponding features between the second set of calculated 3D positioning features in the sensing device coordinate system and the set of reference 3D positioning features in the object coordinate system; transforming the set of 3D surface points into a set of transformed 3D surface points in the object coordinate system using the second transformation parameters. According to another broad aspect, there is provided a method for obtaining three-dimensional surface points of an object in an object coordinate system having two groups of steps. The method first comprises providing a set of target positioning features on the object. In a first group of steps, acquiring at least a pair of 2D first images of the object, at least a first portion of the set of target positioning features being apparent on the pair of first images, extracting, from the 2D first images, at least two first sets of 2D positioning features from a reflection of the target positioning features of the first portion on the surface; calculating a first set of calculated 3D positioning features in the sensing device coordinate system using the first sets of 2D positioning features; computing first transformation parameters for characterizing a current spatial relationship between the sensing device coordinate system and the object coordinate system, cumulating the first set of transformed 3D positioning features to provide and augment the set of reference 3D positioning features. In a second group of steps, providing a projected pattern on a surface of the object using a pattern projector; acquiring at least a pair of 2D second images of the object by the cameras, the projected pattern and at least a second portion of the set of target positioning features being apparent on the pair of second images, extracting, from the 2D second images, at least one set of 2D surface points from a reflection of the projected pattern on the surface, and at least two second sets of 2D positioning features from a reflection of the target positioning features of the second portion on the surface; calculating a set of 3D surface points in the sensing device coordinate system using the set of 2D surface points; calculating a second set of calculated 3D positioning features in the sensing device coordinate system using the second sets of 2D positioning features; computing second transformation parameters for characterizing a current spatial relationship between the sensing device coordinate system and the object coordinate system, transforming the set of 3D surface points into a set of transformed 3D surface points in the object coordinate system using the second transformation parameters. According to another broad aspect, there is provided a method for obtaining three-dimensional surface points of an object in an object coordinate system. The method comprises providing a projected pattern on a surface of the object using a pattern projector; providing a set of target positioning features on the object, each of the target positioning features being provided at a fixed position on the object, the object coordinate system being defined using the target positioning features, the target positioning features being provided by one of a set of external fixed projectors projecting the features and affixed features; acquiring at least a pair of 2D images of the object by at least a pair of cameras with a known spatial relationship, the projected pattern and at least a portion of the set of target positioning features being apparent on the images, each of the 2D images being acquired from a view point referenced in a sensing device coordinate system; using an electronic chip in communication with the at least the pair of cameras for implementing the steps of: extracting, from the 2D images, at least one set of 2D surface points from a reflection of the projected pattern on the surface, and at least two sets of 2D positioning features from a reflection of the target positioning features on the surface; calculating a set of 3D surface points in the sensing device coordinate system using the set of 2D surface points; calculating a set of calculated 3D positioning features in the sensing device coordinate system using the sets of 2D positioning features; computing transformation parameters for characterizing a current spatial relationship between the sensing device coordinate system and the object coordinate system, by matching corresponding features between the set of calculated 3D positioning features in the sensing device coordinate system and a set of reference 3D positioning features in the object coordinate system, the reference 3D positioning features being cumulated from previous observations; and transforming the set of 3D surface points into a set of transformed 3D surface points in the object coordinate system using the transformation parameters. In one embodiment, the electronic chip further implements the steps of: transforming the set of calculated 3D positioning features into a set of transformed 3D positioning features in the object coordinate system using the transformation parameters; and cumulating the set of transformed 3D positioning features to provide and augment the set of reference 3D positioning features. According to another broad aspect, there is provided a system for obtaining three-dimensional surface points of an object in an object coordinate system. The system comprises a set of target positioning features on the object, each of the target positioning features being provided at a fixed position on the object, the object coordinate system being defined using the target positioning features; a sensing device having a pattern projector for providing a projected pattern on a surface of the object, at least a pair of cameras each for acquiring at least one 2D image of the object, each of the 2D images being acquired from a viewpoint referenced in a sensing device coordinate system; an image processor for extracting, from the 2D images, at least one set of 2D surface points from a reflection of the projected pattern on the surface apparent on the images, and at least two sets of 2D positioning features from a reflection of at least a portion of the target positioning features on the surface apparent on the images; a 3D surface point calculator for calculating a set of 3D surface points in the sensing device coordinate system using the set of 2D surface points; a 3D positioning feature calculator for calculating a set of calculated 3D positioning features in the sensing device coordinate system using the sets of 2D positioning features; a positioning features matcher for computing transformation parameters to characterize a current spatial relationship between the sensing device coordinate system and the object coordinate system, by matching corresponding features between the set of calculated 3D positioning features in the sensing device coordinate system and a set of reference 3D positioning features in the object coordinate system, the reference 3D positioning features being cumulated from previous observations; a 3D surface point transformer for transforming the set of 3D surface points into a set of transformed 3D surface points in the object coordinate system using the transformation parameters. Having thus generally described the nature of the invention, reference will now be made to the accompanying drawings, showing by way of illustration a preferred embodiment thereof, and in which: The present device allows simultaneously scanning and modeling the object's surface while accumulating a second model of the positioning features in real-time using a single hand-held sensor. Furthermore, by fixing additional physical targets as positioning features on an object, it is possible to hold the object in one hand while holding the scanner in the second hand without depending on the object's surface geometry for the quality of the calculated sensor positions. Referring to Sensing Device The system comprises a sensing device The positioning features are secured on the object such that the object can be moved in space while the positioning features stay still on the object and, accordingly, with respect to the object's coordinate system. It allows the object to be moved in space while its surface is being scanned by the sensing device. Image Processor The image processor For a crosshair laser pattern, one can benefit from the architecture of the apparatus described thereafter. In this configuration with two cameras and a crosshair pattern projector, the cameras are aligned such that one among the two laser planes produces a single straight line in each camera at a constant position. This is the inactive laser plane for a given camera. These inactive laser planes are opposite for both cameras. This configuration, proposed by Hébert (see P. Hébert, “A Self-Referenced Hand-Held Range Sensor”. in proc. of the 3rd International Conference on 3D Digital Imaging and Modeling (3DIM 2001), 28 May-1 Jun. 2001, Quebec City, Canada, pp. 5-12) greatly simplifies the image processing task. It also simplifies the assignation of each set of 2D surface point to a laser plane of the crosshair. While the sets of surface points 3D Positioning Features Calculator Since the sensing device is calibrated, matched positioning features between camera viewpoints are used to estimate their 3D position using the 3D positioning features calculator 3D Surface Point Calculator The 3D surface point calculator It is also possible to avoid associating each 2D point to a specific structure of the laser pattern. This is particularly interesting for more complex or general patterns. In this case, it is still possible to calculate 3D surface points using the fundamental matrix and exploiting the epipolar constraint to match points. When this can be done without ambiguity, triangulation can be calculated in the same way it is applied by the 3D positioning features calculator The 3D surface point calculator Positioning Subsystem The task of the positioning subsystem, mainly implemented in the positioning features matcher At the beginning of a scanning session, the set of reference 3D positioning features After creation of the initial set of reference 3D positioning features The input to the positioning features matcher where ε is a predefined threshold which is set to correspond to the accuracy of the sensing device. This constraint imposes that the difference in distance between a corresponding pair of points in the two sets be negligible. This matching operation is solved as a combinatorial optimization problem where each segment of points from the set O is progressively matched against each segment of points in the set R. Each matched segment is then expanded by forming an additional segment using the remaining points in each of the two sets. If two segments satisfy the constraint (1), a third segment is formed and so on as long as the constraint is satisfied. Otherwise the pair is discarded and the next one is examined. The solution is the largest set of segments satisfying (1). Other algorithms (see M. Fischler and R. Bolles, (1981) “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography”, Communications of the Assoc. for Computing Machinery, (June 1981), vol. 24, no. 6, pp. 381-395.) can be used for the same purpose. As long as the number of elements in the set of reference 3D positioning features This means that if the calculated 3D positioning features Alternatively, exploiting spatial correlation of sensing device position and orientation can be used to improve matching speed. By assuming that the displacement of the sensing device is small with respect to the size of the set of positioning features, matching can be accomplished by finding the closest reference feature for each observed positioning feature. The same principle can be used in 2D, that is, by finding closest 2D positioning features. Once matching is done, the two sets need to be aligned by computing the optimal transformation parameters [M T], in the least-squares sense, such that the following cost function is minimized:
The transformation parameters consist of a 3×3 rotation matrix M and a 3×1 translation vector T. Such a transformation can be found using dual quaternions as described in M. W. Walker, L. Shao and R. A. Volz, “Estimating 3-D location parameters using dual number quaternions”, CVGIP: Image Understanding, vol. 54, no. 3, November 1991, pp. 358-367. In order to compute this transformation, at least three common positioning features have to be found. Otherwise both positioning features and surface points are discarded for the current frame. An alternative method for computing the rigid transformation is to minimize the distance between observed 2D positioning features
where p 3D Positioning Features Transformer Once the rigid transformation is computed, the 3D positioning features transformer 3D Surface Point Transformer The processing steps for the surface points are simple once the positioning features matcher Having described the system, a closer view of the sensing device is now detailed. In For a hand-held device, the baseline D While illustrated in the block diagrams as groups of discrete components communicating with each other via distinct data signal connections, it will be understood by those skilled in the art that the preferred embodiments are provided by a combination of hardware and software components, with some components being implemented by a given function or operation of a hardware or software system, and many of the data paths illustrated being implemented by data communication within a computer application or operating system. The structure illustrated is thus provided for efficiency of teaching the present preferred embodiment. One skilled in the art should understand that the positioning features, described herein as retro-reflective targets, could alternatively be provided by light sources, such as LEDs, disposed on the surface of the object to be scanned or elsewhere, or by any other means that provide targets to be detected by the sensing device. Additionally, the light sources provided on the sensing device could be omitted if the positioning features themselves provide the light to be detected by the cameras. It should be understood that the pattern projector hereinabove described as comprising a laser light source could also use a LED source or any other appropriate light source. It will be understood that numerous modifications thereto will appear to those skilled in the art. Accordingly, the above description and accompanying drawings should be taken as illustrative of the invention and not in a limiting sense. It will further be understood that it is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains and as may be applied to the essential features herein before set forth, and as follows in the scope of the appended claims. Patent Citations
Non-Patent Citations
Referenced by
Classifications
Legal Events
Rotate |