CA2078503C - Moving body recognition apparatus - Google Patents

Abstract

A moving body recognition apparatus recognizes a shape and movement of an object moving in relation to an image input unit by extracting feature points, e.g.
a peak of the object and a boundary of color, each in said images captured at a plurality of instants in time for observation by the image input unit. The moving body recognition apparatus comprises an image input unit for capturing images of an object as a moving body having a rotation, a feature point extraction unit for extracting feature points from the images inputted by the image input unit, a feature point storage unit for storing known position data of the extracted feature points, and a shape/movement recognition unit for calculating the actual positions and movements of the feature points of the object in the images by using the known position data of the extracted feature points outputted from the feature point storage unit.

Description

~ 20785~3 .

A MOVING BODY RECOGNITION APPARATUS

Background of the Invention Field of the Invention This invention pertains to a moving body recognition apparatus. A moving body recognition apparatus in many cases uses a TV camera as an image input unit receiving an image of an external object.

Description of the Related Arts Image processing devices are used widely, e.g.
for an FA (factory automation) inspection, an automatic monitoring device, and a visual sensor for an automatically operated vehicle, generally as devices capable of processing visual information as with a human being. There are potential demands for them and their research and development is actively conducted. Although an image processing device originally processed a still image as its object, recently it has put a greater emphasis on processing a moving image. Especially, those capable of recognizing a moving body have come to take a large share of all image processing devices.

*' 2078~03 A moving body recognition apparatus generally recognizes the shape of a moving body and its relative movement, when the image input unit of the moving body recognition apparatus is moving in relation to the object. That is, even if an object is not actually moving, when an image input unit of a moving body recognition apparatus moves, the moving body recognition apparatus recognizes the shape of the ob;ect standing still and the movement of the image input unit. For example, in a case of a visual sensor for an automatically operated vehicle, its image input unit is loaded on top of the vehicle, and the moving body recognition apparatus recognizes the environment in which the vehicle is running.
A moving body recognition apparatus must be compact and responsive. Compactness is critical, especially when a moving body recognition apparatus is loaded on a vehicle in its application to a visual sensor of an automatically operated vehicle.
Responsiveness is crucial, because a realtime processing similar to a human vision is required .
A conventional moving body recognition device captures an object by two [2] image input units. By establishing the correspondences between the feature points of the ob;ects in the two [2] images captured 2a78503 by the two [2] image input units, the shape of the object is captured at every certain instant in time for observation by applying a principle of a triangulation, and then the movement of the ob;ect is calculated.

Figure 1 shows a concept of a conventional moving body recognition apparatus.
A first image input unit 2 and a second image input unit 3 input images of an object 1 to a moving image recognition unit 4. The moving body recognition unit 4 detects feature points of an ob;ect 1 from the two [2] images. By matching a same feature point between the two [2] images, the position of a feature point is calculated by applying a principle of a triangulation. Here, a feature point refers to a point representing a particular part of the object 1. When there is a peak, an outline point or a pattern, a feature point may be a dot in a pattern or on a color boundary. The moving body recognition unit 4 calculates the movement of a feature point and the object 1 from a shift of the feature point in a time series. The moving body recognition apparatus outputs as a recognition result 5 the position and movement of a feature point and the ~078~3 movement of an object.

Figure 2 shows a configuration of a conventional moving body recognition apparatus.
A first feature point extraction unit 6 extracts a feature point from an image inputted by the first image input unit 2 and supplies it to a feature point correspondence unit 8. Likewise, a second feature point extraction unit 7 extracts a feature point from an image inputted by the second image input unit 3 and supplies it to the feature point correspondence unit 8. The feature point correspondence unit 8 matches the same feature points from among the feature points extracted from the first feature point extraction unit 6 and the second feature point extractlon unit 7.
A feature point position calculation unit 9 obtains the positions of feature points by relating the positions of the matched feature points with the positions of the first image input unit 2 and the second image input unit 3, and stores the result in a feature point position storage unit 10. The positions of feature points at plural instants in time for observation stored in the feature point position storage unit 10 are sent to an ob;ect movement calculation unit 11, which calculates the movement of `~

an object and stores the result in an object movement storage unit 12.

However, a conventional moving body recognition apparatus as explained in the description of Figures 1 and 2 have the following problems.
(a) Because two [2] feature point extraction units need to individually extract feature points extracted from two [2] images captured respectively by two [2]
image input units, the process load for extracting a feature point is twice as much as that by using a single TV camera.
(b) An additional process of matching features from two [2] images captured differently is required.
The feature point correspondence unit 8 is required as shown in Figure 2. Because the positions of two [2]
image input units are different, they capture the object l differently. This makes it difficult to match feature points of the object 1. Hence, the feature point correspondence unit 8 requires a large workload for searching for corresponding feature points. (The closer the positions of two [2] image input units, the easier it is to make correspondences between feature points of an ob;ect, but the less accurate a recognition of the shape of an object becomes.) ~785~3 Typically, processing in (b) is impossible when a feature point of an object captured by one [1] image input unit cannot be captured by the other, where no correspondence between those feature points can be made.

Figure 3 shows an example in which a conventional moving body recognition apparatus fails to establish correspondences between feature points captured by different image input units.
The object 1 has two [2] feature points, for instance. However, there is a case in which both the first image input unit 2 and the second image input unit 3 can capture only one [1] of the two [2]
feature points.

Summary of the Invention A prime object of this invention is to recognize a moving body by extracting a feature point from images at plural instants in time for observation inputted from a single image input unit. In order to recogni~e the movement of an ob;ect most generally in a movement coupled with a revolution, i.e. a rotation, from the position data of the four [4]
feature points of an object each in orthogonally `~
2078~3 projected images captured at any three [3] instants in time for observation, this invention invokes a determination e.g. that the four [4] feature points are not on a single plane, and then a calculation of the actual positions of the four [4] feature points.
However, since the required processes are complicated in a general case like this, this invention also aims at more easily recognizing the movement of a moving body despite some restrictions.
The above aims can be summarized as follows:
This invention aims at recognizing the movement of an obJect moving on a single plane coupled with a rotation at a constant rotating speed, from the already known position data of two [2] feature points each in images of the object captured from a direction perpendicular to the axis of rotation at three [3]
instants in time for observation having equal time intervals.
Also, this invention aims at recognizing the movement of an object moving on a single plane coupled with a rotation, from the already known position data of three [3] feature points forming a right angle each in images of the object captured from a direction perpendicular to the axis of rotation at any two [2]

~78503 instants in time for observation.
Further, this invention aims at recognizing the movement of an object moving on a single plane coupled with a rotation, from the already known position data of three [3] feature points each in images of the object captured from a direction perpendicular to the axis of rotation at any three [3] instants in time for observation.
Additionally, this invention aims at most generally recognizing the movement of a moving body.
It is to recognize the movement of an object moving three-dimensionally coupled with a rotation, from the already known position data of four [4] feature points each in images of the object captured at any three [3]
instants in time for observation, after determi ni ng e.g. that the four [4] feature points are not on a single plane.

This invention configures a moving body recognition apparatus to comprise a single image input unit for capturing an image of an ob~ect, a feature point extraction unit for extracting feature points in an image outputted from the image input unit, and a feature point storage unit for storing the extracted feature points, thereby enabling the movement of a `~ 2~78503 moving body to be recognized from the known posltlon data of the feature polnts.
According to one aspect, the present lnventlon provldes a movlng body recognltlon apparatus for recognlzlng a movement of a movlng ob~ect by positions of features polnts ln an lmage correspondlng to sald moving ob~ect, comprislngs lmage lnput means for capturlng three lmages of said movlng ob~ect at three instances ln tlme at equal tlme intervals, sald three images belng observed from a dlrectlon perpendlcular to an axls of rotation of sald ob~ect moving on a single plane and said ob~ect havlng a rotation at a constant rate; feature point extractlon means for extracting two feature points each ln sald three lmages captured by sald lmage lnput means, sald feature polnt extractlon means lncludlng space fllter means for extractlng an outllne lmage from sald three lmages captured by said image input means and edge polnt extractlon means for extractlng an edge polnt from sald outllne lmage; feature polnt posltlon storage means for storlng known positlon data correspondlng to said extracted two feature polnts and for storlng said edge polnt as feature polnt dataS and shapetmovement recognition means for calculating actual positions of said extracted two feature points and a movement of said ob~ect from said known position data.
A feature of a second form of thls invention resides in a moving body recognition apparatus for g ~785~3 recognizing a movement of a moving body by positions of feature points on the moving body, comprising: an image input unit for capturing images of the moving body as an ob;ect; a feature point extraction unit for extracting feature points in the images captured by the image input unit; a feature point position storage unit for storing known position data of extracted feature points; and a shape/movement recognition unit for calculating the actual positions and movements of feature points of the object from known position data of three [3] feature points forming a right angle of the object each in the images captured at any two [2]
instants in time for observation from a direction perpendicular to the axis of rotation, i.e.
revolution, of an object moving on a single plane coupled with a rotation by using an output from the feature point position storage unit.

A feature of a third form o this invention resides in a moving body recognition apparatus for recognizing a movement of a moving body by positions of feature points on the moving body, comprising: an image input unit for capturing images of the moving body as an ob~ect; a feature point extraction unit for extracting feature points in the images captured by 11 2~78503 the image input unit; a feature point position storage unit for storing known position data of extracted feature points; and a shape/movement recognition unit for calculating the actual positions and movements of feature points of the object from known position data of three [3] feature points of the ob;ect each in the images captured at any three [3] instants in time for observation from a direction perpendicular to the axis of rotation, i.e. revolution, of an object moving on a single plane coupled with a rotation by using an output from the feature point position storage unit.

A feature of a fourth form of this invention resides in a moving body recognition apparatus for recognizing a movement of a moving body by positions of feature points on the moving body, comprising: an image input unit for capturing images of the moving body as an ob~ect; a feature point extraction unit for extracting feature points in the images captured by the image input unit; a feature point position storage unit for storing known position data of extracted feature points; and a shape/movement recognition unit for calculating the actual positions and movements of feature points of the object from known position data of four [4] feature points of the object each in the 207~5~3 lmages captured at any three [3] lnstants ln tlme for observatlon, by determlnlng that the four [4] feature polnts do not exist on a single plane, that the axls of rotatlon, i.e. revolutlon, of the ob~ect ls not parallel to the directlon of an orthogonal pro~ectlon of the ob~ect between any two [2] of the three [3] instants ln time for observatlon, and that a rotation of the ob~ect between any two [2] of the three [3] in~tants in time for observation is not a rotatlon by an angle of one hundred and eighty degrees [180] around an axis parallel to a plane on which the orthogonal pro~ection is made.
According to another aspect, the present lnvention provides a moving body recognition apparatus for recognizing a movement of a moving ob~ect by positio~s of feature polnts ln images of said movlng ob~ect, comprlslng: lmage lnput means for capturlng three lmages of sald movlng ob~ect at three instances ln t~me at equal tlme lntervals, said image input means located at a posltion perpendicular to the axis of rotatlon of sald ob~ect moving on a slngle plane and sald ob~ect havlng a rotatlon at a constant rate; feature polnt extractlon means for extractlng two feature polnts each ln said three images captured by said image input means, said feature point extraction means including space filter means for extracting an outline image from said three images captured by said image input means and edge point extraction means for extracting an edge polnt from said outllne lmage;
feature point positlon storage means for storing known position data of said extracted two feature points and for ` ~ 2~78~3 storlng said edge point a~ feature polnt datai and movement recognltion means for calculating the movement of ~ald ob~ect from ~ald known posltlon data.

- 12a -L

~ .
13 ~ 3 Brief Description o the Drawings One of skill in the art can easily understand additional features and objects of this invention from the description of the preferred embodiments and some of the attached drawings. In the drawings:
Figure 1 shows a concept of a conventional moving body recognition apparatus;
Figure 2 shows a configuration of a conventional moving body recognition apparatus;
Figure 3 shows an example in which a conventional moving body recognition apparatus fails to establish correspondences between feature points captured by different image input units;
Figure 4 is a block diagram of a moving body recognition apparatus of this invention;
Figure 5 shows a universal relation between an object and its observation plane in an image input unit pursuant to the first form of this invention;
Figure 6 shows a relation between an object and its observation plane pursuant to the first form of this invention, in which feature point O is fixed to the origin of a three-dimensional coordinate system;
Figure 7 illustrates the orthogonal projections of feature points O and 1 shown in Figure 6 on the XY
plane at three [3] instants in time for observation pursuant to the first form of this invention;

2~7~3~

Figures 8A and 8B show two [2] sets of solutions forming mirror image transformations of each other with respect to the X axis pursuant to the first form of this invention;
Figure 9 is an explanatory chart showing the concept of a moving body recognition apparatus of this invention;
Figure 10 is a block diagram showing the global configuration of a moving body recognition apparatus o this invention;
Figure 11 is a block diagram illustrating in detail the confiyuration of a feature point extraction unit 25 shown in Figure 10;
Figure 12 is a block diagram illustrating in detail the configuration of the shape/movement recognition unit 27 shown in Figure 10;
Figure 13 illustrates an embodiment of sensor connections for the known data input unit 31 shown in Figure 12;
Figure 14 is a first one of flowcharts showing in a three part series the whole processes of the movement/shape recognition unit 27 pursuant to the first orm of this invention;
Figure 15 is a second one of flowcharts showing in a three part series the whole processes of the 15 207~7S~3 movement/shape recognition unit 27 pursuant to the first form of this invention;
Figure 16 is a third one of flowcharts showing in a three part series the whole processes of the movement/shape recognition unit 27 pursuant to the first form of this invention;
Figure 17 is a flowchart of an embodiment of recognition disablement process {1} pursuant to the first form of this invention;
Figure 18 is a flowchart of the movement calculation unit 34 pursuant to the first form of this invention;
Figure 19 is an explanatory chart for the method of selecting the appropriate one [1] of two [2]
rotation matrices R and R-l;
Figure 20 is a flowchart of the shape calculation unit 35 pursuant to the first form of this invention;
Figures 21A and 21B are explanatory charts for concave/convex data of an object;
Figure 22 is a flowchart of an embodiment of recognition disablement process ~2} pursuant to the first form of this invention;
Figure 23 shows a universal relation between an ob~ect and its observation plane in an image input unit pursuant to the second form of this invention;

16 2078~03 Figure 24 shows a relation between an object and its observation plane pursuant to the second form of this invention, in which feature point 0 is fixed to the origin of a three-dimensional coordinate system;
Figure 25 shows that edges forming a right angle in a single image cannot produce a set of deinite solutions pursuant to the second form of this nventlon;
Figure 26 illustrates the orthogonal projections of feature points 0, 1 and 2 shown in Figure 34 on the XY plane at two [2] instants in time for observation pursuant to the second form of this invention;
Figure 27 shows two [2] sets of solutions forming mirror image transformations of each other with respect to the X axis pursuant to the second form of this invention;
Figure 28 illustrates the meaning of expression {7} or use in theorem 9;
Figure 29 illustrates a method for determining a value range for angle a;
Figure 30 illustrates a method for determining a value range for angle ~;
Figure 31 illustrates a method for determining the sign of sin~, when m-n is odd;
Figure 32 illustrates a method for determining A

17 2078S~3 the sign of sin~, when m-n is even;
Figure 33 shows a relation among m, n and the sign of sin~, when m-n is even;
Figure 34 illustrates a method for determining the sign of sin~, when m-n-0;
Figure 35 illustrates a method for determining the sign of sin~, when m-n=2;
Figure 36 illustrates a method for determining the sign of sin~, when m-n=-2;
Figure 37 is a first one of flowcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the second form o this invention;
Figure 38 is a second one of flowcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the second form of this invention;
-~ Figure 39 is a third one of flowcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the second form of this invention;
Figure 40 is a flowchart of an embodiment of recognition disablement process {1} pursuant to the second form of this invention;
Figure 41 is a flowchart of the movement ~ .
18 2 07 85 0~

calculation unit 34 pursuant to the second form of this invention;
Figure 42 is a flowchart of an embodiment of determining the sign of sin~;
Figure 43 is a flowchart of the shape calculation unit 35 pursuant to the second form of this invention;
Figure 44 is a flowchart of an embodiment of shape calculation process {1};
Figure 45 is a flowchart of an embodiment of shape calculation process {2};
Figure 46 is a flowchart of an embodiment of shape calculation process {3};
Figure 47 is a flowchart of an embodiment of recognition disablement process {2} pursuant to the second form of this invention;
Figure 48 illustrates the orthogonal projections of feature points 0, 1 and 2 shown in Figure 24 on the XY plane at three [3] instants in time for observation pursuant to the third form of this invention;
Figures 49A and 49B show two [2] sets of solutions forming mirror image transformations of each other with respect to the X axis pursuant to the third form of this invention;
Figure 50 is a first one of flowcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the third _orm of this invention;
Figure 51 is a second one of flowcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the third form of this invention;
Figure 52 is a third one of flowcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the third form of this invention;
Figure 53 is a flowchart of an embodiment of recognition disablement process {1} pursuant to the third form of this invention;
Figure 54 is a flowchart of the movement calculation unit 34 pursuant to the third form of this invention;
Figure 55 is a flowchart of the shape calculation unit 35 pursuant to the third form of this invention;
Figure 56 is a flowchart of an embodiment of recognition disablement process {2} pursuant to the third form of this invention;
Figure 57 shows a universal relation between an object and its observation plane in an image input unit pursuant to the fourth form of this invention;
Figure 58 shows a relation between an object and 078~03 its observation plane pursuant to the fourth form o thls invention, in which feature point 0 is fixed to the origin of a three-dimensional coordinate system;
Figure 59 shows orthogonal projections of feature points on the XZ plane pursuant to the fourth form of this invention;
Figures 60A and 60B show two [2] sets of solutions forming mirror image transformations of each other with respect to the Xz plane on which an image is proiected pursuant to the fourth form of this invention;
Figure 61 is a block diagram of a shape/movement recognition unit 27 pursuant to the fourth form of this invention;
Figure 62 is a first one of 1Owcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the fourth form of this invention;
Figure G3 is a second one of flowcharts showing in a three part series the whole processes o the shape/movement recognition unit 27 pursuant to the fourth form of this invention;
Figure 64 is a third one of flowcharts showing in a three part serles the whole processes of the shape/movement recognition unit 27 pursuant to the 21 207~5~3 fourth form of this invention;
Figure 65 is a flowchart of an embodiment of recognition disablement process {l} pursuant to the fourth form of this invention;
Figure 66 is a first one of flowcharts showing in a two part series the whole processes of the movement/shape calculation unit 39 pursuant to the fourth form of this invention;
Figure 67 is a second one of flowcharts showing in a two part series the whole processes of the movement/shape calculation unit 39 pursuant to the fourth form of this invention;
Figure 68 is a flowchart showing processes for calculating elements at intersections between the first and second rows and the first and second columns of a rotation matrix R comprising three [3] rows and three [3] columns;
Figure 69 is a flowchart of an embodiment of recognition disablement process {2} pursuant to the fourth form of this invention;
Figure 70 is a block diagram of a computer system embodying a moving body recognition apparatus of this invention; and Figure 71 shows an environment for an experimental program verification.

207~503 Description of the Preferred Embodiment Overview of the Underlying Principles Figure 4 is a block diagram of a moving body recognition apparatus of this invention.
The moving body recognition apparatus comprises a single image input unit 15 for capturing an image of an object, a feature point extraction unit 16 for extracting feature points in an image outputted from the image input unit 15, and a feature point storage unit 17 for storing the extracted feature points, thereby recognizing the movement of a moving body from the known position data of the feature points.
Because first, second, third and fourth forms of this invention operate under similar principles, they are all explained by referring to Figure 4.

A First Form A shape/movement recognition unit 18 recognizes a shape and a movement of an object. The shape/movement recognition unit 18 in a first form of this invention calculates the actual position and movement of the object in a three-dimensional space, e.g. moving on a single plane coupled with a rotation at a constant speed, from the known position data of 23 2~78~3 two [2] feature points, e.g. the X coordinate value of a feature point of an object moving on the XY plane, each in images of the object captured at three [3]
instants in time for observation having equal time intervals from a direction perpendicular to the axis of rotation.
The first form of this invention assumes, for example, that the direction of an axis of rotation is the same as the direction of the Z axis, that the direction of the movement of the rotating object is the same as the direction of the X axis, that an object moves on the XZ plane, and that the object is observed from the Y axis perpendicular to the image plane, i.e. the Xz plane. Because of the earlier described assumption of an orthogonal projection, a displacement in the X axis direction is the same as a displacement observed on the image plane, although a displacement in the direction of the Y axis is unknown.
After one [1] of the two [2] feature points moves to the origin, the shape/movement recognition unit 18 calculates the angle of rotation around the origin of the other one [1] of the two C2] feature points.
That is, the shape/movement recognition unit 18 in the first form of this invention obtains the X coordinate 2~ 2~78503 value of a first feature point from the position data of an input image, and then calculates the Y
coordinate value of a second feature point after the first eature point moves to the origin and the angle of rotation around the origin of the object, thereby obtaining the actual positions and movements of the first and second feature points.

A Second Form The image input unit 15, the feature point extraction unit 16, and the feature point storage unit 17 in a second form of this invention operate essentially the same manner as those in the first form of this invention. However, the shape/movement recognition unit 18 in the second form of this invention calculates the actual positions and movements of three [3] feature points forming a right angle of an object in a three-dimensional space, from the known position data of three [3] feature points each in images captured at any two [2] instants in time for observation from the direction perpendicular to the axis of rotation of the object moving on a single plane coupled with a rotation.
The basic differences of the second form of this invention from the first form are that the rotating 2~785~3 speed of the object may not be constant and that ob~ect has three [3] feature points forming a right angle. Accordingly, the second form of this invention allows the actual positlons and movements of feature points of an ob;ect to be calculated from the known position data of the three [3] feature points of the object at any two [2] instants in time for observation.
The second form of this invention uses the same coordinate system as that of the first form of this invention. ~s in the first form of this invention, the displacement of a feature point in the direction of the X axis is the same as the displacement observed on the image plane, although the displacement in the direction of the Y axis is unknown.
Also, as with the shape/movement recognition unit 18 in the first form of this invention, after one [l]
of the three [3] feature points at the right angle moves to the origin, the shape/movement recognition unit 18 in the second form of this invention calculates the Y coordinate values of the other two [2] feature points, the angle of rotation of the object around the origin from a first instant in time for observation to a second instant in time for observation, thereby obtaining the actual positions 20~8503 and movements of the three [3] feature points in the three-dimensional space.

A Third Form As with the third form of this invention, the image input unit 15, the feature point extractlon unit 16, and the feature point storage unit 17 in a third form of this invention operate essentially the same manner as those in the first form of this invention, whereas the shape/movement recognition unit 18 in a third form o~ this invention only operates differently from the shape/movement recognition unit 18 in the first form of this invention. The shape/movement recognition unit 18 in the third form of this invention calculates the positions and movements of the feature points o an ob~ect in a three-dimensional space, from the known position data of three [3]
feature points of an ob;ect each in images captured at three [3] instants in time for observation from a direction perpendicular to the axis of rotation of the ob;ect moving on a single plane coupled with a rotation.
The third form of this invention can be said to correspond to a more generic case in the third form of this invention, in which three [3] feature points of 207~3 an object do not necessarily form a right angle and the known position data at three [3] instants in time for observation are used for recognizing a moving body.
In other words, the second form of this invention is premised on a right angle formed among three ~3]
eature points. However, even with this premise, the second form of this invention can be applied to many cases. For instance, it is applicable to a case in which a TV camera attached to a self-propelled vehicle observes a sidewall along its right of way, because in many cases a sidewall stands straight, thus having a right angle. Not only a sidewall, but also many artificial buildings have their sides form right angles. Hence, the third form of this invention can be applied to a case in which a TV camera observes such a building.
The third form of this invention assumes the same coordinate system as that used in the first form of this invention, where the direction of the movement of an ob~ect is the same as the direction of the X axis, the direction of the observation of an image is the same as the direction of the Y axis, and the direction of the axis of rotation of the ob;ect is the same as the direction of the Z axis.

Of the three [3] feature points, one [1] moves to the origin, and the Y coordinate values of the other two [2] feature points and the angles of rotation of the object around the origin from a first instant in time for observation to second and third instants in time for observation are calculated, thereby obt~inin~
the actual positions and movements of the three [3]
feature polnts in the three-dimensional space.

A Fourth Form A fourth form of this invention represents the most generic case for recognizing the movement of a moving body. The shape/movement recognition unit 18 in the fourth form of this invention calculates from the known position data of four [4] feature points of an obJect moving with a rotation each in orthogonally projected images at captured any three [3] instants in time for observation, thereby obtaining the actual positions and movements of the four [4] feature points in the three-dimensional space.
Also, before calculating the positions and movements of the four [4] feature points, in order to confirm that the positions and movements can in fact be calculated, the shape/movement recognition unit 18 determines that the four [4] feature points are not on 29 2~785 03 a single plane, that the axis of rotation of an ob~ect is not parallel to the projecting direction of an orthogonal projection between any two [2] instants in time for observation of the three [3] instants in time for observation, and that the rotation of the object between any two [2] instants in time for observation of the three [3] instants in time for observation is not a rotation of one hundred and eighty degrees [180] around an axis parallel to a plane on which an image is orthogonally projected.
The basic differences of the fourth form of this invention from the first, second and third forms of this invention are that the movement of an object is not restricted to a single plane, and that neither the relation between the axis of rotation and the direction of observing the ob;ect nor the rotating speed has any limitation.
Because of these differences, the fourth form of this invention requires the known position data of the four [4] feature points of the object each in images captured at three [3] instants in time for observation selected arbitrarily.
The fourth form of this invention uses the same coordinate system as that of the first, second and third forms of this invention, where the direction of 2 ~

observing the object is the same as the direction of the Y axis. As in other forms of this invention, because of the assumption of an orthogonal proJection, the displacements in the directions of the X and Z
axes are the same as the displacements observed on the image plane, although the displacement in tha direction of the Y axis is unknown.
~fter moving any one [1] of the four [4] feature points to the origin, the shape/movement recognition unit 18 calculates the Y coordinate values of other three [3] feature points, the angle of rotation of the ob~ect around the origin from a first instant in time for observation to a second instant in time for observation, and the angle of rotation of the ob;ect around the origin from a first instant in time for observation to a third instant in time for observation, thereby obt~i n; ng the actual positions and movements of the four [4~ feature points in the three-dimensional space.
Detailed Explanation of Actual Embodiments This invention is based on a new theory, which this applicant originally submits and proves. The new theory shows that the shape (positions of feature points) and movement of an ob~ect can be calculated 8 ~ ~ 3 under limited circumstances, as long as the feature points can be extracted at several instants in time for observation even if there is only one [1] image input unit.

The First Form First, the new theory on the first form of this invention is explained in detail.
The first form of this invention assumes that an ob;ect moves on a single plane coupled with a rotation around a predetermined axis of rotation and that an image input unit observes an object from a direction perpendicular to the axis of rotation of the object. The first form of this invention can be applied to a lot of cases under such assumptions.
For instance, as described earlier, it is applicable to a case in which the image input unit is attached to a self-propelled vehicle running along various objects such as a wayside wall along a highway, in which case an edge e.g. formed by the curb can be used as the feature points.
Also, the image captured by the image input unit is approximated by an orthogonal projection. An approximation by an orthogonal projection is pretty good especially when an image input unit can capture 2~785~3 an ob;ect in a narrow scope, because the distance between the image input unit and the obJect is large.
In addition, this theory is premised on the ob;ect having a constant rotating speed.
Many a moving body is deemed to be standing still during a very short lapse of time. Therefore, an assumption that a movement is constant is natural.
Besides, strictly speaking, a condition "The rotating speed of an object is constant." is sufficient for a condition "A movement is constant.", because parallel displacement components of a movement of an ob;ect can be easily obtained from the movements of feature points on an image plane.
When a rotating speed is constant, the number of feature points to be in correspondence can be reduced to two [2]. That is, when two [2] feature points each in images captured at three [3] instants in time for observation having equal time intervals can establish correspondences, a moving body can be recognized. (This relates to theorem l to be described later.) A single feature point does not allow the shape of an object to be obtained. Also, three [3] images are necessary to confirm a status that the movement is constant. Hence, two [2]
feature points and three [3] images are necessary 31-2 2~7~03 premises.
.

Figure 5 shows a universal relation between an ob;ect and its observation plane in an image input unit pursuant to the first form of this invention.
The direction of the Z axis on the image plane is defined as the direction of the axis of rotation of the object, and the X axis is set in a direction perpendicular to the Z axis. More specifically, the direction of the X axis is parallel to the direction of the movement of a feature point. The origin on the image plane is by definition the point at which the X axis crosses the Z axis. The Y axis is perpendicular to the image plane, and passes through the origin on the image plane. Because of an assumption of an orthogonal projection, the XYZ
coordinate system can be displaced parallelly in any direction.

Figure 6 shows a relation between an object and its observation plane pursuant to the first form of this invention, in which one [1] of feature points of the object is fixed at the origin of a three-dimensional coordinate system.
Also, because of the assumption of an orthogonal 31-3 ~07~3 pro;ection, the displacement in the direction of the X
axis is the same as the displacement observed on the image plane, although the displacement in the direction of the Y axis in unknown. As such, one [1] of the feature points, i.e. a feature point 0, can be considered as fixed to the origin 0, as shown in Figure 6.

Figure 7 illu$trates the orthogonal projections of feature points 0 and 1 shown in Figure 6 on the XY
plane at three [3] instants in time for observation pursuant to the first form of this invention.
To obtain data on the shape and movement of an object, it is sufficient to calculate the displacement of a feature point having a correspondence, i.e.
feature point 0, and the angle of rotation around the feature point. Because the displacement of feature point 0 can be obtained self-evidently, as described earlier, it is sufficient to calculate an angle of rotation around feature point 0.
A moving body recognition apparatus pursuant to the first form of this invention obtains an X axis value and a Y axis value from the known data of feature points of an input image, and calculates a Y
axis value of another feature point after feature 31-4 2 0 78~ 03 point O moves to an origin and an angle of rotation of the ob~ect around the origin.
The following is a description of the codes shown in Figure 7.
Feature point O is a feature point of an object having moved to the ori~in 0.
Feature polnt 1 is a feature point other than feature point 0.
Ul iS a two-dimensional vector representing a feature point 1 at a first instant in time or observation.
Vl iS a two-dimensional vector representing a feature point 1 at a second instant in time for observation.
Wl iS a two-dimensional vector representing a feature point 1 at a third instant in time for observation.

32 2078S~3 [Expressions 8]
Ull 1 Vll Wll Ul = [U12 ~ Vl = [V12 ] Wl = [W12 Elere, although a subscript 1 in the above vector expression, e.g. in Ul, iS not necessary, it is put here for maintaining consistency with explanations for other forms of this inventions. Also, a first vector component represents an x element, and a second vector component represents a y element.
Rotation matrix R represents a rotation of an object around the origin from a first instant in time for observation to a second instant in time for observation.
This is equal to the rotation of an object around the origin from a second instant in time for observation to a third instant in time for observation, due to an assumption of a constant rotatirlg speed.
[Expression 9]
r cOs~ - sin~

A

207~03 L sin~ cos~
They have the following relations, Vl = R ul, Wl = R2Ul which represent A first A second A third instant instant instant in time R in time R in time R
for ob- for ob- for ob-servation servation servation 1 0 Ul > Vl ~ Wl Accordingly, the problem can be formulated as follows:
[Problem Of Recognizing A Body Moving On A Single Plane]
Ul, Vl, Wl are two-dimensional vectors whose first elements are known.
R is a two-dimensional rotation matrix.
Vl = R Ul, Wl = R~ Ul Obtain from the above, R and second elements of Ul, Vl, Wl.
A next theorem 1 gives conditions for solving this problem.
[Theorem 1]
The conditions for solving the above movement/structure recognition problem are Vll ~ O '''' {1}

2078~3 Vll = +Ull and Wll = Ull are not outstanding concurrently. {2}
These conditions comprise data solely on the observed X coordinate values.
Condition {1} means that the observation point mapped by Vl does not fall on the origin. That is, it means that two [2] feature points relative from an observation point do not fall on the second instant in time for observation.
Condition {2} may not be clearly understood.
However, it is actually equivalent to a next condition

{3}
[Theorem 2]
Condition {2} in theorem 1 is equivalent to condition {3}.
~ ~ n ~ (where n is an integer) ~ ~ {3}
Condition {3} means that the angle of rotation of an object between a first instant in time for observation and a second instant in time for observation is neither zero degrees [0] nor one hundred and eighty degrees [180].
A next system 2.1 is used in a proof of theorem 3.
[System 2.1]
[Expression 10]

207850~

n ~ (where n is an integer) {3 _ R1 being regular - ( R2 ) 1 -Here, a subscript 1 represents the first row of a matrix.
A next theorem 3 presents an expression for calculating a set of solutions.
[Theorem 3]
Under the conditions of theorem 1, the next set of definite solutions can be finalized.
[Expression 11]

[ cos~ -sin~]
sin~ cos~
and y component of a vector can be calculated by the following equations.
COS ~3 = ( Ull+W12 ) /2Vll sin ~ = +(1 - cosZe)~
Vll U12 = [2cos2~-l -cos~3~ _ sin~ - Wll -V12 = [ sin~ cos~][ ], Wl2 = t sin~ cos~][ ], A next system 3.1 defines a relation between two [2] sets of solutions.
tSYstem 3.1]

207~S03 Assuming one [1] set of solutions comprises R, Ul r Vl and Wl r the other set of solutions corresponding to it comprises:
[Expression 12]

[ ] R [ ] is an inverse rotation matrix of R.

[ 1 0] is a mirror image transformation of u~
0 -1 with respect to the X axis.
[ ] Vl iS a mirror image transformation of Vl 0 -1 with respect to the X axis.
[ ] Wl iS a mirror image transformation of Wl 0 -1 with respect to the X axis.
Theorem 3 and system 3.1 correspond to a fact that there are two [2] sets of solutions for the earlier described moving body recognition problem, i.e. a fact that the Y coordinate value of the other feature point is positive or negative, when feature point 0 in an orthogonally pro~ected image falls on the origin. They also mean that the surface of an object seen from a TV camera as an image input unit forms a convex or a concave.

Figures 8A and 8B show two [2] sets of solutions forming mirror image transformations of each other with respect to the X axis pursuant to the first form 37 207~Q3 of this invention.
Figure 8A shows a projection of a feature point shown ln Figure 6 to an XY plane. Figure 8A
lllustrates a case in which the surface of an object forms a convex. Figure 8B illustrates a case in which the surface of an object forms a concave, which is obtained by a mirror image transformation of the solution shown in Figure 8A with respect to the X
axis.
The proofs of theorems 1, 2 and 3 are illustrated in a sequence of a proposition 1, system 2.1, theorem 2 and theorem 3.
[Proposition 1]
Expression {1} in theorem 1 and expression {3} in 15 theorem 2 are necessary conditions for finalizing a solution.
Vll 7~ 0 '''' {1}
~ ~ n ~ (where n is an integer) ~3}
[Proof Of Proposition 1]
To reiterate the conditions, Vl = R u1 , Wl = R2 Ul (1) Proof that expression ~1} is a necessary condition By assuming that Vll = O, it is shown that no set of definite solutions can be obtained.

38 20785~3 [Expression 13]
Conditions Ull Vll Wll rVll ~:-z~ [ ] =R-l ~ ] ' ~ ] = R ¦
Ul2 LVlZ 'W12 'V12 Ull sin~ Wll 1 r~sin~31 _ _ = _ _ Vl2, _ = V12 -Ulz - -cos~3- -Wlz - - cos~3-Ull -sin~ -wll -Ull _ _ = l Vl2, ] _ -U12 - -cos~3~ -W12 - U12 Therefore, an equation [Ull ] [sin~]
U12 cos~3 needs to be solved.
The unknowns are ~, U12 and V12. Since the number of unknowns are three [3], which is greater than the number of equations, no set of definlte solutions can be obtained.
(2) Proof that expression {3~ is a necessary __ condition By assuming that ~ = n ~(where n is an integer), it is shown that no solution can be finalized.
Since cos~ = +1, and sin~ = 0, R = +I.
It is necessary to solve conditions ~ V1 '= +U1 ' and equations V12 = +U12' W12 U12 2~7~3 The unknowns are U12, Vl2 and Wl2. Since the number of unknowns are three t3], which is greater than the number of the equations, no set of definite solutions can be obtained.
[End of Proof]
[Proof of system 2.1]
[Expression 14]
The contraproposition of a proposition R, cO~a. -~in~ R
lo [ ] _ , ---sln(2~ ln~
(R s) l ~os2 ~, -sln 2 ~ - (~

is proved.

[ R ' ~ is not regular ~ ~ln ~ ~ O ~ ~ ~ n tR~)I
(where n is an integer) ..... ~3~' Therefore, r R, L (R2)l ] is regular ~ ~ ~ n~

(where n is an integer) {3}
[End of Proof]
[Proof of theorem 2]
[Expression 15]
The contraproposition of a proposition that Vll = iUll and Wll = Ull ~ = n ~ is prcved.

~ .
2078~3 (~=) Because cos~ = +l and sin~ =0, R = +I, Vl = +Ul, Wl = Ul, Vll = +Ull and wll = Ull.
( ~ ) - U~ cosc~

is ~f~f; nf~l -u12- ~ sina- ~ -v11 = cos ( a~), w11 = cos (~ 20 ) (i) when Vll = Ull, Wll = Ull COS ~ ~ COS ~, COS ( ~ ~ OS
[~ 2 n ~
~ 2 ~--i~+ 2m~ (2) ~ = 2n ~ or m ~, when either (1) or (2) is positive.

[2 ~ ~ 3 2 n ~ tl) 2 ~ 2 0 = 2 m ~

when both (1) and (2) are negative.
(2) ~ = 2(m-n) ~

(ii) when Vll = -Ull, Vll = Ull 2~503 cos (~Y+ O) z-- cos~, cos (cx+ 2 ~) ~ cos~

cr+ ~ â= f ~ 2m7~ 2).
when either (1) or (2) is positive ~ n~ 1 ) ~. m when both (1) and (2) are negative [~ = t2 n ~
2 cr ~ ~ ~ = 2 m ~1 (2) (2) -tl) ~ m- 2 n~

[End of Proof]
[Proof of theorem 3]
tExpression 16]
From conditions Vl = R Ul and Wl = R2 Ul [v ~ [

12 t~
from system 4.2.1 [ ] is regular, (R~), 12~ [(R~)~] G11 ~ . ... (1) [ R~ ]~' r C09~1 -9in~ r sln2~, -91n~1 (Rr)~ Lcos2~, -sin 28~ sin8 L co92 8, -co3âJ
By using the first component of (1), Ull = 2cos~ Vll - Wll From a condition of theorem 1, since vll ~ 0 Co:~ ~9 ~ ( U11 +W11 ) / ~ 2 V11) sln ~q--+ ( 1-- cos2 ~

` ! 2078~3 By using1the seoond component of (1) 12 1 ~ (2 cos ~ cos ~) [ ]

Frc~ conditi~ w V1 e R u, a~l 1CR~ u I

v1 2s (s l n ~ ~ cos ~ , w1 2 = (s i n ~, cos t~ ) [ 1 1 ]

(End of Pr~of) [Proof of system 3.l]

[Expression l7]
,1 l r1 l f-1 -~ r1 0 Since Lo -11 Lo -1~ L o 11 1 ~ Lo -1 J

r1 -1 0 lo -1 1 1 are mirror image transformations of R, Ul, Vl and Wl with respect to the X axis, they satisfy the - conditions for a set of solution. Therefore, they are in fact a set of solutions. Since there are two -[2] sets of solutions according to theorem 3, it is known that the set of solutions other than the above lS r1 01 l-1 l r1 ~ r1 1 1 0 O -1-1 Lo -1~ 1 Lo -1~ 1 ' [ ~ w1 [End of Proof]

[Proof 2 of system 3.l]
[Expression l8]
If one solution is R, Ul, Vl and Wl from theorem 3, it is known from the calculations that the other solution is r 1 01 r 1 01 r 1 01 r 1 01 r 1 ol -2 ~ 3 L ~RL ~ L ~U1 L ¦V1 L ~W1 [End of Proof]
So far, a new theory for use in the first form of this invention has been explained in detail, by having 5 the shape/movement recognition unit 18 shown in Figure

4 apply the new theory to a body moving on a single plane coupled with a rotation having a constant speed of rotation, based on the extraction result in an image of two [2] feature points at three [3] instants in time for observation having e~ual time intervals, the actual positions and movements of those feature points in a three-dimensional space are calculated, thereby enabling the movement of a moving body to be recognized.
Figure 9 is an explanatory chart showing the concept of a moving body recognition apparatus of this invention.
An embodiment of the first form of this invention assumes that an image input unit 21, provided singularly, recognizes the movement of a moving body rotating around a single axis, e.g. the Z axis, by observing the moving body from a direction perpendicular to the axis of rotation.
In Figure 9, the image input unit 21 captures 2~7g~3 images of an object 20, and transmits them to a moving body recognition unit 22. The moving body recognition unit 22 extracts feature points of the object 20 from an image, and recognizes the shape and movement of an ob;ect from the positions of feature points at plural instants in time for observation. The moving body recognition unit 22 outputs the result of recognizing the object 20, as a recognition result 23.

Figure 10 is a block diagram showing the global configuration of a moving body recognition apparatus of this invention.
In Figure 10, an image input unit 24 captures an input image and outputs it to a feature point extraction unit 25. The feature point extraction unit 25 extracts feature points from the input image, and outputs the position data of the feature points in the input image. The feature point storage unit 26 outputs to a shape/movement recognition unit 27 known its stored position data of the feature points at plural instants in time for observation. The shape/movement recognition unit 27 calculates the positions of the plural feature points in an actual three-dimensional coordinate space and the movement of an object, and stores the result indicating the 2~7~3 movement of an object in an object movement storage unit 28 and the result indicating the three-dimensional positions of feature points in a feature point position storage unit 29.

Figure 11 is a block diagram illustrating in detail the configuration of a feature point extraction unit 25 shown in Figure 10.
In Figure 11, the feature point extraction unit 25 comprises a space filter 25a and an edge point extraction unit 25b. There are a number of methods for extracting a feature point, e.g. by an edge and by a color. The moving body recognition apparatus of this invention may use any one [1] of or any combination of feature point extraction systems, and is not restricted to use a particular method for extracting a feature point.
A typical method for extracting a feature point is to extract an edge point of an object. As shown in Figure 11, the space filter 25a is applied to an input image for extracting an outline image, enabling the edge point extraction unit 25b to detect as feature point data a kink of an edge and an intersection. For instance, in recognizing an object on a single plane, the intersections between 207~5~3 the horizon on an image plane and an edge in the vertical direction may be used as feature points.

In Figure 10, the shape/movement recognition unit 27 uses the earlier described new theory for calculating the actual positions of feature points and the movement of an object.

Figure 12 is a block diagram illustrating in detail the configuration of the shape/movement recognition unit 27 shown in Figure 10.
In Figure 12, the shape/movement recognition unit 27 comprises a known data input unit 31 for receiving known data about the movement of an ob;ect e.g. from a sensor, a feature point normalization position unit 32 for obtaining the relative positions of other feature points when one [1] of plural feature points moves to the origin of a three-dimensional coordinate space, i.e. the normalized positions, a shape/movement determination unit 33 for determining a recognition enablement or a recognition disablement of the movement of an object, a movement calculation unit 34 for calculating the angle of rotation around the axis of rotation of the ob;ect by using an output from the feature point normalization unit 32, a shape 2~78~3 calculation unit 35 for obtaining unknown position data of feature points other than the feature point on the origin, and a feature point position reconstruction unit 36 for outputting the movement of the object the and the positions of feature points in the three-dimensional coordinate space respectively to the object movement storage unit 28 and the feature point position storage unit 29, by using outputs from the movement calculation unit 34 and the feature point position normalization unit 32.

Figure 13 illustrates an embodiment of sensor connections for the known data input unit 31 shown in Figure 12.
Figure 13 shows two [2] connecting systems (1) and (2). In connecting system (1), an external sensor 37 is connected directly to the known data input unit 31. In connecting system (2), an external memory 38 is connected to the known data input unit 31 and the external sensor 37 is connected off-line to the external memory 38. That is, the external memory 38 stores data from the external sensor 37, the external memory disconnects itself from the external sensor 37. The external sensor 37 may be a special electrical sensor, an optical sensor or ~7~3 an ultrasonic sensor for measuring an object.
However, such a sensor is premised on a special relation between an object and the moving object recognition apparatus of this invention and is not a general purpose sensor almi.ghty for any obj ect.

The configuration of the moving body recognition apparatus illustrated in Figures 9 through 13 can be realized almost "as is" by a computer system. First, the image input units 21 (shown in Figure 9) and 24 (shown in Figure 10) are realized by a TV camera, and the known data input unit 31 is connected with a keyboard and a sensor other than a TV camera, as described earlier. The feature point storage unit 26, the object movement storage unit 28 and the feature point position storage unit 29 comprise a memory or a secondary storage apparatus such as a disk storage apparatus, and the feature point extraction unit 25 is realized by image processing hardware including the space filter 25a. The shape/movement recognition unit 27 is realized by a generic CPU, and the recognition result 23 (shown in Figure 9) corresponds to the movement of an object and the positions of feature points stored respectively in the object movement storage unit 28 and the feature point 49 2~7~3 position storage unit 2g.

Figures 14, 15 and 16 are flowcharts showing in a three part series the whole processes of the movement/shape recognition unit 27 pursuant to the first form of this invention.
In Figure 14, on starting the process it is determined in step S40 whether or not the known data input unit 31 shown in Figure 8 has received an expression {3}', i.e. ~ = n ~, where n is an integer.
Here, expression {3}' is generally determined to be outstanding or not only after a calculation of rotation matrix R, because various sensors (not explained here) are attached to the moving body recognition apparatus of this invention, it is determined by signals from those sensors whether or not the rotation of an ob~ect from a first instant in time for observation to a second instant in time for observation is a rotation by zero degrees [0] or one hundred and eighty degrees [180]. Here, the prime sign for expression {3}' indicates that expression {3}
used in theorem 2 is transformed on an equivalent basis.
When the shape/movement recognition unit 21 determines in step S40 that the known data input unit 2078~3 31 has received expression {3}', in step S41 the known data input unit 31 outputs a deactivation signal to the feature point normalization unit 32 and stores in the movement calculation unit 34 recognition disablement data, thereby invoking a recognition disablement process {1}.

Figure 17 is a flowchart of an embodiment of recognition disablement process {1} pursuant to the first form of this invention.
On start of recognition disablement process {1}, the movement calculation unit 34 sends recognition disablement data to the feature point position reconstruction unit 36 in step S42. Then, the feature point position reconstruction unit 36 stores the recognition disablement data in the object movement storage unit 28 and the feature point storage unit 29 in step S43, thereby ending the process.

Returning to Figure 14, when the shape/movement recognition unit 27 determines in step S40 that the known data input unit 31 has received expression {3}', it determines in step S44 whether or not the feature point storage unit 26 has stored the positions of two [2] feature points each in images captured at three ~ 2078503 [3] lnstants ln tlme for observation. The shape/movement recognltlon unit 27 repeats step S44 untll lt determines that the feature polnt storage unit 26 has stored the positlons of two [2] feature polnts each in lmages captured at three [3]
instants ln time for observation. When the shape/movement recognitlon unlt 27 determlnes affirmatively (YES) ln step S44, the feature polnt storage unlt 26 sends an actlvatlon slgnal to the feature polnt position normalizatlon unlt 32, thereby actlvatlng the feature point position normalization unit 32.
Contlnuing on to Flgure 15, ln step S46, the feature point position normallzation unlt 32 recelves the posltlons of two [2] feature points 0 and 1 stored in the feature polnt storage unlt 26 by the feature polnt posltion normallzatlon unit 32. Then, the flrst components Ull, Vll, and Wll of Ul, Vl and Wl are obtained from the lmage coordlnate values of feature polnt 1, whlch is dlfferent from feature polnt 0, whlch falls on the orlgln. The shape/movement determinatlon unlt 33 determlnes ln step S47 whether or not those flrst components Ull, Vll, and Wll satlsfy expresslons {1} and {2}. If at least either one [1] of them ls not satisfled, the shape/movement 52 2~7~5~

determination unit 33 sends recognition disablement data to the movement calculation unit 34 in step S48, thereby invoking recognition disablement process {1}
shown in Figure 17.
As already explained in the description of theorem 2, expression {2} is equivalent to expression {3}. Therefore, it can be said that the check of expression {3} in step S40 and the check of expression {2} in step S47 form a duplication. However, since a sensor can perform the check in step S40, such a duplication ls instituted. Also, no further calculation is performed, when in step S47 the shape/movement determination unit 33 determines a recognition disablement by using only the positions of feature points in an image.
If the shape/movement determination unit 33 determines in step Sg7 that both expressions {1} and {2} are outstanding, the shape/movement determination unit 33 activates the movement calculation unit 34 in step S49, thereby invoking respective processes of the movement calculation unit 34 and the shape calculation unit 35, in accordance with theorem 3.

Figure 18 is a flowchart of the movement calculation unit 34 pursuant to the first form of this 2~7~3 invention.
On starting the process, the movement calculation unit 34 calculates rotation matrix R in step S50, and determines in step S51 whether or not any of the calculation results satisfies the known data regarding the movement of an ob;ect inputted to the ~nown data input unit 31. If rotation matrix R does not satisfy the known data, recognition disablement process {1} shown in Figure 17 is invoked.
As described earlier, a sensor (not illustrated as part of the moving body recognition apparatus) inputs known data about the movement of an object.
Assume here, for instance, that an object stands still on a single plane, and that a TV camera loaded on a running vehicle moves on the same single plane.
Because the moving vehicle may slip, the movement of the TV camera cannot be determined precisely.
However, the direction of the relative movement of the running vehicle as against the still body can be determined. The relative movement o the still body is supplied as known data of the ob;ect via an external memory to the known data input unit 31.
The movement calculation unit 34 obtains rotation matrix R and its inverse rotation matrix R-1, as two [2] sets of solutions , the constraint on the ~7~5~

direction of a relative movement allows only one [1]
set of solutions to be selected as the appropriate set of solutions.

Figure 19 is an explanatory chart for the method of selecting the appropriate one [1] of two [2]
rotation matrices R and R-1.
More specifically, Figure 19 shows a case in which a vehicle loaded with a TV camera is observed straight from the abover and illustrates how to capture the image of a still object facing a direction perpendicular to the moving direction of a running vehicle. It is assumed that a sensor e.g. loaded on the running vehicle has already detected the moving direction of the running vehicle. If the running vehicle moves to the left, the TV camera observes that the object rotates clockwise, which means that it is sufficient to select, as the calculated rotation matrix, either R or R~l that represents a clockwise rotation. If, on the other hand, the running vehicle moves to the right, as opposed to the direction shown in Figure 19, the TV camera observes that the object rotates counterclockwise, in which case it is sufficient to select, as the calculated rotation matrix, either R and R~1 that represents a --/ 2~78~3 counterclockwise rotation.
When the movement calculation unit 34 determines in step S51 that it can select rotation matrix R
satisfying known data of the ob;ect inputted in known data input unit 31, the movement calculation unit 34 sends rotation matrix R as its result to the shape calculation unit 35 and the feature point position reconstruction unit 36.

Figure 20 is a flowchart of the shape calculation unit 35 pursuant to the first form of this invention.
According to the first form of this invention shown in Figure 20, on invoking the process, the shape calculation unit 35 obtains U12, Vl2 and W12 as the second components of two-dimensional vectors Ul, Vl and Wl in step S53. Then, in step S54, the shape calculation unit 35 sends the second components U12, Vl2 and Wl2 to the feature point position reconstruction unit 36.
After the shape calculation unit 35 shown in Figure 15 completes its processes, the feature point position reconstruction unit 36 executes its processes in step S55 shown in Figure 16. Here, from the two-dimensional vectors Ul, Vl and Wl ànd rotation matrix R, as the calculation result of the shape and movement 2~7~5~3 of the ob~ect, the feature point position reconstruction unit 36 selects the values matching the known data about the movement of the ob;ect inputted to the known data input unit 31. If no such values exist, a recognition disablement process {2} is invoked.
Here, for explaining an example of known data, it is assumed that the TV camera loaded on a vehicle moves on a single plane. It is further assumed here that it is known already whether the surace shape o the still object observed from the TV camera is a convex or a concave. For instance, a telegram pole has a convex surface. The concave/convex data on an object are supplied as known data o the still object to the known data input unit e.g. via an e~ternal memory. Although the shape calculation unit 35 obtains two [2] sets of solutions, which are mirror image transformations of each other with respect to the observation plane, the concave/convex data on the object enables only one [1] set of solutions to be selected as the appropriate set of solutions.

2~7~3 .

Figures 21A and 21B are explanatory charts for concave/convex data of an ob;ect.
Because two [2] feature point are captured in the first form of this invention, if the relative distances of the two [2] feature points observed by a TV camera are known, either of the two [2] sets of solutions can be selected as the appropriate set of solutions.
Figure 21A shows a case in which feature point 0 is closer to the TV camera than feature point 1, and Figure 21B shows a case in which feature point 1 is closer to the TV camera than feature point 0. The known data input unit 31 receives as the knowledge about the object either these relative distances "as isll or the results measured e.g. by an ultrasonic sensor.

Figure 22 is a flowchart o an embodiment of recognition disablement process {2} pursuant to the first form of this invention.
In step S56, the feature point position reconstruction unit 36 stores the recognition disablement data in the object movement storage unit 28 and the feature point position storage unit 29, 2~8~03 thereby ending the process.

Returning to Figure 16, when the feature point position reconstruction unit 36 selects a set of solutions satisfying the known data in step S55, the feature point position reconstruction unit 36 stores in the ob~ect movement storage unit 28 the values of elements of rotation matrix R in the selected set of solutions and the in-image coordinate values of feature point 0 in step S57, and in the feature point position storage unit 29 the second component values U12, V12 and W12 of the two-dimensional vectors Ul, Vl and Wl also in the selected set of solutions and the in-image coordinate values of the two [2] feature points 0 and 1 stored in the feature point position normalization unit 32 in step S58, thereby ending the process.
In the above described embodiment, the movement calculation unit 34 obtains two [2] sets of solutlons simultaneously in step S50 (shown in Figure 18) of calculating rotation matrix R. However, the above embodiment can be reconfigured, such that the movement calculation unit 34 obtains only one [1] set of solutions in step S50.
In such an alternative embodiment, as an initial 2~7~503 step, the feature point position reconstruction unit 36 calculates inverse rotation matrix R~l inverse to rotation matrix R, which is in the other set of solutions not selected.
[Expression 19]

[ coS~, sin~]
-sin~, cos~
~ lso, two-dimensional vectors Ul ~, Vl I and Wl ~ in the other set of solutions, which are the mirror image transforma~ions of two-dimensional vectors Ul, Vl and Wl in the selected set of solutions with respect to the X axis are calculated.
[Expression 20]

UllVll l r Wll U 1 = [ ] , [~ W 1 L

This allows the feature point position reconstruction unit 36 to obtain two [2] sets of solutions.
As described above, the first form of this invention enables the movement of an ob;ect moving on a single plane coupled with a rotation at a constant rotating speed to be recognized rom the positions of two [2] feature points each in images captured at three [3] instants in time for observation having equal intervals. Also, as explained in the 2078~3 description of Figure 15, the shape/movement determination unit 33 can immediately determine the recognition enablement or the recognition disablement from the in-image positions of feature points.

The Second Form The following is a description of the second form of this invention. As described earlier, the second form corresponds to a special case of a first form of this invention, in which three [3] feature points of an ob;ect forms a right angle. In such a case, the positions and movements of feature points are calculated from the known position data at three [3]
feature points at two [2] instants in time for observation.
The rotating direction, the moving direction and observing direction of an object in relation with the coordinate axes are essentially the same as those of the first form, and they are not explained again.
The general relation between an ob;ect and an observation plane is similar to what is shown in Figures 23 and 24.
The only difference is that feature point 0 of the three [3] feature points 0, 1 and 2 falls at the peak point of a right angle formed as the intersection 61 2Q~g5~3 between the line segment between feature point 1 and feature point 0 and the line segment between feature point 2 and feature point 0.
Next, the new theory for the second form of this invention is explained below. As shown in theorem 8 described later, the second form of this invention allows a moving body to be recognized by making correspondences between three [3] feature points in two [2] images. Here, the one [1] of the feature points at the peak of the right angle is displaced to the origin of the coordinate axis, and the Y
coordinate values of other two [2] feature points and the rotating angle of the object around the origin is calculated. By putting one [1] of the feature points on the origin, three [3] feature points form a right angle on the XY plane.
To observe the condition that edges form a right angle, three [3] feature points are necessary.
Also, a single image cannot allow the shape of a moving body to be determined. Therefore, three [3]
feature points and two [2] images are necessary conditions.

Figure 25 shows that edges forming a right angle in a single image cannot produce a set of definite solutions pursuant to the second form of this invention.
The actual positions of feature points 1 and 2 are on a single straight line of the observed X
coordinate values. If feature point 1 is arbitrarily is selected, one [1] position of feature point 2 satisfying the condition is determined.
This is because the triangle formed by feature point 1, the origin (feature point 0) and feature point 2 is a right angled triangle. That is, there is an infinite number of combinations between feature points 0 and 1.

Figure 26 illustrates the orthogonal projections of feature points 0, 1 and 2 shown in Figure 25 on the XY plane at two [2] instants in time for observation pursuant to the second form of this invention.
More specifically, Figure 26 shows a status of the actual positions and movements of the feature points from the orthogonally projected points to the image axes of two [2] feature points other than the origin of an object captured at two [2] instants in time for observation.
Signs used in Figure 26 is explained.
Feature point 0 is a feature point of an object ~.

moved to the origin.
_ Feature points 1 and 2 are feature points on an object other than feature point 0.
Ul and U2 are two-dimensional vectors on the XY
plane from the origin respectively to feature points l and 2 at a first point~in time for observation.
Vl and vz are two-dimensional vectors on the XY
plane from the origin respectively to feature points 1 and 2 at a second point in time for observation.
[Expression 11]
Uil Vil - ui = , vi = , (where i = 1, 2) ---Uil ---Vi2 Rotation matrix R represents a two-dimensional rotation of the object on the XY plane around the origin from the first instant in time for observation to the second instant in time for observation.
[Expression 12]
R s [ cose -sine]
sin~ cos~
U2 represents a vector obtained by rotating Ul by ~/2 or by -~/2. The solutions of these two [2]
cases have the relation of a lemma 1. Lemma 1 describes two [2] sets of solutions, which form mirror image transformations of each other with respect to the observation plane, as already explained 64 20785~3 in the description of the first form of this invention.

[lemma 1]
The following solutions (1) and (2) are mirror image transformations of each other with respect to the X axis.
( 1 ) U2 iS a solution obtained by rotating Ul by ~/2.
(2) U2 iS a solution obtained by rotating Ul by -~/2.
Figure 27 shows two [2] sets of solutions forming mirror image transformations of each other with respect to the X axis pursuant to the second form of this invention.
Figure 27 shows that there are cases in which Y
coordinate values of other feature points are positive and negative, when feature point 0 in the orthogonally proJected image falls on the origin. This corresponds to a case in which the surface of an ob;ect forms either a convex or a concave.
Therefore, in the second form of this invention, a recognition solution <2> is a mirror image transformation of a recognition solution <1> with respect to the X axis, where recognition solution <2>
is a solution to the problem in which U2 iS obtained 2078~3 by rotating Ul by -~/2 and recognition solution <1> is a solution to the problem in which U2 is a vector obtained by rotating Ul by ~/2.
[Expression 13]
Therefore, when Ul iS defined as:
cosa [
sin~

Then, U2 is defined as: (a) cos( a+r~/2 ) -sinal uz=dz ] - dz [
-sin( a~-~/2 ) cosa [Expression 14]

Also, when Vl iS defined as:
cos ~ cos ( a+ ~ ) vl=dl = dl ....
-sin~- -sin( a+~ )-Then, V2 is defined as: (b) cos( a+rT/2~ sin( a+~3 ) v2=d2 = dz ~ ....
-sin( a+~/2+~ )- - cos( a+~ ) A first A second instant instant in time R in time for ob- for ob-servation servation Ul ~ Vl U2 ~ V2 By their relations can be stated as 2~7~503 vi = R UL ( where i = 1, 2) Therefore, the problem can be formulated as follows:
[A Recognition Of An Object Having A Right Angle At An Edge On A Plane]
ui (where i = 1, 2) are two-dimensional vectors whose first components are known.
U2 is a vector obtained by rotating Ul by ~/2.
R is a two-dimensional rotation matrix.
vi (where i = 1, 2) are two-dimensional vectors whose first components are known.
vi = R ui (where i = 1, 2) At this time, obtain the second components of u and vi (where i = 1, 2) and R.
The second element of vi (where i = 1, 2) can be obtained directly from R, ui and the relation vi = R
ui. Accordingly, it is sufficient to obtain the second component of ui (where i = 1, 2) and R. That is ~, d1, d2 and a.
A next theorem 4 gives condition for solving the problem.
[Theorem 4]
The condition of allowing the above movement structure recognition problem is 207~3 Vll ~ _Ull ' ' ' ' {4} or V21 7~ +U21 {5}.
If the coordinate values to be observed from the X axis satisfy conditions {4} and {5}, it is determined that a unique set of solutions can be obtained.

Figure 28 illustrates the meaning of expression ~7} for use in theorem 5.
A next theorem 5 gives reasoning of conditions {4} and {5} of theorem 4.
[Theorem 5]
Conditions {4} and {5} are equivalent to next conditions {6} and {7}.
~ ~ n ~ {6} and a + ~ ~ n ~ - a {7}
(where n is an integer.) Condition {7} is equivalent to a next condition {8}.
Ull V21 + U21 Vll 7~ 0 ' ' ' { 8}
Condition {6} means that the ob;ect does not rotate by 0 (zero degrees [0]) or ~ (one hundred and eighty degrees [180]).
To make the meaning of condition {7} clearer, its 25 negation a + ~ = n ~ - a is considered. Since a shift of a + ~ by 2~ (three hundred and sixty degrees [360]) does not produce any difference, a first case of a + ~ = - a and a second case of a + ~ = - a +
need only be considered, as shown in Figure 28.
In the first case of a + ~ = - a, feature point 1 moves symmetrically with respect to the X axis, and feature point 2 moves symmetrically with respect to the Y axis. The second case of a + ~ = - a + ~
corresponds to a one hundred and eighty degree [180]
rotation of the first case of a + ~ = - a.
Condition {8} is used for theorem 6.
Described below is the deduction of formulae for calculating ~, d1, dz and a. The ranges of values a and sin~ need to narrowed for a substantive numerical calculation Figure 29 illustrates a method for determining a value range for angle a.
Figure 30 illustrates a method for determining a value range for angle ~.
Lemma 2 is for narrowing the value of angle a.
tLemma 2]
Assuming ui (where i = 1, 2) are two-dimensional vectors from the origin and U2 is a vector obtained by rotating Ul, the angle of rotation a from the X axis .

has a value within a value range for ~/2 according to the signs of the X coordinate values Uli and U2i of Ul (where i = 1, 2) as shown in Figure 29.
By replacing Ul with Vl and a with ~, the value of ~ = a + ~ can be narrowed.
In lemma 3, the value of sin~ is narrowed by using the results of lemma 2.
[Lemma 3]
From Figures 29 and 30, a pair of integers m and n (= 0, 1, 2, 3) that satisfies two [2] sets of ine~ualities, (~/2)m S ~ < (~/2)(m~1) and (~/2)n S a < ( ~/2)(n+1) is selected.

Figure 31 illustrates a method for determining the sign of sine, when m-n is odd.
Figure 32 illustrates a method for determining the sign of sin~, when m-n is even.
A comparison between the values of m and n and a comparison between the values of Ull and Vll allow the sign of sin~ to be determined from cases (1) and (2) below.
(1) When m-n is odd, an integer p is defined as p = (m-n-1)/2 The sign of sin~ is determined from Figure 31.
(2) When m-n is even, the sign of sin~ is determined 2~7~Q3 from Figure 32. When the equalities are outstanding in the above sets of inequalities for m and n, sin~=0.
After being prepared by lemmas 2 and 3, theorem 6 for calculating the set of solutions is shown.
[Theorem 6]
Under condition of theorems 4 and 5, the solution is obtained as follows, when U2 is a vector obtained by rotating Ul by ~/2.
The following equations allow e, d1, dz and a to be calculated.
COS~ = ( UllU21 + VllV21)/(UllV21 + U21V11) sin~ = +(l-cos2~)1/2, where the sign is determined by lemma 3.
(1) When Ull ~ O and U21 ~ 0 tana = (u11 cos~ - Vll ) / Ull sin~. Comblning with the value range for a obtained rom lemma 2, the value of a can be uniquely determined.
dl = ull/cosa dz =-uz1/sina (2) When Wl =- 0 [Expression 15]

~/2 (when u21 < 0) a =
-3~/2 (when U21 > 0) (from lemma 2) dl = ¦vll/sin~I
d2 = ¦U21¦

~1 ~

207~5~3 (3) When U21 = 0 [Expression 16]
0 (when Ull > O ) ~ (when Ull < O ) ( from lemma 2) d1 = ¦u11¦
d2 = ¦vzl/sin~¦
As described in lemma 1, the set of solutions, in which U2 iS a vector obtained by rotating Ul by -~/2, is obtained by a mirror image transformation of the solution by theorem 6 with respect to the X axis.
System 6.1 describes this more concretely.
[System 6.1]
The set of solutions, in which vector U2 iS
obtained by rotating vector Ul by -~/2, is obtained as described in (1) and (2).
(1) Assuming that the set of solutions, when vector U2 in theorem 6 is obtained by rotating vector Ul by ~/2 is R, ui and vi (where i = 1, 2), the set of solutions, when vector U2 iS obtained by rotating vector u1 by -~/2 is represented by the next formulae.
[Expression 17]

[ o -1] [ 0 -L]
which represents inverse rotation matrix of R.
r 1 01 r 1 ol 2 0 7 ~ ~ ~ 3 L ~ui L J

which represents the mirror image transformations of Ul and Vl .

(2) Assuming that the set of solutions, when vector U2 in theorem 6 is obtained by rotating vector Ul by ~/2, is ~, di, d2 and a, the set of solutions, when vector U2 is obtained by rotating vector Ul by -~/2, is -~, di, d2 and -~.

Proofs of lemmas 1, 2 and 3, theorems 4, 5 and 6, and system 6.1 are described below.
[Proof of Lemma 1]
Assuming that (1) is a set of solution when vector U2 is obtained by rotating vector Ul by ~/2, (2) is a set of solution when vector U2 iS obtained by rotating vector Ul by -~/2, sets of solutions (1) and (2) are mirror image transformations of each other with respect to the X
axis, as shown in Figure 27.
[Expression 18]

.. cos sin~
~ ..... (~) 73 2~7~5~

cos (~ ~/2) ~_s~n~ j U 2 G dz - _ .d 2 . ...!
. -sin t~-~/2)- ~ cos~-Since vectors Vl and V2 are obtalned by rotatingvectors Ul and U2 by angle ~, [Expression 19]
cos ( cY + ~ ) -Vl =dl [ ] .....................
s ~ n ( ~x ~- O ) (b) - ~ cos (~ 7~/2 ~ O) - sin (~
lo sln (~ -~ ~c/2 ~ cos (~ I O ) components of vectors ui and vi (where i = 1, 2) are known.
d, coscY . --.u " . . . - (1) -d l sina --u 21 ~ (2) dI cos (~+~) =v" (3) -d 1 sin (c~ + ~ ) = v ~, .....
The values of the right sides of formulae (1) through (4) are already known.
The following are proofs of lemmas 2 and 3, as well as theorems 4, 5 and 6, in a sequence of proposition 2, theorem 5, lemmas 2 and 3, and theorem 6.
[Proposition 2]
Expressions {6} and {7} in theorem 5 are conditions necessary for having a set of definite solutions.

.~

~7~ ~3 [Proof of Proposition 2]
To reiterate the conditions, vi = R ui and wi = S ui, where i = 1, 2.
(1) Proof that expression {6} is necessary By assuming that {6} is not outstanding, it is shown that there is no definite solution.
By substituting ~ = n ~ into formulae (3) and (4)t + d , coscy= v " ................... (3), lo -+d 2 sinc~--v 21 - (4) ~
where the signs are in the same order.
Since formulae (3)' and (4)' can be transformed into formulae (3)" and (4)" by formulae (1) and (2), formulae (1), (2), (3) and (4) are equivalent to formulae (1), (2), (3)" and (4)".
d, coscY=tlll . . . (1) --d2 sincY=u2~ (2) ~ u " ............................ (3) V21=~ U21 - - - - (4) where the signs are in the same order.
That is, the conditions are only formulae (1) and (2), whereas the unknowns are d1, dz and ~.
Therefore, the solutions are indefinite.
(2) Proof that condition {7~ is necessary By assuming that {7} is not outstanding, it is ~ . ~

2~7~Q3 shown that there is no definite solution.
By substituting (where n is an integer) into the left side of formulae (3) and (4), d, cos (~-t ~ d I cos~

5 --d 2 sin (cr+ ~ d 2 sino~
~d, cosa!=vI, -, - (3) ~
d 2 s i n o~--v ~ 4) ' Therefore, formulae (1), (2), (3) and (4) are equivalent to formulae (1), (2), (3)" and (4)".
o d, cos~= u " (1) --d2 sincy=u2~ .... (~) v " = + u " . (3) v 2,=-+u 11 ' (4) That is, the only remaining conditions are formulae (1) and (2), where the unknowns are d1, d2 and a. Therefore, the solutions are indefinite.
[End of Proof]
[Proof of theorem 5]
It is illustrated first that {4} or {5} is equivalent to {6} or {7}.
The contraproposition of this proposition is proved, which is as follows.

V I l=+ U l 1, V 21 - 1- U 21 (in the same order) ~ ~ = n 1r or ~ -t ~ = n 1r--cY
Therefore, ~ tO78503 [Expression 20]
-v I I = ~ u ~ v zl = ~- u zl ~ O _ (in the same order) (in the same order) -v,t-~-u~, v~ z, ~ ~ n ~ -(1) and (2) below will illustrate the above.
(3) will illustrate that {7} and {8} are equivalent.
V l l=+ U I I, V 21=+ U ~ = n (1) A proof that (in the same order) (~) (3)" and (4)" can be deducted from proof (1) for proposition 2.
10 (~) The formula numbers in proof (1) of proposition 2 are used.
(3)" and (4)" can be transformed into (3)' and (4)' by using (1) and (2). Combining~them with (3) and (4), tExpression 21]
d, cos (~-1 o) =~ d t COS~
-dz sin (c~ O) =-~d z sincr f ~;
That is, ~-cos ~-1 0) L s;n (cc-l- O) -- 1 ~Ino~
By using these formulae, sin~ is calculated as follows sin~ = sin ( (~ ~ 0) - ~}
= sin (cY~ ~) c~so~-- cos (cY+ ~) sin~
=+ { sino~ cos~-- cos~r sin~}
I'h Therefore, ~ - n ~

" 2û78~

(2) A proof that ll - U~ hV~ +U 2d ~ ~ + O = n~ - K
(~) This has already been proved in proof (2) o~
proposition 2.
(~) The formula numbers in proof (1) of proposition 2 are used.
(3)" and (4)" can be transformed into (3)' and (4)'. Combining them with (3) and (4), [Expression 22]
dl c~s ~ d, c~s~
o -d 2 sin (a~ O) = 1 d 2 sintY
r That is, -sin (~I-a) =-~ sin~
sin (2 ~ ~) ~ sin ( (~J 0) I
-- sin (~I 0) ~,o~l co~ 0) ~in~
~ 0' ' ~-1- 0 = n 7~
(where n is an integer.) (3) A proof that {7} and {8} are equivalent.
Substituting (1) through (4) into u " v 21+ u 2, v "
=--d, d 2 { coscY sin (~ sina C~1S (~+ o) }
=--d, d2 sin (2 cY-~O) = n ~ - ~ (where n is an integer.) 2 ~ ~- O = n ~ (where n is an integer.) ~ sin (2 CY~ ) = O
~Illl V21+ U21 v "= O

2~785~3 a + ~ = n~ - a (where n is an integer) {7}
UllVZl+U21V117~0 . . ~ . { 8 }
[End of Proof]
[Proof of lemma 2]
Figure 29 summarizes the result of examining signs of Ull and U21, when the value of a is a=0, O<a<~/2, a=~/2, ~/2<a<~, ae~, ~<a<3~/2, a=3~/2, 3~/2<a<2~.
[End of Proof]
[Proof of lemma 3]
(~/2)m ~ ~ < (~/2)(m+1) and (~/2)n $ a < (~/2)(n+1) where m and n are integers satisfyiny O ~ m e 3 and O ~ n e 3 Because ~ = ~ - a, (~/2)(m-n)-~/2 ~ ~ < (~/2)(m-n)+~/2 ~ (*) (1) If m-n is odd, it can be transformed into m - n = 2p + 1, where p is an integer.
That is, p = (m - n - 1) / 2.
Substituting this into (*), p ~ < ~ < (p +l) ~.
Therefore, Figure 31 is obtained.
(2) If m-n is even, because O e m ~ 3 and O ~ n e 3, -3 e m-n e 3 Since, m-n is even, there are only three [3] cases, which are m-n = O, 2 and -2.

Figure 33 shows a relation among m, n and the sign of sin~, when m-n is even.
More specifically, Figure 33 shows the conditions for determining the sign of sin~, by listing the possible combinations of m and n for these three [3]
cases.
When Ull and Vll have an equality in the conditions shown in Figure 33, the value of sin~ is zero [0].
As described earlier, Figure 32 shows a method for determining the sign of sin~, when m-n is odd.
Figure 32 is obtained as a summary of Figure 33.

Figure 34 illustrates a method for determining the sign of sin~, when m-n=0.
The following is a description of the process for obtaining Figure 33. Basically, depending on the values of m and n, the ranges of positions of Ul and Vl are drawn, thereby obtaining the inequalities between Ull and VlL capable of narrowing the value range for angle ~.
(a) When m-n=0, from (*), -~/2 < ~ < ~/2. There are four [4] cases shown in Figure 34.
By comparing the value of Ull with the value of 25 Vll, the value of ~ can be narrowed.

20785~3 Figure 3.5 illustrates a method for determining the sign of sin~, when m-n=2.
(b) When m-n=2, from (*), ~/2 < ~ < 3~/2. There are two [2] cases shown in Figure 35.
By comparing the values of ¦ Ull ¦ with the value of ¦ Vll ¦, the value of ~ can be narrowed.

Figure 36 illustrates a method for determining the sign of sin~, when m-n=-2.
(c) When m-n=-2, from (*), -3~/2 < ~ < -~/2.
There are two [2] cases shown in Figure 36.
By comparing the value of ¦ Ull ¦ with the value of ¦ Vll ¦, the value of ~ can be narrowed.
[End of Proof]
[Proof of theorem 6]
By a cross multiplication between (1) and (3), UllCOS ( a+~) = v11cosa ~(5) By a cross multiplication between (2) and (4), u21sin(a+~) = v21sina ' ''(6) (1) When U11~0 and U21~0 Since a is not integer times of ~/2 from Figure 29, cosa ~ 0, sina ~ 0 By expanding (5) and dividing both sides by cosa, U21( cos~-tanasin~) = Vll -Ulltansinl3 Vll-UllCOS~ ' ( 5 ) I
By expanding (6) and dividing both side~ by slna, u~, ( cos~- cot~ sin~) =v u 21 cotcr sinO--v 21--U ~I C0S(~ ~ ' (6)~
~y multiplying bath sides of (5)' by corresponding sides of t6)', --U l l U 2 1 S;~2 0-- ( V l l--U l l C05 ~ ) ( V l I--U ~ t C0s ~ ) ( U I I V 2 1 -~- U 2 1 ~r I I ) C0S 0 --U I I U 2 ~ V I I V ~ (7) Because Ullv21-1 U21Vll~O ' ' ' ' ( 5) COS ~7-- ( U I I U 2 1 ~- V I I V 2 1 ) / ( ~1 1 1 V 1 1 ~ U ~ I V I I ) ' . ' ' (~3) i sin~ cos2 o) l/2 , ~, , (9) The sign of (9) can be obtained from lemma 2.
Since sin~ ~ 0 from (5)' and {6), tana( UllCOS~-Vll ) /UllSin~
By combining the value range for a obtained from lemma 3, only one [1~ value of a can be determined.
From (1), dl ~ Ull /cosa ~ ( 10 ) From (2), dz =-U21 /sina - (11) (2) When Ull = 0 [Expresslon 23~
From lemma 2, a = Tl/2 ( when U21 < 0 ) 3T~/2 (when U21 > 0 ) Substituting cos~ = 0, sin~ = ~1 into (1~ through (4), ~ ~.

O = Ull ' ' ' ' (1) 1 -+ d ~ = u ~, . . . (2) -+d, sin~--v " .....
-+ d ~ cos O--v 21 . . , . . (4) That is, the left side of (7) = d1 d2 sin~ cos~ =
the right side of (7).
Therefore, (7) is outstanding in this case, too.
As well, (8) and (9) are outstanding.
Since sin~ ~ 0 in (3)' (from {6}) dl = ¦vll/sin~ (10) ' from (2)' d2 = ¦U21¦ ( (3) When U21 = 0 [Expression 24]
From lemma 2, a = 0 (when Ull > O ) = ~ (when u11 < 0) Substituting cosa = +l and sina = 0 into (1) through (4), d, = u "
O = u z, . . . (2) ~ d, cosO--v " , ~+d 2 sin~ = v 21 . . . (4)~
That is, the left side of (7) = - d1 d2 sin~ cos~
= the right side of (7).
Thereforer (7) is outstanding in this case, too.
As well, (8) and (9) are outstanding.

83 2~78503 From (1)", d1 = ¦u11¦ ~ (10)"
In (4)", since sin~ ~ 0 (from {6}) d2 = ¦v21/sin~¦ (11)"
[End of Proof]
So far, the new theory for use in the second form of this invention has been described in detail. By having the shape/movement recognition unit 18 shown in Figure 4 apply the new theory to a recognition of an object moving on a single plane coupled with a rotation, based on the result of extracting three [3]
feature points forming a right angle each in images captured at two [2] points in time for observation, it becomes possible to calculate the actual positions and movements of these feature points within a three-dimensional space and to recognize the movement of a moving body.
The concept of a moving body recognition apparatus in the second form of this invention is similar to that in the first form of this invention.
Also, an embodiment of the second form of this invention and the detailed configuration of the shape/movement recognition unit 27 are similar to those of the first form of this invention, which are shown in Figures 9 through 13. Accordingly, their descriptions are omitted.

Figures 37, 38 and 39 are flowcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the second form of this invention.

In Figure 37, when the processes start, the shape/movement recognition unit 27 determines in step S60 whether or not the known data input unit 31 shown in Figure 12 has received at least one [1] of expressions {6}', {7}' and {8}'. As with the first form of this invention, a signal from a sensor allows the determination to be made. The relations between expressions {6}' through {8}' and {6} through {8} are the same as those described earlier.
If the shape/movement recognition unit 27 determines in step S60 that the known data input unit 31 has received at least one [1] of expressions {6}', {7}' and {8}', the known data input unit 31 outputs in step S61 a deactivation signal to the feature point position normalization unit 32, and stores recognition disablement data in the movement calculation unit 34, thereby invoking a recognition disablement process {1}
shown in Figure 40. The flow of recognition disablement process {1} pursuant to the second form of " .

this invention shown in Figure 40 is essentially the same as that shown in Figure 17 pursuant to the first form of this invention.

Figure 40 is a flowchart of an embodiment of recognition disablement process {l} pursuant to the second form of this invention.
On start of recognition disablement process {1}, the movement calculation unit 34 sends recognition disablement data to the feature point position reconstruction unit 36 in step S62. Then, the feature point position reconstruction unit 36 stores the recognition disablement data in the object movement storage unit 28 and the feature point storage unit 29 in step S63, thereby ending the process.

Returning to Figure 37, if the shape/movement recognition unit 27 determines in step S60 that the known data input unit 31 has received none of {6}', {7} t and {8}', the shape/movement recognition unit 27 determines in step S64 whether or not the eature point storage unit 26 has stored all data on the positions of three [3] feature points in each of the images captured at two [2] instants in time or observation. The shape/movement recognition unit 27 ....

86 2~78~3 repeats step S64 until it determines that the feature point storage unit 26 has stored the positions of three [3~ feature points each in imayes captured at two [2] instants in time for observation. When the shape/movement recognition unit 27 determines in step S64 that the feature point storage unit 26 has stored all data, the shape/movement recognition unit 27 sends an activation signal to the feature point position normalization unit 32 in step S65, thereby activating the feature point position normalization unit 32.

Continuing on to Figure 38, the feature point position normalization unit 32 stores in step S66 data on the positions of three [3] feature points 0, 1 and 2, which the feature point position normalization unit 32 stores in the feature point storage unit 26, and obtains first components of two-dimensional vectors u and vi (where i= 1, 2) as X coordinate values of feature points 1 and 2, which are different from feature point 0, after feature point 0 moves to the origin and the other feature points 1 and 2 are parallelly displaced. Therefore, the X coordinate values of feature points 1 and 2 after the displacement in parallel with feature point 0 are obtained by subtracting the X coordinate value of 2n785~3 feature point 0 before the parallel displacement from the X coordinate values of eature points 1 and 2 before the parallel displacement.
Then, the shape/movement recognition unit 27 has the shape/movement determination unit 33 determine in step S67 whether or not these first components satisfy either expression {4} or expression {5}. If they satisfy neither expression {4} nor expression {5}, the shape/movement determination unit 33 sends recognition disablement data to the movement calculation unit 34 in step S68, thereby invoking recognition disablement process {l} shown in Figure 40.
As explained in the description of theorem 5, expressions {4} and {5} are e~uivalent to expressions

{6} and {7}. Therefore, it can be said that the check in step S67 and the check in step S60 form a duplication. However, since the check in step S60 can be performed b~ a sensor, such a duplication is instituted. Also, no further calculation is performed, when the shape/movement determination unit 33 determines in step S67 a recognition disablement (leading to recognition disablement process {1}) by using only the positions of the three [3] feature points each in images captured at two [2] instants in time for observation.

If the shape/movement determination unit 33 determines in step S67 that either expression {4} or expression {5} is outstanding, the shape/movement determination unit 33 activates the movement calculation unit 34 in step S69, thereby invoking respective processes of the movement calculation unit 34 and the shape calculation unit 35, in accordance with lemma 3 and theorem ~.

Figure 41 is a flowchart of the movement calculation unit 34 pursuant to the third form of this invention.
On start of its process, the movement calculation unit 34 calculates an integer n for determining the value range for a by using Figure 29 in step S70, and an integer m for determining the value range or ~ by using Figure 30 in step S71. Then, the movement calculation unit 34 executes a process for determining the sign of sin~ in step S102 or S105 shown in Figure 42 before step S76.

Figure 42 is a f lowchart of an embodiment of determining the sign of sin~.
After completing step S71 shown in Figure 41, the movement calculation unit 34 determines in step S72 A

~ 2078S~3 whether an integer m-n is odd or even. When the movement calculation unit 34 determines in step S72 that integer m-n is odd, the movement calculation unit 34 obtains the value of an integer p in step S73, and determines in step S74 the sign of sin~ by using Figure 40 depending on~whether integer p is odd or even. When the movement calculation unit 34 determines in step S72 that integer m-n is even, the movement calculation unit 34 determines in step S75 the sign of sin~ by using Figure 32.

Returning to Figure 41, the movement calculation unit 34 obtains all elements of rotation matrix R in step S76, and determines in step S77 whether or not rotation matrix R or its inverse rotation matrix R~1 satisfies the known data of an object inputted to the known data input unit 31. When the movement calculation unit 34 determines negatively (N0) in step S77, recognition disablement process (1) shown in Figure 40 is invoked. When the movement calculation unit 34 determines affirmatively (YES) in step S77, the movement calculation unit 34 sends data on angle a and rotation matrix R to the shape calculation unit 35 and the feature point position reconstruction unit 36 in step S78.

As with the second form of this invention, the selection of either one [1] of two [2] sets of solutions, e.g. a selection between a rotation matri~
R and its inverse rotation matri~ R~1 in the second form of this invention, is pari passu the description of Figure 19 pursuant to the first form of this invention.

Figure 43 is a flowchart of the shape calculation unit 35 pursuant to the second form of this invention.
On start of its process, the shape calculation unit 35 determines in step S80 whether or not both Ull and U21 have non-zero values. If the shape calculation unit 35 determines affirmatively (YES) in step S80, the shape calculatlon unit 35 invokes shape calculation process {1} shown in Figure 44 before continuing on to step S94.

Figure 44 is a flowchart of an embodiment of shape calculation process {1}.
The shape calculation unit 35 obtains the angle a between two-dimensional vector Ul and the X axis, in step S81, and then the absolute values d1 and d2 of two-dimensional vectors Ul and U2, in step S82.

~ .

Returning to Figure 43, if the shape calculation unit 35 determines negatively (NO) in step S80, the shape calculation unit 35 determines in step S83 whether or not Ull=O. If the shape calculation unit 35 determines affirmatively (YES) in step S83, the shape calculation unit 35 invokes shape calculation process {2} shown in Figure 54 before continuing on to step S94.

10Figure 45 is a flowchart of an embodiment of shape calculation process {2}.
The shape calculation unit 35 determines in step S84 whether or not U21 is smaller than zero [O]. If it determines affirmatively (YES) in step S114, the shape calculation unit 35 sets the value of angle a equal to ~/2 in step S85. If it determines negatively (NO) in step S84, the shape calculation unit 35 sets the value of angle a equal to 3~/2 in step S86. On completing either step S85 or step S86, the shape calculation unit 35 obtains the absolute values d1 and d2 in step S87.

Returning to Figure 43, if the shape calculation unit 35 determines negatively (NO) in step S83, the shape calculation unit 35 invokes shape calculation ~, process {3} shown in Figure 46 before continuing on to _ step S94.

Figure 46 is a flowchart of an embodiment of shape calculation process {3}.
The shape calculation unit 35 determines in step S90 whether or not Ull iS smaller than zero [0]. If it determines affirmatively (YES) in step S90, the shape calculation unit 35 sets the value of angle a equal to zero [0] in step S91. If it determines negatively (NO) in step S90, the shape calculation unit 35 sets the value of angle a equal to ~ in step S92. On completing either step S91 or step S92, the shape calculation unit 35 obtains the absolute values d1 and dz in step Sl23.

Returning to Figure 43, after completing shape calculation process {1}, {2} or {3}, the shape calculation unit 35 calculates the second components Ull and U21 of two-dimensional vectors Ul and U2 in step S94 and the second components Vll and V21 of two-dimensional vectors Vl and V2 in step S95. Then, the shape calculation unit 35 sends those second components to the feature point position reconstruction unit 36 in step S96.

Returning to Figure 38, when the shape/movement recognition unit 27 completes the process of the shape calculation unit 35, the shape/movement recognition unit 47 continues on to step S97 shown in Figure 39.
The feature point position reconstruction unit 36 identifies the transferred set of recognition solutions as a set of recognition solutions <1> in step S97, obtains a set of recognition solutions <2>
in step S98, and determines in step S99 whether or not either one [1] of the sets of recognition solutions <1> and <2> satisfies the known data stored in the known data input unit 31. The feature point position reconstruction unit 36 selects either recognition solution <1> or recognition solution <2>
by concave/convex data of an object.
If the feature point position reconstruction unit 36 determines in step S99 that neither set of recognition solutions <1> and <2> satisfies the known data stored in the known data input unit 31, the shape calculation unit 35 invokes recognition disablement process {2} shown in Figure 47, which is essentially the same as those pursuant to the first and second forms of this invention shown in Figures 22 and 33.

Figure 47 is a flowchart of an embodiment of 207~5Q3 recognition disablement process {2} pursuant to the second form of this invention.
In step S101, the feature point position reconstruction unit 36 stores the recognition disablement data in the object movement storage unit 28 and the feature point position storage unit 29, thereby ending the process.

Returning to Figure 39, i the feature point position reconstruction unit 36 determines in step S99 that either of the sets of recognition solutions <1>
and <2> satisfies the known data stored in the known data input unit 31, the feature point position reconstruction unit 36 identifies it as a selected set of recognition solutions. Then, in step S102, the feature point position reconstruction unit 36 stores in the object movement storage unit 28 rotation.matrix R in the selected set of recognition solutions together with the in-image coordinate values of feature polnt 0 stored in the feature point posltion normalization unit 32. Thereafter, in step S103, the feature point position reconstruction unit 36 stores in the feature point position storage unit 29 the second components of the two-dimensional vectors in the selected set of recognition solutions together 207~503 with the in-image coordinate values of feature points 0, 1 and 2 stored in the feature point position normalization unit 32, thereby completing its process.
Thus, the shape/movement recognition unit 27 ends its process.
Although set of recognition solutions <1> is obtained first in the above embodiment, it goes without saying that there is another embodiment in which set of recognition solutions <2> is obtained first, which requires onl~ the X axis to have the opposite direction.
As described above, the second form of this invention allows the movement of an object moving on a single plane coupled with a rotation to be recognized from the positions of three [3] feature points forming a right angle each in images captured at two L 2 ]
instants in time for observation.
Also, as explained in the description of Figure 31, the shape/movement determination unit 33 can immediately determine the recognition enablement or recognition disablement of the moving body from the in-image positions of feature points.

The Third Form Figure 23 shows a universal relation between an ob;ect and its observation plane in an image input unit pursuant to the third form of this invention.
Figure 24 shows a relation between an object and its observation plane pursuant to the third form of this invention, in which one [1] of feature points of the object is fixed at the origin o a three-diloensiollal coordinate system.
Embodiments of the third form of this invention are explained below. As described earlier, the third form of this invention is different from the first form of this invention, in that the actual position and movement of feature points in an object are obtained from the known position data of three [3]
feature points of an object at any three [3~ instants in time for observation, because the rotating speed of an ob;ect does not have to be constant and because the three [3] feature points do not form a right angle.
The movement and the rotation of an object and the relation between the direction of observing an obJect and the coordinate axes are the same as those in the first and second forms of this invention.
The relations between an ob~ect and its observatlon plane pursuant to the third form of this invention are the same as those shown in Figures 23 and 24 pursuant to the first and second forms of this _ ~ .

invention, except that the three [3] feature points do not form a right angle.
The assumption that images are obtained as orthogonally projected images observed from the direction of the Y axis is the same as that for the first form of this invention. Therefore, the displacement in the direction of the X axis is the same as that observed on the image plane.
Therefore, as explained in the description o Figure 24, after one [1] of the three [3] feature points, e.g. feature point 0, moves to the origin, the Y
coordinate values of other two [2] feature points, e.g. feature points l and 2, the angle of rotation around the origin of the object from a first instant in time for observation to a third instant in time for observation and the angle of rotation around the origin of the ob;ect from the irst instant in time for observation to a third instant in time for observation are calculated.
Described below in detail is the new theory used in the third form of this invention. It is necessary first to analyze the number of feature points and the number of images that must exist to establish correspondences. A theorem 7 gives the answer to the analysis.

20785~)3 [Theorem 7]
To be able to have a definite solution, it is necessary to establish correspondences of three ~3]
feature points each in three [3] images.
A system 7.1 clarifies the meaning of theorem 7.
[System 7.1]
A theorem 7 satisfies following propositions (1) and (2).
Proposition (1) Two [2] images produce no set of definite solutions, no matter how many feature points there are.
Proposition (2) Two [2] feature points produce no set of definite solutions, no matter how many images there are.
[Proof of theorem 7]
Assuming that m feature points each in n images have correspondences, the number of equations need to be greater than the number of unknowns. The following illustrates why conditions of m and n are m23 and n~3.
As described later, assume that a feature point 0 moves to the origin.
Codes are defined as follows.
Vk~ represents a vector from the origin to a j-th 25 feature point in a k-th image, where k=1, , n and 207~03 j=1, , m-l.
Rotation matrix Ri represents a rotation of an object from a first image to an i-th image, where i =
2, , n.
That is, a relation such as A first Ri An i-th instant instant in time in time for ob- for ob-servation servation Vl ~ > Vi;
(where i= 2, ~ , n and j=1, , m-1) can be expressed as vi~ = Ri Vlj (where k=1, , n and j=1, - , m-1) (1) Since unknowns are ~i, which is an angle of rotation represented by rotation matrix Ri, (where i =2, , n,) and third components (i.e. Y components) of vectors Vk~, (where k=1, , n and j=1, , m-1,) the number of unknowns is (n-1) ~ n(m-1) = mn-1 Because two [2] linear equations are obtained from each combination of i and j in expression (1), the number of equations is 2(m-l)(n-1). The number of equations must not be less than the number of unknowns to have a set of definite solutions.
That is, the difference obtained by subtracting `~
2078~3 the number of unknowns from the number of equations must not be negative. Therefore, (the number of equations) - (the number of unknowns) = 2(m-1)(n-1)-(mn-1)=(m-2)(n-2)-1 2 This is transformed into (m-2)(n-2) 2 1 Since both m and n are positive integers, m-221 and n-221. Hence, m23 and n23.
[End of Proof]
Although system 7.1 is a direct result of theorem

7, it can be directly proved. The direct proof is explained below.
[Proof of System 7.1]
(1) A proof that two [2] observations of n points produce no definite solution.
The relations between two-dimensional vectors for n-1 points other than the origin on a rigid body and a rotation represented by rotation matrix R are expressed as follows:
[Expression 14]
A first A third instant instant in time in time for ob- for ob-servation servation [Xl ] R [Xl ~ ti=l n~ [ ~1 -Conditional equations are transformed into [y"~ n[ ] (~ , n-1) That ls, ~ ] ~ CO~a, -sin 0~ [Xl ~ "~-- n-l~
yl' sina, cos O Yl The unknowns are Q, yi and yi', where i = l, and n-l.
Since the number of conditional equations is 2n-2 whereas the number of unknowns is 2n-1, there is no set of definite solutions.
(2) It is shown that n observations at two [2] points produce no set o deinite solutions.
The relation between two-dimensional vectors of a single point other than the origin on a rigid body and a rotation are expressed as [Expression 15]
A first A third instant instant in time in time or ob- for ob-servation servation rxI ~r* J ~ rcos~ sln 0 l ] ~ 2 ~ n)~ Rj L slnO3. cos O~

Conditional equations are [YJ ~ RJ [ ] ~=Z~ . n) That is, ~XJ ~ ~ CosOJ. -g1n ~J ] ~xl ] (~ 2 n)' YJ sin~3, co~ a~ y, The unknowns are ~, y1 and y~, where j=2, --- n.
Since the number of equations is 2n-2 whereas the number o unknown variables is 2n-1, there is no set of definite solutions.
[End of Proof]

Figure 48 illustrates the orthogonal pro;ections of feature points 0, 1 and 2 shown in Figure 24 on the XY plane at three ~3] instants in time for observation pursuant to the third form of this invention.
To calculate the data on the shape and movement of an ob~ect, it is sufficient to calculate the displacement of feature point 0 to the origin and the angle of rotation around the origin. Since the displacement of feature point 0 can be easily obtained as described earlier, the angle of rotation around feature point 0 need only be calculated.
A moving body recognition apparatus pursuant to the third form of this invention calculates and 2~78~03 outputs an angle of rotation of an object around the origin and a ~ coordinate value of a feature point when feature point 0 moves to the origin, in addition to the X and Z coordinate values of feature points in the input images.
The following is a description of the codes shown in Figure 48.
Feature point 0 is a feature point of an object moved to origin 0.
Feature points 1 and 2 are feature points on the object other than feature point 0.
Ul and U2 are two-dimensional vectors respectively representing feature points 1 and 2 at a first instant in time for observation.
Vl and V2 are two-dimensional vectors respectively representing feature points 1 and 2 at a third instant in time for observation.
Wl and W2 are two-dimensional vectors respectively representing feature points 1 and 2 at a third instant in time for observation.
R is a rotation matrix representing a rotation of the object around the origin from the first instant in time for observation to the third instant in time for observation, which is expressed as:
[Expression 16]

207~503 [R2] [ -r12 r11]
S is a rotation matrix representing a rotation of the object around the origin from the first instant in time for observation to the third instant in time or observation, which is expressed as:
[Expression 17]

S = [Sl ] [ Sll Sl2 ]
S2 -S12 Sll These are related by the following expressions:
vi = R ui and wi = S ui, where i = 1, 2.
That is, A first R A second A first S A third instant instant instant instant in time in time in time in time for ob- for ob- for ob- for ob-servation servation servation servation Ul ~` Vl Ul ~ W
U2 > V2 U2 ~ W2 Therefore, the problem can be formulated as follows:
[A Problem Of Recognizing A Moving Body On A
Single Plane]
ui, vi and wi are two-dimensional vectors whose first components are known, where i = 1, 2.

~7~03 R and S are two-dimensional rotation matrices.
vi = R ui and wi = S ui, where i = 1, 2.
Assuming the above, obtain third components of ui, vi and wi, where i = 1, 2, and the rotation matrices R and S.
A next theorem 8 presents conditions for solving this problem.
[Theorem 8]
The conditions of determining a set of solutions for a problem of recognizing a moving body on a single plane are to let {9}, {10} and {11} be all satisfied.
~Expression 18]

Ull U21 [ ] is regular. {9}
Vll V21 - Vll V21 ] is regular. --- {10}
- Wll W21 Wll W21 [ ] is regular. {11}
Ull U21 Here, that a square matrix is regular should be construed as follows:
a matrix being regular ~ its inverse matrix existing ~ the value of its determinant not being equal to zero [0]
Therefore, when determinants of matrices {9}, {10} and {11} are not all zero [0], the shape and movement of an object can be determined.
The proofs of theorems 8, 9 and 10 will be shown later in a batch.
A lemma 4 for use in proving theorems 9 and 10 is shown below. Lemma 4 shows a relation between the two [2~ two-dimensional rotation matrlces R and S.
[Expression 19]

R = [RZ ] S = [S2 ]

rRl L ] is regular. ~ R ~ + S

Here, a ~ sign indicates a necessary and sufficient condition. A next theorem 6 defines the meanings of conditions {4}, {5} and {6}.
[Theorem 9]
Conditions {9}, {10} and {11} in theorem 8 are equivalent to next conditions {12} through {15}.
[Expression 20]
R ~ +I '''' {12}
S ~ +I {13}
R ~ +S ~ ~ ' ' { 14}
[Ul, U2] being regular ~ [Vl, V2] being regular [Wl, w2] being regular {15}
where [
0 1 is an identity matrix.

207~5~3 Condition {12} means that the rotation of an object from a first instant in time for observation to a third instant in time for observation is neither zero degrees [0] nor one hundred and eighty degrees [180~.
Condition {13} means that the rotation of an object from a first instant in time for observation to a third instant in time for observation is neither zero degrees [0] nor one hundred and eighty degrees [180].
Condition {14} means that the rotation of an object from a third instant in time for observation to a third instant in time for observation is neither zero degrees [0~ nor one hundred and eighty degrees [180]. (This is because condition {14} is equivalent to RS-l = +I, where RS-l represents a rotation of an object from the third instant in time for observation to the third instant in time for observatlon.) Condition {15} means that the three C3] feature points 0, 1 and 2 are not on a single straight line, i.e. that the three [3] feature points 0, 1 and 2 form a triangle.
A next theorem 10 presents formulae for calculating a set of definite solutions.

~078~03 [Theorem 10]
Conditions of theorem 8 produce the next set of definite solutions.
tExpression 21]

R = [R2] [ -r12 rl1]
and = [Sl ][ Sll Sl2 ]
Sz -S12 Sll can be calculated by the following expression.

[Expression 22]

Define first that:

[a2 ] [ a2l a22 ] [Sl ]
Calculate al according to:
- Vll Vl2- -a1 = [all a2l] = [ Ull U21]
Wll W21 Rl and Sl can be obtained from components al1 and alz of vector a1 and the next expressions.
R1 = [(l+a112-a122)/(2a11) +(1-r112)1/2]

S1 = t(l-a11r11)/a12 -a11r12/a12]
Since the values of elements of a first row matrix R1 have been calculated, the values of all elements of rotation matrix R are obtained.
The values of all elements of rotation matrix S

are obtained similarly.

` `~
20785~3 Second components Ui2, Vi2 and W12 of two-dimensional vectors ui, vi and wi, where i - 1, 2 are calculated by the next expressions.
[Expression 23]
According to the already calculated values of R
and S1, a2 is calculated by the next expressions.
R -a2 = the second row of [ S1]

The following expression allows the second component of ui, where i = 1, 2, to be calculated.
Vll V21 [ U12 U22 ] = a2 [ ]
Wll W21 The next expressions allow the second components of vi and wi, where i = 1, 2, to be calculated.
Vi2 = Rz ui and wiz = Sz ui (i = 1, 2) [Expression 24]
Theorems 9 and 10 require the inverse rotation matrices of R1 Vll VZl and exists.
-S1 -Wll W21 This is se~uentially guaranteed by lemma 4 and theorem 8.
A next system 10.1 defines the correspondences between two [2] sets of solutions by theorem 10.
[System 10.1]
When one [1] set o the solutions is R, S, ui, v and wi, where i = 1, 2, the other set of solutions is [Expression 25]

[ ] R [ ] is inverse rotation matrix of R, [ ] S [ ] is inverse rotation matrix of S, 10[ 1 0] is a mirror image transformation of Ul 0 -1 with respect to the X axis, [ 1 0] is a mirror image transformation of Vl 0 -1 with respect to the X axis, and [ ] is a mirror image transformation of Wl 150 -1 with respect to the X axis.
System 10.1 corresponds to a case in which there are two [2] sets of solutions for an earlier described problem of recognizing a moving body. This shows that there are two [2] sets of solutions respectively corresponding to cases in which the Y coordinate values of other feature points are positive and negative as shown in Figure 49 when feature point 0 in an orthogonally projected image moves to the origin.
They also correspond to the surfaces of an object observed from a TV camera as an image input unit being 111 207~503 a convex and a concave.

Figures 49A and 49B show two [2] sets of solutions forming mirror image transformations of each other with respect to the X axis pursuant to the third form of this invention.
Figure 49A shows that feature points shown in Figure 47 are pro;ected over to the XY plane as a set of solutions, in which the surface of an object forms a convex, whereas Figure 49B shows the mirror image transformation of the set of solutions shown in Figure 49A, in which the surface of an object forms a concave.

A next system 10.2 allows an easier calculation of r11 by theorem 10.
[System 10.2]
When it can be assumed that the rotation of a rigid body is constant, i.e. when S = R2, r11 = a11/2.
The following are proofs of theorems 8, 9 and 10, in a sequence of proposition 3, lemma 4, theorem 9 and theorem 10.
[Proposition 3]
Expressions {12}, {13}, {14} and {15} in theorem 9 are conditions necessary for having a set of definite solutions.
[Proof of Proposition 3]
To reiterate the conditions, vi = R ui and wi = S ui, where i - 1, 2.
(1) Proof that condition {12} is necessary By assuming that R = +I, it is shown that there is no set of definite solutions.
Condition vi = R ui is transformed into Vl = +Ui.
Thus, the only remaining condition is wi = S UL.
As already explained in the description of proposition (1) in theorem 7, there is no set of definite solutions.
(2) Proof that condition ~13} is necessary This is exactly the same as the proof for {1}.
(3) Proof that condition {14} is necessary Assuming that R = +S, it is shown that there is no set of definite solutions.
vi = R ui = +S ui = +wi Hence, condition vi = R ui is reduced to vi = +wi.
Thus, the only remaining condition is wi = S ui.
As illustrated in the description of (1) in system 7.1, there is no set of definite solutions.
(4) Proof that condition ~15} is necessary Assuming that [Ul, U2] iS irregular, it is shown that there is no set of definite solutions.

2~78503 Since Ul and U2 are linearly dependent on each other, a1u1 ~ azu2 = 0, where (al, az) ~ 0.
Because U1 ~ 0 and U2 ~ O, 1 ~ O, and az ~ 0, it is described t~at u~ - aul.
Condition V2 = R U2 iS reduced to V2 = aVl.
Condition wz = S Ul iS reduced to W2 = aWl.
Thus, all the conditions vi = R ui and wi = S ui where i = 1, 2, are reduced to Vl = R Ul and Wl = S Ul.
~s already explained in the description of (2) in system 7.1, there is no set of definite solutions.
(5) Proof that condition {15} is necessary From conditions, [Expression 26]
[Vl, V2] = R [U1, uz] and [Wl, WZ] = S [Ul, UZ]
Since ~ alld S arc regular, [Ul, UZ] being regular ~ [Vl, VZ] being regular [Wl, WZ] being regular [End of Proof]
[Proof 1 of lemma 4]
[Expression 27]

[ Rl] [ cos~ -sin~ ]
Rz sin~ cos~

[ Sl] [ cos~ -sin~ ]
Sz sin~ cos~

114 207~3 By using -the next relation, being regular ~ 0 - Sl- Sl S
is calculated first.
Rl cos8 -sin8 = =-cos~sin~sin~cos~=sin(~
Sl cos~ -sin~
''' (1) Also, cos(8~ sin(8-~) RS-- sin(8-~) cos(~
(2) Rl Sl = 0 ~ sin(8-~)=0 [from (1)]

(8-~)=n~ (where n is an inteyer) ~ RS-l = ~I
~ R = +S
The contraproposition of the above proposition ls:
[ ~1] being regular ~ R ~ ~S
Sl [End of Proof]
~Proof 2 of lemma 4]
By proving tlle contraproposition, an original proposition is proved.
L Exp~ession ~8]

[ Rl] being irregular Z5 Sl ` ~

~ vectors R1 and Sl being linearly dependent with each other a certain number a satisfying R1 = a S
~ Rl = +Sl, (q'~lis is ~ccause t~lC lcllgtl}s o two-dilllellsional vectors Rl and S1 are both one Ll], since R and S are both rotation matrices.) R - +S
This is because R1=+Sl =~ R2=+Sz =~ R=+S, which is due to the generic form of a two-dimensional rotation matrix, whlch is rcos~ -sin~l sin~ cos~J.
[End of Proof]

[Proof of Theorem 9]

To reiterate the conditions, Vl = R Ui and wi = S ui (where i = 1, 2) (*) From (*), ui = S-1 wi and vi = ~ ui = R S-l wi.
[Expression 29]
Ull U21 ( S l ) l = [Wl W2] ' ' ' ' (1) - Vll V21 - ( p~S-l ) 1 From (*), Vll VZl R
_ _ = _ _ [Ul U2] ~ ' ' ' (2) - Wll W21 - - Sl -From (*), wl = SR-I vi and Ul = R-l Vi Wll W21 - ( SR~
_ _ = l [Vl V2] ' ' ' ' ( 3) - Ull U21 - _ ( R-l ) 1 From (1) through (3), {9} through ~11} are equivalent to next (4) through (9).
(s_l) is regular. (4) - ( RS-~
[Wl w2~ iS regular. ` (S) [ ] is regular- ~ - (6) [Ul U2] iS regular.~ (7) - (SR-l)l -is regular. ~ (8) (R-l) [Vl v2] iS regular. (9) From Lemma 1, (4), (6) and (8) are equivalent respectively to {12}, {14} and {13}.
S-1 ~ +RS-1 ~ R ~ +I {12}
R ~ +S ~ {14}
SR-1 ~ +R-1 ~ ~ S ~ {13}
(7) is equal to {15}, and is equivalent to (5) and (9) as illustrated in the proof of proposition 3.
Therefore, (4) through (9) are equivalent to {12}
through {15}.

/~

117 207 8~ ~3 [Proof of theorem 10]
[Expression 30]
vi = R u1 and WL = S Ui ( where i = 1, 2) Vll V21 R1 _ _ = - - [Ul U2] ''''(1) - Wll W21- - Sl -From conditions of theorem 9, which is R ~ +S
{14} and lemma 4, Rl [ Sl ] is regular, By combining condition {10} in theorem 8 and condition {15} in theorem 9, all the matrices in expression (1) are regular.
r 1 - r al 1 r all al2 1 L Sl J l a2 ~ l a2l a Because al Vll VZ1 -1 [ ] = [UlU2] [ ]
az Wll W22 Vll V21 al = [UllU21] [ ] ' ' . ' ( 2) Wll WZ2 Because parameters on the right side are known, a1 can be calculated.

11~ 2078~03 t Expression 32 ]
By definition, [ I ] [ ' ]

~l [rll ~2] [1 O] ~ 3) By oxpanding~
ra 1l r ~ a ~z s ~ tO
La~r,z~alts~ 5) [a ~ a " r 1 l ~ (6) a ~ z s I t =--a ~ I r ~

Ey squaring both two expressions, a ~ 1 2 --1 + a ~ ~ 2 --2 a " r "
a l ~ r " = ( ~ a " 2 --a l 2 2 ) / 2 - - (8) a , 119 ~78503 [ Expression 33 ]

This is because defining a~-0 from (3), a 12 [91~ 5 IZ] [I O]
a lz~- 0 ~herefore~St~ 0 S - ~ I This contradicts to the condition of theorem 9.
From (8), r "= ( l -~ a ~ a ,2~ ) / ( 2 a l ,) ~ (9) . rl2=~ (1--rl~2 ~ 1~2 . ~ 0) from (9) and (10), R1 can be caIculated.
a, 2 ;~

[ Expression 34 ]

This is because defining a ~z= O
from (3)~ [ ] [l 0]

a " ~ 0 therefQrerr~3 0 Hence, ~ =~ 1 This contradicts to condition of theorem 9.
From (6) and (7)s 1 l - ( 1 - a " r " ) / a 12 S ~2=~ a 1~ r ,2/a 12 . ~ (12) From (11) and (12), S1 can be calculated.

lZ0 2078~ 03 [ Expression 35 ]
A general expression of a two-dimentional matrix is, r co~ -sln~l L sin ~ . cos ~ ~
From R1, S1, the all ele,llen~s ~ r~tation matrises R and S can be calculated.

[ Expression 36 ]
al Rl ~-' Also, [a~ ~ [S~ ~ can be determined.

From (1), because [ ~ [~I ]~I [Vl~ v 2l~

r 1 r~ V2~ 3) therefore, I_U 121 U 2~ = a ~ L J

vi2=R~ u,, Wl2=52 Ul (i=l. Z) . . . (1~1) From (13), (1~), secvllu u~,mponets of V Ui, Vi, Wi ( i = 1,2 ) are finalized [ End of proof ]
[ Proof 1 of system 10.1 ]

From conditions Vi=Rui, Wi=Sui ( i - 1,2 ), ~7~5~3 [ Expression 37 3 [ -1~ 1 ( [ -1] [ -1] [ -1]

[ ] W ~ - ( [ ] S [ 1] ) ~ [ ] U 1 ) Sincc [ -1~ [ -1] [ -1] S [ 1] are rotation matrises, the first compone of [ - 1] [ - 1] [ - 1]

are e~ual t~3 Ji, Vi Qnd Wi, the ter~s ~xpr~ssion (1) form another set of solutions.
[ End of proof ]

1 Proof 2 of system 10.1 ]

2~ A next direct proof can be set forth from theorem 10.

120-2 207~03 [ Expression 38 ]

That the other set of solutions corresponding to rotatio~ matrises can be understood 5[ -1] ~ [ -1~' [ -l~ S [ -1]
from the general expression of a rotation matrix and an expression of theorem 10.

Matrix corresponding to [a ~ ] is [ a ] ~.
This is because 10( [ ] [ 1] ) [ -1] [5 l ] [~

The other set.of solution correspondin~ to ~, ul. ~I. w~ (l ~ 1. 2) is, 15[ ] ~ I ~ [ ] V L ~ [ ] W ~ . 2) . v2l Because, - [ulz ~2Z] - ( - az ) [ ]
Wl 1, W2~

[ l] ~ [ -1] ui ) ~2 U~ V

[ l] [ t] U~) = -S~ ul = - w [ End of proof ]

[ Supplement to the proof of system 10.1 ]

' 120-3 2078aO3 [ Expression 39 ]

For e;aniple, assuming S R [Cs~' -s~n~]
31n~, cos~ /

[ ] ~ [ ] _ [ COS0~ 9inO] [ CoS(-~) -9ln(-~
-1 -1 -slnO, cos~sin(- ~) C09(-~) is th~ inverse of rotation matrix R.

[ End of proo:' ]

[ I'roof of System 10.2 ]

[ E~pression 40 ~
cos O -sin Ocos ~ ~ -sin ~ o DefiningR = _ S = _ _ - sin~ cosO- , - ~In2 O co~Z ~- .
ra t 1- a 1 2l r cOs o . -sin Ol ~ ' Beca~s~L ~ l ~
a ~ " a 22 COS 2 ~ , -s in 2 O
a "---sln 2 O /-sln ( 2 a--~ ) = 2 c .' r l,= a l,/Z

[ End of proof ]

2~785~3 [End of Proof]
So far, the new theory for use in the third form of this invention has been described in detail. By applying this new theory to the shape/movement recognition unit 18 shown in Figure 4 for its recognition of an object moving on a single plane coupled with a rotation, based on a result of extracting three [3] feature points each in images captured at three [3] instants in time for observation, the actual movements and positions of those feature points in a three-dimensional space are calculated, for enabling the movement of a moving body to be recognized.
The concept of a moving body recognition apparatus pursuant to the third form of this invention is similar to that pursuant to the first form of this invention. Also, an embodiment of the third form of this invention and the detailed configuration of the shape/movement recognition unit 27 are similar to those o the first form of this invention, which are shown in Figures 9 through 13. Accordingly, their descriptions are omitted.

Figures 50, 51 and 52 are flowcharts showing in a three part series the whole processes of the ~ 122 ~$5~3 shape/movement recognition unit 27 pursuant to the third form of this invention.
In Figure 27, when the processes start, the shape/movement recognition unit 27 determines in step S60 whether or not the known data input unit 31 shown in Figure 12 has received at least one [1] of the expressions {12}' through {15}', (1), (2) and (3).
Here, generally, a satisfaction of expression {12}' is determined only after the calculation of rotation matrix R. However, the moving body recognition apparatus pursuant to the third form of this invention also has various sensors not explained in detail here, and determines whether the angle of rotation of the object from a first instant in time for observation to a second instant in time for observation is either zero degrees [0] or one hundred and eighty degrees [180]- Here, the prime signs in {12}', {13}' and {14}' represent e~uivalent transformations of {12}, {13} and {14} illustrated in the description of theorem 9.
If the shape/movement recognition unit 27 determines in step SllO that the known data input unit 31 has received at least one [1] of expressions {12}' through {15}', (1), (2) and (3), the known data input unit 31 outputs in step S111 a deactivation signal to s lZ3 the feature point position normalization unit 32, and stores recognition disablement data in the movement calculation unit 34, thereby invoking a recognition disablement process {l} shown in Figure 53. The flow of recognition disablement process {1} for the third form of this invention is essentially the same as that shown in Figure 17 for the first form of this invention.

Figure 50 is a ~lowchart of an embodiment o~
recognition disablement process {1} pursuant to the third form of this invention.
On start of recognition disablement process {1}, the movement calculation unit 34 sends recognition disablement data to the feature point position reconstruction unit 36 in step S112. Then, the feature point position reconstruction unit 36 stores the recognition disablement data in the object movement storage unit 28 and the feature point storage unit 29 in step S63, thereby ending the process.

If the shape/movement recognition unit 27 determines in step S110 that the known data input unit 31 has received none of {12}' through {15}', (1), (2) and (3), the shape/movement recognition unit 27 " 124 2078503 .the feature point position normalization unit 32, and stores recognition disablement data in the movement calculation unit 34, thereby invoking a recognition disablement process {1} shown in Figure 53. The flow of recognition disablement process {1} for the third form of this invention is essentially the same as that shown in Figure 17 for the first form of this invention.

Figure 50 is a flowchart of an embodiment of recognition disablement process {1} pursuant to the third form of this invention.
On start of recognition disablement process {1}, the movement calculation unit 34 sends recognition disablement data to the feature point position reconstruction unit 36 in step S112. Then, the feature point position reconstruction unit 36 stores the recognition disablement data in the object movement storage unit 28 and the feature point storage unit 29 in step S63, thereby ending the process.

If the shape/movement recognition unit 27 determines in step S110 that the known data input unit 31 has received none of {12}' through {15}', (1), (2) and (3), the shape/movement recognition unit 27 ~. ' '~
~ 125 20785~3 determines in step S114 whether or not the feature point storage unit 26 has stored all data on the positions of three [3] feature points each in images captured at three [3] instants in time for observation. The shapeJmovement recognition unit 27 repeats step S114 until it determines that the feature point storage unit 26 has stored the positions of three [3] feature points each in images captured at three [3~ instants in time for observation. When the shape/movement recognition unit 27 determines positively (YES) that the feature point storage unit 26 has stored all data, the shape/movement recognition unit 27 sends an activation signal to the feature point position normalization unit 32, thereb~
activating the feature point position normalization unit 32.

In Figure 51, the feature point position normalization unit 32 stores in step S116 data on the positions of the three [3] feature points 0, 1 and 2, which the feature point position normalization unit 32 stores in the feature point storage unit 26, and obtains first components Ui2, Vi2 and Wi2 of two-dimensional vectors ui, vi and wi (where i= 1, 2) as X
coordinate values of feature points 1 and 2, which are is instituted. Also, no further calculation is performed, when the shape/movement determination unit 33 determines in step S117 a recognition disablement by using only the in-lmage positions of feature points.
If the shape/movement determlnation unit 33 determines in step S117 that expressions {9}, {10} and {11} are all outstanding, the shape/movement determination unit 33 activates the movement calculation unit 34 in step Sll9, thereby invoking respective processes of the movement calculation unit 34 and the shape calculation unit 35, in accordance with theorem 10.

Figure 54 is a flowchart of the movement calculation unit 34 pursuant to the third form of this invention.
On start of its process, the movement calculation unit 34 calculates a first row vector al in matrix a in step S120, rotation matrix R in step S121, and rotation matrix S in step S122. Then, the movement calculation unit 34 determines in step S123 whether or not at least one [1] of calculation results of rotation matrices R and S satisfies the known data regarding the movement of an ob~ect inputted to the S.

.
127 2~78~03 known data input unit 31. In this determination in step S123, the movement calculation unit 34 selects the appropriate one [1] from the results of calculating rotation matrices ~ and S according to signals inputted from various sensors for detecting the movement of an object. If it determines in step S123 neither rotation matrix R nor rotation matrix S
satisfies the known data, the movement calculation unit 34 invokes recognition disablement process {1}
shown in Figure 53.
The selection of either one [1] of two [2] sets of solutions, e.g. a rotation matrix R and its inverse rotation matrix R-l, is pari passu the description of Figure 19 pursuant to the first form of this invention.
If the movement calculation unit 34 determines in step S123 that there is a set of solutions that satisfies known data about the movement of an object, the movement calculation unit 34 sends the selected one [1] of the results of calculating rotation matrices R and S to the shape calculation unit 35 and the feature point position reconstruction unit 36 in step S124.

Figure 55 is a flowchart of the shape calculation S.
128 2 07g 5 03 unit 35 pursuant to the third form of this invention.
On start of its process, the shape calculation unit 35 obtains the second row of a2 of matrix a in step S125. Based on the result obtained in step S125, the shape calculation unit 35 obtains U12 and U22 as the second components of two-dimensional vectors Ul and U2 in step S126, Vl2 and VZ2 as the second components of two-dimensional vectors Vl and V2 in step S127, W12 and W22 as the second components of two-dimensional vectors Wl and W2 in step S128. Then, in step S129 the shape calculation unit 35 sends to the feature point position reconstruction unit 36 the values of second components Ul2, U22, Vlz, V22, Wl2 and W22 of those two-dimensional vectors Ul, U2, Vl, V2, Wl and w2.

Returning to Figure 52, the feature point position reconstruction unit 36 executes its processes from step S130, on completing the process of the shape calculation unit 35 shown in Figure 51. In step S130, from the two-dimensional vectors u1, uz, Vl, V2, Wl and w2, and rotation matrices R and S obtained as the result of calculating the shape and movement of an object, the feature point position reconstruction unit 36 selects the values matching the known data about ~"s the movement of the ob;ect inputted to the known data input unit 31. If it determlnes negatively (N0) in step S130 that such values exist, the feature point position reconstruction unit 36 invokes a recognition disablement process {2} shown in Figure 56, which is essentially the same as that pursuant to the first form of this invention shown in Figure 22.

Figure 56 is a flowchart of an embodiment of recognition disablement process {2} pursuant to the third form of this invention.
In step S131, the feature point position reconstruction unit 36 stores the recognition disablement data in the ob;ect movement storage unit 28 and the feature point position storage unit 29, thereby ending the process.

The shape calculation unit 35 selects either one [1] from two [2] sets of solutions, which are mirror image transformations of each other, based on the concave/convex data shown in Figure 21.

Returning to Figure 52, when the feature point position reconstruction unit 36 can select a calculation result satisfying the known data in step ` 20785Q3 S130, the feature point position reconstruction unit 36 stores in the object movement storage unit 28 the values of elements of rotation matrices R and S in the selected set of solutions and the in-image coordinate values of feature point 0 in step S132, and in the feature point position storage unit 29 the values of the second components U12, U22, V12, V22, W12 and W22 of those two-dimensional vectors Ul, U2, Vl, V2, Wl and W2 in the selected set of solutions together with the coordinate values of the three [3] feature points each in three [3] images stored in the feature point position normalization unit 32 in step S133, thereby ending the process.
The above described embodiment assumes that a rotation around the origin of an object from a first instant in time for observation to a second instant in time for observation, which is represented by a rotation matrix R, is different from a rotation around the origin of an object from a first instant in time for observation to a third instant in time for observation, which is represented by a rotation matrix S. However, these rotations can be equal, when an object maintains a steady rotating speed and equal time intervals spacing apart those instants in time for observation.

131 ~0~8~3 Under such a circumstance, when the known data input unit 31 receives data that the rotation of an object is constant, the movement calculation unit 34 may substitute the above calculation of rotation matrix R (in step S121 shown in Figure 54) by [Expression 41]
Rl - [rll rl2] = [all/2 +(1-rll2)l/2]
for reducing the processing load..
Further in this case, the calculation in step S120, which is [Expression 42]
Vll V21 al = [ all, al2 ] = [ Ull, U21 ] [ ]
Wll W21 is performed for obtaining only vector al of a partial matrix, thereby calculating rotation matrix S by using an equation S = R2 in lieu of going through step Sl22.
The above embodiment allows two [2] sets of solutions to be calculated simultaneously when the movement calculation unit 34 calculates rotation matrix R in step S121. However, the above embodiment can be reconfigured, such that the movement calculation unit 34 obtalns only one [1] set of solutions in step S121.
In this alternative embodiment, the feature point position reconstruction unit 36 calculates a set of `~
132 2078~03 rotation matrices R~1 and S-1, which are the inverse to the set o rotation matrices R and S, according to the next expressions.
[Expression 43]

[ rll -rl2 ] [ Sll -S12 ]
rl2 rll S12 Sll Also, the other values of two-dimensional vectors ui', vi' and wi' (where i = 1, 2), which are the mirror image transformations of ui, vi and wi (where i = 1, 2) with respect to the X axis are calculated by the next expressions.
[Expression 44]
Uil Vil Wil [--Ui2 ] [--Vi2 ] [--Wi2 ] ( i = 1, 2 ) This enables the feature point position reconstruction unit 36 to obtain two [2] sets of solutions.
As described above, the third form of this invention allows the movement of an object from a positions of three C3] feature points each in images of an object moving on a single plane coupled with a rotation captured at two [2] instants in time for observation. As described in the descrlption of Figure 51, the shape/movement determination unit 33 can immediately determine whether or not a moving body -133 2078~03 can be recognized from the in-image positions of feature points.

The Fourth Form Figure 57 shows a universal relation between an object and its observation plane in an image input unit pursuant to the fourth form of this invention.
Embodiments of the fourth form of this invention are explained below. As described earlier, the fourth form of this invention is for recognizing a moving body in the most generic cases, in which the actual positions and movements of four [4] feature points of an object are calculated by determining, from known position data on the four [4] feature points each in orthogonally projected images captured at any three [3] instants in time for observation, that the four [4] feature points do not exist on the same plane, that the axis of rotation of an object is not parallel to the direction of the orthogonal projection between any two [2] instants in time for observation of the three [3] instants in time for observation, and that rotation of the object between any two [2] instants in time for observation is not a rotation of one hundred and eighty degrees [180]
around the axis parallel to the plane on which the --~- s 134 2073~03 images are orthogonally pro;ected.
As with the first, second and third forms of this invention, in the fourth form of this invention, an orthogonally projected image is used. Except the fact that the direction of observing an image is in the direction of the Y axis, the second form of this invention has no limitations on the rotation and movement of an object, unlike the first, second and third forms of this invention.

Figure 58 shows a relation between an observation plane and an object whose feature point 0 is fixed to the origin pursuant to the fourth form of this invention.
Figure 59 shows orthogonal projections of feature points on the XZ plane pursuant to the fourth form of this invention.
The displacements of an ob;ect in the directions of the X and Z axes are the same as those observed on an image plane. As shown in Figures 58 and 59, the angle of rotation of an ob;ect around the origin and the Y coordinate values of other three [3] feature points (e.g. feature points 1, 2 and 3) when one [1]
feature point (e.g. feature point 0) moves to the origin are calculated by an expression ..
- 2078~03 A new theory or the fourth form of this invention is explained below. The theory discusses a problem of reconstructing the structure (positions of feature points) and movement of an object from obsexvations of four [4] feature points each in orthogonally projected images captured at three [3]
instants in time for observation. As described earlier, one [1] feature point is fixed to the origin for a problem formulation.

Figure 59 showing the relation between an object and the image axis illustrates a problem of obtaining the positions and rotations of three [3] triangles in a three-dimensional space rom three [3] in-image triangles.
Codes are defined as follows.
Feature points 1, 2 and 3 are feature points other than feature point 0, which falls on the origin.
Ul, U2 and U3 represent three-dimensional vectors from the origin respectively to feature points 1, 2 and 3 at a first instant in time for observation.
Vl, v2 and v3 represent three-dimensional vectors from the origin respectively to feature points 1, 2 and 3 at a second instant in time for observation.

Wl, W2 and W3 represent three-dimensional vectors from the origin respectively to feature points 1, 2 and 3 at a third instant in time for observation.
[Expression 10]
The vector expressions are as follows.
--Ull-- --Ul --V1~ --VIW~ W
U ~ 8 U ~ 2_ ~ V t '' V t 2 -- ~ W I n' 1 2 ' tl t 3 ~ --V 1 ~-- --V 1 3 W ~
(~1.2, 3) It is assumed here that the first, second and third components respectively represent X, Z and Y
coordinate values, instead of X, Y and Z coordinate values.
Rotation matrix R represents a three-dimensional rotation of an object around the origin ~rom the first instant in time for observation to the second instant in time for observation.
Rotation matrix S represents a three-dimensional rotation of an ob~ect around the origin from the first instant in time for observation to the third instant in time for observation.
Rotation matrices R and S are expressed as follows.

-r" r 12 r Ir -r l - -R' --s ~ s ~z s 1l- -s l - -S' -R - r21 r22 rZ3 -- r~ -- S-- 5 1l 5 22 SZ~ ' S ~ F
-r~ r,2 r,3- -r3- -r3- -Sll SIZ S~3- -S~- -S3-2~78503 The reason why the Y coordinate values become third components of vectors here unlike a generic expression is because expansions of expressions are easier when the Y coordinate values become the third components of vectors, since it is the Y coordinate values that are unknown.
Accordingly, vector relations can be expressed as vi = R ui and wi = S ui (where i=1, 2, 3). That is, A first R A second A first S A third instant instant instant instant in time in time in time in time for ob- for ob- for ob- for ob-servation servation servation servation Ul ~ Vl Ul ~ Wl UZ > V2 U2 > W2 U3 > V3 U3 ~ W3 Codes of rotation matrices R and S are defined as follows.
20O(n) represents the entire class of orthogonal matrices.
SO(n) represents the entire class of n-dimensional rotation matrices.
A' is a matrix with the last row of A shed when A
25is a matrix having plural rows and plural columns, and " ~ 2078~03 a matrix with the last column (row) shed when A is a matrix having only one [1] row (column).
Ri~ is an n-l-dimensional matrix obtained by shedding the i-th row and j-th column when R is an n-dimensional matrix.
Thus, the problem can be formulated as follows.
[A Movement/Shape Recognition Problem]
Assuming that ui, vi and wi are three-dimensional vectors whose first and second components are known, where i = 1, 2, 3, R and S are three-dimensional rotation matrices, such that vi = R ui, and wi = S ui, where i = 1, 2, 3, obtain the third components of ui, vi and wi, where i = 1, 2, 3 and rotation matrices R and S.
[Expression 12]
vl Rul, w,;Su, (i=l, 2.3) [w, ]-[s ~ul V ' V2 ' V~ '] [R'~ [ul U~ Ul ] ...- -(!) w,'w~'w,' S' u,~ U21 U~
A next theorem 11 gives conditions for solving the problem.
[Theorem 11] Next conditions {16}, {17} and {18} are necessary to prevent the above movement/shape recognition problem from degenerating in a sense of 2078~03 four [4] points and~three [3] observations.

[Expression 13]
- Ul I U2 ~ U3 ~ -rank = 3 {16}
Vl ~ V2 ~ V3 Vl ~ V2 ~ V3 rank ] = 3 {17}
- Wl~ W2~ W3 Wl~ W2~ W3 rank ~ ] = 3 {18}
Ull U2~ U31 A next theorem 12 presents reasons of the conditions.
[Theorem 12]
{16}, {17} and {18} in theorem 11 are equivalent to {19} through {22}.
[Expression 14]
{19} R ~ -U

;
- dct U-(where U is a two-dimensional orthogonal matrix.) {20} S ~ -U

- det U-(where U is a two-dimensional orthogonal matrlx.) - {21} S ~ -U R

- det U-~`~
20783 ~3 (where U is a two-dimensional orthogonal matrix.) {22} Ul, U2 and U3 are linearly independent;
Vl, V2 and V3 are linearly independent; and Wl, W2 and W3 are linearly independent.

The meaning of {19} is as follows.
"~otation matrix R does not represent a rotation around the Y axis, which is the third axis, or a rotation whose angle around the axis of rotation on the XZ plane is one hundred and eighty degrees [180]."
The meaning of {20} is as follows.
"Rotation matrix S does not represent a rotation around the Y axis, which is the third axis, or a rotation whose angle around an axis of rotation on the XZ plane is one hundred and eighty degrees [180]."
The meaning of {21} is as follows.
"There is neither any rotation around the Y axis, which is the third axis from the second instant in time for observation to the third instant in time for observation, nor any rotation whose angle around the axis of rotation on the XZ plane is one hundred and eighty degrees [180]."

"~ 2078503 The meaning of {22} is as follows.
"The object has a three-dimensional shape."
The meanings o {19}, {20} and {21} will be described later in proofs of proposition 5 and lemma

8.
A next theorem 13 describes the equations for calculating the solution.
[Theorem 13]
Under conditions of theorems 11 and 12, a set of solutions is inalized according to next steps.
[Expression 15]
A first step is to calculate - a 1 -~ V I ~1 2 V 3 When is regular and -WIV W2V W~D ~ _ - -Vl ' V2 ' ~ -a" a~2 al~-u~ uz u~, --' a2~ azs az~
- - ~WIV W2D W3P ~ -a 31 a 32 a 33~
Select a single value for p (l~p~2), such that ~a ll a 12--a2, a2z-becomes regular. When a selection of p fails, an attempt is made to select p by rotating the XY
coordinate system by other than integer times of ninety degrees [90].
[Expression 16]
A second step is to calculate [ ]
S p 14~

There is an h (l~h~2), such that ah3 ~0.
The values of ~i (l~i~2), ~ and r are determined by next expressions.
a~ a ~2 - a l3 2) /Z (1~ 1 ~ 2) 2 ) ~- ~"a23--a2~a~3, alza2a--az2ala) r-= (r~ r2 T3 ) - (a2a~ -al~, a~
{1} in second step is to calculate [r ~I r 12]
r21r when a13 ~ 0 and a23 ~ 0, ~ I] [~ ] ~~ [r I ]
When a13 = 0 and a23 ~ 0, rrl~] [~1 /aZ~] [ ] [ ] [ ]
Lr 2~ 123 r 2Z a 2 ' C~
When all ~ 0 and a23 = 0, [ ] ~ ~ [ ] [ r l 2] [ ~ / a l 3]
r 21 -a I' ar~ r 22 -~2 /a [Expresslon 1/~
{2} in second step is to calculate [r la]

r~ rl~Z --r~Z2 ) ~2 r2l- when rl3 ~ 0; - ~I rI3/ a2 when rl3 = o; ~tl--r z~2 -- r 22r)~2 5 ~ "-- ~ a h k r k ~ ) / a h 3 ~ 1 5 J ~3) A third step is to calculate the third component of Ul, which is the third row of a 3 - [

`~ 2078~3 l4~
- rvl~ v 2' '13' ~11 1 3 U z ~ ~1 3 3~ = a 3 (l ~ q ~ 2, A fourth step is to calculate ~' , , -I
r ~ r~ z U 3 Sp lWIq W2~ W3~ a3 LUI-~ U23 ~33 A fifth step is to calculate R and S by next formulae.
r3J=(-I)3~J det R~J, s~J=(~ 'J det S3J

A sixth step is to calculate the third components of vi and wi by next formulae.
' v ,~= r3 u I, w l~=S~ U I
A next system 13.1 defines the relation between two [2] sets of solutions.
[System 13.1]
When one [1] of the two [2] sets of solutions in theorem 13 is [Expression 18]
~, S, [Ul3 UZ3 U~3]~ [Vl~ v2a vaa~, [Wl3 ~i3 W13]
Then, the other set of solutions is [ -1 ] [ - I ] ' [ - I ] S [ - I ] ' --[U l ~ U Z J IJ ~ [ V l ~ V 21 V ~3~], --~W I 3 W 2~ W IJ~
where I2 is a two-dimensional identity matrix.

Figures 60A and 60B show two [2] sets of `~ 20785~3 I ~
solutions forming mirror image transformations of each other with respect to the XZ plane on which an image is projected pursuant to the fourth form of this invention.
The proofs of the above theorems are presented in the sequence of proposition 4, theorem 12, theorem 13 and system 13.1.
[Proposition 4]
{19} through {22} in theorem 12 are necessary to prevent the above movement/shape recognition problem from degenerating in a sense of four [4] points and three [3] observations.
[Proof of Proposition 4]
It is shown that the movement/shape recognition problem degenerates into a problem in which the number of instants in time for observation is less than three [3] or the number of observations is less than four [4], when conditions are not satisfied.
[Expression 19] -U
{19} R ~ .

- det U-v ~ =~ u I
(where U is a two-dimensional orthogonal matrix.) ~ U ~
~_~ . ................. .~
tlel U- -U 12 2078~03 > j- V I --- U - U I
!-V 13 ~ C~ U - U 13 (d~t U--~ l ) Since Vi3 iS unknown, there are no data for Ui3.
Therefore, this degenerates into a shape/movement recognition problem comprising only wi = S ui.
[Expression 20]
{20} -U

- ~5~ U-(where U is a two-dimensional orthogonal matrix.) As with the case of {19}, this degenerates into a shape/movement recognition problem comprising only vi = R ui. -U
{21} S = . ..

- dc~ U-Wl --S U~
(where U is a two-dimensional orthogonal matrix.) ~-WI ' 1 rU 1 -Wl ' - -U - ~ ;~t ' -U vl ' Rul -w~ - - deL U- ~ Wl~ --dct U ~ v~ L

Since Wi3 and Vi3 are unknown, there are no data for Ui3. Therefore, this degenerates into a shape/movement recognition problem comprising only vi = R Ul.

20785~3 [Expression 21]
{22} When Ul, U2 and U3 are not linearly independent, the shape/movement recognition problem degenerates into:
ui is a linear combination of Ul, '', Ui-l, Ui~l, '''', 0 U3, for an existlng i (where l~i~3), and vi = R ui and wi = S ui (i = l, 2, 3) <
vi = R ui and wi = S ui (i = 1, , i-1, i~1, , 3) [End of Proof]
[Proof of Theorem 12]
15 [Expression 22]

~v~ ~ V2 ~ v, W I Yf 2 W 3 [S ] [U l ~1 2 U ~ ] ' From Wl = S Ul = S R-l Vi and ui = R~1 vi, w ' w ~ w,~ ~
U I ' 112 ~ U~ I .
[R-" ][vlv2v~
From ui = S-1 wi and Vl = R ui = R S-l wi, u,-s-lwl~vl Ru, RS-Iwl rUI ~ U~ ' U~ ' 1 L v, v 2 v, [ ] [Wl W2 W

Therefore, {16}, {17} and {18} are equivalent to <1> through <6>. (This is because for a matrix product AB, rank AB ~ rank A, rank B.) R' [ S ' ] <1>

U1, U2 and U3 are linearly independent. <2>
(SR-l) I
rank - 3 <3>
_ R-l Vl, V2 and V3 are linearly lndependent. <4>

rank [ ] = 3 .... <5>
(RS-l)' Wl, W2 and W3 are linearly independent. <6>

[Vl V2 V3~ R[Ul U2 U3]-.

[Wl W2 W~ ] 8S [Ul U2 U~ ] ' - 2û78~ 03 1 4~
Therefore, <2>, <4> and <t;> are equivalent to each other. {22}
From lemma 6, <1>, <3> and <5> are equivalent to {21}, {20} and {19}.
S [ I dct.U] It ~ (2 [ - ~ S R - ~ ~ S ~ [ - ~
de~U detU ~(2 0 ) [ ! dcLU~ S R [ i dëLU~ (1 9) [End of Proof]
[Expression 24]
VlVZ ' V3 ' ~
When [ J is regular, and WlrW2~ W3~

Vl ' V2' V~' -I all al~ a~ .
[U~ ù, u~ ~ [ ~--[al, a22 a2~1 Wl, W2~ W~, a " a ,2 a ~J

select one [1] value of p (l~ps2), such that ;. ~, [a2l a22]
is regular.
When it cannot select one [1~, select a value of p by rotating the XY coordinate system by other than integer times of ninety degrees [90~]. Lemma 7 shows a value o~ p can be thus selected.

[W ~, W, W " ~ [ 5 ] [111 U 2 U ~ - - tZ) R' ~v, ' V2 ~ V~ f [S~ [ J llV~D ~2~ W~r ~
By expressing rR~ -a, -a" a,2 al3-_ a2 . . a2, a22 a2, s,. a~ a" a,2 a,~

2~78503 all a,z a2~ a2Z ?$0 -al v~ ' v~ ' V~
] ~ [U, U U~ ~ [ ]
-a 2 ~lr~ r2D W~

-R' -The formulae for calculating can be deducted from this. However, since the deductive process is long, it is shown at the end of the proof.
(Refer to steps 1 through 6.) Therefore, a3 can be calculated.
[Expression 25]
From (2), -R~']-ILvl~v2'vl~] [Ul~u2~
ls,. wl, r~2" W~ Ul~ U~ U~t Therefore, [u~3 U2~ u~]~a~ [ ]
~ 2~
[Vl ~ V2 ~ V~ ~ ] [R~ ] [Ul ~ U2 ~ U~ ~ ]
From (1) w~ ' w~ ' w, ' S' u,, u,. u [n''] [V~ ~ V2~ V~ ] [U~ ~ U2~ U~
S' Wl ' W2 ~ W~ ~ Ul~ Ul~ J
Therefore, sq, such that l~q~2 and q~p, is obtained from u, u ~ u, - - ' S.l -' L~ W2~ Wl~l l J U I ~ U ~I U ~ -Since R' and S' are obtained, R and S are obtained from lemma 5.
From conditional expressions, vl~!nrl Ul, Wl~ 5l ~ Ul Therefore, [vl, V23 V~3]~ [~ 2~ ] iS also - - 2078~03 obtained.
Steps 1 through 6 below describe the procedure for deducting the remaining formulae for calculating rRl l L Sp J.
First, rl is determined by deducting simultaneous linear equations only of ri, whose result is used for obtaining sp [Expression 26]
By definition, -a ~ - -r ~ - -I O 0-r, -a2- -s.~ ~ I ~~
The i; component is ~:la,hr"J-~al~p3~lJ (1~1~;2, l~;J~3) (3) Here, when i=j, ~IJ- I. and when i~ = 0 By transforming (3), ~ 1 3 ~ v 3 -- ~ I 3 ~;~ ~ I k r k J ~ 1 ~ 1 ~ 2, 1 ~ ~ ~ 3 ) ( 3)~
[Expression 27]
Step 1 -r 1l-a.~ ; 2 ) ~~ 21-where ai = (~ a ~ 32 ) /2 k = I

By squaring both sides of (3)', 1 51 2078so3 (al~5~) 2 ~ J--~ a~l~r~

When an addition is performed with respect to i, ~ (al~ 5rl) ~ ~Xl (ISIJ--~J~ r~
The left side = a 1~ ( s ~ s The right side = ~ lJ~ x a,~r~,2-2~"~_lal~r~l ~2allal2rllr~) ~ IJ2 -~ ~ a l~2 ~ r ~J~ - 2 ~,~lJ~lal~r~J ~2allal2JrlJr~J

~ ~1 a ~ ~ 2 --2 ~ a ~ ll r ~ l That is, 2 a 1~2 . L ~ ~l a 112--2 ~l a li r ~1 ~ al~r~ ~ a~Z --a~ ) /2 (a~l; alz) [ ] ~rl al [r2~]
( I ~i I ~; Z ) [Expression 29]
Step 2 rrl l a~l a a ¦ ~ r ---- (r)) P ~ ~al~a,~ - a,~alz az~ az~
P-- (~, Rz ) ~ (a"az.--az,a,.: al~,--a"al~) r (r, r2 r, ) - ~az" -al, O) By replacing i with 1 and 2 in (3), for j, which is 1~ 3, 207~ 03 [Expression 30] 1 5 2 -~ ai k r k~ a l3S~3~

;1 Zk r kJ-I- a 2~ S r~= ~ 2~
sp~ is eliminated by (the upper equation) x a23 -(the lower equation) x a13, 2 ~ Ik ~ 13 ~ ~J a l~
;1 2k a z3 ~ ~ J a z3 10 ;~ ~k r ka--r - . (P ~ ~Z ) ~-r 1 ] ( r [Expression 31]
Step 3 ~lrk r~h~ 2) Assuming l~i, h~2, i~h and 1~j~3, From (5) ~l r IJ~I ~1, r I~J - r J
~ ,~ r ,~ r ~ r ~ l [Expression 32]
Substituting into ~l r lJ r Since ~, r~ ~r~ r~ r3 rlJ--Because r ~ = O , ~ r ~t r ~
k - I
by further replacing ~ with k, [Expression 33]

~Step 4 1 53 al3~0 or a23~0 If al3~0, (where 1~is2) [Expression 34]
5The third column of -r~- -al - -l 0 0~ -r 13- --r2 a2 = O 1 0 ~5 r2a ~33= 0 -s p - -a 3 - -O O 1- -S P3 Because sp3a33=1, a33~0 10[rl~ ~ ] I
- ~dclU-This contradicts Wit~l the assumption.
[Expression 35]
-r 1l r 12-Step 5 L j;
r z, ~ ~.2 15When al3~0 and a23~0, [r~,] [~l' ] [~l ] (1~i~2) When al3=0 and a23~0 [r 2l] [a 2 /a Z8] [r 2Z][;~ ;, ' ] [~Xz ]

When al3~0 and a23=0 [r 11] [~12 ] -t [r I ] ~-r ~2] -- [ -aZ /a 13~
-r 13- r 13=+ ( 1--r I IZ --r 12Z ) 1~2 ; r 23_~When rl3~0 - ~ I r ~ 2 -r z~~ Lwhen rl3=0 ; ~ (l - r 212 - r Zz2 ) 20~8503 ` When one set of the solutions is expressed as ~r ~
r z~

Then the other set of solutions is expressed as ~r, ~]
r,3 5 To reiterate (4), (5) and (6), [Expression 36]
-r 1l- -r ~ -a I ' - ~ 2 ) (~) a -r ---(5) - r 2~- - r 2 -( r ' . r ~ ' ? a ~ 1 5 2 ) (~) Because r' (rl.r,) (a", -a") I-a2~r~--a~r 12--a, .... (G)' a22r2,--a l,r22~R~
From columns 1 and 2 of (4) and (5), [Expression 37]

[~ [ ] [ ] (7) (l~i~2) When [- 2 1]

can be calculated uni~uely.
al2 ,172 a~ al2~ a~2l a~--¦ [ ~] al2_[al2] al~ a~
a~ a" at~ a21 a21 1 a" a2,¦a21 a22 a2.
a~l al2 I a~ a~tl When i=1 a~, I ' i al;
a,l a~, I a" al, When i=2 a z~ a " a "
I a21 a,2l ¦ a~ a,2¦

[Expression 38]
When a13~0 and a23~0, From ~7), r rll rl2 L ~ 1 55 r2l r2z is obtained uniquely.
When a13=0 and a23~0, [ ] [
rzl t~2/~23 From i=2 in (7), [r ~2~ iS determined.
r z2 When a13~0 and az3=0, From i=1 in (7), [ ~ is determined.
~2l ( ~ [ ] --r r ~z -~ 2 /a î 3 (any combinations) 2 2 1~2 rr~3-l r,3=~ (1--rl~ --r~
Lr 23~ r 2~a ( 1- r 21Z - r z22 ) I~2.
From column 3 in ~5), ~1 r~ 2 r23--~ (. ra ==U) Because ~ 3 a 2z a 2~
From the relation of an inverse matrix, ,B2 5 0 ~ rl3 = O
When rl3~0, ~2~o, From r23 = -~1 rl3/ ~2 The sign of r23 is determined.

"` ~ 207~03 - i56 Therefore, when one set of solutions for r13 and r23 is expressed as (r13, r23), every set o~ solutions can be expressed as (+r13, +r23).
[Expression 39]
Step 6 For an i (l~i~2) such that al3~0, because S~J= (~ alkrk~) / a~ 3) sp is ob~ained from r1 and r2.

When [r ] [r~~ r~ ~ r2~] Sp - ~S~I Sp2 ~Sp~

The first half is obtained from (3)', while s is obtained from nc~ (k = 1, 2) in the same row.
[Expression 40]
Especially when j=3, S V~=--~ a lk (~1~ r k3) /a h=l Therefore, S 1> [S Pl ~ ~-2 ~ p~]
[End of Proof of Theorem 13]
[A Proof of System 13.1]
[Expression 41]
From step 6, -- r, - r 1 ~ r ~ 2 ~1~ t' I l-r~ = rz~ r22 ~ r2~
- s p - - s ~ p ~ S r 3~
A set of solutions corresponding to the positive 2~78~3 signs are expressed as R, S, ~ ~ - - -U 13 llz3 U~ , Vl ~ V2~ V;~ , Wl~ Wza W3:

Then, R corresponding to the negative signs is r~

(from lemma 5) -1-Because by definition, ~' -a ~ -~ a2 From the relation of an inverse matrix, a I
a2 in case of the negative siyns -a 3 -- al-a 2 From a proof of theorem 13, [Expression 42] -Vl V 2 V 3 U13 U23 tl3~ =a3 (1p ~p \~/3p in case of the negative signs is U13 Uz3 U33 _ U13 Uj!3 1~3:~

Also, from the proof of theorem 13, ` ~ S 2~78~3 [Expression 43]
U~ U2 U~
Since S ~ W~ W~' W3' ~ l3 U~
In case of the negative signs, U~ z~ ll ttl~ U2~ U~
has the signs in ltS n-th column inverted, S' has the signs in its third column inverted. As with the case of R, Rotation matrix S corresponding to the negative 1 2 - - I z signs is (from lemma 1) [Expression 44] -v~ R'- -u~ '- -S'- -Ul'--~ , (l~l,Z,3) -v~ r,- -Ul2 - -wl2 - -S~- -Ul~ -In case of the negative signs, the first expression is -- I 2 -- -- I 2 -- -- U ~ I 2 U l ~ -- V I ~ ----1- -- --U 13 - -- --1-- U 1~ V 1~
Similarly, the second expression is [ _ ] S [ -1] [; ] - [ _J S [ ] ~ ]
Therefore, ln case of the negatlve signs, V IJ Vz~ V ~ W2~ ~:13 become --~ Vl~ V2~ V~, -- Wl~ W2~ W~3 15q [End of Proof]
The lemma used in the proof of the above theorem is proved below.
[Lemma 5]
When n22, [Expression 45]
In an n-dimensional rotation matrix (l)R, _~ ~, r In o R3 ~ rl~-------r n l r nn When (n-1)-dimensional matrix obtained by eliminating the i-th row and j-th column is expressed as Rij, ~J~ J dct R IJ

[ Proof ]
For a regular matrix A, generally, ~ J det~J
Substituting R into A, since R~1 = Rt r IJ= (--~ ) I IJ detR IJ
[End of Proof]
[Proof 2]
A direct proof is presented.
After the case in which i, j = n is proved, the - ` t 2078~03 satisfaction of other cases are illustrated.
(When i, j = n) By applying the components of the k-th column to the n-th row of R~ and by applying two-dimensional rotation to the components of n-th column, in sequence, the components of k-th column can be made to have values zero [0]. That is, by expressing the (n-2)-dimensional identity matrix as In-2~ -[Expression 46]
- -- cosO I ~ 0, -I~-2! ~ ---.. ........... * *
O I n _2 0 = - ---- -.. -O ~ c~/9 n-l 21l-t~n., ....... ..................... O
~InOr. ~ CO~IOn~ InOI U c0~30~ _ _ By expressing (n-1)-dimensional matrix obtained by eliminating the n-th row and n-th column as Rn-l, [Expression 47]
Rn I O
~ .. . .. ..
~ d~L Rn.l That is, R - [
O del It~ l ~
- cos O, 0 -sin O, - - -In 2 O
x 0 1~ 0 c~o~,sin0.1 _ s(ng ~ O co~O ~ sinO ,~ 1 cvsO
- -- -- -- C~6 0 1 i7, ~_1 0 COSO 2 *
O cosO n-l -O dct Rn.l ~ ~ * c090 1 cOct1n 1 ~

r ~s O ~
R n n ~ R n l L c~s o ~ ~k O C~On-l dctR nn= ( ilcl R n-I ) cosO I - cosOn- I --r nn . r nn= (~I)n~n dct Rnn 2fl78~3 ~61`
(When i~n and j~n) The matrix obtained by swapping the i-th row with the n-th row and the j-th column with the n-th column in rotation matrix R is [Exp~ession 48]
j-th column i-th row -rl~ rln rlJ-- r nl r nn r n.
~ r n_l ~ n-l -rl, rln rll-Because this is a rotation matrix, j-th column -r" rln I i-th row r ~Dd~ r nl r nn rn-lln-l ~

Because columns needs to be swapped n-1-j times or rows need to be swapped n-1-i times to transform the matrix in the matrix determinant on the right side into Ri~, (When i=n and j~n) A matrix obtained by swapping the j-th column and n-th column by changing the sings in the n-th column is ` ~ 1 2078aO3 [Expression 49~
j-th column -r ~ -r,n r tl -!rn- n- .... ' ~rnl ~tnn tnJ
Because this is a rotation matrix, j-th column ~r 1l -r~"
r A~--de~ j _ r n ~
To transform the matrix in the determinant on the right side into R~, the columns need to be swapped n-i-j times after converting the signs of j-th column.
(_ I ) l~n-l~J det RnJ= (--I ~ n~J det RnJ
(When i~n and j=n) This is proved as in the case (when i=n and j~n) CEnd of Proof]
[Lemma 6]
R and S are n-dimensional rotation matrices where n22, CExpression 50]
By expressing - r, - -R ' - - 51 - - S ' -R- - - ,s8 ...

-r n ~ -r n ~ ~S n ~ ~S ~ ~
The necessary and sufficient condition for - S ' is that (n-1)-dimensional orthogonal matrix U exists, which is expressed as R.
- dc~U-~` S 20783~3 [Proof~
(Necessity) Because rank R' = n-l, S' = U R', where U is an (n-l)-dimensional square matrix. By expressing the i-th column of U as ui, because (si, si) = 1 and (ui, ui) = 1 When i~j, because (si, s~) = 0 and (ui, uj) = 0 Therefore, U is an (n-1)-dimensional orthogonal matrix.
From lemma 5, [Expression 51]
~nl (-11n-~ dcLSnl ~( l)n-l deL (URnl) dct tV) (-I)n~' dctR
dcL (U) r Therefore, Sn 8 de~ tU) r~
-S ' - -U R ' - .-U
S ; ~ -- R
(Sufficiency) -s n ~ ~ det ( U ) r n ~ ~ d~t U-This is self-evident from S' = U R'.
[End of Proof]
[Lemma 7]
Premised on conditions {16}, {17} and {18} in theorem 11, next propositions (1) and (2) are outstanding.

~ S i 2~78~3 Proposition (1) {23} and {24} are equivalent for p (= 1 or 2).

[Expresslon 52~y tv ~v ~-{23} When is regular, and --W ¦ D W 21- W 3, - - -v,~vz~ VJ ' - -I ~a~ a~Z a~~
u~ uz u~ i a2~ a~2 ;~
-a ~ a 12- _ - -W~, W2D W~D ,- -a 31 a 12 a 3~-is regular.
~a 21 a 22-{24} Sp37~0, and ~s ~
is regular.
Proposition (2) If the XY coordinate system is rotated by other than integer times of ninety degrees [90], a value of p (= 1 or 2) satisfying {19} can be selected.
15 [Proof]

[Expression 53]
-Vl'V2'V,'- -R'- --- Ul U2 U
W I, W 2D W 3D -- S D --Because [U1, U2, U3] iS regular, --V I Y 2 V ~ --is regular ~ is regular.
--W I V W 2~, W~ ~ ~ S
-R' - ~' - - -v, ' v2 ' v, ' - -At this time, u, U2 U~
--5: P ~ ~ ~ --W l~7 W2-- W3~
By defining this matrix as A = -a ll a 12 a ll-a2~ a2z a2~
-a 31 a 3Z a 31-25 From a general formula for obtaining an inverse ``` S 2078503 matrix, [Expression 54]
Because 5"~ _ a~l alZ S~3~0 ~ a~l a~z a2, a2z a21 a22 Therefore, {23} and {24} are equivalent.
(2) It need only be shown that p (=1 or 2) satisfying {24} can be selected from (1).
As described in theorem 12, note that {16}, {17}
and {18} are equivalent to ~19} through {22}. What should be illustrated is that a value of p (=1, 2) can be selected, if the XY coordinate system is rotated other than by integer times of ninety degrees [90], when p (=1 or 2) satisfying {24} cannot be selected.
The negation of {24} is a next external 1.
[External 1]
{25}

[Expression 55] -rl~
{25} For p such that l~p~2, Sp3=0 or is not regular.
{25} can be classified into next cases (10.1), (10.2) and (10.3).
( 10. 1 ) Sl3=S23=O, -r~- -r~-(10.2) are not regular.
-S~- -sz 1 66 ` 2078~03 - 1~ ' 'I
( 10 . 3 ) Sl3=0 and I is not regular.

-R ' -Alternatively, .S23=0 andis not regular.
However, cases ~10.1) and (10.2) contradict conditions {19} and {21}. -U
In case (10.1), because S = -- - --this contradicts condition {19}. - d~t U -In case (10.2), because sp (l~p~2) is a linear combination of r1 and r2, it intersects orthogonally with r3. That is, (sp , r3)=0 (l~p~2) Since SR-1 = SRt, the p3 element of SR-1 is obtained as (sp, r3), which is zero [0].
[Expression 56] --U
Because this is, S ~~l~ ~~ ~~~~~~~
- dctU-it contradicts condi~ion {21}.
Therefore, only case (10.3) remains.
Especially, when S13=0 and S23=0. Because this is not case (10.1), both of them cannot be zero [0].
It is shown that Sl3~0 and S23~0 can be achieved, if the XY coordinate system is rotated other than by integer times of ninety degrees [90].
[Expression 57]
Expressing w = S u when a coordinate system in ` ~ 2078~03 . .

.
which w = S u is rotated around the Z axis by ~~, since an object rotates around the Z axis by ~ in a relative sense, [r (~) ] ~ ~r (O) ;

rr (O) r t-O) Therefore, S~L l]s[ IJ

=~ r (O) [ ] r(-O) r (~) [52 [5~ S22] r(-O) s,~

0 ~S ~ 23] [S Zl]

When only one of S13 and S23 iS zero [O], by selecting ~ other than integer times of ninety degrees [90 ], S13~0 and S23~0 can be achieved. Conditions {19} through {22~ are outstanding even if an XY
coordinate system is rotated.
Therefore, when p (=1 or 2) satisfying {24~
cannot be selected, a rotation of an XY coordinate system by other than integer times of ninety degrees [90] enables p satisfying C24} to be selected.
[End of Proof]
[Expression 58]
A next [NOTE] allows " is regular." in lemma 7 to be replaced by (sp, r3)~0, which is an easier expression.

207~03 [NOTE]
[Expression 59]
" is regular." in condition {24} is equivalent to (sp, r3)~0.
[Expression 60]
As a proof of lemma 7, its contraproposition is proved, which is [ ~is not regular. ~ (sp, r3)~0.
(=~) Since sp is a linear combination of rl and r2, it intersects orthogonally with r3.
(~=) If an expression sp = ~ bl ri (where bl is a constant) is substituted into (sp, r3)=O, b3=O. ~, Since sp is~a linear combination of r1 and r2, is not regular. -s.-[End of Proof]
Explained below is a proof of the representations in {19}, {20} and {21} by proving proposition 5 and lemma 8.
[Proposition 5]
[Expression 61] -U
Three-dimensional rotation d t U ( where U is a two-dimensional orthogonal matrix.) (*) represents a rotation either (1) or (2).
(1) a rotation around a third axis (2) a rotation around an axis on the plane formed by . ~ ~ 2078~03 first and second axes, whose angle is one hundred and eighty degrees [180]. Conversely, rotations (1) and (2) can be expressed by a rotation matrix (*).
[Proof]
Because det U = +1, rotation matrix (*) can be classified into (1) and (2).
[Expression 62]
I -Ul (1) I l_ (where U is a two-dimensional rotation matrix.) This represents a rotation around the third axis.
(2) -- I
(where U represents a two-dimensional orthogonal matrix, and det U=-1.) It is shown below that this represents a rotation around an axis on the plane formed by first and second axes, whose angle is one hundred and eighty degrees [180]. To obtain the axis of rotation of the matrix in question, an equation (a) --u -- --x-- --x--- -1- -2- -~- ' ' ' ' (a) is solved. Equation (a) is equivalent to an x x equation (b) u a~ Z = 0 [Expression 64]
From lemma 8, - c~;~ slnO-u (0~<2~) - sln~ -ccs~-`` 2078503 1 7 0 -x c~
Since a set of solutions of (b) is ~~
-~- - sln (O/Z) -(where a is any real number.) .
-x- a r ~ (o/2) 1-Since a set of solutions of (a) is ~ - L sin ( 0~) J

(where a is any real number.) [Expression 65]
-U
Therefore, 1 0 - _ I J
represents a rotation around an axis expressed by - c~ ( ~ /2) -a vector sln ~) on the plane formed by first and second axes. U - -_~
Because ~~~ transfers a point on the Z axis ~ -to a symmetrical point o , the angle of rotation is one hundred and eighty degrees [180].
As described above, rotation matrix (*) represents both rotations (1) and (2). Hence, conversely, rotations (1) and (2) are expressed as rotation matrix (*).
~End of Proo]
[Lemma 8]
When two-dimensional orthogonal matrix U is not a rotation matrix, [Expression 66]

1 71 2~85~3 [ cos ~ ~In O ~ ( 0~<2~ ) slnO -co~

The set of solution for a vector equation U ~ ] ~ ; ]
~Xl r c~ (0 / 2) 1 is lyJ l sln (~ / 2) J where a is any real number.) That is, U is a mirror image transformation with respect to an axis expressed by a vector [ C~ (a / 2) ]
sln ( O/Z ) [Proof]
[Expression 67]

Becauseo(z)-so(2)~S0(2) [ ]'direct sum of sets), rl l U = S ~
(where S is a two-dimensional rotation matrix.) Because rotation matrices S and R can be expressed as S [ c~ ~ ~in~
sln~ c~aJ (0~<2T~) cosO sillO
[ ~;ln~ -cos~] ( 0~<2~) By transforming the expression in question, ~Expression 68]

~~~
t 72 20785~3 - c~so-i s~n~ - -x- ()-=

( c ) - s}n~ -c~â -1- -Y~
By applying the formula of a double angle, , --2 sln~ (O/Z), 2 sln (0/2) cc6 (~/2) - -x- -~---x-- -- c~ ( O /2) -- 2 sln (a/2) cos (~m . -2 c~' (0/2~ - -y- -O---a - - - (e) - sin (0/2) - - ~ln (0/2) cos (~/2) - -x- -O- -y- - sln (a/2) -x ~ d) c~ ( ~m - - ~in ( 0~) ~ ( 0/2) ~ o ( where a is any real number.) satisfies (d) i.e. (c). Conversely, it is shown that (e) represents every solution of (c).
I. When ~0 or ~, since sin~ ~0, sin~ = 2 sin(~/2) cos(~/2), sin(~/2)~0 and cos(~/2)~0.
Hence, (d) is reduced into [Expression 69]
- ~In (~/2) cos (a/2) - -x- -U-- sln ( 0/2) ~os ~ ~2), -Y- -O-. Accordingly, (e) represents every solution of ( c ) .
II. When ~=0, [Expression 70]
~ ~ ~X~
-- (c) -X- -1- 0 2 -y- -0-is a solution of (c), which matches (e).

Y --O----2 0- -x- -n-III. When ~=~, ....- (C) --~ U- --y~ --U-`` 173 207850~

[Expression 71] --2 ~- -X- -~-X O ~ (C) a -~ 0- -y~
~Y~ -1- is a solution of (c), which matches (e).
[Expression 72]
Since (e) is a solution of (c), the axis of U is -C~ t~/2) ~
a vector -sln(~/2)-Since U represents an orthogonal transformation, the length and angle remain unchanged.
[Expression 73]
Therefore, U represents a mirror image transformation with respect to axis expressed by a - c~ /Z) vector The fina~ résult can also be directly proved by showing U = (a two-dimensional rotation of angle ~/2) (a mirror image transformation with respect to the first - 20 axis) (a two-dimensional rotation of [the angle of rotation - ~/2]) That is, [Expression 74]
- c~ (0/2) ,-sln(0/2) - -1 - - cos ~- ~/2),-sln(- a ~2) -U =
- sin ( O /2), c~( O /2) - ~ ln (--0 /2) , c~ ~- 0 ~2) -needs to be shown.
The right side =
r coS~ ( a /z~ -si,.r ( o /z) æ~in~ ~ /2) c0~ 2) L 2sln(O/~) c~ /Z), --co~(O/2) +~ 2 ( O/2) J
= r c~ sln~l = The left side L sin~ -cos~J [End of Proof]
So far, the new theory for use in the fourth form of this invention has been discussed in detail. By having the shape/movement recognition unit 18 shown in Figure 4 apply this new theory to an ob;ect moving with a rotation, based on the result of extracting four [4] feature points each in images captured at three [3] instants in time for observation, the actual positions and movements of those feature points in a three-dimensional space are calculated, thereby enabling the movement of a moving body to be recognized.

Fiyure 61 is a block diagram of a shape/movement recognition unit 27 pursuant to the fourth form of this invention.
The concept of a moving body recognition apparatus in the fourth form of this invention is - similar to that in the first form of this invention.
Also, an embodiment of the fourth form of this invention and the detailed configuration of the shape/movement recognition unit 27 are similar to those of the first form of this invention, which are shown in Figures 9 through 13. Accordingly, their descriptions are omitted. The only difference is tha~ the movement calculation unit 34 and the shape calculation unit 35 (illustrated in Figure 12, which is a block diagram of the shape/recognition unit 27 shown in Figure 11) are consolidated into a movement/shape calculation unit 39 shown in Figure 61.

Figures 62, 63 and 64 are flowcharts showing in a three part series the whole processes of the shape/movement recognition unit 27 pursuant to the fourth form of this invention.
In Figure 62, when the processes start, the shape/movement recognition unit 27 determines in step S140 whether or not the known data input unit 31 shown in Figure 12 has received at least one [1] of the expressions {19}' through {22}', (l), (2) and (3).
As with the first, second and third forms of this invention, a signal from a sensor allows the determination to be made. The relations between expressions {19}' through {21}' and {19} through {21}
are the same as those described earlier.
If the shape/movement recognition unit 27 2~78503 determines in step S140 that the known data input unit 31 has received at least one [1] of expressions {19}' through {22}', (1), (2) and (3), the known data input unit 31 outputs in step S141 a deactivation signal to the feature point position normalization unit 32, and stores recognition disablement data in the movement calculation unit 34, thereby invoking a recognition disablement process {1} shown in Figure 64. As with the flow of recognition disablement process {1} for the second and third forms of this invention, the flow of recognition disablement process {1} for the fourth form of this invention is essentially the same as that shown in Figure 17 for the first form of this invention.
If the shape/movement recognition unit 27 determines in step S140 that the known data input unit 31 has received none of {19}' through {22}', (1), (2) and (3), the shape/movement recognition unit 27 does not determine (N0) in step S144 that feature point storage unit 26 has stored all data on the positions of four t4] feature points each in images captured at three [3] instants in time for observation. The shape/movement recognition unit 27 repeats step S144 until it determines that the feature point storage unit 26 has stored the positions of four [4] feature points each in images captured at three [3] instants in time for observation. When the shape/movement recognition unit 27 determines (YES) in step S144 that the feature point storage unit 26 has stored all data, the shape/movement recognition unit 27 has the feature point storage unit 26 send an activation signal to the feature point position normalization unit 32 in step S145 (shown in Figure 63), thereby a~tivating the feature point position normalization unit 32.

Continuing on to Figure 63, the eature point position normalization unit 32 stores in step S146 data on the in-image positions of four [4] feature points 0, 1, 2 and 3, which the feature point storage unit 26 has stored, and obtains first and second components of ui, vi and Wl ~ where i= 1, 2, 3) as X and Z coordinate values of feature points 1, 2 and 3, which are different from feature point 0, after feature point 0 moves to the origin and the other feature points 1, 2 and 3 are parallelly displaced.
Therefore, the feature point position normalization unit 32 obtains the X and Z coordinate values of feature points 1, 2 and 3 after a parallel displacement by subtracting the X and Z coordinate values of feature point 0 before the parallel displacement from the X and Z coordinate values of feature points 1, 2 and 3 before the parallel displacement.
Then, the shape/movement recognition unit 27 has the shape/movement determination unit 33 determine in step S147 whether or not these first and second components stored by the feature point position normalization unit 32 satisfy all of expressions {16}, {17} and {18}. If the shape/movement determination unit 33 determines negatively (N0) in step S147, i.e.
that they don't satisfy at least one [1] of expressions {16}, {17} and {18}, the shape/movement determination unit 33 sends recognition disablement data to the movement/shape calculation unit 39 in step S148, thereby invoking recognition disablement process {1} shown in Figure 65. The flow of recognition disablement process {1} pursuant to the fourth form of this invention shown in Figure 65 is essentially the same as that shown in Figures 17, 30 and 49 pursuant to the first, second and third orms of this invention, except that the movement/shape calculation unit 39, instead of the movement calculation unit 34 sends recognition disablement data to the feature point position reconstruction unit 36.

- - 1 79 2~78503 Figure 65 is a flowchart of an embodiment of recognition disablement process {1} pursuant to the fourth form of this invention.
On start of recognition disablement process {1}, the movement/shape calculation unit 39 sends recognition disablement data to the feature point position reconstruction unit 36 in step S142. Then, the feature point position reconstruction unit 36 stores the recognition disablement data in the ob;ect movement storage unit 28 and the feature point storage unit 29 in step S143, thereby ending the process.

As explained in the description of theorem 12, expressions {16}, {17} and {18} are equivalent to expressions {19} through {22}. Therefore, it can be said that the check in step S147 and the check in step S140 form a duplication. However, since a sensor can perform the check in step S140, such a duplication is instituted. Also, no further calculation is performed, when the shape/movement determination unit 33 determines in step S147 a recognition disablement by using only the positions of feature points in an image.
If the shape/movement determination unit 33 determines positively (YES) in step S147 that 1`80 20~8503 expressions {16}, {17} and {18} are all outstanding, the shape/movement determination unit 33 activates the movement calculation unit 34 in step S149, thereby invoking respective processes of the movement calculation unit 34 and the shape calculation unit 35, in accordance with theorem 13.
As described earlier, a sensor performs a check in step S140, aside from a process of a moving body recognition apparatus pursuant to the fourth form of this invention. If the elements of R and S are known, the shape/movement recognition unit 27 can determine the receipt o expressions {19}', {20}' and {21~' snd calculate two-dimensional orthogonal matrix U.
As for expression {19}', [Expression 75]
-r~ll` r.l2.- 0 ~
~= 1 21 r 22 (1 --O O r 33~

That is, rl3=r23=r3l=r32-0 is a necessary and sufficient condition for - de~ U-(where U is a two-dimensional orthogonal matrix.) At this time, r ll 12 ~

-rSl r,r.~

- ~ -18 1 2078a O3 S-- S~l S2~ 0 Next, as for expression {20}', - o o s"-[Expression 76]
That is, rl3=r23=r3l=r32=0 is a necessary and sufficient condition for -U~ -S - .... ... .
- det U-(where U is a two-dimensional orthogonal matrix.) -s 11 S lZ--At thls time, U~

Next, as for expression {21}', U
[Expression 77]
S-- -U-(where U is a two-dimenslonal orthogonal matrix.) U, is e~uivalent to SR-I

- dctU-(where U is a two-dimensional orthogonal matrix.) By defining SR-l=P
[Expression 78] -Pll P l2 ~ -, ~)= pz~ pzZ () _ ~) O p31-That is, pl3=p23=p31=p32-0 iS a necessary and sufficient condition for 18~

-u S R
- dct U~
(where U is a two-dimensional orthogonal matrix.) --S l l S 12--At this time, U =

Figures 66 and 67 are flowcharts showing in a two part series the whole processes of the movement/shape calculation unit 39 pursuant to the fourth form of this invention.
On start of its process, the movement/shape calculation unit 39 selects in step S150 a value of p defined in expression {23}, as a first procedure.
Then, as a second procedure, the movement/shape calculation unit 39 obtains in step S151 a value of h such that ah3 is not zero [O], and determines in step S152 values of ~, ~ and r, for calculating the values of r11, r12, r21 and r22 among elements of rotation matrix R.

Figure 68 is a flowchart showing processes for calculating elements at intersections between the first and second rows and the first and second columns of a rotation matrix R comprising three [3] rows and three [3] columns.
In step 5158, the movement/shape calculation unit 39 determines in step S155 whether or not both a13 and a23 are zero [Q]. If it determines in step S155 that both of them are not zero [0], the movement/shape calculation unit 39 calculates r11, r12, r21 and r22 in step S156. If it determines in step S155 that either al3 or a23 is zero [0], the movement/shape calculation unit 39 determines in step S157 whether or not a13 is zero [0]. If it determines a13=0 in step S157, the movement/shape calculation unit 39 calculates r11, r12, r2l and r22 in step S15~. If it determines a13~0 in step S157, the movement/shape calculation unit 39 calculates r1l, r12, r2l and r22 in step S159.

Returning to Figure 66, the movement/shape calculation unit 39 calculates r13 in step S160, on completing the calculation of r11, r12, r2l and r22 in step S156, S158 or S159 shown in Figure 68. Then, the movement/shape calculation unit 39 determines in step S161 whether or not rl3=0. If it determines rl3~0 in step S161, the movement/shape calculation unit 39 calculates r23 in step S162. If it determines r13=0 in step S161, the movement/shape calculation unit 39 calculates r23 in step S163.
Thus, the movement/shape calculation unit 39 has 1 84 2~785~3 completed calculating the values of the elements in the first and second rows in rotation matrix R.
Continuing on to Figure 67, as the final process of the second procedure, the movement/shape calculation unit 39 calculates sp~. Then, as a third procedure, the movement/shape calculation unit 39 calculates a3, which is a third row of matrix a in step S165, and the third component of a three-dimensional vector Ul in step S166.
As a fourth procedure, the movement/shape calculation unit 39 calculates sq in step S167, thereby completing the calculation of the first and second rows of rotation matrix S.
As a fifth procedure, the movement/shape calculation unit 39 calculates in step S168 the third row r3 of rotation matrix R and the third row S3 of rotation matrix S.
As a sixth procedure, the movement/shape calculation unit 39 calculates in step S169 third components of three-dimensional vectors vi and wi, and sends to the feature point position reconstruction unit 36 those calculation results, i.e. the values of rotation matrices R, S, and three-dimensional vectors Ui, Vl and wi (where i=1, 2, 3) representing three [3]
feature points at three [3] instants in time for ~` 1 85 2078503 observation.

Returning to Figure 64, the feature point position reconstruction unit 36 begins its processes from step S171, when the movement/shape calculation unit 39 completes lts processes in step S170 shown in Figure 67. In step S171, the feature point position reconstruction unit 36 selects the values matching the known data of an object inputted to the known data input unit 31 from the results of calculating the movement and shape of an object, i.e. from the values of rotation matrlces R, S, and three-dimensional vectors ui, vi and wi (where i=1, 2, 3).
Also, the feature point position reconstruction unit 36 selects either rotation matrices R and S or their inverse rotation matrices, as well as either one [1] of two [2] sets of solutions, which are mirror image transormations of each other with respect to an observation plane, as with the first, second and third forms of this invention.
When it fails in selecting such values in step S171 due to their non-existences, the feature point position reconstruction unit 36 invokes a recognition disablement process ~2} shown in Figure 69, which is essentially the same as those pursuant to the first, . --second and third forms of this invention shown in Figures 22, 33 and 56.

Figure 69 is a flowchart of an embodiment of recognition disablement process {2} pursuant to the fourth form of this invention.
In step S172, the feature point position reconstruction unit 36 stores the recognition disablement data in the object movement storage unit 28 and the feature point position storage unit 29, thereby ending the process.

Returning to Figure 64, when it succeeds in selecting such values in step Sl71, the feature point position reconstruction unit 36 stores in the object movement storage unit 28 the values of selected rotation matrices R and S together with the in-image coordinate values of feature point 0 stored in feature point normalization unit 32 in step S173, and in the feature point position storage unit 29 the values of third components of the selected three-dimensional vectors ui, vi and wi (where i=1, 2, 3) and the in-image coordinate values of four [4] feature points stored in the feature point position storage unit 29 in step S174, thereby ending its processes.

~ 187 2078~03 Thus, the shape/movement recognition unit 27 completes its whole processes.
In the above embodiment, the movement/shape calculation unit 39 simultaneously calculates two [2]
sets of solutions in obtaining inverse rotation matrix R-l, as shown in Figure 65. However, it is also posslble to structure another embodiment in which the movement/shape calculation unit 39 calculates only one [1] set of solutions according to the steps shown in Figure 66.
The other embodiment is such that the feature polnt position reconstruction unit 36 calculates, as the other values of rotation matrices R and S, R~l and S-1, which are inverse matrices of R and S, according to next formulae.
[Expression 79]

- r 1~ r l2 -r ~ s ~ s 12 -S 13-R' --- r2~ r22-rz~ . S' = sz~ Sz2-Sz:l - -r 31 -r 3z r ~3- - -S 31 -S 32 S 3a-The feature poin-t position reconstruction unit 36 calculates by next formulae the other set of three-dimensional vectors ui', vi' and vi' (where i = 1, 2, 3), as the mirror image transformations of the set of three-dimensional vectors ui, vi and vi (where i = 1, 207~503 2, 3) with respect to the XZ plane on which feature points are orthogonally pro~ected.
[Expression 80]
- Ull- - Vll- - Wll-~1 1 ' = U 1 2 , V I ' -- V 1 2. ~ W ~ z -tlll- Vl3- - Wl~
(where i = 1, 2, 3) Accordingly, the feature point position reconstruction unit 36 can obtain -two [2] sets of solutions.
As described above, the fourth form of this invention allows the moving body recognition apparatus to recognize the movement of an object moving with a rotation from four [4] feature points each in images captured at three [3] instants in time for observation. Also, as explained in the description of Figure 63, the shape/movement determination unit 33 can lmmediately determine the enablement or disablement of recognizing a movement from the in-image positions of feature points.
Figure 70 is a block diagram of a computer system embodying a moving body recognition apparatus of this invention.
As has already been described in the descriptions of Figures 9 through 13, a TV camera 180, image 2078~03 18q processing hardware including a space filter 181, a first memory 182 and a CPU 183 correspond respectively to the image input unit 24, the feature point extraction unit 25, the feature point storage unit 26 and the shape/movement recognition unit 27 shown in Figure 10. The CPU 183 connects with a sensor 184 and the first memory 182. The sensor 184 comprlses e.g. a keyboard for supplying known data to the known data input unit 31. The CPU 183 outputs a result of recognizing a moving body to a second memory 185 corresponding to the object movement storage unit 28 and the feature point position storage unit 29.
A moving body recognition apparatus having such a system configuration can have the TV camera 180 capture images of an object and the second memory 185 store the result of recognizing the movement of an object. The moving body recognition apparatus can be an automatic monitoring device, an visual sensing device for an automatically running vehicle, or an FA
inspection device.

Figure 71 shows an environment for an experimental program verification.
An experiment is performed to verify a program for processes of a moving body recognition apparatus 1 qo 2078~03 of this invention by using as an input feature point correspondence data created internally in a computer.
The program comprises two hundred and eighty [280]
steps, and the average duration of execution is four point six milliseconds [4.6ms]. This attests that the moving body recognition apparatus of this invention, even if it is a system based on a personal computer, can recognize a moving body at a rate faster than a video speed.

As described in detail above, this invention allows the shape and movement of an object to be recognized based on the feature points extracted from images of an object captured by a single image input unit. As such, compared with a prior art in which two [2] image input units are used, this invention enables the numbers of image input units and feature point extraction units to be slashed by half, thereby miniaturizing a moving body recognition apparatus.
Also, because feature points of an object need not have correspondences, unlike the prior art of using two [2] image input units, this invention sheds the time required for establishing the correspondences, thereby having an advantage of reduced processing time.

.l lql - 2078~03 This invention enables a single TV camera to recognize a moving body. A moving body recognition apparatus can be applied for use in various industries requiring a recognition of a moving body by an image processing, such as factory automations and various monitoring devices.
In addition, this invention is applicable not only to a visual recognition of an object but also to an acoustic recognition of an object, in which case a moving body recognition apparatus is configured mutatis mutandis e.g. by using an ultrasonic sensor as a sound input unit in lieu of a TV camera as an image input unit.

Claims (63)

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1. A moving body recognition apparatus for recognizing a movement of a moving object by positions of features points in an image corresponding to said moving object, comprising:
image input means for capturing three images of said moving object at three instances in time at equal time intervals, said three images being observed from a direction perpendicular to an axis of rotation of said object moving on a single plane and said object having a rotation at a constant rate;
feature point extraction means for extracting two feature points each in said three images captured by said image input means, said feature point extraction means including space filter means for extracting an outline image from said three images captured by said image input means and edge point extraction means for extracting an edge point from said outline image;
feature point position storage means for storing known position data corresponding to said extracted two feature points and for storing said edge point as feature point data; and shape/movement recognition means for calculating actual positions of said extracted two feature points and a movement of said object from said known position data.

2. The moving body recognition apparatus according to claim 1, wherein:
said shape/movement recognition means processes said images of said object captured from said direction perpendicular to said axis of rotation as orthogonally projected images of said object.

3. The moving body recognition apparatus according to claim 1 or 2, further comprising:
feature point position storage means for storing the actual positions of said feature points outputted from said shape/movement recognition means; and object movement storage means for storing the angle of rotation of an object outputted from said shape/movement recognition means.

4. The moving body recognition apparatus according to claim 3 wherein:
said image input means includes a television camera;
said feature point extraction means includes image processing hardware including a space filter;
said feature point position storage means includes a first memory;
said shape/movement recognition means includes a central processing unit;
said feature point position storage means includes a second memory and said object movement storage means includes said second memory; and said moving body recognition apparatus further includes a sensor for outputting to said CPU known data regarding the shape and movement of said object.

5. The moving body recognition apparatus according to claim 3, wherein:
said image input means captures images of said object monitored by a visual device of an automatically running vehicle or an image of said object inspected in an inspection of factory automation (FA); and said feature point position storage means and said object movement storage means store the results of recognizing the movements of said object being monitored or inspected.

6. The moving body recognition apparatus according to claim 3, wherein said shape/movement recognition means comprises:
known data input means for receiving from an external sensor known data including movements and positions of said feature points of said object;
feature point position normalization means for obtaining as normalized known position data the relative positions of one of said two feature points when the other one of said two feature points of said object in said images captured at each of said three instances in time for observation having equal time intervals falls on the origin of a three-dimensional coordinate space;
shape/movement determination means for determining an enablement or a disablement of recognizing said movements and positions of said feature points of said object by using an output from said feature point position normalization means;
movement calculation means, activated by a determination by said shape/movement determination means of an enablement of recognizing said movements and positions of said feature points of said object, for calculating an angle of rotation of said object around said origin by using an output from said feature point position normalization means;
shape calculation means for obtaining unknown position data other than said known position data of said two feature points of said object by using outputs from said movement calculation means and said feature point position normalization means; and feature point position reconstruction means for outputting a movement of said object by combining an angle of rotation around said origin outputted from said movement calculation means with said position data of said other one feature point falling on said origin in said images outputted from said feature point position normalization means, and for outputting positions of feature points by combining said unknown position data outputted from said shape calculation means with said position data of said two feature points in said images outputted from said feature point position normalization means.

7. The moving body recognition apparatus according to claim 6, wherein an external memory means for temporarily storing said known data outputted from said external sensor on an offline basis is provided before an input to said known data input means.

8. The moving body recognition apparatus according to claim 6, wherein said shape/movement recognition means performs the following functions:
a first step of determining whether or not said known data input means has received .THETA. = n .pi. (where n is an integer), when the relations between two-dimensional vectors u1, v1 and w1 are expressed as:
v1 = R u1 and w1 = R2 u1 , where the X axis is the direction of a parallel displacement of said feature points, the Z axis is the direction parallel to the axis of rotation of said object and, the Y axis is a direction perpendicular to an image plane, which is the XZ
plane, u1 = [u11, u12 ]t, v1 = [v11, v12 ]t, w1 = [w11, w12 ]t on the XY plane representing relative positions of feature point 1 (one [1]

not falling on the origin of said two [2] feature points 0 and 1) from the origin on which feature point 0 falls respectively at said first, second and third instants in time for observation, and rotation matrix R

R = [ cos.THETA. -sin.THETA. ]
sin.THETA. cos.THETA.
represents a two-dimensional rotation on the XY plane around the origin of an object from said first instant in time for observation to said second instant in time for observation;
a second step, invoked when said shape/movement recognition means determines positively in said first step, of executing recognition disablement process {1}
after said known data input means outputs a deactivation signal to said feature point position normalization means and recognition disablement data to said movement calculation means;
a third step, invoked when said shape/movement recognition means determines negatively in said first step, of determining whether said feature point position storage means has stored positions of said two [2] feature points 0 and 1 each in said images at three [3] instants in time for observation, which is repeated until a positive determination is obtained;
a fourth step, invoked when said shape/movement recognition means determines positively in said third step, of having said feature point position storage means send an activation signal to said feature point position normalization means, thereby activating said feature point position normalization means;
a fifth step of having feature point position normalization means store positions of said two [2]
feature points 1 and 2 each in said images stored in said feature point position storage means and obtain first components of vectors u1, v1 and w1 as the X
coordinate values by subtracting from the X coordinate values of feature point 1 in said images the X
coordinate values of feature point 0 in said images, which falls on the origin of said XZ plane;
a sixth step of having said shape/movement determination means determine whether said first components of vectors u1, v1 and w1 obtained by said feature point position normalization means satisfy both of next two [2] conditions v11 ? 0 v11 =u11 and w11 = u11 are not outstanding concurrently;
a seventh step, invoked when said shape/movement determination means determines negatively in said sixth step, of having said shape/movement determination means output recognition disablement data to said movement calculation means, thereby executing a recognition disablement process {1};

an eighth step, invoked when said shape/movement determination means determines positively in said sixth step, of having said shape/movement determination means activate said movement calculation means for executing a process of said movement calculation means;
a ninth step of having shape calculation means execute its process;
a tenth step of having said feature point position reconstruction means determine whether or not there is a set of R, u1, v1 and w1 satisfying said known data regarding said movement and said shape of said object received by said known data input means;
an eleventh step, invoked when said feature point position reconstruction means determines negatively in said tenth step, of having said feature point position reconstruction means execute a recognition disablement process {2};
a twelfth step, invoked when said feature point position reconstruction means determines positively in said tenth step, of having said feature point position reconstruction means store in said object movement storage means rotation matrix R satisfying said known data and the coordinate values of feature point 0 in said images outputted from said feature point position normalization means; and a thirteenth step of having said feature point position reconstruction means store in said feature point position storage means second components of u1, v1 and w1 satisfying said known data and coordinate values of feature points 0 and 1 in said images outputted from said feature point position normalization means, thereby ending the whole processes.

9, The moving body recognition apparatus according to claim 8, wherein said movement calculation means:
calculates said rotation matrix R by using expressions cos.THETA.= (u11+w11 ) /2v11 sin.THETA.=(1-cos2?);
determines whether or not calculated rotation matrix R satisfies said known data of said object received by said known data input means;
invokes said recognition disablement process {1}
when no rotation matrix R satisfies said known data;
and sends rotation matrix R satisfying said known data to said shape calculation means and said feature point position reconstruction means.

10. The moving body recognition apparatus according to claim 8 or 9, wherein said recognition disablement process {1} comprises:
a step of having said movement calculation means send to said feature point position reconstruction means recognition disablement data; and a step of having said feature point position reconstruction means store said recognition disablement data in said object movement storage means and said feature point position storage means, for a process completion.

11. The moving body recognition apparatus according to claim 8, wherein said shape calculation means obtains said second components of vectors u1, v1 and w1 by using expressions u12 = 1/sin.THETA. [2cos2.THETA.-1 -cos.THETA.] [v11 w11]t v12 = [sin.THETA. cos.THETA.] [u11 u12]t w12 = [sin.THETA. cos.THETA.] [v11 v12]t

12. The moving body recognition apparatus according to claim 8, wherein said recognition disablement process {2} comprises:
a step of having said feature point position reconstruction means store recognition disablement data in said object movement storage means and said feature point position storage means, for a process completion.

13. The moving body recognition apparatus according to claim 8 wherein:
said movement calculation means obtains said rotation matrix R by only one [1] of the two [2] values of sin.THETA. having positive and negative signs calculated by using expressions cos.THETA. = [u11 + w11] /2v11 and sin.THETA. = (1-cos2.THETA.), and sends said rotation matrix R thus obtained to said shape calculation means and said feature point position reconstruction means;
said shape calculation means obtains said second components of vectors u1, v1 and w1 by using expressions u12 = 1/sin.THETA. [2cos2.THETA.-1 -cos.THETA.] [v11 w11]t v12 = [sin.THETA. cos.THETA.] [u11 u12]t w12 = [sin.THETA. cos.THETA.] [v11 v12]t, and sends said second components to said feature point position reconstruction means; and said feature point position reconstruction means obtains inverse rotation matrix R-1, which is an inverse to rotation matrix R, and vectors u1', v1' and w1', which are mirror image transformations of vectors u1, v1 and w1 with respect to the X axis by using expressions R' = u1' = v1' = w1' =

14. A moving body recognition apparatus for recognizing a movement of a moving body by positions of feature points on said moving body, comprising:
an image input means for capturing images of said moving body as an object;
a feature point extraction means for extracting feature points in said images captured by said image input means;
a feature point position storage means for storing known position data of extracted feature points; and a shape/movement recognition means for calculating the actual positions and movements of feature points of said object from known position data of three [3] feature points forming a right angle of said object each in said images captured at any two [2] instants in time for observation from a direction perpendicular to the axis of rotation of an object moving on a single plane coupled with a rotation by using an output from said feature point position storage means.

15. The moving body recognition apparatus according to claim 14, wherein:
said shape/movement recognition means processes said images of said object captured from said direction perpendicular to said axis of rotation as orthogonally projected images of said object.

16. The moving body recognition apparatus according to claim 14 or 15, wherein said feature point extraction means comprises:
a space filter means for extracting an outline image from said images captured by said image input means; and an edge point extraction means for extracting an edge point from said outline image and for storing it as feature point data in said feature point position storage means.

17. The moving body recognition apparatus according to claim 14 or 15, further comprising:
a feature point position storage means for storing the actual positions of feature points outputted from said shape/movement recognition means;
and an object movement storage means for storing the angle of rotation of an object outputted from said shape/movement recognition means.

18. The moving body recognition apparatus according to claim 17, configured as a computer system comprising:
a television camera in correspondence with said image input means;
image processing hardware including a space filter in correspondence with a feature point extraction means;
a first memory in correspondence with said feature point position storage means;
a central processing unit (CPU) in correspondence with said shape/movement recognition means;
a second memory in correspondence with said feature point position storage means and object movement storage means; and a sensor for outputting to said CPU known position data regarding the shape and movement of said object.

19. The moving body recognition apparatus according to claim 17, wherein:
said image input means captures images of said object monitored by a visual device of an automatically running vehicle or an image of said object inspected in an inspection of factory automation (FA); and said feature point position storage means and said object movement storage means store the results of recognizing the movements of said object being monitored or inspected.

20, The moving body recognition apparatus according to claim 17, wherein said shape/movement recognition means comprises:
a known data input means for receiving from an external sensor known data regarding movements and positions of said feature points of said object;
a feature point position normalization means for obtaining as normalized known position data the relative positions of the other two [2] feature points when one [1] of said three [3] feature points of an object at the peak forming said right angle each in said images captured at any two [2] instants in time for observation falls on the origin of a three-dimensional coordinate space;
a shape/movement determination means for determining an enablement or a disablement of recognizing said movements and positions of feature points of said object by using an output from said feature point position normalization means;
a movement calculation means, activated by a determination by said shape/movement determination means of an enablement of recognizing said movements and positions of said feature points of said object, for calculating an angle of rotation of said object around said origin by using an output from said feature point position normalization means;
a shape calculation means for obtaining unknown position data other than said known position data of said three [3] feature points 0,1 and 2 of said object by using outputs from said movement calculation means and said feature point position normalization means; and a feature point position reconstruction means for outputting a movement of said object by combining an angle of rotation around said origin outputted from said movement calculation means with said position data of said feature point falling on said origin in said images outputted from said feature point position normalization means, and for outputting positions of feature points by combining said unknown position data outputted from said shape calculation means with said position data of said three [3] feature points 0,1 and 2 in said images outputted from said feature point position normalization means.

21. The moving body recognition apparatus according to claim 20 wherein an external memory means for temporarily storing known data outputted from said external sensor on an offline basis is provided before an input to said known data input means.

22. The moving body recognition apparatus according to claim 20, wherein the whole processes of said shape/movement recognition means comprise:
a first step of determining whether or not said known data input means has received at least one [1]
of expressions .THETA. = n .pi. (where n is an integer), .alpha. + .THETA. = n .pi. - .alpha., and u11v21 + u21v11 = 0, when the relations between two-dimensional vectors ui and vi are expressed as:
vi = R ui (where i = 1,2) and u1 = d1 [cos.alpha. sin.alpha.]t, and u2 = d2 [cos(.alpha..pi./2) sin(a.pi./2)]t, where the X axis is the direction of a parallel displacement of said feature points, the Z axis is the direction parallel to the axis of rotation of said object and, the Y axis is a direction perpendicular to an image plane, which is the XZ plane, ui = [uil, ui2]t, vi = [vi1, vi2]t (where i = 1, 2) on the XY plane represent relative positions of a feature points i (where i = 1, 2, i.e. two [2] not falling on the origin of said three [3] feature points 0, 1 and 2) from the origin on which feature point 0 falls respectively at said first and second instants in time for observation and rotation matrix R

R = = represents a two-dimensional rotation on the XY plane around the origin of an object from said first instant in time for observation to said second instant in time for observation, a set of recognition solutions <1>
represents actual positions and movements of feature points 1 and 2 when vector u2 is obtained by rotating vector u1 by .pi./2, and a set of recognition solutions <2> represents actual positions and movements of feature points 1 and 2 when vector u2 is obtained by rotating vector u1 by -.pi./2;
a second step, invoked when said shape/movement recognition means determines positively in said first step, of executing recognition disablement process {1}
after said known data input means outputs a deactivation signal to said feature point position normalization means and recognition disablement data to said movement calculation means;
a third step, invoked when said shape/movement recognition means determines negatively in said first step, of determining whether said feature point position storage means has stored positions of said three [3] feature points 0, 1 and 2 in said images at two [2] instants in time for observation, which is repeated until a positive determination is obtained;
a fourth step, invoked when said shape/movement recognition means determines positively in said third step, of having said feature point position storage means send an activation signal to said feature point position normalization means, thereby activating said feature point position normalization means;
a fifth step of having feature point position normalization means store positions of said three [3]
feature points 0, 1 and 2 each in said images stored in said feature point position storage means and obtain first components of vectors ui and vi (where i = 1, 2) as the X coordinate values by subtracting from the X coordinate values of feature points 1 and 2 in said images the X coordinate values of feature point 0 in said images, which falls on the origin of said XZ
plane;
a sixth step of having said shape/movement determination means determine whether said first components of vectors u1 and v1 obtained by said feature point position normalization means satisfy next condition, v11?u11 or v21=u21;
a seventh step, invoked when said shape/movement determination means determines negatively in said sixth step, of having said shape/movement determination means output recognition disablement data to said movement calculation means, thereby executing a recognition disablement process {1};
an eighth step, invoked when said shape/movement determination means determines positively in said sixth step, of having said shape/movement determination means activate said movement calculation means for executing a process of said movement calculation means;
a ninth step of having shape calculation means execute its process;
a tenth step of having said feature point position reconstruction means obtain set of recognition solutions <1> representing actual positions and movements of feature points 1 and 2 when vector u2 is obtained by rotating vector u1 by .pi./2 by using the second components of vectors ui and vi and rotation matrix R sent from said movement calculation means and shape calculation means;
an eleventh step of having said feature point position reconstruction means obtain set of recognition solutions <2> representing actual positions and movements of feature points 1 and 2 when vector u2 is obtained by rotating vector u1 by -.pi./2 by using expressions R' = ui' = vi' = ( where i = 1, 2) a twelfth step of having said feature point position reconstruction means determine whether or not either set of recognition solutions <1> or <2>
satisfies known data of said object received by said known data input means;
a thirteenth step, invoked when said feature point position recognition means determines negatively in said twelfth step, of having feature point position reconstruction means execute a recognition disablement process {2};
a fourteenth step, invoked when said feature point position reconstruction means determines negatively in said twelfth step, of having said feature point position reconstruction means store in said object movement storage means rotation matrix R
in the set of recognition solutions satisfying said known data and the coordinate values of feature point 0 in said images outputted from said feature point position normalization means; and a fifteenth step of having said feature point position reconstruction means store in said feature point position storage means second components of ui and vi (where i = 1, 2) in said set of recognition solutions satisfying said known data and coordinate values of feature points 0,1 and 2 in said images outputted from said feature point position normalization means, thereby ending the whole processes.

23 The moving body recognition apparatus according to claim 22, wherein said movement calculation means:

obtains an integer n (= 0, 1, 2 or 3) defining the value range of angle a formed between one [1] of said two-dimensional vectors u1 and the X axis, such that (.pi./2)n ? .alpha. ? (.pi./2)(n+1) in accordance with the sign of the values of first components u11 and u21 of two [2] of said two-dimensional vectors u1 and u2, by a next table obtains an integer m (= 0, 1, 2 or 3) defining the value range of angle .beta. formed between one [1] of said two-dimensional vectors v1 and the X axis, such that (.pi./2)m ? .beta. ? (.pi./2)(m+1) in accordance with the sign of the values of first components v11 and v21 of two [2] of said two-dimensional vectors v1 and v2, by a next table determines the signs of sin.THETA. by using the values of integers n and m defining the value ranges of angles .alpha. and .beta.;
calculates said rotation matrix R by using expressions cos.THETA.= (u11u21+v11v21)/(u11v21+u21v11 ) sin.THETA.=(1-cos2?);
determines whether or not said rotation matrix R
thus calculated or its inverse rotation matrix R-1 satisfies said known data of said object received by said known data input means;
invokes said recognition disablement process {1}
when neither said rotation matrix R nor said inverse rotation matrix R-1 satisfies said known data; and sends the values of angle a and rotation matrix R
or said rotation matrix R-1 satisfying said known data to said shape calculation means and said feature point position reconstruction means.

24. The moving body recognition apparatus according to claim 23, wherein said movement calculation means determines the sign of sin.THETA. by a method comprising:
a step 1 of determining whether the value of an integer (m-n) is odd or even;
a step 2, invoked when the value of integer (m-n) is determined to be odd in said step 1, of determining whether the value of an integer p is odd or even, where p = (m - n = 1) / 2 a step 3, invoked when the value of integer p is determined to be odd in said step 2, of determining that the sign of sin.THETA. is positive;
a step 4, invoked when the value of integer p is determined to be even in said step 2, of determining that the sign of sin.THETA. is positive;
a step 5, invoked when the value of integer (m-n) is determined to be even in said step 1, of determining the sign of sin.THETA. by a next table through a comparison between the values of u11 and v11.

25. The moving body recognition apparatus according to claim 22 and 23, wherein said recognition disablement process {1} comprises:
_ a step of having said movement calculation means send to said feature point position reconstruction means recognition disablement data; and a step of having said feature point position reconstruction means store said recognition disablement data in said object movement storage means and said feature point position storage means, for a process completion.

26. The moving body recognition apparatus according to claim 22, wherein said shape calculation means:
determines whether the first components u11 and u21 of two [2] of said two-dimensional vectors u1 and u2 are zero [0];
executes a shape calculation process {1} on determining that neither one [1] of the first components u11 and u21 of two [2] of said two-dimensional vectors u1 and u2 is zero [0], calculates the second components u12 and u22 of two [2] of said vectors u1 and u2 by using expressions u12 = d1 sin.alpha. and u22 = d2 cos.alpha.
obtains the second components v12 and v22 of two [2] of said two-dimensional vectors v1 and v2 by using expressions v12 = R2 u1 and v22 = R2 u2, and sends to said feature point position reconstruction means the second components u12, u22, v12 and v22 of said two-dimensional vectors u1, u2, v1 and v2;
executes a shape calculation process {2} on determining that the first component u11 of one [1] of said two-dimensional vectors u1 is zero [0], calculates the second components u12 and u22 of two [2] of said vectors u1 and u2 by using expressions u12 = d1 sin.alpha. and u22 = d2 cos.alpha.
obtains the second components v12 and v22 of two [2] of said two-dimensional vectors v1 and v2 by using expressions v12 = R2 u1 and v22 = R2 u2, and sends to said feature point position reconstruction means the second components u12, u22, v12 and v22 of said two-dimensional vectors u1, u2, v1 and v2; and executes a shape calculation process {3} on determining that the first components u11 and u21 of two [2] of said two-dimensional vectors u1 and u2 are respectively non-zero [?0] and zero [0], calculates the second components u12 and u22 of two [2] of said vectors u1 and u2 by using expressions u12 = d1 sin.alpha. and u22 = d2 cos.alpha.

obtains the second components v12 and v22 of two [2] of said two-dimensional vectors v1 and v2 by using expressions v12 = R2 u1 and v22 = R2 u2, and sends to said feature point position reconstruction means the second components u12, u22, v12 and v22 of said two-dimensional vectors u1, u2, v1 and v2.

27. The moving body recognition apparatus according to claim 26, wherein said shape calculation process {1} comprises:
a step of obtaining said angle .alpha. formed between one [1] of said two-dimensional vectors u1 and the X
axis by using an expression tan.alpha. = (u11cos.THETA. - v11 ) / u11sin.THETA.; and a step of obtaining lengths d1 and d2 of two [2]
of said two-dimensional vectors u1 and u2 by expressions d1 = u11 / cos.alpha. and d2 = -u21 / sin.alpha..

28. The moving body recognition apparatus according to claim 26, wherein said shape calculation process {2} comprises:
a step 6 of determining whether or not the value of the first component u21 of one [1] of said two-dimensional vectors u2 is less than zero [0], a step 7, invoked when the value of u21 is determined to be less than zero [0] in step 6, of equalizing the value of angle a to .pi./2 and obtaining lengths d1 and d2 of two [2] of said two-dimensional vectors u1 and u2 by using expressions d1 = and d2 =; and a step 8, invoked when the value of u21 is not determined to be less than zero [0] in step 6, of equalizing the value of angle .alpha. to 3.pi./2 and obtaining lengths d1 and d2 of two [2] of said two-dimensional vectors u1 and u2 by using expressions d1 = and d2 = .

29. The moving body recognition apparatus according to claim 26, wherein said shape calculation process {3} comprises:
a step 9 of determining whether or not the value of the first component u11 of one [1] of said two-dimensional vectors u1 is greater than zero [0], a step 10, invoked when the value of u11 is determined to be greater than zero [0] in step 9, of equalizing the value of angle a to zero [0] and obtaining lengths d1 and d2 of two [2] of said two-dimensional vectors u1 and u2 by expressions d1 = ¦u11¦ and d2 = ¦v21 / sin.THETA.¦; and a step 11, invoked when the value of u11 is not determined to be greater than zero [0] in step 6, of equalizing the value of angle .alpha. to .pi. and obtaining lengths d1 and d2 of two [2] of said two-dimensional vectors u1 and u2 by expressions d1 = and d2 =

30. The moving body recognition apparatus according to claim 22, wherein said recognition disablement process {2} comprises:
a step of having said feature point position reconstruction means store recognition disablement data in said object movement storage means and said feature point position storage means, for a process completion.

31. The moving body recognition apparatus according to claim 22, wherein the whole processes of said shape/movement recognition means comprise:
a first step of determining whether or not said known data input means has received at least one [1]
of .THETA. = n .pi. (where n is an integer), .alpha. + .THETA. = n .pi. - .alpha., and u11v21+u21v11=0, when the relations between two-dimensional vectors ui and vi are expressed as:
vi = R ui (where i = 1, 2), u1 = di [cos.alpha. sin.alpha.]t, and u2 = di [cos(.alpha..pi./2) sin(.alpha..pi./2)]t, where the X axis is the direction of a parallel displacement of said feature points, the Z axis is the direction parallel to the axis of rotation of said object and, the Y axis is a direction perpendicular to an image plane, which is the XZ plane, ui = [ui1, ui2]t, vi = [vi1, Vi2]t (where i = 1, 2) on the XY plane represent relative positions of a feature points i (where i = 1, 2, i.e. two [2] of three [3] feature points not falling on the origin) from the origin on which feature point 0 falls respectively at said first and second instants in time for observation and rotation matrix R

R = = represents a two-dimensional rotation on the XY plane around the origin of an object from said first instant in time for observation to said second instant in time for observation, a set of recognition solutions <1>
represents actual positions and movements of feature points 1 and 2 when vector u2 is obtained by rotating vector u1 by .pi./2, and a set of recognition solutions <2> represents actual positions and movements of feature points 1 and 2 when vector u2 is obtained by rotating vector u1 by -.pi./2;
a second step, invoked when said shape/movement recognition means determines positively in said first step, of executing recognition disablement process {1}
after said known data input means outputs a deactivation signal to said feature point position normalization means and recognition disablement data to said movement calculation means;
a third step, invoked when said shape/movement recognition means determines negatively in said first step, of determining whether said feature point position storage means has stored positions of said three [3] feature points 0, 1 and 2 each in said images at two [2] instants in time for observation, which is repeated until a positive determination is obtained;
a fourth step, invoked when said shape/movement recognition means determines positively in said third step, of having said feature point position storage means send an activation signal to said feature point position normalization means, thereby activating said feature point position normalization means;
a fifth step of having feature point position normalization means store positions of said three [3]
feature points 0, 1 and 2 each in said images stored in said feature point position storage means and obtain first components of vectors ui and vi (where i = 1, 2) as the X coordinate values obtained by subtracting from the X coordinate values of feature points 1 and 2 in said images the X coordinate values of feature point 0 in said images, which falls on the origin of said XZ plane;
a sixth step of having said shape/movement determination means determine whether said first and second components of vectors ui and vi ( where i =1, 2) obtained by said feature point position normalization means satisfy next condition, v11 ?u11 or v21 =u21;
a seventh step, invoked when said shape/movement determination means determines negatively in said sixth step, of having said shape/movement determination means output recognition disablement data to said movement calculation means, thereby executing a recognition disablement process {1};
an eighth step, invoked when said shape/movement determination means determines positively in said sixth step, of having said shape/movement determination means activate said movement calculation means for executing a process of said movement calculation means;

a ninth step of-having shape calculation means execute its process;
a tenth step of having said feature point position reconstruction means obtain set of recognition solutions <2> representing actual positions and movements of feature points 1 and 2 when vector u2 is obtained by rotating vector u1 by -.pi./2 by using the second components of vectors ui2 and vi and rotation matrix R sent from said movement calculation means and shape calculation means;
an eleventh step of having said feature point position reconstruction means obtain set of recognition solutions <1> representing actual positions and movements of feature points 1 and 2 when vector u2 is obtained by rotating vector u1 by .pi./2 by using expressions R' = ui ' = vi' = ( where i = 1, 2) a twelfth step of having said feature point position reconstruction means determine whether or not either set of recognition solutions <1> or <2>
satisfies known data of said object received by said known data input means;

a thirteenth step, invoked when said feature point position recognition means determines negatively in said twelfth step, of having feature point position reconstruction means execute a recognition disablement process {2};
a fourteenth step, invoked when said feature point position reconstruction means determines negatively in said twelfth step, of having said feature point position reconstruction means store in said object movement storage means rotation matrix R
in the set of recognition solutions satisfying said known data and the coordinate values of feature point 0 in said images outputted from said feature point position normalization means; and a fifteenth step of having said feature point position reconstruction means store in said feature point position storage means second components of ui and vi (where i = 1, 2) in said set of recognition solutions satisfying said known data and coordinate values of feature points 0, 1 and 2 in said images outputted from said feature point position normalization means, thereby ending the whole processes.

32. A moving body recognition apparatus for recognizing a movement of a moving body by positions of feature points on said moving body, comprising:
an image input means for capturing images of said moving body as an object;
a feature point extraction means for extracting feature points in said images captured by said image input means;
a feature point position storage means for storing known position data of extracted feature points; and a shape/movement recognition means for calculating the actual positions and movements of feature points of said object from known position data of three [3] feature points of said object each in said images captured at any three [3] instants in time for observation from a direction perpendicular to the axis of rotation of an object moving on a single plane coupled with a rotation by using an output from said feature point position storage means.

33. The moving body recognition apparatus according to claim 32, wherein:
said shape/movement recognition means processes said images of said object captured from said direction perpendicular to said axis of rotation as orthogonally projected images of said object.

34. The moving body recognition apparatus according to claim 32 or 33, wherein said feature point extraction means comprises:
a space filter means for extracting an outline image from said images captured by said image input means; and an edge point extraction means for extracting an edge point from said outline image and for storing it as feature point data in said feature point position storage means.

35. The moving body recognition apparatus according to claim 32 or 33, further comprising:
a feature point position storage means for storing the actual positions of feature points outputted from said shape/movement recognition means;
and an object movement storage means for storing the angle of rotation of an object outputted from said shape/movement recognition means.

36. The moving body recognition apparatus according to claim 35, configured as a computer system comprising:
a television camera in correspondence with said image input means;
image processing hardware including a space filter in correspondence with a feature point extraction means;
a first memory in correspondence with said feature point position storage means;
a central processing unit (CPU) in correspondence with said shape/movement recognition means;
a second memory in correspondence with said feature point position storage means and said object movement storage means; and a sensor for outputting to said CPU known position data regarding the shape and movement of said object.

37. The moving body recognition apparatus according to claim 35, wherein:
said image input means captures images of said object monitored by a visual device of an automatically running vehicle or an image of said object inspected in an inspection of factory automation (FA); and said feature point position storage means and said object movement storage means store the results of recognizing the movements of said object being monitored or inspected.

38. The moving body recognition apparatus according to claim 35 , wherein said shape/movement recognition means comprises:
a known data input means for receiving from an external sensor known data regarding movements and positions of said feature points of said object;
a feature point position normalization means for obtaining as normalized known position data the relative positions of the other two [2] feature point when one [1] of said three [3] feature points of said object each in said images captured at any three [3]
instants in time for observation falls on the origin of a three-dimensional coordinate space;
a shape/movement determination means for determining an enablement or a disablement of recognizing said movements and positions of feature points of said object by using an output from said feature point position normalization means;
a movement calculation means, activated by a determination by said shape/movement determination means of an enablement of recognizing said movements and positions of said feature points of said object, for calculating an angle of rotation of said object around said origin by using an output from said feature point position normalization means;
a shape calculation means for obtaining unknown position data other than said known position data of said three [3] feature points of said object by using outputs from said movement calculation means and said feature point position normalization means; and a feature point position reconstruction means for outputting a movement of said object by combining an angle of rotation around said origin outputted from said movement calculation means with said position data of said feature point falling on said origin in said images outputted from said feature point position normalization means, and for outputting positions of feature points by combining said unknown position data outputted from said shape calculation means with said position data of three [3] feature points in said images outputted from said feature point position normalization means.

39. The moving body recognition apparatus according to claim 38, wherein an external memory means for temporarily storing known data outputted from said external sensor on an offline basis is provided before an input to said known data input means.

40. The moving body recognition apparatus according to claim 38, wherein the whole processes of said shape/movement recognition means comprise:
a first step of determining whether or not said known data input means has received at least one [1]
of six [6] expressions, R=I, S=I, R=S, (where I is an identity matrix) u11u22-u21u12=0, v11v22-v21v12=0 and w11w22-w21w12=0, when the relations between two-dimensional vectors ui, vi and wi (i= 1, 2), are expressed as:
vi = R ui and wi = S ui, (i=1, 2) where the X axis is the direction of a parallel displacement of said three [3] feature points 0, 1 and 2, the Z axis is the direction parallel to the axis of rotation of said object and, the Y axis is a direction perpendicular to an image plane, which is the XZ
plane, ui = [ui1, ui2]t, vi = [vi1, vi2]t and wi = [wi1, wi2]t on the XY plane representing relative positions of feature points 1 and 2 (two [2] not falling on the origin of said three [3] feature points 0, 1 and 2) from the origin on which feature point 0 falls respectively at said first, second and third instants in time for observation, and a pair of rotation matrices R and S

R = S = represent two-dimensional rotations on the XY plane around the origin of an object from said first instant in time for observation respectively to said second and third instants in time for observation;
a second step, invoked when said shape/movement recognition means determines positively in said first step, of executing recognition disablement process {1}
after said known data input means outputs a deactivation signal to said feature point position normalization means and recognition disablement data to said movement calculation means;
a third step, invoked when said shape/movement recognition means determines negatively in said first step, of determining whether said feature point position storage means has stored positions of said three [3] feature points 0, 1 and 2 in said images at three [3] instants in time for observation, which is repeated until a positive determination is obtained;
a fourth step, invoked when said shape/movement recognition means determines positively in said third step, of having said feature point position storage means send an activation signal to said feature point position normalization means, thereby activating said feature point position normalization means;
a fifth step of having feature point position normalization means store positions of said three [3]
feature points 0, 1 and 2 each in said images stored in said feature point position storage means and obtain first components of vectors ui, vi and wi (where i = 1, 2) as the X coordinate values by subtracting from the X coordinate values of feature points 1 and 2 in said images the X coordinate values of feature point 0 in said images, which falls on the origin of said XZ plane;
a sixth step of having said shape/movement determination means determine whether said first components of vectors ui, vi and wi (where i = 1, 2) obtained by said feature point position normalization means satisfy all of next three [3] conditions is regular.

is regular.

is regular.

a seventh step, invoked when said shape/movement determination means determines negatively in said sixth step, of having said shape/movement determination means output recognition disablement data to said movement calculation means, thereby executing a recognition disablement process {1};
an eighth step, invoked when said shape/movement determination means determines positively in said sixth step, of having said shape/movement determination means activate said movement calculation means for executing a process of said movement calculation means;
a ninth step of having shape calculation means execute its process;
a tenth step of having said feature point position reconstruction means determine whether or not there is a set of R, S, u1, v1, w1, u2, v2 and w2 satisfying said known data regarding said movement and said shape of said object received by said known data input means;
an eleventh step, invoked when said feature point position reconstruction means determines negatively in said tenth step, of having said feature point position reconstruction means execute a recognition disablement process {2};
a twelfth step, invoked when said feature point position reconstruction means determines positively in said tenth step, of having said feature point position reconstruction means store in said object movement storage means a pair of rotation matrices R and S
satisfying said known data and the coordinate values of feature point 0 in said images outputted from said feature point position normalization means; and a thirteenth step of having said feature point position reconstruction means store in said feature point position storage means second components of u1, v1, w1, u2, v2 and w2 satisfying said known data and coordinate values of feature points 0, 1 and 2 in said images outputted from said feature point position normalization means, thereby ending the whole processes.

41. The moving body recognition apparatus according to claim 40, wherein said movement calculation means:
calculates a1 = [a11 a12] = [u11 u12] rotation matrix R by the result of calculating a by using an expression R1 = [r11 r12] = [(1+a112-a122)/2a11 (1-r112)1/2]
rotation matrix S by using an expression S1 = [s11 s12] = [(1-a11r11)/a12 -a11r12/a12]
determines whether or not a calculated pair of rotation matrices R and S satisfies said known data of said object received by said known data input means;
invokes said recognition disablement process {1}
when no pair of rotation matrices R and S satisfies said known data; and sends said pair of rotation matrices R and S
satisfying said known data to said shape calculation means and said feature point position reconstruction means.

42. The moving body recognition apparatus according to claim 40 or 41, wherein said recognition disablement process {1} comprises:
a step of having said movement calculation means send to said feature point position reconstruction means recognition disablement data; and a step of having said feature point position reconstruction means store said recognition disablement data in said object movement storage means and said feature point position storage means, for a process completion.

43. The moving body recognition apparatus according to claim 40, wherein said shape calculation means:
calculates a2 by using an output from said movement calculation means and by using an expression a2 = [a21 a22] = second row of ;

calculates second components of said two-dimensional vectors u1 and u2 by using an expression [u12 u22] = a2 ;

calculates second components of said two-dimensional vectors v1 and v2 by using an expression vi2 = R2 ui = [-r12 r11] (where i = 1, 2);

calculates second components of said two-dimensional and vectors w1 and w2 by using an expression wi2 = S2 ui = [-S12 S11] (where i = 1, 2); and sends to feature point position reconstruction means the calculated values of second components u12, u22, v12, v22, w12 and w22 of said two-dimensional vectors u1, u2, v1, v2, w1 and w2.

44. The moving body recognition apparatus according to claim 40, wherein said recognition disablement process {2} comprises:
a step of haying said feature point position reconstruction means store recognition disablement data in said object movement storage means and said feature point position storage means, for a process completion.

The moving body recognition apparatus according to claim 40, wherein said movement calculation means:
calculates only all by using an output from said feature point position normalization means and an expression a1 = [a11 a12] = [u11 u12] ;

calculates rotation matrix R by the result of calculating a1 by using an expression R1 = [r11 r12] = [a11/2 (1-r112)1/2]
rotation matrix S by using an expression S1 = first row of rotation matrix R2; and sends said calculated pair rotation matrices R
and S to said shape calculation means and feature point position reconstruction means.

46. The moving body recognition apparatus according to claim 40, wherein:

said movement calculation means calculates a1 by using an output from said feature point position normalization means and an expression a1 = [a11 a12] = [u11 u12] calculates rotation matrix R by using only one [1] of positive and negative values of r12 R1 = [r11 r12] = [(1+a112-a122)/2a11 (1-r112)1/2]
calculates rotation matrix S by using an expression S1 = [s11 s12] = [(1-a11r11)/a12 -a11r12/a12], and sends said pair of rotation matrices R and S to said shape calculation means and said feature point position reconstruction means;
said shape calculation means calculates a2 by using an output from said movement calculation means and an expression a2 = [a21 a22] = second row of calculates second components of two [2] of said two-dimensional vectors u1 and u2 by using an expression [u12 u22] = a2 calculates second components of two [2] of said two-dimensional vectors v1 and v2 by using an expression vi2 = R2 ui = [-r12 r11] (where i = 1, 2) calculates second components of two [2] of said two-dimensional and vectors w1 and w2 by using an expression wi2 = S2 ui = [-s12 s11] (where i = 1, 2), and sends to feature point position reconstruction means the calculated values of second components u12, u22, v12, v22, w12 and w22 of said two-dimensional vectors u1, u2, v1, v2, w1 and w2;
said feature point position reconstruction means calculates a pair of rotation matrices R' and S' inverse to said pair of rotation matrices R and S by using expressions R' = S' = calculates the mirror image transformations ui', vi' and wi' of (i = 1, 2) said two-dimensional vectors ui, vi and wi (i = 1, 2) by using expressions ui' = vi' = wi' = (i = 1, 2).

47. A moving body recognition apparatus for recognizing a movement of a moving body by positions of feature points on said moving body, comprising:
an image input means for capturing images of said moving body as an object;
a feature point extraction means for extracting feature points in said images captured by said image input means;
a feature point position storage means for storing known position data of extracted feature points; and a shape/movement recognition means for calculating the actual positions and movements of feature points of said object from known position data of four [4] feature points of said object each in said images captured at any three [3] instants in time for observation, by determining that said four [4] feature points do not exist on a single plane, that the axis of rotation of said object is not parallel to the direction of an orthogonal projection of said object between any two [2] of said three [3] instants in time for observation, and that a rotation of said object between any two [2] of said three [3] instants in time for observation is not a rotation by an angle of one hundred and eighty degrees [180°] around an axis parallel to a plane on which said orthogonal projection is made.

48. The moving body recognition apparatus according to claim 47, wherein said feature point extraction means comprises:
a space filter means for extracting an outline image from said images captured by said image input means; and an edge point extraction means for extracting an edge point from said outline image and for storing it as feature point data in said feature point position storage means.

49. The moving body recognition apparatus according to claim 48, further comprising:
a feature point position storage means for storing the actual positions of feature points outputted from said shape/movement recognition means;
and an object movement storage means for storing the angle of rotation of an object outputted from said shape/movement recognition means.

245 The moving body recognition apparatus according to claim 49, configured as a computer system comprising:
a television camera in correspondence with said image input means;
image processing hardware including a space filter in correspondence with a feature point extraction means;
a first memory in correspondence with said feature point position storage means;
a central processing unit (CPU) in correspondence with said shape/movement recognition means;
a second memory in correspondence with said feature point position storage means and said object movement storage means; and a sensor for outputting to said CPU known position data regarding the shape and movement of said object.

51 The moving body recognition apparatus according to claim 49 wherein:
said image input means captures images of said object monitored by a visual device of an automatically running vehicle or an image of said object inspected in an inspection of factory automation (FA); and said feature point position storage means and said object movement storage means store the results of recognizing the movements of said object being monitored or inspected.

52. The moving body recognition apparatus according to claim 49, wherein said shape/movement recognition means comprises:
a known data input means for receiving from an external sensor known data regarding movements and positions of said feature points of said object;
a feature point position normalization means for obtaining as normalized known position data the relative positions of the other three [3] feature point when one [1] of said four [4] feature points of said object each in said images captured at said three [3] instants in time for observation falls on the origin of a three-dimensional coordinate space;
a shape/movement determination means for determining an enablement or a disablement of recognizing said movements and positions of feature points of said object by using an output from said feature point position normalization means;
a movement/shape calculation means, activated by a determination by said shape/movement determination means of an enablement of recognizing said movements and positions of said feature points of said object, for calculating an angle of rotation of said object around said origin and unknown position data other than said known position data of said four [4] feature points of said object by using an output from said feature point position normalization means; and a feature point position reconstruction means for outputting a movement of said object by combining an angle of rotation around said origin outputted from said movement/shape calculation means with said position data of said feature point falling on said origin in said images outputted from said feature point position normalization means, and for outputting positions of said four [4] feature points by combining said unknown position data outputted from said movement/shape calculation means with said position data of four [4] feature points in said images outputted from said feature point position normalization means.

53. The moving body recognition apparatus according to claim 52, wherein an external memory means for temporarily storing known data outputted from said external sensor on an offline basis is provided before an input to said known data input means.

54. The moving body recognition apparatus according to claim 52, wherein the whole processes of said shape/movement recognition means comprise:
a first step of determining whether or not said known data input means has received at least one [1]
of R = , S = , S = R, (where U is a two-dimensional orthogonal matrix), u1, u2 and u3 are not linearly independent, v1, v2 and v3 are not linearly independent, and w1, w2 and w3 are not linearly independent, where the three-dimensional vectors ui, vi and wi (i = 1, 2, 3) representing the relative positions of feature point i (three [3] of four [4] feature points not falling on the origin) from the origin on which feature point 0 falls respectively at said first, second and third instants in time for observation, are expressed as ui = = vi = = wi = = and where their relations are expressed as:
vi = R ui and wi = S ui (i = 1, 2, 3) in which the Y axis is a direction perpendicular to an image plane, which is the XZ plane, and a pair of rotation matrices R and S represent three-dimensional rotations of said object around said origin from said first instant in time for observation respectively to said second and third instants in time for observation, which are expressed as R = = = S = = = a second step, invoked when said shape/movement recognition means determines positively in said first step, of executing recognition disablement process {1}
after said known data input means outputs a deactivation signal to said feature point position normalization means and recognition disablement data to said movement/shape calculation means;
a third step, invoked when said shape/movement recognition means determines negatively in said first step, of determining whether said feature point position storage means has stored positions of said four [4] feature points 0, 1, 2 and 3 each in said images at three [3] instants in time for observation, which is repeated until a positive determination is obtained;
a fourth step, invoked when said shape/movement recognition means determines positively in said third step, of having said feature point position storage means send an activation signal to said feature point position normalization means, thereby activating said feature point position normalization means;
a fifth step of having feature point position normalization means store positions of said four [4]
feature points 0, 1, 2 and 3 in said images stored in said feature point position storage means and obtain first and second components of vectors ui, vi and wi as the X and Z coordinate values obtained by subtracting from the X and Z coordinate values of feature point i in said images the X and Z coordinate values of feature point 0 in said images, which falls on the origin of said XZ plane;
a sixth step of having said shape/movement determination means determine whether said first and second components of vectors ui, vi and wi obtained by said feature point position normalization means satisfy all of next three [3] conditions rank = 3 rank = 3 rank = 3 a seventh step, invoked when said shape/movement determination means determines negatively in said sixth step, of having said shape/movement determination means output recognition disablement data to said movement/shape calculation means, thereby executing a recognition disablement process {1};
an eighth step, invoked when said shape/movement determination means determines positively in said sixth step, of having said shape/movement determination means activate said movement/shape calculation means for executing a process of said movement/shape calculation means by fixing one [1]

value of p in an expression [u1 u2 u3] = = where is regular;

a ninth step of having movement/shape calculation means execute its process;
a tenth step of having said feature point position reconstruction means determine whether or not there is a set of results, calculated by said movement/shape calculation means, comprising a pair of rotation matrices R and S and three-dimensional vectors u1, u2, u3, v1, v2, v3, w1, w2 and w3 satisfying said known data regarding said movement and said shape of said object received by said known data input means;
an eleventh step, invoked when said feature point position reconstruction means determines negatively in said tenth step, of having said feature point position reconstruction means execute a recognition disablement process {2};
a twelfth step, invoked when said feature point position reconstruction means determines positively in said tenth step, of having said feature point position reconstruction means store in said object movement storage means a pair of rotation matrices R and S
satisfying said known data and the coordinate values of feature point 0 in said images outputted from said feature point position normalization means; and a thirteenth step of having said feature point position reconstruction means store in said feature point position storage means third components of said three-dimensional vectors u1, u2, u3, v1, v2, v3, w1, w2 and w3 satisfying said known data and coordinate values of feature points 0, 1, 2 and 3 in said images outputted from said feature point position normalization means, thereby ending the whole processes.

55. The moving body recognition apparatus according to claim 54, wherein said movement/shape calculation means:
selects a value of p (1?p?2) such that becomes regular, if necessary by rotating the XY
coordinate system by other than integer times of ninety degrees [90°];

obtains a value of h (1?h?2), such that ah3 ? 0;

defines the values of .alpha.i (1?i?2), .beta. (= [.beta.1, .beta.2]) and r (= [r1, r2, r3] ) by using expressions ai = (1 + - ai32) / 2 (1?i?2);

.beta. = [a11a23-a21a13, a12a23-a22a13], and r = [a23, -a13, 0]
calculates the values of elements r11, r12, r21 and r22 of rotation matrix R;
calculates the values of element r13 of rotation matrix R by using an expression r13 = (1 - r112 - r122);
determines whether or not the value of element r13 is zero [0];
calculates the value of element r23 of rotation matrix R by using an expression r23 = -.beta.1r13/.beta.2, if r13 =/ 0;
sends a set of wi to feature point position reconstruction means;
calculates the values of element r23 of rotation matrix R by using an expression r23 = (1 - r212 - r222), if r13 = 0;
calculates the values of elements in p-th row of rotation matrix S by using an expression Spj = (.delta.hj- ) / ah3 (1?j?3);

calculates the third row a3 of matrix a as calculates third components of three-dimensional vectors u1, u2 and u3 by using an expression [u13 u23 u33] = a3 ;

calculates the q-th row Sq of rotation matrix S
(1?q?2, q?p) Sq = [w1q w2q w3q] = a3 ;

calculates the values of elements in the third rows R3 and S3 of a pair of rotation matrices R and S, r3; = (-1)3+i det R3j and S3j = (-1)3+j det S3; (1<j<3);
calculates the values of third elements vi3 and wi3 of three-dimensional vectors vi and wi by using expressions vi3 = r3 ui and wi3 = s3 ui (1?i?3); and sends to feature point position reconstruction means the calculated set of rotation matrices R and S
and three-dimensional vectors ui, vi and wi.

56. The moving body recognition apparatus according to claim 55, wherein said movement/shape calculation means calculates the values of elements r11, r12, r21 and r22 of said rotation matrix R, by a method comprising:
a step 1 of determining whether or not the values of elements a13 and a23 in said matrix a are zero [0];
a step 2, invoked when said movement/shape calculation means determines in step 1 that neither the value of element a13 nor the value of element a23 in said matrix a is zero [0], of calculating the values of elements r11, r12, r21 and r22 by using expressions = (1?i?2);
a step 3, invoked when said movement/shape calculation means determines in step 1 that either the value of element a13 or the value of element a23 in said matrix a is zero [0], of determining whether or not a13 is zero [0];
a step 4, invoked when said movement/shape calculation means determines in step 3 that the value of element a13 in said matrix a is zero [0], of calculating the values of elements r11, r12, r21 and r22 by using expressions = and = ;
a step 5, invoked when said movement/shape calculation means determines in step 3 that the value of element a13 in said matrix a is not zero [0], of calculating the values of elements r11, r12, r21 and r22 by using expressions = = .

57. The moving body recognition apparatus according to claim 55,wherein said recognition disablement process {1} comprises:
a step of having said movement/shape calculation means send to said feature point position reconstruction means recognition disablement data; and a step of having said feature point position reconstruction means store said recognition disablement data in said object movement storage means and said feature point position storage means, for a process completion.

58. The moving body recognition apparatus according to claim 55, wherein said recognition disablement process {2} comprises:
a step of having said feature point position reconstruction means store recognition disablement data in said object movement storage means and said feature point position storage means, for a process completion.

59. The moving body recognition apparatus according to claim 54, wherein said movement/shape calculation means:
selects a value of p (1?p?2) such that becomes regular, if necessary by rotating the XY
coordinate system by other than integer times of ninety degrees [90°];
obtains a value of h (1?h?2), such that ah3 ? 0;
defines the values of .alpha.i (1?i?2), .beta. (= [.beta.1, .beta.2]) and .GAMMA. (= [.GAMMA.1, .GAMMA.2, .GAMMA.3] ) by using expressions .alpha.i = (1 + -ai32) / 2 (1?i?2);

.beta. = [a11a23-a21a13, a12a23-a22a13], and .GAMMA. = [a23, -a13, 0]
calculates the values of elements r11, r12, r21 and r22 of rotation matrix R;
calculates one [1] of the value of element r13 of rotation matrix R by using an expression r13 = (1 - r112 - r122);
determines whether or not the value of element r13 is zero [0];
calculates the value of element r23 of rotation matrix R by using an expression r23 = -.beta.1r13/.beta.2 if r13?0;
sends a set of wi to feature point position reconstruction means;

calculates one [1] of the value of element r23 of rotation matrix R by using an expression r23 = (1 - r212 - r222), if r13 = 0;
calculates the values of elements in p-th row of rotation matrix S by using an expression Spj = (.delta.hj- ) / ah3 (1?j?3);

calculates the third row a3 of matrix a as calculates third components of three-dimensional vectors u1, u2 and u3 by using an expression [u13 u23 u33] = ;

calculates the q-th row Sq of rotation matrix S
(1?q?2, q?p) Sq = [w1q w2q w3q] ;

calculates the values of elements in the third rows R3 and S3 of a pair of rotation matrices R and S, r3j = (-1)3+j det R3j and s3j = (-1)3+j det S3j (1?j?3);
calculates the values of third elements vi3 and wi3 of three-dimensional vectors vi and wi by using expressions vi3 = r3 ui and wi3 = s3 ui (1?i?3);
sends to feature point position reconstruction means the calculated set of rotation matrices R and S and three-dimensional vectors ui, vi and wi;
has said feature point position reconstruction means calculate a pair of rotation matrices R' and S' inverse to said pair of rotation matrices R and S by using expressions R' = S' = and has said feature point position reconstruction means calculate the mirror image transformations vi', ui' and wi' (i = 1, 2, 3) of three-dimensional vectors vi, ui and wi (i = 1, 2, 3) with respect to the XZ plane by using expressions ui' = vi' = wi' = (where i = 1, 2, 3).

60. A moving body recognition apparatus for recognizing a movement of a moving object by positions of feature points in images of said moving object, comprising image input means for capturing three images of said moving object at three instances in time at equal time intervals, said image input means located at a position perpendicular to the axis of rotation of said object moving on a single plane and said object having a rotation at a constant rate;

feature point extraction means for extracting two feature points each in said three images captured by said image input means, said feature point extraction means including space filter means for extracting an outline image from said three images captured by said image input means and edge point extraction means for extracting an edge point from said outline image;
feature point position storage means for storing known position data of said extracted two feature points and for storing said edge point as feature point data; and movement recognition means for calculating the movement of said object from said known position data.

61. A moving body recognition apparatus for recognizing a movement of a moving body by positions of feature points on said moving body, comprising:
an image input means for capturing images of said moving body as an object from a direction perpendicular to the axis of rotation of an object moving on a single plane coupled with a rotation;
a feature point extraction means for extracting three [3] feature points in said images captured at any two [2] instants in time for observation by said image input means;

a feature point position storage means for storing known position data of extracted feature points; and a movement recognition means for calculating the movement of said object from known position data of said three [3] feature points forming a right angle of said object each in said images captured by said image input means, by using an output from said feature point position storage means.

62. A moving body recognition apparatus for recognizing a movement of a moving body by positions of feature points on said moving body, comprising:
an image input means for capturing images of said moving body as an object from a direction perpendicular to the axis of rotation of an object moving on a single plane coupled with a rotation;
a feature point extraction means for extracting three [3] feature points in said images captured at any three [3] instants in time for observation by said image input means;
a feature point position storage means for storing known position data of extracted feature points; and a movement recognition means for calculating the movement of said object from known position data of said three [3] feature points of said object each in said images captured by said image input means, by using an output from said feature point position storage means.

63. A moving body recognition apparatus for recognizing a movement of a moving body by positions of feature points on said moving body, comprising:
an image input means for capturing images of said moving body as an object;
a feature point extraction means for extracting feature points in said images captured at any three [3] instants in time for observation by said image input means;
a feature point position storage means for storing known position data of extracted feature points; and a movement recognition means for calculating the movement of said object from known position data of four [4] feature points of said object each in said images captured by said image input means, by determining that said four [4] feature points do not exist on a single plane, that the axis of rotation of said object is not parallel to the direction of an orthogonal projection of said object between any two [2] of said three [3] instants in time for observation, and that a rotation of said object between any two [2] of said three [3] instants in time for observation is not a rotation by an angle of one hundred and eighty degrees [180°] around an axis parallel to a plane on which said orthogonal projection is made.