WO1993001080A1

WO1993001080A1 - Method and apparatus for assembly of car bodies and other 3-dimensional objects

Info

Publication number: WO1993001080A1
Application number: PCT/CA1992/000296
Authority: WO
Inventors: Timothy R. Pryor
Original assignee: Sensor Adaptive Machines Incorporated
Priority date: 1991-07-12
Filing date: 1992-07-13
Publication date: 1993-01-21
Also published as: US5380978A; EP0592585A1

Abstract

This invention is particularly concerned with the manufacture of car bodies - particularly what is called the ''Body-in-white'' (BIW), the as - welded body without the doors, engine, transmission, etc., and major sub-assemblies thereof. While particularly oriented to this problem, the invention is also usable for assembly of other ''3-D'' structures, such as planes, ships, trucks, buildings, telephones, and the like. Some applications even exist in space, for manned and unmanned operation. The manufacture of car bodies is a complex process, and the tooling required to make the various component pieces and assemble them into a dimensionally correct, aesthetic shape consumes a great deal of money and time. Current trends in the industry are to ever shorten the time involved and to lower cost. This invention assists this goal. An important and preferred element of the invention is the use of optically visible datums (in UV, visible, and IR wavelengths) on a tool or on the part which are either functional (that is used for assembly or other purposes, such as a hole) or simply for the purpose of optically guided positioning, either manually or automatically. These datums, typically holes, painted, engraved, or retroreflective targets, either by themselves, or in combination with other features of the part such as surfaces or edges, are observed by optical sensor/camera systems in and/or surrounding the work area. One or more positioning means either automatic, semi-auto, or manual, are guided by the signals from the cameras to cause the requisite portion of a tool, or the parts themselves to be positioned in the proper location.

Description

METHOD AND APPARATUS FOR ASSEMBLY

OF CAR BODIES AND OTHER 3-DIMENSIONAL OBJECTS

Referenced US Patents

US 3,968,558 Sekine et al, 5,010,634, 5,005,277, (Uemura e al), 4,691,905 Tamura et al, 4,654,949 Pryor, 4,453,085 Pryor, 4,639,878 Day et al, 4,796,200 Pryor, 4,736,515, Catena, 4,396,945, DiMatteo et al, 4,044, 377 Bowerman , 4,613,692 Chen. Introduction

Automobile bodies today are assembled using what is commonly referred to as "fixture tooling" usually in volumes over one hundred thousand per year, per plant.

This tooling, which is typically hand fabricated from thic steel, is fixed in place - ie "hard", and is used to position the parts precisely for assembly (typically welding) . It is very costly since it is custom built, difficult to maintain and is further of exceptionally long lead time since it must be precisely made. In addition it is not immediately reliable, and often takes months to "debug" along with associated metal fit problems of the panels provided. Finally such tooling is "Dumb", providin no process feedback.

These factors impede the manufacturer*'s ability to come out with new car models in a timely manner to meet market needs. They furthermore saddle the manufacturer with a large tooling amortization cost which, though practical in large production quantities such as 200,000 bodies of the same type per year, may be totally impractical when amortized over low volumes of 10,000-20,000 per year. These numbers seem small but several cars with quite popular name plates have unfortunately been selling in just these quantities recently and numerous articles have been written over the fragmentation of the industry into "niche" markets.

The hard tooling not only takes a long time to build, it also takes a long time to get running properly. This is because the parts seldom fit together as they are supposed to and the tooling is often changed after the fact, idling the rest of the line in the process. Finally even on a running line there is substantial maintenance associated with keeping the various parts of today's tools operational.

The invention solves many of these problems. A first group of embodiments addresses a new form of Machine vision based fixture tooling construction based to a degree on the invention of US Patent 4,851,905 by the inventor. Code named here "RER" or Rapidly Erectable/Reconfigurable tooling, this system can allow very quick set up and change over of conventional locating and clamping fixtures. Some elements are also disclosed in a general line control system for conventional lines, capable of monitoring critical locator and robot locations. Others can be used for assembly of vehicles having space frame construction and the like, by accurately positioning locating pads for panels which attach to the space frame.

A second group of embodiments concerns partly and fully "fixtureless" assembly versions of the invention which facilitate production in very low volumes. While the fully flexible system described herein can be operated in a totally automatic manner, humans can be used to loosely position and tab/slot or otherwise temporarily join the parts together, using the invention to provide final positioning and rigid/semi-rigid holding while welding or joining occurs. (Semi-rigid is made possible by the laser joining processes, as no forces are exerted on the parts to effect the weld, only to hold them in contact.)

Where parts themselves are directly positioned, they are can be joined, using means such as resistance or laser welders, or fastened with screws. Adaptive control of position can include many variables such as forces, previous history ("learning"), etc. This process can be self checking and adaptive in the sense that errors or distortions created in the structure by joining at one location can be accounted for at one or more further locations by repositioning of the pieces or distorting them in the proper manner to remove, or at least ameliorate, the previous distortion. This reduces error propagation and can intelligently be used to force error to zero in the body build. A major advantage of the invention is that it can be used to produce different models. If one model doesn't sell you don't have its assembly tooling cost (and critical plant space) tied up, you simply switch over. The amortization period of the new type of tooling will be limited only by wear and not the life of the vehicle itself. This means that maybe three or possibly four vehicles lifetimes could be experienced compared to the situation which exists today. In addition, running changes can be easily incorporated where only a portion of the vehicle is changed to "freshen it".

With the invention, construction of prototype bodies can be accomplished with the same equipment as final production, where for the first time fabrication techniques and programs checked out in the prototype phase maybe directly used in volume production. While suited perhaps best for the lower volumes, such flexible tooling can be scaled up, if the car begins to sell, to produce higher volumes to buy time if needed while more conventional tooling might be constructed, at which point the tooling of the invention can be turned over to build other models. The invention is particularly suited for: a) Construction of fabricated assemblies with rapidly erectable and reconfigurable tools; b) Assembly with flexibly configured tools augmented by human or robotic means; c) Construction of dimensionally correct assemblies substantially by robotic or human means, or mixture of the two - without resort to fixed "hard" tooling. Background In the process of developing a car today, first pieces produced are assembled into prototype vehicles which can then be used for consumer evaluations, engineer- ing tests and safety, emissions, fuel economy tests with the government . The sooner such prototypes are ready, the shorter the process becomes.

After the prototypes, the first run of pilot vehicles are built, next, it is then imperative to get the first vehicles of an approved project to the showroom. The question then is how to produce these vehicles in the shortest possible time frame realizing however, that they have to be dimensionally correct - often times to a few thousandths of an inch (0.1 mm), in many of the critical sealing or chassis dimensions.

In bygone days when dimensions were not so critical, tooling jigs could be hastily slapped together to support the various pieces of the car such that the vehicles could be produced. Slowly as the production evolved, one could adjust the jigs to make the body better to a degree. Today however, it is impossible to sell a car that has poor body dimensional fits. Furthermore the consumer magazines and opinion leaders test these early vehicles and form judgments that may linger for the vehicle lifetime.

Today, there are at least two different stages of potential tooling production; one for the initial proto¬ types; e.g. 200-300 units, and, production tooling used to produce however many cars will sell. In this latter regard it is noted that while many cars have been built with tools that could produce 200,000 vehicles a year, the actual market has not been so kind and the cost of such tooling become an enormous burden if sales are only 15,000 for example per year.

It not generally known in the art how to produce dimensionally accurate bodies without resulting to expen¬ sive fixed hard tools of the high production kind, at least if "world class" in dimensional precision is needed. (0.1 millimeter precision for the location of the various mating surfaces and hole to hole locations are critical to the function of the body and its chassis components.) Another problem with present production results if one does build such dimensionally accurate "production hardened" tools, one is often precluded from changing to other bodies rapidly if the intended one does not sell. Although certain attempts have been made with interchange¬ able tool plates such as the Comau "Robogate" or the Renault systems; these solutions are "brute force", and in any case are still practically limited to a relatively small number of different bodies e.g. 4. And too, in the case of the conventional tooling for example a body is often thought to be different even though it might be the same car but in a sedan or coupe version, or a different name plate, with slight differences in the basic platform. While at least two attempts worldwide have been made to provide completely programmable body tools (the

Volkswagen "Geobox", and the Nissan IBAS) , these are very complex, expensive and in any case pose the question of exactly where in location the programmable details are, i.e. a control problem, (the IBAS system as shown in U.S. Patent 5,010,634 does however provide a means for sensing workpiece position in a work cell to a structure during assembly) .

Few similar attempts have been made to flexibly produce the smaller sub-assemblies, such as doors, under- bodies, side frames, etc., which often form a critical bottleneck.

This invention seeks to solve all of the above problems and particularly forms an economic and unitary flexible production system for all bodies from the earliest prototypes all the way through to production of at least 5,000 units. The system further is expandable into low cost rapidly changeable tooling which can produce 200,000/yr. Figure 1 : Description of the Prior Art Figure 1 illustrates one aspect of the assembly of the body-in-white as is conventionally performed today (see also for example US PAT 3,968,558, Sekine et al.). While approaches vary throughout the world there are really only two fundamental principles in common use for volume production. Welded Unibody construction is by far most prevalent, and is largely treated here, although the inven- tion is not limited thereto.

Conventional High Volume Practice

In the first approach, shown in Figure 1, the two parts should be, let us say, the side to the underbody of a car are to be assembled. The side of the car, 2, (called a Body Side) is positioned by locator block 7 and attached fixture 4 and clamped by clamp 3, against the member 5 of underbody 6, and against various front and rear members as well, typically. Although only one member, a front cowl structural member 5, is shown here for clarity in a posi- tion to be assembled. While the side of the car is shown clamped at point "P" by clamp 3, it is actually held at perhaps as many as thirty locations depending on the situa¬ tion. Points are held together with additional clamps, such as those of Figure 2, which are pneumatically actuated against the metal to keep it together so that the spot welds in between the clamping locations can be affected properly, for example by guns shown in Figure 2b.

Dimension of the assembly is controlled by locator blocks such as faces 8 and, in certain locations by pin means, such as pin 9 in hole 10.

The side may be "tabbed" into the underbody or cowl to hold it temporarily in place using for example tab 15 of side 2, and slot 16 of the underbody 6 (a process commonly called "toy tabbing") . On command a spot weld gun 20, (carried for example a robot 21) comes in and pulls the metal together into contact and resistance welds it. Welds are often placed at specially provided metal flanges on the parts in question.

This mode of operation is used throughout the world, with differences being whether or not fixed weld guns or robotically positionable weld guns such as 20 are utilized. Fixed guns have the advantage that they are fixed in (or swung into) one location and can not drift off (however, this often is not necessarily the case due to mechanical problems) , and that numerous guns can weld simultaneously. However such mechanical fixtures are cumbersome and notoriously difficult to maintain and setup. They are also not usable for other assemblies, and are generally scrapped after use on a single car model run . It is virtually impossible to build another body on the fixture with a large number of fixed guns. After the point at which the structure is physically joined at key locations, the remaining welds on the structure are typically put in, in the case of the body-in-white with robots, and in the case of the sub-assemblies with fixed guns although many lower volume manufacturers in Europe, for example, also use robots here as well.

While shown here with implications for the assembly of the complete body from major subassembly components, such as the side frames, underbody, motor compartment, roof, etc., the same technique can also be used for each of these assemblies, which may have 15-50 individualized pieces, which themselves may be subassem- blies. In subassembly welding, less robotics is often used than for the final body framing operation. There are multiple problems with the conventional prior art assembly techniques as described above. First, it is very difficult to change from car line to car line; that is let us say from a Cadillac to a Saturn; because the fixed weld tools, such as locator block s(also called an 'NC Block) also including clamps - and fixed guns mounted in relation thereto, and actuated to slide in and weld the metal flanges in the vicinity of the clamps, simply cannot be moved. One approach to at least allowing this to be done for similar styles has been the use of movable gates which are assemblages of such locators which can be changed en mass. However, there is still no mobility in most lines to change complete types of cars (car lines) either from job to job, after a shift, or even over a weekend. Figure 2

Figure 2a further illustrates the function of locators and clamps. NC Block 7 whose surface center is located at cartesian point X,Y,Z, has a machined surface at angle θ (and other angles θ & γ as needed) , cut from a CAD model of the part, positions the part at the correct location. Clamping force F is required to offset the springback of the part(s) 2 and 5 being welded.

In real life, the clamping force is supplied by pneumatic pressure, and in many cases clamps distort the metal in whatever direction necessary in order to force the piece against the block face and against each other. In a sense one is actually re-bending the metal to accommodate the fact that it wasn't stamped correctly in the first place (or was distorted during shipment) . In some cases, the part is located on a pin, to locate it in the plane against the metal. This can only be at one location on the part, let us say, because of the problem of inaccuracies of hole locations, typically.

Figure 2b illustrates a typical spot weld gun 20, welding flanged metal parts 21 and 5 between 2 clamping locations 31 and 32. The gap W is closed by virtue of the clamping action of the pneumatically activated gun elec¬ trodes 35 and 36.

It is now desired to weld the two parts, who would have been so clamped together at discrete locations. Typically they are clamped together at multiple locations, along the length and height of the part. The larger the part, the more clamps and locators. In a typical situa¬ tion, there is a flange between the two pieces of metal which acts at the point at which these spot welds are placed. The robot comes in as shown in Figure 2B, located itself over the flange, and clamps the two together with the pincers of the robot electrodes, and through a high current, resistance welds the two pieces together in a "spot".

There are several problems that occur in this process. First of all, the robot location is uncertain (flange widths 'h' being made large enough to accommodate some error, but at a cost of body weight and aesthetics) , and if the robot is off location, during this process, the metal is pulled away from its proper position. This can distort the metal, and can cause the clamps nearby to open, thereby creating a major error in "fit" of the body. Often times too, once a clamp is so opened up it can stay opened up for the whole welding of the body, since there may be no intelligence to say that it has been opened (see also discussion below) . Robot location is uncertain because of the accuracy of the robot in 6° of freedom space holding the gun, which itself is heavy. Even if the robot per se was accurate, which it isn't, the gun weight in different orientations would be enough to throw it off. Secondly however, there are problems with the fact that the metal itself has to be clamped together by the electrodes. In other words,, if one tries to weld it without such clamping action, no weld occurs, because of the air gap 'w' between the metal, in between the clamp zones. In other words the clamping is required to make a zero gap.

This is a major impediment to laser welding, which is a one sided weld process, not using two squeezing electrodes, and has inhibited the use of laser welds in industry. Today, in order to laser weld, for example, one virtually has to have"perfect" sheet metal, placed perfect¬ ly in the tools (a near impossibility) , or one has to have the same kind of problem clamp location that replicates the robotic spot weld tools. The invention, as will be described later, solves this problem through intelligent positioning of tooling or metal, and if necessary, control¬ led deformation of the metal. Misposition of the robot can often cause the body metal to be pulled so hard as to break open one or more clamps, causing gross body distortion. Even minor errors can cause stresses, bending, and other problems. aAnother major disadvantage of the above prior art systems is that they cannot guarantee that the individual metal locations between the clamping locations (or in many cases even at the clamping locations) are indeed fitted together; that is in physical contact such that a laser for example coming in from one side can weld them together.

Without such contact, i.e. with a gap "W" as in Figure 2b, the use of otherwise advantageous laser welding (which can be used to reduce the weight of the car through strengthen¬ ed individual welds and is much more accurate) cannot be used. It is the intent of this invention to show systems that ensure such fit up and indeed maximize the value of laser systems for car body manufacture.

Another major problem, not apparent per se from the drawings of the conventional equipment, is that it takes a long time for these systems to be set up. Both because they physically locate the body at multiple points at once and are related to mechanical tool co-ordinates that have to be precisely set up mechanically. They also require a substantial amount of debug time, particularly with the conventional non robotic tools, the positions have to be changed often in order to keep the body together especially in the presence of dimensionally incorrect panels that do not fit properly (a "bad metal" condition) . In this case, in a conventional system, one rapidly loses touch with exactly where the locators are in space. Only that "wherever they are today, that's what makes the metal that we are getting today, work" (or not work, as the case might be) . There is thus little or no intelligence that can be brought into the process since almost nothing is known about the actual physical locations of the locators. Only after the fact - at the end of the line, are the body gages utilized to determine the whole body dimensional fit complete with all the spot welds that have been put in. This is not necessarily indicative of what it takes to control the original welding operation used to establish the positions. This same argument on a smaller scale, also holds for the sub-assemblies (doors, sides, underbody motor compartments, etc) .

Finally another disadvantage of prior art systems is that there is no commonality and no carry over from one part of the process to another, or from year to year, model to model, plant to plant. And there is no real economy of scale in capital cost, training or maintenance, nor is there direct "learning" between the construction of proto¬ types and the experience of building them, to the final high production build process. Goals of the Invention

The prior art greatly limits the quality, flexibility, and timeliness of autobody and other part production. Accordingly:

Further disclosed are modification of joining (for example welding) , programs as a function of the location or forces of objects being assembled, or with respect to additional variables, such as laser power, steel thickness and the like.

1. It is a goal of this invention to provide a tooling system for positioning parts for joining or other assembly tasks, which can be readily and economically constructed, and changed over for other parts. This tooling system can have optional abilities to determine part or robot positions and forces.

2. It is a further goal of the invention to provide means for determining the position of the tools and for being able to relate the finished product quality to the tool locations. 3. It is a further goal to provide systems which can flexibly position: one part relative to an arbitrarily held second part; one part relative to one or more other parts, where at least one part is loosely pinned and/or supported on a surface; one part having a tab, into another having a slot; one part relative to a flexibly held second part; 4. It is also a goal to provide means for accurately joining objects which are slipped together or have eccentric holes;

6. It is an even further goal of the invention to show application of force control and optically based position knowledge to intelligently position, and if necessary deform sheet metal parts to fit together in an overall assembly;

7. It is a further goal to provide a unified system for controlling a line containing tools, robot joining devices, and gages, in order to reduce flange widths, improve uptime, etc.

In Addition :

1. It is a goal of this invention to provide systems which can self inspect and feed back data in real time after or even during the individual joining operations. It is a further goal to optimize the build of the body via feed forward of data, best fit models, and other means, to create higher quality and reduce cumulative errors;

2. It is a goal of the invention to provide assembly systems which provide quick accurate manual or automatic changeover between different models or assemblies;

3. It is a goal of the invention to show a common system which can be used all the way from prototypes to higher volume production with common tools, sensing, intelligence, and learning through the process, and to provide a unified system for assembly of sub assemblies, the complete body, and even the components to be added to it, such as doors, tail lights, wheels, batteries and the like; 4. It is a goal of the invention to provide systems which can be used to accurately join parts by sequential rather than parallel clamping, and reducing the complexity and debug time of assembly structures; 5. It is a further goal to provide programmable weld clamping and steady rest devices, also usable with the vision locating system;

6. It is also a goal of the invention to provide means for learning from past assembly experience, including position and force data, to better control instant and future assembly operations.

7. It is a further goal of the invention is to provide a method of assuring fit up for laser welding, and other joining purposes, even in the presence of less than perfect components, and to provide methods for one sided welding with slip fit parts;

8. It is a further goal of the invention to provide photogrammetric vision systems comprising multiple cameras which can be accurately calibrated and maintained in plants.

9. It is a further goal of the invention to illustrate means for dynamically altering the dimension or location of joining details of mating parts (eg. tabs and slots) as a. function of determined or estimated dimension or location of the parts.

10. It is a still further goal of the invention to provide means for target datum application verification and data base storage;

11. It is an even further goal of the invention to show application of force control and optically based position knowledge to intelligently position, and if necessary deform sheet metal parts to fit in an overall assembly;

13. It is also a goal of the invention to provide means for learning from past assembly experience, including position and force data, to better control future assembly operations;

14. It is a further goal to provide temporary part holding means of novel configuration; 15. It is a further goal to provide systems for driving of assembly errors to zero using programmed deformations, programmed weld paths, or sensor controlled slip fit assembly; 16. It is a further goal to provide a combination of manually positionable tools with automatically positionable robot part positioners, including those dynamically guided. Also disclosed is sensing of datums on parts, as well as on robotic positioners for multi-robot positioning of parts; 17. It is a further goal to disclose the use of calibration or master objects loaded into robotic assembly cells, which have datum points with respect to each other, and known shapes. Location of these points are then determined using the system relative to the locating points, either flexibly robotically positioned, or otherwise, and the positions of the parts are monitored before, during, and after welding, and robotic positioning, to create an ability to teach the system, and to provide for diagnosis of system function. Description of Figures Illustrating the Invention

Figure 1 - detail of a conventional framing station weld system of the prior art as widely practiced. Figure 2 - illustration of fixed and robotic clamping, locating and welding details in common practice and fit up problems which can occur between locator points. Figure 3 - illustrates a tooling embodiment of the invention (RER) .

Illustrated is manual positioning of locators in Space using a visual display, deriving as well from my co-pending applications with preferred optical and computer image processing embodiments;

3a illustrates a basic tool plate with 3 tool locations adjusted in location using a display driven from the optical sensor system of the invention. A preferred sliding base with 2 axes of manual or motorized movement and roll axis measurement is shown;

3b illustrates a complete tool of the invention with parts,having motorized micro adjustment of tool position for tuning and learning purposes, also including force sensing;

3c detail of targeted "NC BLocks" of a tool, including motorized positioners for accommodating different parts;

3d illustrates a functional block diagram of the embodiment;

3e illustrates a 5-6 axis embodiment with locking ball joints;

Figure 4 illustrates an intelligent tool embodiment using sensor controlled robot loading and working, and measurement of part variables as a result of joining. Also illustrated are some automatic tool functions for either reconfiguration or process learning.

4a illustrates parts in a robotically loaded tool, including guidance to preclude the robot from moving or smashing the tool, and for optimizing quality, and diagnostics;

4b illustrates a special tool with one fixed and one conforming locator,and robotic laser welding and clamping; 4c illustrates conforming holding and clamping, including pushing a part into a generic holding device, and using said device to hold the part for clamping and welding operations, to include a unique two axis flexible holding device and magnetically stiffened holding devices.

4d illustrates an embodiment with at least one fixed pin or surface location, and other locations flexibly positionable.

Figure 5 illustrates a control systems for tools and robots

5a illustrates a calibration system of the invention, used in an AGV body line 5b illustrates an optical control system for complete body in white line including tools, robot welders and gages. Such a control system can allow maximum up time, highest accuracy of body build and decreased flange widths. Optional are force and weld parameter monitoring Figure 6 illustrates a "fixture-less" automatic

(or manual) assembly cell according to the invention, including force as well as optical sensing.

In this case parts are held without benefit of precisely located mechanical details, and optically guided robots according to the invention, are utilized to position the parts, and to clamp and weld them. This is a fully flexible, "fixtureless" approach, which represents the ultimate in flexibility, since software essentially defines the tool. It also is disclosed in manual and semiautomatic modes.

6a illustrates Components of the 1/4 panel of a car being assembled according to the invention;

6b illustrates a detail of welding two components together in the previous embodiment; 6c illustrates a body in white assembly station of the invention to obtain best fit Figure 7 - Calibration 7a illustrates factory calibration of photogrammetric camera modules using a CMM; 7b illustrates an interferometric displacement based apparatus for electro-optical determination of member position, both as an alternative to fig 3, and a means to calibrate the systems of this invention. Also shown is an alternative manual or auto theodolite calibration; Figure 8 illustrates an alternative tool positioning method, using tool mounted cameras Figure 3 - The First Embodiment of the Invention

The previous two embodiments basically describe the prior art of conventional sheet metal body assembly (commonly termed unibody construction) , which have been used to build both the finished body of the car and the sub-assemblies, such as doors, sides, underbody, etc. relating thereto. These systems generally have fixed locator positions for the resting of one piece; generally the largest, with clamps that are brought into clamp the second mating piece, usually smaller, to it in an accurate manner, with accuracy generally derived from mechanical locations, such as with precision tooled NC blocks, gage pins, etc.

Changeover between one car type to the next can only be done by completely changing the physical mechanical tools either on a sliding fixture (Robogate) , rotary fixture (Renault) , or a total interchange of tooling sets typically called "gates". This physical/mechanical exchange has historically been very expensive, both to build in the first place, qualify, debug, diagnose, ware- house, etc. Clearly as the number of parts or different styles goes up in any one plant, as it needs to for maximum flexibility for the market, this tooling cost and complex¬ ity escalates almost exponentially in the effect on the plant, since the requirement for change exists at many different assembly and sub assembly locations given that several hundred sheet metal pieces of the car need to be joined together to make the unibody.

It is the particular goal herein to disclose practical applications with respect to sheet metal body construction, since this represents the largest field of present car body manufacture. This invention is not only applicable thereto however, and indeed applies to space frame construction, and other types of assembly tasks.

Note that generally speaking, we are not correcting gross misalignments, but fine misalignments, both to tool or robot misalignments and the part shape itself. In addition,tooling locations may be off, and the question of distortion due to the process and whether or not one can reform the materials, so to speak.

My previous invention (U.S. Patent No. 4,851,905) has disclosed a means for the construction of fixture tools. This application consists of many improvements to that invention, including additional capabilities for positioning of tools, detection of part location in the tool, positioning of parts in the tools, and improved methods for manually or automatically configuring the tool. Also disclosed in this invention are numerous further improvements to the operation of the process of making the parts within the tool, such as the inspection of parts and control of robot positions during manufacture, the control of weld gun locations , determination of clamping locations and forces, and the automated micro-repositioning of the tool surfaces to account for production and material variation.

Figure 3 illustrates a method according to the invention for the rapid and reconfigurable configuration of body framing/tooling. It is realized that this is useful for sub-assembly welding as well. (e.g. side frame, under¬ body, motor compartment, doors etc.)

The invention is illustrated first in the reconfigurable tool embodiment of Figure 3a, which shows a tool base 50, according to the invention, typically made of steel, on which a tool for assembling small sheet metal parts of the car is to be built up. The operator is to take various pieces of the tool and place them on the tool base under the instructions presented to him on display 52, driven by computer 60, which itself is connected to an electro-optical measuring system 58 capable of determining where the tool position is in a 3D reference coordinate system, preferably using optically visible reference points thereto, of sufficiently high contrast for the image processing or other system being used to be accurately distinguished and their position determined. In this particular case, this electro-optical measuring system is comprised by fixed stereo cameras, represented by 61 and 62. Where larger tools are involved, or more resolution, optional cameras 65 & 66, for example, can also be employed, with a suitable delineation in the computer as to which cameras apply to which area of the tool.

The goal of the procedure is to mount a sheet metal locating block 70, commonly called an "NC block"in the trade (as its metal location surface 71 is NC machined in precision juxtaposition to its mounting surfaces) , at a position in 3D space, such that its surface face contacting sheet metal 71, will be in the correct position proscribed by the tool model 75 in computer memory, derived from the CAD data base of the car body, 76. A goal is to look at a reference point on a block, such as retroreflective reference point target 80, with the electro-optical measur¬ ing system, and determine the location of the target in 3D space, and instruct the operator with suitable commands on display 52 where to position the block. The block face 71 is known through the machining process of the block to a sufficient degree of accuracy to the target 80 (typically ±0.12mm)

In the perfectly general case, an additional reference point, such as circular target 84, is also required in order to establish the roll position of the surface in the direction around the z axis of the riser post 85, as it is called, holding an L shaped bracket 90 to which a block fits. If other means are available to position the tool, such that it is always in the correct plane, relative to the face 71, then the positioning by the operator in the roll direction around the z axis is not required. In other words, if all blocks are always placed such that the L plates 90 are parallel to the cartesian axis, then the surface of the block 71 can simply be cut at a compound angle to fit the piece of metal that might be requiring same, otherwise it can be cut at a simple angle - one angle relative to one cartesian axis,and the sensor system herein used to determine the rotation required to position the surface at the correct roll angle.

It is noted that generally the reference point datum is desired on a face generally in a direction perpendicular to the tool base, although other cameras such as optional side facing cameras 91 and 92 can also be used to give enhanced accuracy in the Z direction (the Z being the weakest axis to the limited angle between the two cameras forming a stereo effect) . Where occlusions prohibit viewing of the top target for example due, cables, wires, various other problems that can ensue in a factory, additional cameras can also be used to supplement.

To make a tool that could be positioned in an infinite number of x,y, z locations within a working area of the tool, there is a question of how the riser base 100 is positioned on the tool base 50. Presented here are 3 ideas. One, is the use of tapped holes on the base, at known locations, where the block is simply locked down into these tapped holes. In this case a degree of motion in the y and z direction, using same micrometer adjustment screws or shim blocks is required, in order to place the tool in the proper location. Shim blocks while rigid, are to be avoided if possible, due to the difficulty and tediousness of positioning the tool. These shim blocks are also routinely placed between the NC locator block, and the L plates in conventional practice today.

A second alternative is to place linear dovetail ways into the tool base, such as those 110 shown. Here in this case the riser block base 100 with suitable dovetail bottom can be slid along in the infinite number of x positions. This too, is a sturdy type of system. However in y, again an adjustment would be required. It is noted that the dovetail slots might typically be parallel to body lines of the car.

The most versatile of all is to have a infinitely adjustable x,y position of the block itself, by using a magnetic base, particularly for example if the block 100 is magnetic, and the base of steel, one simply can activate a lever such as 101 to lower the magnetic portion down, as it is well known, to lock the base down. The magnetic version for many applications as will be discussed, can be strong enough to support the welding. This however, assumes that sufficient control is present for the robot to keep the parts from being pushed or otherwise stored in such a manner that they would lose some of the magnetic holding force. The block 100 can have an electro magnet, which is controllable, and provides additional force. This could be energized for example to save power, only when an actual working operation was to be done.

The z axis can move up and down, and for example, be locked by set screw mechanism 120, or other means, such as hydraulics, etc., where an expanding collet can be used, for example. Optical operation

In order to operate the system, the camera units pick up the image of the target datums, which are typically of either retro-reflective material (using on axis light sources 61a and 62a, for example) , or illuminated with general light source 63, if they are painted, for example, or other bright or dark relative to their surroundings. It is generally desirable to have high contrast, such as the target positions are immediately visible on the screen to the operator. Suitable systems to see the targets are Cohu model 1200 cameras (RS 170), with a Matrox Image 1200 frame grabber and image processor. Targets can be found using binary thresholding, when of high contrast, or with grey level image processing when less distinct from the background. Sub pixelization to achieve accuracy can be done by fitting circular (or other as desired) curves to the target, and sensing edges, for example using the technique of US PAT 4,394,683.

When retroreflectors are used, there is virtually no problem finding the targets. Otherwise where there is some difficulty the operator can put a window around the target location, such as 85, by signalling it with a light panel, or whatever, using no means, and this area then is the window in which the camera system looks for that particular target in order to determine its location. Using suitable target tracking algorithms in the computer, these targets can be tracked as the tool is slid around, even if many other targets are present in the field. Segmentation of the image in this way too, has the additional advantage that it makes a much smaller volume of image to process. For example representing only 100 by 100 pixels in a field of view of, let us say, 700 x 500 for each of the cameras, and possibly less, and therefore greatly speeds up the solution, and therefore the inter¬ action of the operator. In the particular case of two stereo cameras, the two cameras each see the target, but from a slightly different view, because of the known angle between the two cameras, which is either known a priori, or essentially calibrated by means of test targets in the field (discuss below) two positions at which the target is found are correlated together, and the x,y z distance found by known photogrammetric principles.

The goal is then for the operator to move the tool signified by point p, which he has say designated by putting a window 77 around it using known software means, to a direction indicated on the display. This display then would have a, let us say, a point to which he is to move it, P' indicated. The operator then simply watches the display as he moves it to P' . However the display itself is only in x & y. What about z? In this case, there are many possibilities, including have 3D glasses for the operator, using known means, by which he can actually see in 3D, because of the stereoscopic cameras.

The other possibility is simply for him to get it into the point, in which the x, y location is correct, indicated by the x, y , z location, delta x,y, z shown on the screen. When the delta x and delta Y are sufficiently less than some thresholds, such a .12 mm, or whatever is chosen, he then simply moves the z axis of the tool up and down, until he gets in the correct location. If the system is calibrated accurately , there will be no change in the x & y. If there is some change, due to change in vantage point, or something, he would have to then re-adjust x & y. However, every attempt is made to calibrate out these errors so that there is no "cross coupling" effects. When the operator is complete, he moves on to the next tool locator, as will now be shown in Figure 3B. Before moving forward, it is of interest to note that one can use a different type of 3D measuring system to deter¬ mine the location of a block, such as automated theodolite cameras, as shown in the referenced application, and auto¬ matic laser tracking devices as shown in the text below.

It should also be noted that the operator in the case of distinguishable targets, may not need to designate the target initially at point p, because, for example, if there is an imaged target such as p and q corresponding to points 80 and 84, a roll type target, there could also be other indicators in the field that would essentially code this as being a unique block in the field, and the computer could find this first, by suitably searching the field of view for that code, through known template correlation techniques.

It is also contemplated that the display would also instruct the operator which blocks to choose for any given riser. In other words, he would select them from a bin or box that had the ones for that particular part, that was to be set up. It is noted that the particular goal of this is to rapidly "build" a fixture, and very importantly, provide a way to reconfigure it into another. The goal is to use common components which can be rapidly reconfigured to suit a new part design, such as from another vehicle, or even another part of the same vehicle.

Required, is an unobstructed view of the tool targets in setting up, and if used during production, when the robots or other automation extend to the vicinity of the desired locations. This means the cameras may have to be at an angle to (ie above, to the side, etc) , or behind the datum surface. In addition, the datum surface may not be desired for target location due to wear and other factors. In this latter case, the targets are for example, located on the top of the block or an appendage of the block 150 using machined in dowel holes to locate the glass bead or cube corner targets that are in relationship to the front surface 154.

Alternative tool targeting schemes are to use painted or retro-reflective dots, made with retro- reflective tape (e.g. Scotchlite 7615), at known locations on the tool face relative to the datum surface. One such dot will give xyz position, and three such dots allow the face to be observed. The tool targets can be retro- reflectors or inserted retro-reflectors, painted dots, or any other datums which can be recognized and measured by the photogrammetric system. Figure 3b

Figure 3b shows a completed simple tool having 3 such NC blocks on risers 161, 162 and 163, the parts 195 and 198 to be mated together, shown in dotted lines placed on the NC blocks, and clamped by clamps attached thereto. It is noted that for more complex tools, such as a door tool, for example, perhaps as many as 25 such blocks might be required in different locations in different directions. To assist in positioning of the material, and to determine where it is for checking purposes or to for other intelligence applications as discussed below, a projected laser spot projector 165 is shown, which allows a datum to be placed on the part 166, which is visible in x, y, z by the cameras. Typically such parts have very few recogniz- able reference points such as holes and slots and the like, and it is contemplated that the invention could use, in addition to the natural reference points of the object, the surface references typically found by the triangulation of the stereo, in this case, of the laser spot projector, or from laser engraved scribed lines, or other specialized optical targets, such as those 167 shown.

It is also noted that we have shown for reference in Figure 3b, the use of the optional cameras to provide a view of the other end of the parts, the scribe lines 167 are seen with the optional cameras, whereas reference hole 170 of the part and surfaces 175 and 176 (which using 3 points or more on each define the intersecting planes which created an edge reference) is seen with the main cameras, also with the aid of the programmable spot projector 165 to create a target where none exists on the bare metal surface. In this way, there is sufficient number of points to establish the positions of both parts, before and after mating, to provide further control of the process, as discussed herein. The spot (or other zone or pattern) projector is programmable by computer 60 to place spots as desired on the surface for mensuration of part location. The part is usually rough located first by use of its natural features, or by tool locations, and then the desired spot locations generated. In addition, the robot placing the part in, the tool can be so guided, by the invention, as can any welding devices used. For example robot 180 with weld gun 181 having four target set 185, is brought in to spot weld part 168 to another part 195. Control computer 60, reads the location of the robot in up to 6 axes, relative to the tool locators, and if desired, the parts to be welded, and corrects the robot location to place the spot weld precisely centered on the area of the mating flanges. Where the parts are indexed down a line, as typically done for the large parts such as side frames of the car, for example, or doors, the cameras may be located off axis, such as 210 and 220, so as to clear any sort of automation of robots coming in from the top or the transfer, etc.

It is noted therefore that not only can the sensor system of the invention be brought in, let us say temporarily, to set up the tool, it can also be slid in, let us say over a line on some sort of a shuttle, or taken around to different individual tools by an operator, or as discussed extensively herein, left in place, since the camera units are inexpensive, such that it can be used to determine many different features of the positioning of the parts, and the robot, both in the space relative to the tool locations. For example, when the part is obviously not in place, one can check and upgrade the tooling locations, just to make sure things haven't moved, or if they are moved by an operator update the data base of where they know are, something that is lost in the factor after a few weeks of operation.

When the part is in place, then the part itself can be sensed and positioned, as can any device either loading or working on it, using suitable reference points thereon. On sheet metal parts, reference points can typically include holes, slots, bumps (or depressions) , edges, surfaces, and even special marks such as laser engraved targets (circles, concentric rings, crosshairs, etc.) and painted on targets. Other parts could have numerous choices including lettering, decorative marks, etc. which can be used as reference points.

Show in figure 3b as well, is the presence of target points on the tool, which can also be constantly referenced by the cameras to assure that the cameras themselves have not been moved relative to the fixed base of the tool. In addition, they can be used to update the calibration of the cameras if there should be thermal drift, or other problems. Tooling, such as locators, clamps, robot end effectors (e.g. weld guns), etc, can best use painted or better, retroreflective precise and distinct targets. Suitable retroreflective material is 3M Scotch light 7615, and discreet reflectors such as glass beads and corner cubes.

The camera systems typically would come pre-calibrated from the factory, or be calibrated in place on the tool base, at the time of tool configuration and build. Other portable calibrating systems are also possible, using automatic theodolites, manual theodolites or scanning laser interferometers as described below. Disclosed in this embodiment is the use of multiple cameras typically, but not necessarily fixed in position to analyze the position of parts or tools, and particularly a variety of tool locations or different parts that may be put into the work cell, generally for mating purposes. Where long slender parts for example are utiliz¬ ed, cameras are utilized to look at opposite ends of a part as no one camera may have the whole part in its field of view. This is in order to maximize the accuracy of any one camera on the position of the part. Similarly where stereo pairs of cameras are utilized as is often the case, the groups of cameras can be selected as to which two are utilized to make the stereo match of a particular grid. Consider for example a grid of x 8 cameras in a generic work cell. Cameras 1 & 2 can be chosen for parts that are small to one side. Other cameras can be brought to bear for longer parts or wider parts, and the results combined. The part location is therefore determined more precisely, and any angular rotation error is minimized, particularly in the moment from one end to another. Figure 3c Figure 3C illustrates a close up of the clamping by pneumatically actuated clamp 226 of one piece of metal 227 relative to another 228 to an NC block230, as typically used (not shown in above for clarity) . Shims, 240 and 241 which generally are to be avoided in such systems, are shown. Typically these are placed in later if need be, to move the metal to allow it to function better than the original design.

Also illustrated in this drawing is an optional force cell 250 to sense clamping force on the metal, or in this case the reaction thereto. Micro-step motors 255 and 260 are also illustrated to move the NC block in x, y (and z if desired, not shown) to essentially replace the shim system. This can be used for "tuning " the operation, discussed below.

The clamp 226 can also have a target point 252 for viewing by the optical system, to allow its correct position to be determined and monitored. Figure 3d Figure 3d illustrates a functional block diagram of the embodiment. Figure 3e

Figure 3e illustrates a variation which does not use an NC block with known surfaces, but instead simply a member with a f$at planar surface 280 or other contour, positioned in 6 of freedom, using a rotary ball joint 281 and the x, y , z cartesian motions previously described. The stereo camera system in this case looks not just at one point for x, y, z, or two points for x, y, z and roll, but 3, or better even 4 points to determine all 6 degrees of freedom of the plate and its location. Where the plate is relatively small, and many surfaces on the sheet metal are d flat anyway, it can act as an "NC block" of a totally variable nature. In order to lock, a mechanical locking mechanism to expand the inside ball out against the outer ball, can be accomplished, under command of electric, or a hydraulic clamping device can also be used to expand the inner to the outer, and therefore secure the ball location. The hydraulics while messy is interesting in that the same hydraulic s can lock the Z axis as well. This relieves the requirement in many cases of having precision machined NC blocks, and is particularly useful for prototype work where such blocks would not be easily obtained. The manual set up embodiment of the invention disclosed can be utilized both for construction of fixture tooling and its changeover.

The invention allows such tooling to be erected in place very rapidly, "tuned" to early sheet metal condition, and to be later reconfigured as needed for example to change body styles overnight, say. Here again the rapid and accurate sensing embodiments of the inventions for determination of component location in 3D space are utilized, preferably using fixed or movable stereo TV cameras with computer readout and added laser pointing devices as appropriate. This has major advantage in cutting the cost and time of getting lines running and changed over. Rapid reconfiguration makes possible economic production of small volumes using more or less conventional part holding and joining.

If tool is in a running line, like a side body line, the invention here can be used to reconfigure a second tool while the first tool is running, shuttling in the second tool to replace the first during line changeover. The Principle advantage of reconfiguring the tool while the other is running is to buy time. You don't have to have the line down while changing over, even though there are, of course, normal times such as third shift, weekends, etc., when the line may be down anyway. This however does allow you to intermix production on a much more rapid basis, for example hour by hour. Figure 4

The disclosed invention describes a method for rapid tool set up, and reconfiguration to new model changes. Upon finishing the set up, a sensor system checks tool component locations and records same in the data base for that tool., and assures that it is correct. Further changes in the tool are indeed updated in the data base. The sensor system then can be further operated for tool location in two modes: a) to take the system away after tool location is set up, and use it to set up another tool, or; b) leave it in place, or at least a camera system in place, for operation during actual production.

This application goes on to describe in the following sections, the in production application, both of measuring tool location and two other important verifica¬ tions - sheet metal position and forces. For in-line operation of the tooling sensor device, the camera units can be able to verify the position of each tool during operation, and any movement of the tool that occurs. However, there is more to in-line measurement and line optimization than this. For example, if one is to change over lines, from one body style to another, or car line to another (i.e. Cadillac to Chevette) , one must first of all put in place the mechanisms to get the tooling changed over to the new model, which is physically moving around the NC blocks, replacing some of the NC block surfaces, etc., and on top of that, try to "home in" and "tune" the line in the quickest possible time for the new model, and for the metal condition of the panels being presented to the line at that time. The latter requirement occurs because the metal from the sheet metal forming processes, due to steel varia¬ tions in both thickness and metallurgical content, and die wear, etc. is never quite the same, and can be degraded from the time one runs the line at one time to another. This situation is also made more difficult because of the need for quick die change, for example, in the stamping process, and "just-in-time", where the body metal made for the cars being produced today, if made on a press line with certain dies, which are then changed to make the body metal for the line running the next day, or in the case of out¬ side suppliers to sell to a completely different customer. With just-in-time inventory deliveries there can be days, or a week or more, between the times of different manu¬ facture (and this relates to the steel variants) , and there can be different dies, and their own locational problems and variations in the press runs. In other words, the press shop itself has to get the new part running in an expeditious manner (another story, and aided greatly by the D-Sight invention of which the inventor is partially responsible) .

To get the product running on lines with fixed or individual assembly cells with more or less conventional types of fixtures, we have in this invention presented a way to quickly re-position the fixtures for the different bodies. But what about getting the line running and keeping it running? Illustrated below are several additional features of the invention which assist in this function.

In the first instance, if a camera system-located overhead, on a lay down body side frame line-has determined the position of the tools, and the clamping devices that are used to hold the metal in place. This is preferably done using target datums on the tools and clamping surfaces. This monitoring if left in place, according to the invention, can be used to continually monitor their locations and ensure that no change is occurring, or if there is a change to record this into the data base, and present trend analysis, and at times for maintenance, on these lines. This presents the major problem that occurs, where things get out of place, clamps get broken, etc., and the line goes on producing scrap, or at best poor quality product.

Also of importance is the position of the metal after welding. Electro-optical sensors of the invention look at certain key positions of the metal, and determine that excessive spring back is not occurring after welding indicative of over forces or other indigenous problems.

With such "in-tool" inspection, the requirement at the end of the line, or off-line, for related inspection fixtures and systems, may be obviated, or at least miti¬ gated - in other words a relatively infrequent co-ordinate measuring machine check of the product should be sufficient to control the line, when used in conjunction with the in-tool sensors here disclosed. The metal can be looked at, either at specific datums already present in the metal, or what might often be the case, specialized datums, such as laser marks etc., put onto the metal specifically for this purpose. It is noted that the stereo cameras overhead looking at the tools, can if suitably equipped for grey level imaging of the panels, also be use for panel location determination as well, assuming the features are of interest. For example in the lay down body side frame system, the outer panel surfaces which are face down, and these may be the ones that are of interest for the measur¬ ing purposes, requiring below looking up sensors. However there are more features on the inner sides of the panels that can be monitored, and if these are sufficient they can seen from overhead.

From the sensor protection point of view, it is always desirable to have the sensors overhead or to the side, but not underneath if at all possible due to the contamination which can fall on them. This can be various oils, wash waters, etc. used in the process, or the odd piece of trapped metal. It is not a major problem, and can be dealt with, as pointed out elsewhere, with shutters, and built-in air blow-offs if desired. These can actually be cued by sensor malfunction indications provided by the system if necessary. The sensor windows are desirably sufficiently close to the camera to render contamination thereon out of focus. Also of interest in this system is the monitoring of forces. Basically in this case the force of clamping, and potentially as well any distortion forces that occur during welding. This essentially provides two methods of looking at the result of the weld operation; namely the distortion physically, with resultant deformation measured optically, and the distortion due to actually measured force value.

Two force sensors as shown; the first measuring the reaction force in the locator due to the clamping force F. This clamping force F is constant if a constant force is resisted by the metal itself, and the resultant force Fm is equal to the reaction force on the locator.

The other type of sensor is on the clamp itself, where a variable force is utilized sufficient to clamp the part down under any and all conditions, but that the inflection point at which the part is deemed to be located is recorded, as the rapid rise in force, and the amount of force is determined at which the inflection occurs . The goal here is to rapidly home in on the right positions of the locators, and the metal to enable satis¬ factory welding. The locators is they are off position, even though in the theoretical CAD position, could be off positioned for the particular situation of the day, and small incremental changes can be made therefore to cause residual forces to be lessen, and the forces of clamping to be optimized. This system is aimed at obtaining this data in the shortest possible fashion.

Finally, we also note the desirability of sensing the location of, and optically force applied by at the welding robots as well. These can have an effect on forces by pulling the metal which can be directly mounted by any force sensing in the system in the clamp forces, or reac¬ tion forces to clamping. The other things robots can do is simply be off location and pull the metal off, or otherwise cause difficulties. This too can be instantly monitored. Figure 4

Figure 4 illustrates further issues surrounding the measurement of tool location, robot location, and part location as a functioning tool in the course of production, as well as illustrates optional methods for holding the part, beyond the use of NC locator type blocks or other fixed members, for a particular part.

Figure 4a illustrates the invention utilized, much the same as Figure 3b above, to weld one part 301 to another 302. This illustrates further the spot weld locations put on to weld the two parts together by the spot weld gun 300 moved by robot 301, having target 305, or 4 target set 306.

It is further noted in this case that the clamps holding the metal down can also be targeted, such as that on clamp 310, which further allows the , since the clamp shoes are of known widths, allows the determination where the top piece of the metal closest to the clamp is located. Dynamically in operation, it can help of determine that the clamp did indeed close all the way, and was not either faulty or pulled open by the robot urging the metal against it.

While the sensor system used to set up the fix¬ ture can be removed, there are several other interesting aspects of leaving the sensing in the fixture. First, is that the panel itself after welding, and after un clamping can be gaged. This, in essence, will allow the degree of spring back in the material to be determined, in its post weld condition. In addition the actual weld points can be sensed, and the camera system can make sure that they in¬ deed are all there. (See also below for control systems for these operations) . It can also, given suitable resolution, determine that they are of a certain diameter or larger, for example, indicative of what could be construed to be a good weld. Naturally, if they are not, some sort of a reject signal could be provided to the automation, or to an operator. In addition to seeing the material after welding, one could also look at the two panels as the first one is placed in, and then as the second one is placed in before clamping to assure that they are in the right location, and they are not too "sprung". Also after the clamps come down the shape can be sensed, and any variations in the shape of the panel that are sensed can be feed to the weld gun to cause it to change its location.

This particular issue has a great deal to do with how the ability to use smaller flange widths, as does the control of the robot. For example, in a conventional tool, the clamps and locators have a certain degree of error in their location, especially after they have been moved by operators in various arbitrary locations that are unknown. In addition, the loading devices of the metal also have a certain degree of error, and there is always the possibility that the metal doesn't seat right next to the second piece, which is being welded, or to the locator block. All of these issues make a variance in the location of the flanges.

In addition, the robot itself as has been noted, also has a variance in its location, which typically is on the order of ±lmm, but can be worse as the robots wear, get out of tune. It is widely known that robots use for carrying heavy spot weld guns, must be returned almost daily if they are to achieve the proper accuracies, and even then it could be argued that one might be uncertain in their location to within ±2mm.

The sum total stack up then of all the potential errors, clearly has to be accounted for by using weld flanges that are large enough to accommodate the mismatch, and still achieve a suitable weld. With the disclosed invention however, these problems are eliminated.

For example, the electro-optical sensor system, such as in Figure 3 or 4 herein, on sensing that the metal is somewhat out of position can feed an appropriate signal to the robot to move to this position, that the robot moves into this position, the sensor unit again senses now the robot, and determines that it is somewhat off in its calibration, and needs to be moved so as to put the weld down. It is considered to that the weld flanges such as those of width W_χ and W₂ of parts 301 and 302 in this manner can be satisfactorily reduced by at least a factor of 2 in their width from today, and perhaps as much by a factor of four, indeed, down to literally the size of the weld spot itself, plus a small clearance dimension. This particular move can increase, since it is known in the trade that the weld flanges vehicle can weigh up to ninety pounds. This has an untold saving in weight, fuel economy, handling characteristics, and other environmental advant- ages. In addition, aesthetically it can make for a better looking car, because the flanges themselves have to hidden or covered over with gaskin material or other artifices to ensure that there otherwise objectionable appearance are not present to the customer. We all have seen cars that have flanges that are have weld spots a 1/4" displaced from each other, along that line, and even that can be seen through the paint and looks unappealing.

The control system of the invention may control not only the welding, but the loading of the parts, if automatically done with one or more additional robots, or other automation devices (which may or may not be them¬ selves controllable as robots are) , and assures that these devices are in the correct position to load the fixture, and without damage. It is of interest to consider what would be the case if we utilize the automated motorized clamping block positions, shown in Figure 3 above, as well as the force sensing capabilities. We clearly can have a system that controls the position of the robots relative to the tool block locations, the clamp locations, and the parts rather. It now can be seen that if we weld the part, and we unclamp, it is then in its free state, and certainly the clamping locations are typically the referenced surfaces for the part, and certain holes or other surfaces on the part are also comprised of the gaging references for this part. However, is the part good? Clearly, by using the control system as previously described in above, it should be a lot better than it would have been. However, some¬ times, especially during the early runs of a model (and this is much more prevalent if one is changing over often) , the sheet metal itself, coming from the presses is variant, in its spring back, in its shape, etc. Often times the metal does indeed distort. In some cases, even so much so that the clamps can hold the two pieces together, or if they do, they hold them loosely. Clamping Control

If we build force sensing into the clamps, the control system of computer 60, can sense the clamping pressures. If the maximum clamp pressure of the device is exceeded by the resistance offered by the material, there will be no clamp pressure sensed by the force sensor under¬ neath the material, and if it goes below a minimum value it can signal problems that need correcting clearly, either due to low pneumatic clamping pressure or damage clamps or severe metal distortion. If the force sensor is located in the clamp, it then senses the reaction of the metal as the clamp closes down. As the clamp is closed, the actual spring back force of the metal can be determined. This can be useful in determining how distorted the metal is from its supposed nice flat condition in the region of the clamps if the part is laying properly. It can also determine if the part is not laying properly, although this also can be determined optically.

If there is a high clamping force required to hold the parts together, it could be that the clamp itself needs to be moved. In this case, the motorized positioners, such as 255in Figure 3, can be used to move the clamp in x,y, or z in the NC block, in order to perhaps better position the device for the particular metal being run that day. The range of positional variation however, needs to be controlled within limits, so that the overall geometry of the part produced is not unduly distorted. It is an object of the invention therefore to also make these micro movements automatically, such that the positions of the part are more optimally arrived at as viewed by the sensor camera, to make the clamping force more optimal, or, and in the final analysis this is the more principle one, to assure that the finished part that comes out of the fixture is of the correct shape once welded.

A degree of intelligence can be added, by putting in the computer program an intelligent mechanism for moving the clamps in a known manner, such that the system homes in, and drives the error to as near zero as possible, given the state of the metal being presented to it. In order, to arrive at a stable location, however, a substantial number of panels have to be run since there is variation in all sets of panels and statistics need to be built up. For this reason, statistical programs are also included in the system which can be analyzed to determine the trends in the metal and make adjustments thereto after, let us say, 10-20 parts have been produced from a given lot, etc. In other words, it is not thought that movement of the NC blocks with the micro positioners, should be done at every part. Having said this however, it still can be done only within limits, if there is a definite seating problem of a part. This would generally however, only be done during some sort of early prototype build or the like, where one would be willing to except a less than potentially optimal result in order simply to get a part to be used to build the vehicle. It is clearly therefore, a function of the invention, to determine the finished part location after welding with the clamps off to sense its status and better regulate the clamp location, the robot gun locations, including even where the spot welds are placed, although this is often determined by the engineering department, and also to determine the clamping pressures required, which are optional in this invention, made variable, etc.

From a diagnostic point of view again, this provides a much higher level of control in this process, controlling now any or all of the following: clamping pressure, robot location, robot gun clamping pressure (not the case in the laser welding situation) , part location, clamp location, tool location, part after welding dimension and location, welding parameters (e.g. laser power, weld current), etc.

It is finally noted that we could also monitor the part surface, the clamps, and robots during welding. This is noted if any sort of untoward, progressive distortion is taking place on the part on the two pieces being welded together to form a single part, so as to potentially change the control path of the robot, or the number of spot welds, for example, adding more spot welds if the distortions need to be corrected more, for example, or a high level of forces are required, which could be deleterious to the welds, etc.

Note that the sensor system can sense the part position during tool loading, and instruct the robot loading the part to nudge it a little if it is not quite fallen into place.

Figure 4b includes a somewhat different embodi¬ ment than the above. In this case, a first part, in this case a long channel section type, 350 is located at one end in a specialized form of NC block 351 on the flanges of the channel. At the other end, of the part, for illustration purposes here, is located in a new form of 2 dimensional, flexible positionable holding device, 355 which holds the part, not rigidly, but in a more or less correct location, simply by having the part placed down into it. Put another way, the part is being held at one end, at points that are fixed by the design of the tool, and at the other end in a way that is generic to many different parts that might be placed within it. The particular part holding device is a "bed of nails "type, which I have found to be uniquely suited, not only because it can take the shape of multiple different types of pieces, but because of the action of the pins 360 against the side walls of some of these pieces, can actually serve to hold it semi-rigidly in position - what I feel is a new development. This is due to the side forces exerted by the pins on the channel or other shaped member (Figure 4c) .

To this channel section, is to Be added, another channel section 370 however, inverted 180 , so that the two welded together form a quasi tubular type of part over a certain length, commonly seen in many portions of the undercarriage or interior of the vehicle.

In this case, again, the clamping portion can have a specific flange contact such as that shown 380, or it can have again, the bed of nails type of face, such as 381, or other deformable surface capable of generic loca¬ tion. It should be noted too, that a variant, is to pin the device at one end, as will be shown below.

When the bed of nails arrangement is utilized, for holding one piece of metal such as 365 relative to another such as 350, it clearly is not in a completely known position, but it is stable. In this case, another device, in this case the weld gun, must exert enough force to force the two together, if they are distorted, or a programmable clamp, such as 370 on robot 390 can be utilized to clamp the metal together, wherever is desired, while it is stabilized, and in this case, an integrated YAG laser welder with a fiber optic beam delivery and motor drive can be used to make the welds. Such a device allows a higher clamping pressure to be used, as well as desirable laser welds. Again however, the key is the control of the robot, because it now becomes the dimensional reference for where these parts are clamped, and where the x,y, z location of the clamping point is. In other words the robot has become a programmable NC block of a sort. This leads, as discussed below, to fixtureless assembly. Also discussed below are other variations for this. It has been found that in trying to seat sheet metal parts on a "bed of nails", that due to the generally inclined shape in channel sections, the pins that are on the incline tend to force against the part, in such a way that it holds it side to side, and even to some in and out (along the longitudinal axis of the pins) . If the pins (nails) can be locked, this means that the holding force may be able to be kept constant, and in fact they can furthermore be urged against the part,to increase the holding force at least to some degree. A relatively universal holding fixture can be thus provided which doesn't precisely locate the part, but once the part is there, keeps it more or less there as it is being assembled or worked on (e.g. laser cut, or other process not applying too much force to overcome the holding effect of the bed) . Another such conformable member would be stiffen¬ able magnetic members such as magnetic fluids, which on excitation of a field becomes stiff. Here again we note that the member itself, be it a magnetic material or whatever, deforms under the weight of the part. As well, the part can be urged against it, by a robot loading the part, to take the basic shape of the part, generally in two axes, but it could be in a linear section. Figure 4d

Figure 4d Illustrates a further manifestation, wherein the mechanical holding of two parts 410 and 411 is accomplished at one end by a pin 400 and surface 401, and at the other end by a lamping robot 420 with optional target 421 viewed by the electro-optical sensor system freely holding the part (or if desired, alternatively resting the part on a surface such as 425 shown (dotted lines) or on the bed of nails or other conformable tool as above. The Robot in this case is schematically illustrated as a truss type with motors such as 422 in the joints to provide a fast, rigid system of small range. Any suitable robot can be used.

Noted is that pre-punched holes, let us say, in both mating parts can actually be aligned by the action of the sensor system guiding the robot to load the parts in sequence over the pin. Because the optical system can guide the positioning of the part for welding, the pin can be a loose fit in the hole, with other slack taken up by robot as it aligns the parts into position. This same approach can be used for other types of assembly such as fastening, where a bolt could be put through oversize holes.

Figure 4D also illustrates the use of motorized positioners to move the tool member, in this case a pin, to a new location, to suit another part. Riser base 430 moves in two axes on a linear motor base 50, having a grid lmm x 1mm capable of programmable positioning to 0.1mm or better, controlled by computer 60. Motor 435 moves the pin base suitcase 401 in z, (on command of computer 60) . Miscellaneous points

The tooling system described above is desirably constructed of standard components which can be used (and re-used) in many factories and therefore need be designed but once.

For use in controlling automatic positioners, such as shown in Figure 4d or in Figure 3b/c, in 3 or more axes, the control system can function as follows. For example the automatic positioner positions the part in what it feels is correct location for that NC block. This then is observed by one or mores aspects of the photogrammetric camera system; and an error signal if any generated, which is then fed to the appropriate manipulators to re-position a block. The new re-positioned block is then looked at again, and any further error determined, and so on the process is reiterated until the resulting positional error in whatever axes are desired is under some set limit. One can also position parts in the same manner, by observing the tool or robot that holds the part and manually, or automatically causing the tool position to be moved to re-position the part in the desired location. Where one looks at the part directly one can also go through the same control sequence. It is generally speaking more difficult to look at the part images (and datum points thereon) for such positions, and the error bands may have to be a bit larger than when actually positioning the tools, such as NC blocks, which can have precise high contrast specialized targets.

The tooling of the invention can, as has been noted above, be erected in place. Because of the precise dimensional set up possible with the camera or laser scanner sensor units overhead, (which can be calibrated using known photogrammetric techniques) it is not necessary to build a fixture tool in a build shop, then tear it down, send it to the plant and re-erect it. This is also made possible because of the relatively simplicity, and standard nature of the tooling.

To reconfigure the tooling for a different body all that is necessary is to simply lay out the new positions of the tools using the guidance system just as one did to build it, and reprogram, verify, and correct if needed, the robotic welding.

Since the sensor unit(s) overhead can be used to check the position of the tools and robots, the risk in repositioning a tool to a new body is eliminated. One can indeed reposition it to within the accuracy of the system right back to where it was in the first place.

In terms of running of a plant with reconfigurable tools of the invention, there are several possibilities. First would be to change completely the car body style being made, let us say from a large car to a small one, over a weekend. To do this economically probably requires that the tooling be able to accommodate different style variants of any particular car line being run e.g. sedans, coupes, and wagons for example. However a large variation could be accomplished on a weekly change¬ over basis. The idea here is to keep the tooling in place at any one time as simple as possible, and not to require enormous hardened fixtures loaded with all kinds of interfering tools such as those used today. Operator Displays

There are numerous displays which can be used to indicate to a manual operator, how he should make a move to position the tool, or the part, in the correct location for assefibly. They range from a bar chart type display of up to 6 of freedom, where the zero offsets of each bar, and the rate of movement of the bars with each of the variables is set by the photogrammetric co-ordinate system computer, such that a particularly easy to obtain situation like all the bars lined up in a row on a fiducial mark is shown in the figure is obtained when the part or tool is in the correct location. Displays with Pseudo or real 3D can be used as well. The block diagram of Figure 3d where the operator looks at the display, determines that, let us say, one variable, such as angle, needs to be repositioned, moves that part or tool in that angle, at the same time influenc¬ ing some of the other variables, and jockeying the part around in position until all variables are known to be within tolerance.

A second type of display is a gun sight type display, in which the part or tool is moved in such a manner as to cause a displayed gun sight reticle to close on a cross hair indicating the desired location. x,y, and roll can be easily displayed, with z shown by size of the reticle (or a separate readout) . Pitch and yaw can be displayed by exaggerated foreshortening of the reticle. Beyond these relatively simple displays, there are more sophisticated displays such as to use the control computer to generate a 3-D CAD, even colour model re¬ presentation of the tool or part position, and where another part is involved ,the mating part position as well. The operator then homes these in on each other, simply by watching the display, and when they get close the shading of the display, which when it goes to a uniform shade, or colour in any one direction indicates that the part is correctly positioned.

It is also possible to provide a 3D stereoscopic display, using two cameras of the measuring system as input. When viewed with LCD shutter glasses as known in the tool or part and the reference points thereon. The position and closing data to position the part or tool can be displayed on the screen as well.

In many cases where machined contoured locator surface ("NC Blocks") can be made, it is not necessary to position the locator surface angularly since a correctly angled surface can be machined into the locator from the knowledge of the data of the part. In this case a 3 cartesian axis locking positioner may be used (X, Y, Z) . Also only a single data point (e.g. retro-reflected) on a tool block is required in this case (rather than 3 or more where angle is desired) .

In the above examples, leaving the cameras in place on the tool has several advantages:

1. You can track any movements in time and determine if corrective^•action is needed, by mastering the system periodically;

2. The errors are not introduced by positioning and re-positioning the cameras. The set up today is exactly the same as it was on the last time this particular job was run, within the air band of the human positioning in the display;

3. One can also use this to troubleshoot and to double check the situation;

4. If the field of view is large enough, the cameras can look at a master body component, for example, not just a tool setting target . in order to assure position. Some, or for that matter all of the positions to which the operator places the locating details in Figure 3 can be optionally automatically positioned by appropriate servo motors if desired. Naturally if this is done completely it really resembles the prior art of Figure 3. However this is very costly. There could be some details however that could be best done automatically, for example those which are required to be repositioned often. One of which would be the change from a coupe to a sedan on any one car as this could almost occur job to job for certain special portions. The other would be those areas which were difficult to get at for the operators. Figure 5a

Figure 5a illustrates an application of the invention to monitoring of tool and robot locations on a conventional assembly line, including one constructed using tools equipped according to the tooling concepts of the invention. In this case an optical position sensing unit, such as the one shown in Figure 3, or others such as automatic scanning theodolites of the referenced fixture construction invention, etc. are used to locate tools in space and compare their surface locations to the data base of the tool, derived from the base of the designed part. In addition, the same system is utilized to determine the weld robot locations relative to the tool. Optionally, the system is further utilized to compare the data taken to that of the gage systems that may be incorporated at the end of any sort of line, or even off-line gaging systems, such as check fixtures or co-ordinate measuring machines used to check the parts that are made on these tools with the robots involved. Indeed, the invention comprises the comprehensive use of checking of tools, robot, parts, and the combination of any or all of that data with that of an external gages in an optional way.

As illustrated in Figure 5a, a particular application is shown, where an optical sensor checking system 510, such as the stereo photogrammetric camera system of the invention is located on one of the AGVs (Automated Guided Vehicles) 482 utilized in an automotive body plant. Such systems are heavily in use in General Motors and Fiat, for example. However, they have an equivalent in other plants, which use conventional indexing conveyors. Because it is on, in this case, a final body line, the sensing system, is comprised of many cameras, for example 20 or 30 pointing to both sides, plus some pointing overhead, in order to see key reference features of a critical body tooling such as "gases" 490 and welding components. The sensor system, located on an AGV carrier like those of the body itself, proceeds to each station on the body line. The first stop in this case, is the framing station, 500, which typically has numerous clamping devices and NC locator blocks, perhaps 30 on each side, and several on the top, that hold the side frames, relative to the underbody, all of which have been "toy-tabbed together" prior to the station. Robots are utilized to come into this station and tack weld the pieces together. This is common in the RoboGate, Geobox, and IBAS systems. Other systems have fixed guns that are physically connected to the clamps to do this job. Whatever the case, the checking system 510, according to the invention, comes into this station, and "looks" using the electro-optical sensing system, at the various tool, robot, and fixed gun locations, and determines their correct locations relative to where they are supposed to be, according to the tool data base. In the first case, it is simply to monitor these positions, and record them into an updated data base, which is acted on at a future time.

In a more adaptive control, closed loop condition, a computer 520 takes the sensed data computes the best positions for the robots such as.530 or any other device such as motorized locators as shown above, that actually can be controlled in this location, and adjust it accordingly. Clearly, however some of the devices, such as fixed clamps and locators, and fixed weld guns, simply cannot be so adjusted automatically, and signals or displays have to be provided for the operator to allow him to come in and do this.

A typical task for an operator, for example, is to make these adjustments, by putting shims behind the NC blocks, or by reprogramming the robot using a teach pendant to fall into the correct location. This is a particularly difficult issue for robots, in that the teaching actually has to be done from a master part, and is not as easily accomplished from simply touching up the absolute co-ordinates. The invention solves all of this problem, by simply reprogramming the robot to come to a particular new location, which can then be reverified by the measurement system here disclosed, and error removed.

After attending to the framing station, the system moves forward just as a normal car body to other positions, typically ten or so, which constitute the

"re-spot" portion of the body line. In these areas other robots come in to make the remaining welds on the body, and each of these robots too has to be updated in its calibration periodically, and its location, due to drift, etc.

Finally, the system comes to the body gaging system, if used. Many body plants have in-line CMMs or vision based systems, provided by either Perceptron or Diffracto ltd., to determine key body positions and report back there any variations from the desired nominal locations. Generally speaking, these nominal locations have been introduced to the system by use of a "silver body", of known dimension.

The calibration system here can either relate the gage data to the data it has taken, or indeed zero itself off the gage to make sure that the gaging, data relates to all the data that it has received. This zeroing capability can be done, either with respect to fixed datums on the gage, or even, more specifically, can be used relative to each of the sensor units on the gage. In this latter aspect, it is related in some sense to US PAT 4,796,200 which is related to a method for setting up sensor units in such a gage.

While described here relative to the body-in-white, this system is clearly usable for the other lines of the car body plant, as well, including that of the underbody, the side frames, the doors, and indeed any other line where the system would be transferred from position to position on some sort of automation mechanism. The same type of measurement' system, possibly located on a portable dolly, can be used to check out normal "stand alone" tooling fixtures, for production of isolated sub-assembly parts. Figure 5b

Figure 5b illustrates a real time control system based on the above sensing systems. This control system is depicted here, is also applicable for current body production , and utilizes the novel sensing aspects of the invention to insure maximum up-time and efficiency. A central computer 535 monitors the PLCs and robot controllers of the framing and respot stations, and the robots weld parameters. In addition one or more of the following are monitored: a) Robot or weld gun position via targeted datums thereon, with feedback from sensors (e.g. cameras) overhead to control the robots and assure proper programmed location has been reached; b) Sub-assembly dimensions measured with a vision gage or ccm at the end of the line or other monitored section of operation; c) Clamp/locator position with cameras; d) Clamp pressure, (optional) . A major advantage of the invention is to allow the maximization of quality and 'up-time' of body and major subassembly production lines, even those of the prior art (such as those depicted in figures 1-3) . In order to maximize such up-time, it is important that all the critical variables that effect body production be known and controlled. Such control can range from simply knowing that a switch is failing to the actual knowledge of the physical positions of various critical components to the knowledge that the weld currents, etc. that are used, or laser power levels (if laser welded) and other factors are correct.

As shown in figure 5b, a central control unit; for example running on an IBM PC, is connected to the programmable logic controllers (PLC's) that control in the clamping units for the, let us say, body-in-white fixture, and obtains therefrom the knowledge that the clamps have made or have retracted, and if desired clamping pressure. Similarly at the other end of the line, the inputs from the existing optical gages were used, or any other gaging systems that may be applicable, such as CMM's and the like are imputed, together with the knowledge of what particular job number is being worked on, so that all variables create to the production of a single job (i.e. one particular car body) can be linked.

These are the basics of the invention that apply to existing lines. However, very importantly is the addition of further datums that result from optical embodiments here depicted in the figures herein included. These are, for example, optical sensors to determine the position of robots as illustrated in figure 3, 4 •- 5, whether they be positioning robots for locating blocks, or welding robots, and where the targets could be on the gun, as weld guns as shown.

In addition, where visible, targets can also be, as it is illustrated in figure 3, on the locators, and observed by cameras, etc. Thus inputs as to where locator positions are, where robots are in space at the time of weld, and as well as all of the other variables of the line, are imputed into the control system, and a critical analysis made between the various positions and the effects on the dimensional integrity of the body, such as determined by the gages.

Since there are a very large number of variables including several hundred, if not thousands of switches and other details on such a line, as well as the various pressures, weld currents, robot positions in as many as 6 axes each, clamp locator blocks as well, this is a great deal of information. For this reason it is also desired to prioritize this information to track only those critical points at least on a real time job to job basis that cause the problem. For example, let us consider that a dimen¬ sional measuring sensor of the final gage begins to pick up, that a particular position on a critical door opening is trending towards the outer limits. As it does this, all those particular variables that effect that area, which could be 3 or 4 NC locators, positions, several robot positions, welding in that area, certain switch, and clamps, etc. all then become tracked, with other less critical or demanding tests put into the background. A signal is sounded in case locators need moving, or the locators are robots automatically moved, if such feedback is capable.

Figure 6 - "Fixtureless" Assembly

The above embodiments have described operation where at least one portion of two panels (individual sheet metal pieces, or finished subassemblies)to be joined are mechanically fixed, together - typically with clamps to surfaces in known locations, but even loosely about a pin. Fig. 6 however, discusses the most flexible case, generic to large numbers of parts, or groups of assembled parts in a structure, where no part specific mechanical location is used. (Although use of a pin can be generic for those parts which could fit on the pin) . In general, sequential clamping of the pieces to be joined is undertaken at one point at a time, not at multiple points as in Fig. 3b.

One of the major advantages of the invention is that it can allow the body to be built economically, but very accurately while utilizing parts that essentially slip fit together, or utilize other types of "tolerant" assembly such as fastening through oversize holes. The reason for this is that one can build an object up from a data base determined by the sensor system. This can be a "global" data base encompassing the whole volume of the structure to be built up, whether it is a sub-assembly as shown above, or the complete car body, as in the final framing lines, or for that matter, another object entirely, such as an airplane wing, bulldozer or what have you. Once the electro-optical sensing systems are in place to effectively cover the volume required and dimensionally interrelate the positions of objects within it, the actual structure being built up doesn't make too much difference. In addition, turning specifically to the sheet metal car body industry, the question of slip fits, and eccentric holes is a very important one. In order to make bodies fit properly, people have in the past, for example, used concepts which require extremely precise hole locations in stampings, as well as overall stamping quality. This presents an extreme load on the supply structure furnishing these parts, and in fact, often times doesn't work anyway. There are just too many problems.

A better idea is to build the whole body in such a way that the panels themselves do not determine, at least in the main, the positions of the final dimensional structure, but rather that this structure is established by the assembly system, and particularly in this case, the optical sensor devices utilized, to instruct the actual positioning of the parts with respect to each other, so that they can be joined at a position that yields a zero overall error, or within some tolerance band, as near zero as possible. One such example is the locating of the two body side assemblies in parallelism to each other at a know cross car distance, established in this case by slip fitting roof rails and side body location on at least one side to the cowl, motor compartment and shelf areas of the underbody.

Consider for example Figure 6, which illustrates a small inner 1/4 panel support member to a large outer panel member. Part 601 is located on generic support rests, 605, 606, 607 and 608 (not shown) and its position sensed in a reference coordinate system by the electro- optical measuring system 610, in this case located overhead. The part 601 could also be located on magnetically attracting mounts, or on a V rest, or another other location means that would desirably be as generic as possible for this application. An alternative electro -Magnetic Vee 630 is shown in dotted lines.

It is noted that rather than rest the part on such a locator resting on a base, a robot 631 (dotted lines) could have located the part holding it in a generic arbitrary fashion with a similar magnetic or vacuum cup. Where a robot 631 is used as the holding means for a part, it may be desirable to hydraulically lock the joints or other wise stiffen the robot arm. Note that 631 can be used to load the part on a mounting plate and o/b, once welded.

Reinforcement 602 is to be attached to outer part 601. In order to attach it, the 602 is located relative to the 601 by robot device 610, whose positions are instructed from a knowledge of both the position of the part 601, and the second part 602, as determined by the electro-optical measuring system. Generally speaking, one part, such as 601, can be located at rest, while the other one is moved in juxtaposition to it. Alternatively however, the part 601 can also be mounted on a moveable device such as 631 or, for example, the table on which the mounting 605-608 are placed could be located on, for example, a "Stewart" or other joint driven truss type table, movable in up to 6 axes , hich yields a high stiffness, rapid response movement over limited range, relative to, let us say, a fixed robot that would simply hold the second part. In either case, one part is moved relative to the other, in order to get them in correct juxtaposition. If the two parts are close in local shape a spot weld gun, such as 620 held by robot 621, can be used to simply come in and weld the two together, with the clamping action of the spot weld gun on the flanges 623 and 624, providing the necessary forces to bring the two parts in contact.

Problems can occur however, if further clamping is required that is beyond the capability of the spot weld gun. Such clamps have been shown relative to Figure 3 above, for example. In this case, a programmable clamping device may be required to sequentially force the parts together at locations near the instant weld location desired. While fixed clamps could be used, this would destroy the generic nature of this assembly cell. Where programmably positionable clamps, such as that located on robot 420 in Fig. 4d is utilized, it is desired to clamp the metal together close to the point at which the weld is undertaken. The weld then being, poten¬ tially put on with the laser at least in those areas near the point of clamping, or with the spot weld gun having a built in clamping action.

One of the major advantages of this system as presently described is as one deforms the metal to fit, basically, using the action of the clamp, the parts tend to distort, and can change in their position. However, interactively, the clamping force can be applied, and the motion of one part relative to the other moved so as to minimize the error in position when the final clamping force is applied. This would be done through an iterative manner, applying force, seeing what happens with the sensing unit, then relaxing the force, moving the part in a known manner, to account for say, reclamping, and iterating the error down until such time as the final after clamp position is correct. This could be done very rapidly if necessary.

In this way, and very much unlike all other known assembly cells today, one not only has a generic work cell, but one has one that can achieve the lowest possible of error of parts, even slipped fitted together, in the presence of clamping and distortion forces.

As the weld begin to be applied, such distortions can also come in, and the modifications while smaller in size can also be made.

It should be noted too, that the way in which the part is welded is also of interest. One would first clamp one end, let us say, .weld there, and then typically would move to the other end, and perhaps the opposite side to establish that point. In other words, trying to basically tack down the key location points to establish the overall reference, such that further welds could be applied without greatly changing the overall locations, and perhaps without using the iterative clamping and adjusting technique.

At this point the part is correctly positioned but not necessarily in contact at all points desired. If desired, the electro-optical system as well as the optional force monitoring system can determine that the point to be first welded is in contact, or near contact. This is to be done by varying the position of the end effector 440 which, for example, can be magnetically energized, slightly so as to cause the part 602 to tilt toward that particular point 'A', and as well to observe it with the optical camera systems as shown. When it is, in the approximate position the second robot, 621 is directed to come in and put a spot weld (laser weld, etc. or whatever) on to weld these two parts together. If the spot weld gun is used, this gun is capable of clamping the part at this location which further brings them together. The same holds true if an additional programmable clamping robot is used.

c) something else that the part will actually go back down and be welded in the proper place at the proper time.

There is a teach mode to this, where we purposely push down,and cause the robot to make a weld of a good part in the right area, and learn from the resultant position of the parts reacting to this action, what to accept for future weld activity. The taught situation becomes the norm from which a best fit is made where required to use the parts. Possible Actions

1. Push on one side. Account for adjacent panel distortion;

2. Push on one side of part to contact a second to weld. Account for"pop up" of other side due to non-perfect match, and chose weld location or move panel to minimize;

3. Spot weld one end of panel, then go to opposite end and spot it, etc.;

4. In system with multiple components, adjust for misalignments by taking it partially out of each angle or dimension, rather than in any one point;

5. Hole fits - If hole location in a part is out of position, then we can: a) add a little error to mating part weld locations in order to get the hole location as good as we can, or; b) enlarge hole (to fit a bolt say) with laser which might be in the cell anyway for welding.

6. Assures all parts go together to make an acceptable final assembly with minimum overall error and as near zero error as possible in critical areas according to preset or learned rules.

Use of the invention to eliminate flanges

It is desired to have a single assembly being fastened together, composed of inner and outer parts. In car body manufacture these are typically flanged parts, with the two flanges used for spot welding, as illustrated in Fig. 4 above. The part then has a stack up of error of each of the parts, in let us say, the height direction which is also accentuated by angular differences in the flanges. It is desirable for weight/cost saving and error reduction to eliminate weld flanges. The error can be eliminated if slip fit parts are utilized. However, from what reference points can the slip fit be positioned? Today, this is virtually impossible to do in-plant, because there are no physical references. In this case however, the references are optical marks on the part and/or projected points looked at with the cameras as shown.

But how to make the weld? Fig. 6b

For one sided positioning and welding, using a laser, one might wish to use an arrangement shown in Figure 6b. In this case, the 'nose' 670 on the laser robot weld head 671 pushes the part 701 (held fixed, as in Fig. 3, or by one or more robots 700) against the other part 720, with an additional robot 675, being used to assist in this effort, as shown in this case where the back side is reachable, pushing from the back side in a co-ordinated motion.

The laser beam is scanned by a motor driven unit, which scans a laser beam through slot 674 across the weld area in the plane out of the paper.

In order to force part 701 against part 720 at the point at which welding is to take place, some degree of deflection of part 720 may be experienced. In this case the optical sub-systems may serve to monitor this deflec- tion, and assure that not only that it is within known elastic limits, and that any deformation is accountable in the dimensional position, but also that after welding and forces are relaxed by the robots that it springs back to a predicted or satisfactory location. For example, 701 is deformed in the 'x' direction by the action of robot 695 carrying the laser weld head until the force sensing device 697 sees a rise in force due to contact with the part surface 720. At this point the weld is made, and force released.

It is noted that the laser shown can also be used to cut out slots (or tabs or holes) in parts in precisely known places, as all points are determined with the optical systems. These slots may be used to establish a precise reference for a further part to be joined in a subsequent operation. The invention indeed is ideally suited to positioning in space multiple parts in juxtaposition to each other, many or all of said parts having been manually tabbed together or otherwise temporarily joined before reaching the station of the invention. Error reduction with multiple parts (>2)

Now let us consider the problem of building errors to zero with numerous parts in 3 dimensional space, that may themselves include error in shape, hole to hole location, etc. In this case as we begin to build the body or subassembly up from individual pieces, we wish to predict the final location of all key functional reference points after welding, and reduce or eliminate error by moving all parts in juxtaposition to each other until the optimum is reached. This we can do, because we have created an absolute reference system for the whole work are, be it for a door, or body side, or even a whole body, to allow optimum positioning of the pieces to be made.

The optimization program processed by the control computer of the system, can for example follow simple rules, namely that all components re to be positioned such no critical reference point, such as a chassis attachment hole on the underbody be out of position to another criti¬ cal point to which it is functionally related, by more than a given tolerance. Less important are surfaces that create to visual fit perceptions, and least important dimensionally would be those points to be welded which only serve to connect the structure. Such approaches work if the amount of distortion required is relatively low. Figure 6c Optimal fit of Body in white major subassemblies

Figure 6c illustrates the fitting of the major sub-assemblies of the body together in the framing station as presently constructed. This assumes that the major sub-assemblies are brought together at this point, and that the car is not built up piece by piece, as is also possible with the invention.

As shown, the underbody 750, containing the motor compartment and cowl 751, and rear shelf 752 is positioned in the station, tied on a shuttle, A.G.V., or what have you. The right body side 753 is brought in and placed in locating surfaces 755 and 756, and welded to the shelf, and motor compartment areas and underbody rails with the position of this part vertical and parallel to the side of the underbody, using vision systems overhead, 770 particularly those located at each end 770 and 761 as shown, and in the area of the welding. However, in one form it is totally arbitrary where these go, as long as they are within tolerance, as be seen by sensor system 770, overhead and to the side.

For addition of the second side by, the goal is to be parallel to the first, such that the openings for hood and deck form a square rather than parallelogram. The left sidebody 775 is then placed in position, and in one manifestation of the invention, the locating points for this sidebody are laser cut in the underbody to suit tabs on the pre-measured side body. In another instance, the sidebody is simply slip fitted, as is shown here, using flanges that go along the underbody cowl and shelf, that are simply adjusted such that the tilt of the sidebody measured using sensor system 771, as well as the fore aft position, and in-out location are such that it is parallel to the right side, and the correct fore and aft location, as well as side to side cross car location. When all of this is correct, a welding robot (not shown) welds the flanges together, and other welding as needed to fit the side.

pixels extent, and thus would not be desirable by themselves to the accuracies required.

To alleviate this situation two approaches are used. First the camera density can be increased in areas where such datums exist. This can be done by adding cameras and/or by moving a camera or a 'nest' of cameras to the required location, itself accurately known, so as to view the part datum. Where many cameras are used in a cell, it is noted that only a subset of datums needed for the task at hand need be processed in real time (or at the speed needed) .

Second, multiple cameras can be used to view a single part at different locations simultaneously, and create a best fix locations scenario to extract the necessary accuracy.

Additional datums (e.g. laser lines) can also be projected as needed to provide a sufficient number for extraction. Such datums can be projected to create a large 'bank' of applicable data. This also serves to minimize effect of error in any one datum.

It is noted that while one camera from the top and side is sufficient to obtain a 3 dimensional position of the part, two cameras give range data from both direc- tions due to the stereo effect and are preferred. In operation, the cameras feed data to the robots which then position the front body hinge pillar in the correct up/down/fore/aft and side to side location along its length, in other words, in all 6 degrees of freedom. In order to position the part in correct position, the robot may or may not actually have to work against the forces of previously welded on parts to the part in question which may have sprung it out of position slightly due to variances in the part. The amount of force to mate the parts is detected by force sensors in the robot and utilized potentially to modify the actual location to a new known location that takes some of these problems into account. In this case the errors caused by not being in the exact, correct location due to distortion difficulties would be made up for a little bit at each location along the part so that the total overall error was minimized. There are many other error modification schemes that will be discussed below.

Accuracy improvement with best fit model using numerous part location datums

Clearly, the more field of view of given accuracy in locating a point on the surface of the part or tool locator, the less complex and more generic, the sensing system becomes. However, as pointed out above, the accuracy of, let us say, one part in 5,000 to 1 part in 15,000 of the field of view is quite hard to obtain today, given the present restraints on camera size, image members, and the like.

However, the overall goal is positioning accuracy of the part, as a rigid body, or near rigid body. In this case therefore, it is not necessarily only the dimension at any one reference feature that is the issue, but it is, in fact, the ensemble of all sensed points on the part, when fit together, that makes the location. Because one can fit through least square methods, and others, a nominal part surface to an ensemble of points, each one which may have certain areas, the summed error of the fitted surface location can be substantially less than the error at any one point.

For example, if reference point 1 is shifted to the left by an apparent .1mm due to the error in the camera reading, a similar point at the other end of the part, viewed by another camera for example, generally speaking would not be so shifted. The sum total of all errors then, in locating the part surface when viewed as 7 points shown in the diagram would tend toward the nominal location, at least statistically.

The same holds true in the y and z dimension as well. It should be noted that the angular resolution of the part is very much reduced if one uses points that are just evenly spaces, as in opposite ends of the part. It should be noted that one only needs to measure the part at those ends, or other locations, where critical reference points are needed. It is not necessary, for example, to have cameras everywhere, if the part is falling within the cell, only reference points in certain areas.

It is also noted by using, as well, it is also in the invention to only use a portion of the camera field, where the particular part feature is thought to be located. This is particularly easy to do in the tool case, where one is actually teaching the thing, where it is. In this case, a window can be put around the particular area, and only processing done within this window. This speeds up any particular processing by at least a factor equal to the reduction in the area of the camera system so utilized, and in many more complex calculations can be even more so. (Was this included above?)

Finally, it should be noted that one might wish to take a great deal more points, than would normally be required. For example, we have mentioned using 7 above to give us a statistical basis. But one could always have outliers, and therefore the larger the number of readings, the closer the fall as a statistical mean. In short, one might wish to measure 100 points, many of which would not be really accurate reference points with respect to the part function, but might be discernable as features which in the sum, describe the part. This would be particularly easier to do, if we put datums on these parts, that had just for this purpose, to increase the accuracy for exam¬ ple, as in the use of laser engraved systems in a trim operation in the stamping plant for example. Figure 7 - calibration

Figure 7 in order to function, the optical sensor system may require substantial part location accuracies to be derived from the camera video, or other sensor data. When using fixed cameras as shown, there are two basic approaches which may be required to obtain accuracies required.

- Calibrate the sensor, or the complete multi sensor system, at the sensor build source, over the volume in which it is operate in the factory, and/or

- Calibrate the system in situ, in the factory. This may also be necessary in any case to periodically verify function, and in case of damage.

FIGURE 7a illustrates the calibration of the sensor or sensor system itself

Figure 7a illustrates the calibration of the sensor system itself. This is being down to avoid the requirement for in situ calibration, except for calibration update purposes. In this case, a co-ordinate measuring machine 801 is employed, having the volume equal to that of the camera systems desired, (which could be either a simple group of two cameras as shown in 805 & 806 fixed together in a housing 807, or could actually be a complete ensemble of cameras, such as the four shown in Figure 4, or even as many as 20 or 30, used for example to look at the whole side of the car body, as shown, for example, in Figure 5a) . In any case, the whole ensemble is fixed on a plate, whether it be 2 or 20, or other rigid mounting, and is used to look at a target point 802 moved around by the CMM to known locations under control of calibration computer 820.

Since the CMM, over a volume, at least, of 500x400x200mm, is accurate to within .03, this is perfectly adequate to generate a calibration matrix for this camera system. It should be noted that a computer control system drives the CMM to a pre-programmed set of calibration points, probably at least on 25mm centers, through the complete volume. This would yield a complete volume. This would yield about 20x16x10, or 3200 points, which can easily be stored today in a matrix in computer 820, outputted to the control computer of the sensor (e.g. 60 in fig 3 above), generally by tape or disc. ^'The readings then in use that fall between these points would be interpolated between them in 3 dimension space. I have found that pre-calibrating the system, including the lens, light source, etc., is generally speaking more accurate than trying to generate an in situ calibration device, using the in situ conditions that exist.

However, nonetheless the conditions of this pre-calibrated sensors must be maintained.

Figure 7b illustrates the calibration of the sensor in situ. For this purpose, the first application is to show the use of a tracking laser interferometer 870, which can be used to track a retroreflector 871 which is also seen target by the sensor cameras 872 to set them up, and to create a similar calibration table. This is the easiest thing to do in the factory, and indeed, one can actually use the calibration point 880 to lay right on the tool surfaces that are being used, offset by a giving amount.

It should be noted, that the operator in doing this, is performing a dual function. He is one, checking where the locators are, as a double check on any previous work, and secondly he is calibrating the camera or other sensor system. Actually one could use the tracking laser interferometer to set up the tool in the first place, although it could not be left in place to bring the other advantages of this invention.

To set up the tool, the operator would simply put the retroreflector that is being tracked by the tracking interferometer, such as the Leica Co. SMART 310 on an NC block, for example, in a position known with respect to the block surface, and move the block around in space, accord¬ ing to the invention, until it was in the correct position. As the tracking interferometer can only measure in x,y, and z, the block surfaces would have to be cut precisely with these directions. The Smart 310 is a high speed pointing angle device with a laser interferometer ranging unit. This interferometer gives extremely high displacement measurement precision in the radial direction, but must be tracked in angle from a known home position, as is commonly known in the art.

The basic method for utilizing this system in this application is to track a target retroreflector from a "Home" point "A" to each of the sequential locator points such as 910-915 on the fixture to check their location. However, in this invention, we wish not just to check it's location but to physically move the retroreflector such as 871 until the operator display 936 shows that it is in the correct location. For this reason the laser retroreflector is positioned on the locating surface 939 of the tool in a special mount 940; that is a known distance from the retroreflector surface 942. This mount then is moved around in space, tracked by the system, until it is in the correct 3 dimensional location. At that point the retroreflector is taken out and the correct NC block surface with this position on it is placed thereon. To check it, the retroreflector contact point can be touched to different parts of the NC block if desired, to check that the correct block has been placed in and is located correctly. However, this is seldom necessary if care is taken in selection and precision of manufacture of the blocks. The operator then takes the retroreflector and goes to the next, and so forth until all of the points are set up.

When changeover comes, to change over to a different body, the process is repeated with the operator sequentially placing them in the new position and selecting new blocks as required to fit the body (or sub-assembly thereof) now to be produced.

This same procedure can also be accomplished using a tracking automatic theodolite system, such as a Wild ATM. In this case too, the tracking theodolite system gives the position of the block, or the target correction device.

Another means to calibrate the system by using specially targeted master parts, which typically either could be made of special materials as masters, or, this is often the case, simply made of regular stamped parts, which have been checked on a CMM, called a 'silver' part in the trade. However, unique to this invention, is the use of specialized target points, not just the regular surfaces in the material, which can be seen easily the camera system. These points might be quite different, but perhaps nearby to those points normally viewed, let us say, such as 158 and 159 on part 165 n Figure 3b, and/or could be indeed, target points such as those used in the tool. For example they could be retroreflective tape dots put onto the master part in exactly those locations where the tool was, for example, the tool locator surfaces, except that instead of being on a tool locator surface they would be on the opposite side of the body metal, on the other side of the surface. Whatever offset that this represented would have to be figured in, when one wishes to check where the tool location was. However, since the body metal has it on thickness, and it was pre-presumed to rest in the tool, this part could be used to check the sensors that control the tool.

This is not that risky for two reasons. One, the cameras are digital and are generally able to interpolate across the small regions of their field without significant error, and two, in some ways the use of a targeted part, known to be good and of certain dimensions may be a better check of the tool as than to measure the tool itself, as the tool positions may not accurately reflect the finished part positions. However, in reality it really doesn't make much difference, because essentially we are taking known datums wherever they are, and calibrating the sensor system. The part, by definition, for that particular tool, has the target datums in the right places, since they are the places near where the tool locations. Use of the 'silver' parts however, for calibra¬ tion of a generic parts assembly work cell, such as a more esoteric persons in Figure 6, however can create substan- tial problems in that we are actually truly trying to cali¬ brate the sensors over the whole volume, and operable over this volume, for at least the most generic cases. Note that for normal purposes however, where we are simply changing over to do one part today, another part tomorrow let's say, or what have you. For any given day, the calibration really only has to be around those areas where the parts of today fall. It's only when one intermixes parts through the whole volume that one has the problem of calibration over the whole volume at once, that this is an important point, and simplifies substantially the calibra¬ tion issue.

The use of targeted body metal (a 'silver body' of sorts) , whether it is the whole car or sub-assembly, or whatever is of interest in the fixture or assembly opera¬ tion in question, of known dimensional relationship. (For example, checked with a CMM.) This metal is then viewed by the target control system, and used to rezero and recalibrates the cameras at that location. Factory control can be further assisted if one puts through, in the first run of panels through the assem¬ bly system, specially targeted panels which can be loaded onto the tools and observed from the sensors units to determine their dimensional locations. This then provides a double check on the tool positions as well as assures that the panels are indeed where they are suppose to be as a function of other operations such as loading, robotic spot weld, clamping, or the like. Indeed this ability to self check, in particular where targeted panels having known high contrast datum points are utilized is a major feature of the invention.

Also can position the calibration parts so they may be monitored before, during, and after welding, and robotic positioning, to create an ability to teach the system, and to provide for diagnosis of system function. This can be done with normal production parts to verify function of the system and correct conditions that are trending off(eg robot locations), or metal conditions requiring some positional change, clamping pressure change or the like.

For periodic checking of a tool or work cell according to the invention, target datums on the tool surface, or the work area of the cell of Figure 3 - 6 above, or in the work area surrounding either the tool or the work cells, such as on the sides or overhead, which acts as calibration datums, or at lest check out calibra- tion datums for the camera units utilized.

There are numerous places in which this can be used, for example, a camera system 61/62 shown in Figure 3b, points in the correct position to look directly at the fixture base, in which the target plate 945 has been located, which acts as the master for that tool. The target plate may have multiple target datums, different ones for which are used for different car bodies, and may have datums at different known heights as well to better calibrate over the volume. Alternatively or additionally, the target datums may be located across the work cell on some reference plate, or on a reference plate, or other geometric member that is brought into the work cell for the purpose of set up and mastering, as described above. Targeted body metal (ie with optically visible reference points in known locations) can be used to check out any individual station's operations. It is preferred (but not necessary) to utilize the retroreflective targets illuminated by suitable light source(s) which can be very accurately denoted in their positional locations. The system overhead can check not only the position of the parts as loaded and clamped if they are clamped but on top of that can be utilized to check the position after welding as well and feed this data back to the central system, where optimization programs can calculate the new improved locations and tell the operator (or robot) where to move them. The data taken from the inspection process is used to feedback as shown through the control sequence to the alternate positions at which the operator is then instructed to reposition the tool. Figure 8

Shown in figure 8a, is the use of a different form of set up procedure for fixture tools; that is to locate the locator blocks and any associated clamps or the like that go along with it, using optical datums on a master object (noting that heretofore I have disclosed positioning generated from datums being located on the tool and being looked at from overhead with either fixed or moveable camera systems, or other optical ranging type devices) . In this case, we have shown a somewhat reversed situation, in which a miniature TV camera 980 is located in a known relationship, to the tool locator surface 979 itself (e.g.clamped in a miniature V block 946) . The camera can be left in this position or removed and put on another locator to set it up, in any case it is fixed in position by action of the Vs on its cylindrical body relative to center line, and its axial location, due to stop 948.

As shown, the operator looks at the visual dis- play such as 914, which in this case also gives a direct TV (grey level) image as seen by the camera, as well as any target co-ordinates of the object which in this case is targets, such as 947 and 948, or other features on a master component 950 (measured "silver body") or alternative target master plate 960 dotted lines brought in for set up purpose) . This target plate can be a flat plate with a plurality of targets observable by a plurality of tool cameras or a known positionable plate using robotic positioners, or any known targeted geometric body. The operator aims the tool and camera combina¬ tion, after having loosened up on any previous fixing of the tool, until the target datums in question are in the field of view. Then using instructions on the TV display, which have been discussed relative to Figure 4, such as bar graphs, gun sights, or 3-D model displays, he proceeds to move the tool until it is in the correct juxtaposition with respect to the target datum master. When the computer indicates through the medium of the display or otherwise that it is, he then locks it down, and proceeds to the next item.

While the cameras can be removed, the cost of cameras in relation to the total operation is relatively low today (under $1,000 in quantities), and will likely be even less in the future.

Using specially marked target datums on the 'silver' body which can be laid out on CMM's, etc, and put on with retro-reflective material or otherwise, the camera systems can be used to look essentially past the tool at the body, and set the tool essentially from the perfectly measured body condition.

Leaving the cameras in place on the tool has several advantages:

1. You can track any movements in time and determine if corrective action is needed, by mastering the system periodically;

3. One can also use this to troubleshoot and to double check the situation;

4. If the field of view is large enough, the cameras can look at a master body component, for example, not just a tool setting target in order to assure position. There are numerous places in which this can be used, for example, a camera 961 shown in Figure 8b, points in the correct position to look directly at the floor, in which the target plate 265 has been located, which acts as the master for that tool. The target plate may have multi¬ ple target datums, different ones for which are used for different car bodies, but any group of three or four known targets can provide as previously disclosed a single camera photogrammetric solution in X,Y,Z roll, pitch, and yaw of the camera to the position of the tool.

Alternatively or additionally, the target datums may be located across the work cell on some reference plate, or on a reference plate, or other geometric member that is brought into the work cell for the purpose of set up and mastering, as described relative to fig A-C above. Simulation

To assist development of the process, the above procedures can be computer simulated, to include the opera¬ tion of the system given the CAD data of the parts and assemblies presented to it. In addition, one can simulate aspects of the sensing system in various conditions of the assembly process, including accuracies of photogrammetric positioning. One can also simulate robotic operations given in the presence of panel deformations, to include prohibitive force loads on robots and other panel supports, clamps, etc. One can also predict whether parts will be in contact for laser welding, and alter in the simulation various parameters to attain the desired results. The same holds true for accurate results in general, and to achieve the speed of assembly desired. Other Points

In the embodiment of figure 4, any or all positioning axes and functions can be either manual or motorized. The hydraulically (or pneumatically or electri¬ cally) lockable/positionable joints (balls, sliding tubes etc.) are important for function this and other applica¬ tions of this invention. The clamps/locators so positioned can also be targeted to aid determination of their position. These locking joints can utilize electric brakes, or hydraulically or pneumatically actuated brakes, for example. These are momentarily activated on control of the control computer to stabilize the position for welding during which time the normal robot servo axis control is turned off. Locking joints can also be manually positioned as described relative to the rapidly convertible tooling of figure 4 etc.

It is also recognized that the accurately reposi- tionable tooling of Figure 4 can be located on a moving member such as an AGV, which would allow it to be recon¬ figured for a different part in a different location then where the part was actually assembled.

Another feature of the invention is the ability of the invention to essentially inspect its own work. This in essence results because the position of the pieces are known, but before and after the joining operation is completed. This means then that the results of the joining operation, and particularly the body shape at desired locations is automatically known through this method. Not only can statistical data be taken, which relates to the body, but it can be automatically used to correct; not only future bodies but even later portions of the assembly of the same body. This is unheard of with conventional measuring technology, which for a weld system can only hope to measure after the fact, and many times after considerable amount of time at that.

The ability of the invention to assemble numerous smaller stamped parts into an accurate, larger assembly, is important for reducing costs of manufacture (as low cost press line can be used) , and in actually improving quality as over all error can be reduced or eliminated by using the addition of each additional part to drive overall error to zero (especially if parts are pre-measured, and intel- ligently nested together) .

Feature lines with multiple points on them are useful for establishment of critical dimensions. A line in a horizontal direction would be great insofar as determin¬ ing the up/down; a line in a vertical direction the fore/aft. If we have another line at the top of the member then we can see the side to side, also with multiple points on the line. This is not necessarily a light section triangulation issue.

The advantage of targeting the robot (s) in the work area to determine their position and correct them, along with determining the part location using the vision system is important, (see also my patent on Robot

Calibration) . So is the time tagging of data to allow one to associate in the Robot controller, the relationship on an instantaneous basis of the Forces, Part locations Robot axes locations and other variables, in order to better understand and develop the joining process, and assure quality (by for example, recording the signature of good parts, and comparing the relationships to the instant measured variables) .

It is noted that the system of figure 6 etc allows one to accurately build up a total assembly (e.g. a side frame) out of small parts which may be of mediocre accuracy, this is because the error can be reduced to zero as the parts are "added up" in the joining process, as long as their design will allow a sliding or similar reposition- able fit which can be varied to suit. This is the opposite of normal practice which tries to have as little variation as possible. Small parts of modest dimensional quality are the cheapest to stamp.

The ability of the system to inspect its own work allows one to eliminate check fixtures, another major body shop expense, and a notorious accuracy problem.In addition, one can use the system to measure the parts and calibrate the press or earlier assembly operations on an automatic, on going basis. It should be noted that one can in designing the part on CAD, one can call in the machine vision program to analyze the features on the part (holes, slots, edges, etc) and determine from known rules about these features, and the various photogrammetric equations of the hopefully standard workcell camera layout, whether the part can be accurately located. If it can't, then the designer can put in other features under prompting from the CAD to allow a satisfactory solution. Only as a last resort, (ie if added part features are economically unfeasible) would one like to change the cell (eg by adding cameras) , which would damage its generic nature. The fully flexible version of the invention also has a substantial benefit where laser welding and other non-contact joining processes are used. This is because the process adaptively brings the parts together in a way where everything is sensed. That is, where many if not all of the pertinent variables are sensed such as location of each of the parts — the robotic or other manipulators, even humans used to position the part — one can therefore ensure that the parts are indeed in contact which is virtually required for successful laser welding or bonding. This is not the case with a conventional fixture, where one does not know at all what the exact situation is. Conventionally, one assumes that because clamps have clamp¬ ed a piece at various locations, that all locations, including the intermediate locations are indeed in contact. But this is often not the case if the material or the parts are out of specification, bent or what have you. Even at the clamp points, clamp failure can occur. Conventional spot weld guns obviate some of this problem by locally clamping using the gun, but this cannot be easily done with the otherwise very desirable laser weld technique.

The invention provides a common system which can be used all the way from prototypes to higher volume production with common tools, sensing, intelligence, and learning through the process, and to provide a unified system for assembly of sub assemblies, the complete body, and even the components to be added to it, such as doors, tail lights, wheels, batteries and the like. Datums on parts can serve to allow parts to be assembled to one another with local reference points, or with global points. Some Unigue features

Finally, and to reiterate, the invention dis- closed has several features simply not obtainable in today's conventional body-building technology:

1. The ability of each individual operation to build on the dimensional data of the previous operations by feed forward of such data; 2. The ability of the system to "learn" from both past bodies produced for example from an earlier pilot stage and to correct the individual opera¬ tions as a result of learned data. This is particularly of interest where the components (e.g. sheet metal) provided are less than correct and one must make for adjustments for particular runs of components that are deviant in their dimensions. Conventional tooling can't do this at all. People have to come over and try to catch the problem and do something with it manually - a difficult task in a "hard" fixture tool;

3. The system of the invention can inspect its own work and can continually upgrade its own data for the learning process, as well as feedback information to suppliers.

4. Some of the embodiments disclosed are totally reprogrammable;

5. All data points may be known for each part and assembly. Because the system can inspect its own work and learn from it, and previous assemblies, it can also be used to optimally produce a "best fit model" for how the car should be put together and for any dimensional changes that need to made to the pieces in order to enable a better fit (or to provide a customer acceptable fit with less than perfect pieces -a usual start-up problem) . Because it takes data on every piece as the assembly is built up, all data needed is avail¬ able. This is simply unobtainable on any form with today's technology; 6. The system is built on dimensional data which can be interpreted directly through a CAD system without going through the intermediate stage of building a tool from a clay or other mock up body which has been converted to the tool co-ordinates. This saves tolerance stack-up and potential dimensional error from creeping in. Today for example, the body CAD data are used to create a tool design, whose manufacturer creates errors which then propagate back to the body. So the final body produced has had two steps for error propagation - the tool and the body manufacture.

The optical sensor adaptive control system described is usable for both totally programmable robotic part positioning as well as for positioning conventional tooling such as NC blocks and locators used in high volume assembly. Of importance is that variations of each, in¬ corporating some fixed rapidly repositionable details, and some programmably repositionable details are possible with the invention.

A note on Non Precision Holding Devices

The invention allows the use of vacuum suction cups, magnets, and other non precision holding devices to be used to position parts, since actual position can be determined adaptively by the optical sensor system. This allows body panels to be held if desired from an outside surface (e.g. the outside of a door), which is generally not a reference surface.

Also possible is contoured holding devices, such as precontoured blocks, types such as shown in the Tamura patent or "self contouring" types such as nested pins and other two dimensional holding devices such as shown herein In some cases a two stage determination of part location in space is required wherein the camera overhead or to the side determines robot end effector position hold¬ ing the part (rather than the part itself directly) and a second sensor on the robot, generally a camera, or structured light sensor determines part feature location relative to the end effector.

To position a robot, signals are fed from the camera or other sensor system preferably at a rate of 60HZ or better. Once the parts are determined to be in the correct juxtaposition, there is a necessity to hold them there while the laser weld or joining process takes place, which can be a few seconds. To assist this away programmable stabilizing member such as a flat surface or in some cases a pin, can be used to act as a steady rest to back up a programmable urging together of two panels say.

Where reference datums on the parts are used, it is possible to grab the part in any suitable manner, as position can be determined by observation of the part, and feedback to the robot control. Where occlusion of the part occurs in the process, the part, and the robot gripper or other reference location can first be sighted in a non occluded state and the part to robot position determined. Then the sighting can lock on the robot, making a suitable co-ordinated transformation as desired to determine part location at the occluded position.

For parts which cannot be suitably targeted or whose natural features don't provide reliable reference points, the robot tooling can grip the part on known loca- tions (e.g. holes, surfaces) which then can be used as a new reference. Sighting of the robot datums can then effectively give part position (e.g. a set of target points on or near the end effector) .

The electro-optical measuring system of the invention can be of many suitable types. For dynamic control of part positioning, fixed cameras or other sensors are desired to give rapid updates of reference point location. For set up of tools, many types are useful, particularly fixed photogrammetric cameras, automatic theodolites, (possibly located on traversing slides if used for set up of long assembly tools) , and tracking laser interferometers. Laser rangefinders in theodolite mounts can also be used if of sufficient accuracy for the task at hand.

The electro optical measuring system determines location of reference points relative to its coordinate system. This data is then converted to the coordinate reference frame of the desired structure such as a car body assembly or subassembly, or a tool therefore.

Where a rigid object forms part of the assembly, such as a tool base, the reference frame can be that of the tool object. In this case, datums on the base can be used to assist in establishing the reference of the electro- optical system to this coordinate system. CAD Data concerning parts is converted to tool coordinates to locate and assemble the parts in this reference system. Where the reference points are located on parts to be assembled, the reference coordinate system is generally that of the workcell to which robots and other assembly systems are referenced. Alternatively, it can be that of the finished structure, such as a car body. The reference system can even be derived directly from the CAD design, relying instead for assembly operations on the determined part locations in the frame of reference of the electro-optical measurement system.

The invention desirably provides for location accuracy over the measurement the field, making possible "Global" references from datums anywhere. "Local" references can also be used, near a point to be assembled to another. Reference points can be chosen from natural features, such as holes, letters, bosses, dimples, decoration etc. Certain special marks can be covered up by the assembly of the next piece, with the reference transferred to datums on that piece, or from the overall global reference.

For this reason, the invention can be used for many other assembly tasks, including 3D circuit board placement, telephone and computer assembly, aircraft assembly, etc.

The invention is not limited to the exact form of the embodiments described herein.

Claims

CLAIMS :

1. A method for assembly of members comprising the steps of: providing at least two members to be assembled; providing an electro optical measuring system for determining location of points in a reference coordinate system; electro-optically determining the location of reference points on each of said first and second members; determining from the location of said reference points, the 3 Dimensional location of said members in said coordinate system; using said determined location information to position said first member with respect to said second member, and joining or otherwise assembling said members.

2. A method according to claim 1 wherein at least one of said first and second members is also positioned with respect to said reference frame.

3. A method according to claim 1 wherein the determination of the location of reference points on one of said members is made at a first location from which subsequent locations of points on the member are known.

4. A method according to claim 1 wherein said first member is pushed against said second member until a desired face is attained.

5. A method according to Claim 1 wherein one of said members is held in a known manner by a least one position¬ ing member having at least one reference point thereon whose location is determined by said electro-optical measurement system.

6. A method according to claim 1 wherein a part member part containing special datums is used in place of a normal production part and the position of said special datums are determined by said system, and data therefrom used to calibrate said electro-optical determination.

7. A method according to claim 1 wherein said reference point is applied at a previous operation used to make or assemble said member.

8. A method according to claim 1 wherein said electro-optical measurement system contains a plurality of TV Cameras in fixed positions during a given assembly task.

9. A method according to claim 1 wherein at least one of said reference points is applied to a member as a function of other optically measured features on the same member or mating parts.

10. A method according to claim 1 wherein at least one of said members is mechanically fixed by at least one direction in at least one location.

11. A method according to claim 1 wherein at least one of said members is held magnetically, and said magnetic field is de-energized after joining.

12. A method according to claim 1 wherein said first member comprises at least a portion of the frame of a car, and said second member is an attachment device for outer body panels of said car.

13. A method according to claim 1, wherein said electro-optical determination is made at a sufficient plurality of reference points on said member to achieve the accuracy of member location desired.

14. A method according to claim 1 wherein successive members are sequentially positioned and joined to form an accurate assembly.

15. A method according to claim 14 wherein the position of each member is adjusted to determine the best fit based on said determined reference point locations.

16. A method according to claim 14 wherein the location of a member is adjusted in consideration of its dimensions or the dimension of one or more other members.

17. A method according to claim 1 wherein a mating section of a member is drilled, slotted, cut or otherwise modified as a function of dimension and/or locations in one or more members.

18. A method according to claim 1 wherein the location of said second member is optimized in considera¬ tion of the dimension and/ or deformation of said first member.

19. A method according to claim 18 comprising the additional step of placing said second member substantially in contact with said first member at least one point which is then joined.

20. A method according to claim 19, comprising the further step(s) of joining at additional points.

21. A method according to claim 1 wherein forces required to bring said members in substantial contact are determined.

22. A method according to claim 1 wherein positional adjustment is made, or at least one new member is substituted for an old member, if position or force determined is not within the tolerances allowed (due to springback, dimensional variation of the metal or other factors) .

23. A method according to Claim 1 wherein at least one of said reference points are natural features of said members.

24. A method according to claim 1 wherein at least one of said reference points are determined from surface locations determined by said electro-optical system.

25. A method for joining parts comprising the steps of: providing programable locating members for locating each of at least two parts to be assembled; providing programmably positionable Joining members such as weld guns for joining said parts; providing at least one optically visible reference point on each of said parts; providing at least one electro-optical position determining system; determining the location of said reference points, with said system; determining from said reference point locations, the location of said members; comparing said location to a desired location, and indicating the direction and optimum magnitude to move said member, and when a desired location is reached within a certain desired tolerance, joining said parts with said programmable joining means

26. A method for positioning members in space comprising the steps of: providing a base or other frame of reference for said members; providing a member to be positioned, said member having at least one optically visible reference point; providing an electro-optical reference point location determining system, fixed during said member positioning in relation to said frame of reference; determining the location of said reference point or points with said system; determining from said reference point location, the location of said member with respect to said frame of reference; comparing said location to a desired location, and indicating the direction and optimum magnitude to move said member; moving said member in said direction and magnitude, until the desired location is reached within a certain desired tolerance.

27. A method according to claim 26, wherein additional members are so positioned with respect to said reference frame.

28. A method according to claim 26, wherein said system includes a plurality of TV cameras fixed with respect to said frame of reference.

29. A method according to claim 26 wherein at least two of said sensors view said reference point, and the location of said reference point is determined stereoscopically.

30. A method according to claim 26, wherein said members are locating members for use in assembly of parts.

31. A method according to claim 26, wherein said members are clamping members for use in assembly of parts.

32. A method according to claim 26, wherein said members are joining members for use in assembly of parts.

33. A method according to claim 26 wherein said members are attached to a structure and are moved according to the invention to a final desired position relative to said structure.

34. A method according to claim 26 wherein said are members actuated automatically to correct location.

35. A method according to claim 33 wherein said system is also attached to said structure.

36. A method according to claim 26 wherein said members are movable in at least 3 axes.

37. A method According to claim 26, further including the step of determining location or dimension of parts located on said members.

38. A method according to claim 26, further including the step of determining force exerted on parts located on said members.

39. A method according to claim 26 including the further step of comparing measured data on parts produced and adjusting the position of said members accordingly.

40. A method according to claim 26 wherein more than one reference point is determined on said member.

41. A method according to claim 26 wherein said base is magnetic, for ease of member positioning.

42. A method according to claim 26 wherein said move¬ ment is made manually, by an operator receiving instruc- tions from a display, said display having windows which the operator places around the reference points effective for the member being positioned.

43. A method for controlling the assembly of vehicle bodies and components thereof, comprising the steps of: providing one or more members for locating parts of said bodies; providing joining members such as weld guns for joining said parts; providing at least one optically visible reference point on at least one of said joining or locating members; providing an electro-optical measurement system capable of determining of electro-optical reference point positions on members at a plurality of stations in an assembly line; determining the location of at least one reference point on at least one of said members, with said system; determining from said reference point location, the location of said member; comparing said location to a desired location in a data base, and determining the direction and magnitude to move said member to optimize a given operation; moving said member in said direction and magnitude, until said desired location is reached within a certain desired tolerance.

44. A method according to claim 43, wherein said electro-optical determines said reference point position at a plurality of stations simultaneously.

45. A method according to claim 43, wherein said systems TV cameras.

46. A method according to claim 43 wherein said members are actuated automatically to their desired location.

47. A method According to claim 43, further including the step of determining location or dimension of parts located on said members.

48. A method according to claim 43, further including the step of determining force exerted on parts located on said members.

49. A method according to claim 43 including the further step of comparing measured data on parts produced and adjusting the position of said members accordingly.

50. A method according to claim 43 wherein parts are monitored before, during, or after joining or robotic positioning to assist diagnosis of assembly function.

51. A method according to claim 43, wherein the correct position of said reference points or parts is taught to the electro-optical measurement system and measurized for latter comparison to measured points.

52. A method according to claim 43 wherein said

, reference point locations are Memorized and recorded in a data base.

53. A Method for joining parts, comprising the steps of: providing at least two parts to be joined: mechanically locating at least one of said parts in a known location at least one point, in at least one direction, providing an electro-optical measurement system for determining the location of said parts, using said determined locations, locating said parts in relationship to one another at one or more further points, and joining said parts when they are in a desired location.

54. A method for assembling parts, comprising the steps of: providing an electro-optical measuring system; providing a device for holding a part, said device capable of accommodating a plurality of different parts; placing a part in said device so as to cause said device to conform to said part; with an electro-optical sensor system, determining the location of said part; providing means to hold a second part with respect to said first part at one or more points, using said determined location information, and; assembling said parts.

55. An assembly apparatus for 3 dimensional structures comprising: means for electro-optically determining in a 3 dimensional reference coordinate system, the location of reference points on at least two members to be assembled; means for determining positioning data from said reference point locations; means for positioning at least one of said members using said positioning data, and; means to assemble said positioned member to at least one other member using said positioning data.

56. An apparatus for positioning parts for assembly comprising: individual locating members for portions of said parts, said locating members shaped to contact said part portions, and containing at least one reference to said contact portion; a base, including individual support means for said locating members, said support means capable of being positioned in a least one coordinate axis, and; an electro-optical measuring means capable of determining location of said reference points on said location members, and further capable of determining the location of at least one reference point on one of a group consisting of a part to be assembled, a loading means, an assembly or joining means, and a clamping means.

57. An apparatus for positioning parts for assembly comprising: individual locating members for portions of said parts, said locating members shaped to contact said part portions, and containing at least one reference point referenced thereto; a base, including individual support means for said locating members, said support means capable of being positioned in a least three coordinate axes, and; an electro-optical measuring means capable of determining location of said reference points on said location members; individual clamping means, to clamp said parts to said members.

58. A method for monitoring equipment for the assembly of vehicle bodies and components thereof, comprising the steps of: providing one or more members for locating and holding parts of said bodies; providing joining members such as weld guns for joining said parts; providing at least one optically visible reference point on at least one of said joining or locating members; providing a computer data base of said reference points, or member locations derived therefrom; providing an electro-optical measurement system capable of determining a plurality of electro-optical reference point positions at a plurality of stations in an assembly line; determining the location of said reference point on at least one of said members, with said system, and; updating said data base with said location of said reference point, or said member location derived therefrom.

59. A method for assembling parts comprising the steps of: determining from the design of the parts to be assembled, and their assembled condition, desired locations for the parts and reference points thereon; providing an electro-optical measuring system to determine locations of said reference points when parts are in approximate position to be assembled, converting the part design to the reference frame of said measuring system, and determining from said determined locations of said reference points, the movements of said parts to be in correct location indicated by said design of assembly, and; assembling said parts when held in said correct location.

60. Apparatus for positioning assembly tools, comprising: a tool member for locating parts for assembly, said tool member being affixed to a base; an electro-optical measuring system attached to said tool member in a known location thereto; a master artifact containing at least one reference point, located in a known manner with respect to said base, and; a computer, for determining the position of said measuring system related to said artifact, and providing signals indicative of required positioning of said tool member.