|Publication number||US8072581 B1|
|Application number||US 11/655,437|
|Publication date||Dec 6, 2011|
|Filing date||Jan 19, 2007|
|Priority date||Jan 19, 2007|
|Publication number||11655437, 655437, US 8072581 B1, US 8072581B1, US-B1-8072581, US8072581 B1, US8072581B1|
|Inventors||Arlen E. Breiholz|
|Original Assignee||Rockwell Collins, Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (20), Referenced by (13), Classifications (9), Legal Events (2)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of the Invention
The present invention relates generally to measurement of distance to visual landmarks for applications including navigation and mapping, and more particularly navigation and mapping in a Global Positioning System (GPS) denied environment.
2. Description of the Related Art
There is a growing need to provide underground, urban, and indoor navigation without the use of external augmentation aids or GPS. Tracking or navigation systems often make use of measurements from sensors to aid in determining a location (“localization”) or an orientation (attitude and heading) or a pose (position and orientation) of an object such as a person, a vehicle or a robot as it navigates in an environment, such as within the bounds of a building. A variety of types of sensors are available for such systems, including sensors that measure a relative location between a sensor and a target. An example of such a sensor/target combination is a pulsed laser emitter and a photosensor that can determine a distance traveled by the laser pulse by measuring time of flight. Different types of sensors measure different aspects of the relative pose of a sensor and a target, such as a range, direction, or relative orientation.
Location and mapping is a fundamental problem in mobile robotics. While a robot navigates in an unknown environment, it must incrementally build a map of its surroundings and, at the same time, localize itself within that map. One popular solution is to use Simultaneous Localization and Mapping (SLAM) algorithms, treat localization and mapping as an estimation problem and apply a Kalman filter.
As will be disclosed below, the present invention supplies technology components that can supply target range and bearing information to a SLAM algorithm using a very compact, low-power sensor. Further, the disclosed design allows the range of the sensor to be extended at the expense of spatial resolution in cases where the range from sensor to target is too great for the sensor to operate normally.
Flash LIDAR, which illuminates a scene using a single laser pulse that is optically spread to cover the field of view of a gated focal plane array, outputs an image in which each pixel is labeled with both brightness and range. This ability to simultaneously range to all objects in a scene is extremely useful for unmanned ground vehicles as well as for general reconnaissance and surveillance.
The alternative to Flash LIDAR is a scanning LIDAR. This uses a pencil laser beam with a single sensor and a moving mirror assembly that scans the beam across the scene. Aside from the potential difficulties with rapidly moving parts, the fact that all pixels in a scene are sampled sequentially can lead to distortion or misperception if there is relative motion between the sensor and all or part of the scene.
A major drawback to the use of flash LIDAR is that the spreading of the illuminating laser beam introduces a quadratic fall-off in illumination with range. In a GPS-Denied Navigation activity for dismounted troops, the range of the flash LIDAR is limited due to the requirement that the LIDAR be body-worn. Both available battery power and heat that must be dissipated from the unit contribute to this restriction. Heretofore, it has been necessary to sacrifice field of view by using longer focal length optics on the flash LIDAR in order to achieve acceptable range for a man-worn system.
U.S. Pat. No. 6,414,746, issued to R. Stettner, et al., entitled “3-D Imaging Multiple Target Laser Radar,” assigned to Advanced Scientific Concepts, Inc., Santa Barbara, Calif., discloses a device which uses a single pulse from a pulsed light source to detect objects which are obscured by camouflage, fog or smoke but otherwise enveloped by a light-transmitting medium. The device simultaneously operates in two modes, light reflected from the nearest object is processed to form a three-dimensional image by an array of pixels. This first image is based upon the light-pulse transit time recorded in each pixel. Each pixel also contains a high-speed analog memory that sequentially stores reflected signals at a repeated time interval. The first reflection acts as a time base that controls when the analog memory begins or ends the storage sequence. The first return could be from a camouflage net and the amplitudes of the return signals, after the first return, would then be from objects behind the net. Computer processing these amplitudes reveals the three-dimensional nature of the obscured objects. The device consists of the pulsed light source, optics for collecting the reflected light, a sensor for detecting the light and converting it to electrical data, drive and output electronics for timing and signal conditioning of data generated by the sensors and a computer for processing the sensor data and converting it to a three dimensional image. The sensor collects and processes the light data in a unique manner, first converting it to electricity by a number of alternate detector technologies and then using integrated circuit chips which consist of a two dimensional array of electronic pixels also called unit cells. The two dimensional array defines two dimensions of the image. Stored within each unit cells is data associated with the third dimension, ranges of targets, and amplitudes of target reflections. This data are read out of the integrated circuit chip in the time interval between laser pulses to a processing computer. The processing computer corrects the data and, by means of computer algorithms specific to the device, converts the data to a three-dimensional image of one or more targets. This image may be viewed or processed electronically to isolate targets.
Other Advanced Scientific Concepts, Inc. assigned patents include U.S. Pat. Nos. 6,133,989, 5,696,577, and 5,446,529.
None of the aforementioned references provide an efficient flash LIDAR system for laser range finding that is particularly suitable for man worn applications.
In a broad aspect, the present invention is a laser range finding system that includes a light detection and ranging (LIDAR) sensor unit (SU) and a LIDAR processor unit (PU). The LIDAR SU is for transmitting light pulses and receiving resulting input light signals reflected from objects within the field of view of the SU. The LIDAR SU includes a flash LIDAR illuminating laser source for transmitting light pulses. The LIDAR illuminating laser source includes an illuminating laser and zoom optics operatively associated with the laser. A LIDAR receiver receives resulting input light signals reflected from the objects. The LIDAR receiver includes a sensor; and, a flash readout integrated circuit (IC). The flash readout IC measures the transit time of the light pulses. The LIDAR processor unit (PU) is operatively associated with the LIDAR SU and it utilizes flash LIDAR ranging. A power source is operatively coupled to the LIDAR PU. Zooming of the transmitted light pulses results in the received resulting input light signals illuminating a relatively reduced area of the frame, thus providing a flash LIDAR image of relatively reduced resolution but enhanced range by utilization of the transit time measurements.
The LIDAR receiver preferably operates as a passive imager part of the time to allow an operator to point said sensor, allowing an area of interest to be centered in a frame.
The LIDAR illuminating laser source may include an optical view finder coupled with the zoom optics.
The inventive concepts herein may provide an efficient flash LIDAR system for laser range finding that is particularly suitable for man worn applications. The system may include a computer system coupled to the LIDAR PU for receiving processed navigation and mapping information from the LIDAR PU and providing an optimized geo-referenced map of the path traversed. In such an instance the laser range finding system operates as a navigation and mapping system, wherein the LIDAR PU comprises a LIDAR navigation processor unit (NPU) and the LIDAR SU comprises a LIDAR navigation sensor unit (NSU).
Referring now to the drawings and the characters of reference marked thereon,
The SU 12 includes a flash LIDAR illuminating laser source 16 for transmitting light pulses. The LIDAR illuminating laser source includes an illuminating laser 18 and zoom optics 20 operatively associated with the laser 18. A LIDAR receiver, designated generally as 22, receives resulting input light signals reflected from the objects within the field of view. The LIDAR receiver includes a sensor 24; and, a flash readout integrated circuit (IC) 26. The flash readout IC 26 measures the transit time of the light pulses. The LIDAR receiver 22 also includes suitable optics 27.
In the specific application of the invention, shown in
In a personalized handheld application the LIDAR NSU 12 preferably includes a miniature flash Laser Imaging Detection and Ranging (LIDAR) sensor 24 with an integrated Inertial Measurement Unit (IMU) that serves as the primary mapping and navigation sensor for the system 10. It typically includes a GPS antenna. The use of accurate local Flash LIDAR ranging corrects a Dead Reckoning (DR) module when GPS is unavailable. The LIDAR NSU 12 includes the components essential for operation such as a LIDAR transmitter, LIDAR Focal Plane Array (FPA) and LIDAR Pre-Processing.
In a preferred embodiment an embedded GB-GRAM military GPS card provides typical accuracy of 2 to 4 meters and can be used when possible. The basic DR module is composed of a combination of 3 axis gyros, magnetic compasses, and accelerometers incorporated in the Flash LIDAR sensor unit. Personal position estimates from the DR sensors are fused together using technology that uses recursive algorithms to optimize raw sensor data with typical body motion logic to reduce sensor drift.
The LIDAR NSU 12 may include other types of sensors such as imaging sensors, for example, Electro-Optic (EO)/Infrared (IR); Low Light Television (LLTV); Synthetic Aperture Radar (SAR); Ultra Wide Band (UWB); and Chemical, Biological, Nuclear, Radiological, and Explosives (CBNRE). Acoustic sensors may be used such as gunfire, seismic, and sonar. Communication links may be used to share situational awareness between multiple users.
The LIDAR NPU 14 includes a subsystem module which includes a LIDAR power supply and LIDAR post-processing module. Another subsystem module within the LIDAR NPU 14 includes a navigation processor power supply and GB-GRAM GPS receiver.
The computer system 28 is coupled to the LIDAR NPU 14 for receiving processed navigation and mapping information from the LIDAR NPU 14 and providing an optimized geo-referenced map of the path traversed. This computer system 28 is preferably a handheld unit such as a Commercial Off-The-Shelf (COTS)/Military Off-The-Shelf (MOTS) handheld computer with large-format keypad for easy input of position information and user interaction with the remainder of the system 10. A power source 30, e.g. battery, provides power to the LIDAR NPU 14 and the computer system 20.
A foot-mounted sensor may be included to improve accuracy Dead Reckoning (DR). Sensor drift corrections are provided by using Zero velocity Updates (ZUPT) when the foot is stationary. This optional foot-mounted sensor may be provided by a MEMs type small inertial package if required.
The LIDAR NSU and LIDAR NPU may be integrated as a single system. Such an Integrated LIDAR/INS Navigation System (LIINS) combines the concept of a standard INS with a SLAM algorithm. In this system, an IMU provides IMU sensor data (delta velocity and delta angle measurements) to an Inertial Navigation System (INS). The strapdown navigator integrates the sensor data to compute a current estimate of position, velocity, and attitude. The IMU and the strapdown navigator comprise the INS. In order to constrain error growth due to errors in the sensor data, an extended Kalman Filter is provided to estimate errors, system position, velocity, attitude, as well as sensor data errors—bias and scale factor. Measurement inputs to the Kalman filter include range and range rate measurements based upon LIDAR transmissions. The raw reflection data originating from the LIDAR light source (NSU 12) is collected on the focal plane array on NSU 12, read out and provided to a reflection point manager, part of the NPU 14. This function is responsible for identifying usable reflection points, and correlating their respective locations across LIDAR image frames. Once reflection points have been identified, range and range rate measurements to those reflection points are computed and provided to the Kalman filter. These measurements are then used as the basis for estimating errors in the system solution. The IMU is fixed to the LIDAR so that the measurements from the LIDAR may be easily represented in a known reference frame. In addition, the Kalman filter provides reflection point location error estimates to the reflection point manager.
The Kalman filter estimates system solution errors (three position and three velocity), sensor data errors (three accelerometer bias, three gyro bias, three accelerometer scale factor, three gyro scale factor) and reflection point position error (three for each reflection point in use). There are two measurements per reflection point (range and range rate) which provide observability to the system states. Since the range to the reflection points is likely to be relatively small, there could be significant changes in line of sight angles. Therefore, the Kalman filter will be implemented as a Gaussian second-order filter.
The LIDAR NPU 14 preferably solves the simultaneous localization and mapping (SLAM) problem using a thin junction tree filter (TJTF), thereby correcting real time dead reckoning/IMU error drift.
The LIDAR NPU 14 may include other types of navigation systems such as a radio navigation unit such as Very High Frequency Omni-directional Radio-range (VOR); Automatic Direction Finder (ADF); Tactical Air Navigation (TACAN); and/or Instrument Landing System (ILS). It preferably has a GPS navigation system. It may also include an anti-jam countermeasure such as AJ wavelet processing, Space-Time Adaptive Processing (STAP), and/or Space-Frequency Adaptive Processing (SFAP).
The present invention is designed to be extremely compact. Referring now to
The zoom optics 20 may comprise, for example, a variable focal length configuration of lenses and mirrors allowing the angle of coverage of the optics to shift from illumination of the full field of view of the LIDAR receiver 22 down to illumination of only a few pixels of the receiver focal plane array. The LIDAR illuminating laser source may include an optical view finder coupled with the zoom optics. The zoom optics can either have a continuously variable focal length or be of a simpler design that is switched between a limited set of fixed focal lengths. In either case, the system would select only those pixels actually illuminated when creating images. Although the illuminating source has been discussed as being a laser, the illuminating source may be some other type of illuminating source such as a light emitting diode, xeon flash lamp or other suitable flash illuminating source. The LIDAR receiver preferably operates as a passive imager part of the time to allow an operator to point the sensor, allowing an area of interest to be centered in a frame.
The sensor 24 may be, for example, a silicon focal plane array for operation in the visible portion of the spectrum or an indium-gallium-arsenide focal plane array for operation in the short-wave infrared portion of the spectrum.
The flash readout IC 26 may be, for example, a collection of parallel charge-coupled, analog shift registers, one per sensor pixel, that are clocked to capture time-sequential samples of the pixel output in real time and subsequently pass the captured charge to a set of analog-to-digital converters that translate the captured signal into digital form for subsequent processing.
Referring now to
As can be seen in
Zooming only the optics of the illuminating laser provides an efficient system obviating the need to provide complementary zoom optics on both the laser illuminator and on the receiving focal plane array. A system having two sets of zoom optics would be difficult and cumbersome because the field of view would have to be changed in a synchronized manner.
As mentioned above, with the present invention one operational approach is to operate the receiving focal plane array (camera) as a passive imager part of the time. Since the laser pulse repetition rate is apt to be relatively slow to conserve power anyway, this is quite efficient. This allows the user to point the sensor using the passive image so that the area of interest is centered in the frame. Then, the laser can be pulsed through optics that are zoomed to illuminate just that central area of the frame resulting in a flash lidar image of reduced resolution but increased range. An optical viewfinder may be provided to facilitate this aiming. The passive view finder imagery can be used as input to a SLAM algorithm in addition to the flash lidar range data. This is synergistic because SLAM can be utilized with passive imagery. The system can display substantially continuous passive video to the user (and record it as well) while periodically pulsing the laser and deriving range to the illuminated pixels.
Instead, of using an optical viewfinder, the imager may be operated as a TV camera to facilitate aiming, providing some visual indication of what region the laser is illuminating.
Although application of the inventive concepts herein have been described relative to their utilization with a man-worn system, the present invention can be utilized with a large variety of applications including, for example, unmanned ground vehicles and UAV (unmanned air vehicle) sensors.
Other embodiments and configurations may be devised without departing from the spirit of the invention and the scope of the appended claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5365218 *||Sep 14, 1992||Nov 15, 1994||Deutsche Aerospace Ag||System for guarding property including a mobile laser unit|
|US5446529||Feb 9, 1993||Aug 29, 1995||Advanced Scientific Concepts, Inc.||3D imaging underwater laser radar|
|US5696577||Jun 6, 1995||Dec 9, 1997||Advanced Scientific Concepts, Inc.||3D imaging underwater laser radar|
|US6133989||Jun 19, 1996||Oct 17, 2000||Advanced Scientific Concepts, Inc.||3D imaging laser radar|
|US6176837||Apr 17, 1998||Jan 23, 2001||Massachusetts Institute Of Technology||Motion tracking system|
|US6392747||Jun 11, 1999||May 21, 2002||Raytheon Company||Method and device for identifying an object and determining its location|
|US6414746||Nov 24, 1999||Jul 2, 2002||Advanced Scientific Concepts, Inc.||3-D imaging multiple target laser radar|
|US6522396||Jan 8, 2002||Feb 18, 2003||Raytheon Company||Dual mode adaptive threshold architecture for 3-D ladar FPA|
|US6526352 *||Jul 19, 2001||Feb 25, 2003||Intelligent Technologies International, Inc.||Method and arrangement for mapping a road|
|US6741341 *||Feb 3, 2003||May 25, 2004||Bae Systems Information And Electronic Systems Integration Inc||Reentry vehicle interceptor with IR and variable FOV laser radar|
|US6922632||Aug 11, 2003||Jul 26, 2005||Intersense, Inc.||Tracking, auto-calibration, and map-building system|
|US7015831||Dec 17, 2003||Mar 21, 2006||Evolution Robotics, Inc.||Systems and methods for incrementally updating a pose of a mobile device calculated by visual simultaneous localization and mapping techniques|
|US7098999||Mar 5, 2004||Aug 29, 2006||Raytheon Company||High-speed readout circuit and system incorporating same|
|US7649616 *||Jul 8, 2004||Jan 19, 2010||Lockheed Martin Corporation||Fiber laser ladar|
|US7859649 *||Apr 27, 2007||Dec 28, 2010||Quality Vision International, Inc.||Laser range sensor system optics adapter and method|
|US20040119838||Apr 29, 2003||Jun 24, 2004||Andrew Griffis||Compact economical lidar system|
|US20050182518||Feb 10, 2005||Aug 18, 2005||Evolution Robotics, Inc.||Robust sensor fusion for mapping and localization in a simultaneous localization and mapping (SLAM) system|
|US20050234679||Feb 10, 2005||Oct 20, 2005||Evolution Robotics, Inc.||Sequential selective integration of sensor data|
|US20060227316 *||Apr 6, 2005||Oct 12, 2006||Phillip Gatt||Three-dimensional imaging device|
|DE2930937A1 *||Jul 31, 1979||Feb 5, 1981||Messerschmitt Boelkow Blohm||Fixed marking suppression in Doppler laser radar system - distinguishes target from background by variation of laser beam cone geometry|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8744765 *||Jul 26, 2010||Jun 3, 2014||Msa Technology, Llc||Personal navigation system and associated methods|
|US8976340 *||Apr 16, 2012||Mar 10, 2015||Advanced Scientific Concepts, Inc.||Ladar sensor for landing, docking and approach|
|US9041915 *||May 2, 2011||May 26, 2015||Ball Aerospace & Technologies Corp.||Systems and methods of scene and action capture using imaging system incorporating 3D LIDAR|
|US9069061 *||Jul 19, 2012||Jun 30, 2015||Ball Aerospace & Technologies Corp.||LIDAR with analog memory|
|US9304203||Mar 13, 2013||Apr 5, 2016||Google Inc.||Methods, devices, and systems for improving dynamic range of signal receiver|
|US9360554||Apr 11, 2014||Jun 7, 2016||Facet Technology Corp.||Methods and apparatus for object detection and identification in a multiple detector lidar array|
|US20110029241 *||Jul 26, 2010||Feb 3, 2011||Mine Safety Appliances Company||Personal Navigation System and Associated Methods|
|US20120044476 *||May 2, 2011||Feb 23, 2012||Ball Aerospace & Technologies Corp.||Systems and methods of scene and action capture using imaging system incorporating 3d lidar|
|US20120261516 *||Apr 16, 2012||Oct 18, 2012||Patrick Gilliland||Ladar sensor for landing, docking and approach|
|US20130211672 *||Feb 8, 2013||Aug 15, 2013||Audi Ag||Time-of-flight camera for a motor vehicle, motor vehicle and method for operating a time-of-flight camera|
|US20150009485 *||Jul 1, 2014||Jan 8, 2015||Electronics And Telecommunications Research Institute||Laser radar system|
|US20150055119 *||Aug 21, 2014||Feb 26, 2015||Sheltered Wings, Inc.||Laser rangefinder with improved display|
|US20160070265 *||Jul 30, 2015||Mar 10, 2016||SZ DJI Technology Co., Ltd||Multi-sensor environmental mapping|
|U.S. Classification||356/5.01, 356/4.01|
|Cooperative Classification||G01S17/023, G01S7/51, G01S17/89|
|European Classification||G01S7/51, G01S17/89, G01S17/02C|
|Jan 19, 2007||AS||Assignment|
Owner name: ROCKWELL COLLINS, INC., IOWA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BREIHOLZ, ARLEN E.;REEL/FRAME:018809/0028
Effective date: 20070118
|Jun 8, 2015||FPAY||Fee payment|
Year of fee payment: 4