WO2008070205A2 - Obstacle detection arrangements in and for autonomous vehicles - Google Patents

Obstacle detection arrangements in and for autonomous vehicles Download PDF

Info

Publication number
WO2008070205A2
WO2008070205A2 PCT/US2007/070918 US2007070918W WO2008070205A2 WO 2008070205 A2 WO2008070205 A2 WO 2008070205A2 US 2007070918 W US2007070918 W US 2007070918W WO 2008070205 A2 WO2008070205 A2 WO 2008070205A2
Authority
WO
WIPO (PCT)
Prior art keywords
obstacle
prospective
radar
bunched
classifying
Prior art date
Application number
PCT/US2007/070918
Other languages
French (fr)
Other versions
WO2008070205A3 (en
Inventor
William L. Whittaker
Josh Johnston
Jason Ziglar
Original Assignee
Carnegie Mellon University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Carnegie Mellon University filed Critical Carnegie Mellon University
Publication of WO2008070205A2 publication Critical patent/WO2008070205A2/en
Publication of WO2008070205A3 publication Critical patent/WO2008070205A3/en

Links

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08GTRAFFIC CONTROL SYSTEMS
    • G08G1/00Traffic control systems for road vehicles
    • G08G1/22Platooning, i.e. convoy of communicating vehicles
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0238Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
    • G05D1/024Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors in combination with a laser
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/0278Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using satellite positioning signals, e.g. GPS
    • GPHYSICS
    • G08SIGNALLING
    • G08GTRAFFIC CONTROL SYSTEMS
    • G08G1/00Traffic control systems for road vehicles
    • G08G1/16Anti-collision systems
    • G08G1/161Decentralised systems, e.g. inter-vehicle communication
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0257Control of position or course in two dimensions specially adapted to land vehicles using a radar
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/027Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means comprising intertial navigation means, e.g. azimuth detector

Definitions

  • the present invention relates generally to methods, systems, and apparatus for the autonomous navigation of terrain by a robot, and in particular to arrangements and processes for discerning and detecting obstacles to be avoided.
  • brackets - [ ] - refer to the list of references found towards the close of the instant disclosure.
  • millimeter wave radar has emerged as a technology well- matched to outdoor vehicle navigation. It sees through dust and rain, doesn't depend on lighting, senses over a useful range, and can be cheap to mass- produce.
  • Car manufacturers have successfully used radar for adaptive cruise control (ACC) and now offer them as options on luxury models and trucks [1] [2] [3].
  • ACC adaptive cruise control
  • Adaptations for autonomous vehicle navigation through unstructured terrain have been much less successful for a variety of less-publicized weaknesses associated with radar [4].
  • a first data manipulation involves distinguishing between those potential obstacles that are surrounded by significant background scatter in a radar diagram and those that are not, wherein the latter are more likely to represent binary obstacles that are to be avoided.
  • a second data manipulation involves updating a radar image to the extent possible as an object comes into closer range.
  • the first aforementioned data manipulation may be performed via context filtering, while the second aforementioned data manipulation may be performed via blob-based hysteresis.
  • a method of providing obstacle detection in an autonomous vehicle comprising the steps of: obtaining a radar diagram; discerning at least one prospective obstacle in the radar diagram; ascertaining background scatter about the at least one prospective obstacle; classifying the at least one prospective obstacle in relation to the ascertained background scatter; and refining the radar diagram and reevaluating the at least one prospective obstacle; the reevaluating comprising repeating the steps of ascertaining and classifying.
  • a system for providing obstacle detection in an autonomous vehicle comprising: an arrangement for discerning at least one prospective obstacle in a radar diagram; an arrangement for ascertaining background scatter about the at least one prospective obstacle; an arrangement for classifying the at least one prospective obstacle in relation to the ascertained background scatter; and an arrangement for refining the radar diagram and reevaluating the at least one prospective obstacle; the refining and reevaluating arrangement acting to prompt a repeat of ascertaining background scatter about the at least one prospective obstacle and classifying the at least one prospective obstacle in relation to the ascertained background scatter.
  • the patent or application file contains at least one drawing executed in color.
  • Figure 1 shows a schematic of an overall architecture of navigation software in which at least one presently preferred embodiment of the present invention may be employed;
  • Figure 2 schematically illustrates a processing pathway of a radar obstacle detection method.
  • Figure 3 illustrates returns from an exemplary 180 degree radar sweep to a 75m range.
  • Figure 4 shows the application of an energy filter to radar data.
  • Figure 5 shows backscatter returns from a 30 gallon plastic trash can.
  • Figure 6 graphically illustrates a kernel mask that may be employed during context filtering.
  • Figure 7 graphically provides a side-by-side comparison of unprocessed and context-filtered radar data from a desert site.
  • Figure 8 provides a side-by-side comparison of a successive images of an obstacle refined by blob-based hysteresis.
  • FIG. 9 graphically illustrates time indexing in a FMCW radar.
  • autonomous is used to indicate operation which is completely automatic or substantially automatic, that is, without significant human involvement in the operation.
  • An autonomous vehicle will generally be unmanned, that is without a human pilot, or co-pilot. However, an autonomous vehicle may be driven or otherwise operated automatically, and have one or more human passengers.
  • An autonomous vehicle may be adapted to operate under human control in a non- autonomous mode of operation.
  • vehicle refers to any self-propelled conveyance.
  • description of the present invention will be undertaken with respect to vehicles that are automobiles.
  • the use of that exemplary vehicle and environment in the description should not be construed as limiting.
  • the methods, systems, and apparatuses of the present invention may be implemented in a variety of circumstances.
  • the embodiments of the present invention may be useful for farming equipment, earth moving equipment, seaborne vehicles, and other vehicles that need to autonomously generate a path to navigate an environment.
  • Fig. 1 shows a schematic of an overall architecture of navigation software in which at least one presently preferred embodiment of the present invention may be employed.
  • a further appreciation of specific components forming such an architecture may be employed as an illustrative yet non-restrictive environment for at least one presently preferred embodiment of the present invention, may be gleaned from "SOFTWARE ARCHITECTURE FOR HIGHSPEED TRAVERSAL OF PRESCRIBED ROUTES", supra.
  • a radar 202 and binary detection arrangement (comprising, preferably, a pipeline 210 and radar module 230 as discussed herebelow) can preferably be integrated into a navigation architecture as shown in Fig. 1 and advantageously provide radar- based obstacle detection in such a context in a manner that can be more fully appreciated herebelow.
  • FIG. 2 broadly illustrates a processing pathway of a radar obstacle detection arrangement 200, and associated method, in accordance with a presently preferred embodiment of the present invention. Reference to Fig. 2 will continue to be made throughout the instant disclosure as needed.
  • a radar arrangement 202 transmits and receives radar energy (207) in a general sweep (to be better appreciated further below) and as such will rebound from one obstacle 208 after another. Data is then binned (209) by range and azimuth and fed to a "radar pipeline" 210, also to be better appreciated further below, that context-filters data in accordance with at least one particularly preferred embodiment of the present invention before proceeding (229) to a radar module 230.
  • data is preferably transformed from polar coordinates to rectangular coordinates (relative to the vehicle in question) before undergoing, in accordance with at least one particular preferred embodiment of the present invention, an updating and refinement (to the extent possible in view of time and range constraints) via blob-based hysteresis before being transmitted (238) to a remainder of a general navigation system (such as that discussed and illustrated herein with respect to Fig. 1.
  • the radar indicated at 202 can be embodied by essentially any suitable equipment; a very good illustrative and non-restrictive example would be a Navtech DSC2000 77 GHz Frequency Modulating Continuous Wave (FMCW) radar.
  • An advantage of such equipment is that it provides the capability of applying (206) a Fast Fourier Transform (FFT) to data received by an antenna 204, whereupon the data can be binned by an onboard DSP, thus permitting the data to become available over Ethernet.
  • FFT Fast Fourier Transform
  • the data output from such a radar 202 is expressed as intensity of backscatter in bins measuring 1.2 degrees in azimuth by 0.25 m in range.
  • Such a radar provides a vertical beam width of 4 degrees with a scan rate (i.e., rotational velocity) of 360 degrees in 0.4 seconds.
  • data output from radar 208 is in the form of individual packets each containing a radar vector which is an 800-member vector of discretized intensities.
  • a radar vector which is an 800-member vector of discretized intensities.
  • values can range from 0 (minimum intensity) to 144 (maximum intensity) and be indexed by range in 0.25 m increments from 0 to 200 meters.
  • Each radar vector also preferably is recorded with an azimuth at 0.1 degree precision.
  • a full 360 degree sweep can thus include about 310 radar vectors at 1.2 degree separation.
  • antenna 204 is spun at an imprecisely controlled rate and the samples are timed at 1 ms separation, the exact number of radar vectors in a sweep is not necessarily guaranteed. Additionally, the azimuth direction of radar vectors can of course vary slightly between sweeps, so the unique azimuth of any given radar vector should preferably be recorded.
  • the half of the radar field of view facing "backward” can readily be eliminated, only those objects in the 180 degree arc from "straight left” to "straight right” in front of the sensor need be analyzed.
  • the antenna 204 can thus be scanned at a steady rate, with about 0.2 seconds between when the left side data and right side data are recorded during a single sweep.
  • radar vectors will be individually time-stamped with their arrival times to ensure that proper vehicle poses are retrievable.
  • radar vectors are preferably collected into a radar image, which is a wrapper for a matrix that indexes intensity by azimuth and range.
  • This radar image can be expressed as a 180 degree view of radar backscatter intensities, an example of which is shown in Figure 3. More particularly, shown in Figure 3 is an exemplary 180 degree radar sweep to a 75m range with the Navtech equipment discussed above. Green represents areas of low backscatter, while red areas darken in proportion to the strength of backscatter returns.
  • supported radar image functions include the ability to form windowed iterators and writing to image file or over Ethernet.
  • the aforementioned radar images preferably pass through a software pipeline 210 that performs context-filtering operations on the data, the result of which will be more fully appreciated herebelow.
  • the pipeline 210 can preferably contain several software classes, including a reader 212, first filter 214, branch 216, followed in parallel by (a) a second filter 218 and a writer 220 to file (222) and (b) a third filter 224 and a writer 226 to Ethernet.
  • a regulator 228 governing writers 220/226 is also preferably included.
  • pipeline 210 is preferably configured at runtime with a custom scripting language.
  • Reader 212 will preferably receive radar vectors from the radar 202 over
  • Writers preferably transmit data to file, shared memory, or over Ethernet; here, writers 220/226 are shown as writing to file and transmitting to Ethernet, respectively.
  • configurable filters 214/218/224 are shown that can be ordered via the scripting language. All of these classes derive from the Pipe class, which contains a pointer to the previous Pipe in the Pipeline 210 ( Figure 2). Branching is also supported (216), allowing multiple filtering methods on the same data or multiple output formats. Of all the class types- Reader, Filter, Writer, Branch, Regulator- only the reader 212 need be radar hardware specific.
  • First filter 214 will preferably undertake the "context filtering" as broadly understood herein (and as described in more detail herebelow in accordance with at least one preferred embodiment) while second filter 218 and/or third filter 224 can undertake secondary filtering operations such as additional thresholding (e.g., to further increase the likelihood of avoiding false positives).
  • regulator 229 can preferably apply a scheme to avoid or obviate such a contingency.
  • a single radar can be passed "backward” (i.e., towards reader 212) from any "final” element of the pipeline 210 (e.g., either of the writers 220/226).
  • the filters 214/218/224 would not act on the data represented by the image.
  • the reader 212 can fill it with radar vectors and send it back "forward" through the pipeline 210 (i.e., towards writers 220/226), whereupon the filters 214/218/224 would actually act on the data.
  • a pipeline 210 in accordance with at least one embodiment of the present invention will preferably afford some flexibility such that, e.g., filters can be dropped in and out or be reconfigured at runtime.
  • the radar antenna 202 can also be replaced with required changes isolated to only one sub-class (i.e., reader class).
  • the implementation of the radar image wrapper class can be completely reworked without affecting the filter processes.
  • the radar image can be defined as a matrix of Cartesian coordinates, a lookup table of Cartesian coordinates for a polar coordinate storage structure, and directly as a polar coordinate structure.
  • the pipe classes need not be changed to support such modifications.
  • the radar method up to this point is modular, self-contained, and can operate freestanding. Its output (shown here on the output from writer 226) includes the location of obstacles indexed by azimuth and range and marked with data collection time stamps. To transform these values into a real world location of an obstacle, however, requires knowledge of the position antenna 204 at the time of each data collection. On most robots, this information is available to the sensor from the vehicle's estimation of position, orientation in 6 DOF (Degrees of Freedom). To utilize these resources without losing the modularity of the Pipeline approach, an additional radar module (230) process is preferably added that functionally follows pipeline 210.
  • the radar module 230 preferably references the vehicle pose history, and therefore is not stand-alone. It receives input from the end of the pipeline 210 over Ethernet. This input data includes the azimuth and range (relative to the sensor at a collection timestamp) of location bins containing obstacles. These obstacles preferably are reported with binary confidence and are not further classified. Rather, radar module 230 is preferably configured to convert (232) the data from polar coordinates (azimuth range) relative to the sensor to a latitude/longitude location on the Earth's surface. The output 233 of conversion 232 thus preferably takes the form of a rectangular map, e.g., 100 meters by 100 meters, centered on the vehicle at an arbitrary time.
  • each obstacle pixel preferably ends up being converted to Cartesian coordinates, then transformed (236) by a calibration file and the vehicle pose to determine its location in the map. If an object has not already been reported at that location, it is added, and the map is forwarded (238) to the robot's navigation and planning algorithms (see Fig. 1).
  • an updating of data via blob-based hysteresis (234) preferably takes place, to be better understood herebelow.
  • a significant advantage enjoyed in accordance with at least one embodiment of the present invention is the rendering of a priori assumptions that are intuitive after posing radar data in image format.
  • Existing filtering and classification methods can essentially be borrowed from camera-based image processing to identify obstacles representing a significant risk to a vehicle while not reporting false positives that are actually traversable.
  • radar still reports little except range to objects, providing very few discriminable features.
  • LIDAR produces detailed geometry from which shape and roughness can be extracted. Stereovision and visual segmentation allow separation of objects that stick up sharply from the smooth road. Available radar antennas have too low a resolution for shape identification and have too great a vertical beam width to produce height maps.
  • a street gutter is a non-oblique angle in only two directions, so it may return very little energy, while a drain in the gutter could become visible.
  • a road may have few returns until it goes uphill and faces the antenna a little more directly. Especially with rough surfaces and grasses, this undesired ground return can be a significant source of false positives.
  • Thresholding represents an effort to connect intensity of backscatter to the risk associated with hitting an object. High intensity means more danger, and vice versa. Unfortunately, these properties are actually poorly related.
  • Energy filtering is effected by convolving a rectangular kernel with the image. At every pixel, the intensities within the kernel were summed and if they pass a threshold, the pixel was classified as an obstacle.
  • the energy filter works very well at detecting road edges like gutters and berms (common on dirt roads) and larger obstacles like cars and buildings. Unfortunately, it also has a lot of false positives.
  • FIG. 4 shows the application of an energy filter to radar data, and highlights the disadvantages of this approach.
  • the image on the left shows unprocessed radar data, where red corresponds to high backscatter returns.
  • the right image is the same data processed with an energy filter.
  • Green is safe to drive, red represents identified obstacles, white is missed obstacles, and black is false positives. Most of the black region (false positives) is mown grass, which is easily traversed by a HMMWV.
  • the failure of more straightforward methods suggests making use of more sophisticated models of a priori data. Since a sensor in accordance with at least one preferred embodiment of the present invention can support an already navigable vehicle, it might be possible to turn off the radar in situations where the previous methods are known to be untrustworthy. Then, obstacles will only be reported in areas of high confidence, reducing correct detections, but potentially reducing false positives to acceptable levels.
  • the global clutter filter creates a histogram of all values in the radar image. It then selects the intensity at a percentile selected as a parameter and subtracts that intensity from all values in the image. This was optimized by experimentation at the 80 th percentile. If an image is 80% empty of backscatter, a common occurrence in low-clutter regions, this filter will have no effect on the raw data. In areas where more than 20% of the image contains significant backscatter, all returns are decreased. This effectively increases the burden of proof for the next stage of filtering. [65] Combining the global clutter filter with the energy or threshold filters is sufficient to identify many obstacles with few false positives. This method is robust to areas of high clutter and works well at identifying obstacles in otherwise clear areas. Because it only identifies obstacles in low-density environments, however, it misses true obstacles in clutter-filled regions.
  • the antenna is a physical device with a Gaussian distribution of beam intensity in the azimuth direction
  • the beam width is not discrete.
  • the cited 1.2° beam width is the half-power width. This means that a strongly reflecting object, even if it is small enough to fit into one azimuth bin, will "bleed" into the surrounding azimuth bins.
  • the range measurement is a binned result from a continuous FFT, a reflecting object will also bleed intensity into the surrounding range bins. Therefore, a single point object like a fencepost or barrel will actually show intensity in at least 9 bins, and the edges of all objects will be fuzzy.
  • a context filter as implemented and employed in accordance with at least one presently preferred embodiment of the present invention, eliminates objects surrounded by clutter and recognizes that most real obstacles are small and surrounded by intensities very close to zero.
  • the context filter can be employed as a "first filter" indicated at 214 in Fig. 2). As shown in Fig. 6, it may preferably use two kernels of different radii centered on the same pixel ( Figure 6). The inner kernel (here, nine pixels shaded in black) is "positive" space, while the outer annulus (the remainder, or here the thirty-six pixels not shaded in black) surrounding the inner kernel is "negative" space. Intensities within the positive space are summed like an energy filter, while intensities from the negative space are subtracted. The total is then normalized by the number of pixels in the inner kernel:
  • S is the intensity
  • is the number of inner pixels
  • O is the set of outer pixels. If S is greater than zero, the center pixel is set to S, otherwise it is set to zero.
  • This filter distinguishes small objects surrounded by clear space and attenuates objects in close proximity with other objects.
  • a fixed threshold is preferably enforced to further bias the classifier away from false positives; with relation to the illustrative layout shown in Fig. 2 this could, for instance, be undertaken by second filter 218 and/or third filter 224 or, in an embodiment that does not involve branching as shown in Fig. 2, by essentially any filter that is downstream of first filter 214.
  • FIG. 7 graphically illustrates unprocessed and context filtered radar data from a desert scene in Nevada. This comparison demonstrates how obstacles of interest are preserved by this filtering while clutter is eliminated.
  • the context filter is extremely successful at detecting the obstacles in the context and scope of this research. It has very low rates of false positives because it automatically becomes less sensitive in high clutter areas.
  • this filter uses the context of an obstacle, not just its shape or intensity; the former is not recognizable by the low resolution beam and the latter is poorly correlated with obstacle danger.
  • An area with no radar backscatter return isn't always clear, because angled surfaces can be stealth objects or the vertically narrow beam may be aiming above obstacles.
  • Receiving even significant backscatter may mean nothing because it could come from grass, a gentle incline, or rough road surface.
  • a backscattering object surrounded immediately by zero-intensity bins almost always means a significant obstacle in the road that threatens the autonomous vehicle.
  • Calibration parameters define the coordinate transformation between the radar and the vehicle. While the translation of the antenna origin can be physically measured from the vehicle origin, the algorithm is more sensitive to rotation, which occasionally needs recalibration.
  • the Sandstorm vehicle (see, e.g., U.S. Provisional Application Serial No. 60/812,593, supra) poses problems because its vehicle coordinate origin is relative to its electronics enclosure, which is independently suspended from the vehicle chassis. Since the radar antenna is mounted on the chassis, any change to the rest position of the E-box (electronic box, or box containing electronic components) requires a recalibration of the radar. This is accomplished manually through trial and error, usually by driving up to a set of identifiable obstacles and correcting for any that are incorrectly localized.
  • the radar pipeline 210 preferably receives these parameters at runtime from the custom script language that controls it.
  • several other parameters are dictated by the hardware, like the angular width of azimuth bins. Radar devices may have slightly different scan rates and require individually tuned values. Since these values are passed at runtime, there is flexibility to change hardware during testing.
  • navigation software typically polls perception routines like the radar module 230 for maps at a higher rate than the radar obstacle classifier refreshes, so persistence of obstacles is desirable.
  • an obstacle is not always visible in every sweep, instead appearing or disappearing as the vehicle approaches the obstacle. Because of this, a method of maintaining memory of obstacles is very desirable. This cannot be a rigid algorithm recording the location of all previous obstacles, however, because their position and size is refined as the vehicle gets closer.
  • the angular resolution of the radar corresponds to several pixels in the rectangular space of the planner due to magnification.
  • a single fencepost may appear as a several-pixel- wide object of 2 meters width or more.
  • the reported obstacle width will narrow to the correct location.
  • Figure 8 In the left image of Fig. 8 the robot is about 30m from a magnified fence post. In the right image, the robot has approached to about 10m and the object's location and shape is refined. Therefore, the memory should preferably have some flexibility to clear previously reported obstacles in the case of new, better information.
  • physics-based hysteresis takes advantage of the known physics of the context filter. Any obstacle that survives the context filter is surrounded by empty space by definition. Therefore, when an obstacle is reported, any pre-existing obstacle within a certain radius of it can be removed from memory. If a prior obstacle was reported somewhere and a new one appears very close to it, the algorithm assumes the old obstacle was misplaced or its size was overrepresented and uses the more recent information. [85] Because the area around the filter is expressed in polar coordinates, the clearing of this area must also be described by polar coordinates.
  • a blob-based hysteresis method as broadly contemplated herein in accordance with at least one presently preferred embodiment of the present invention, and that operates solely in the Cartesian space of the obstacle map in memory, solves the aliasing problem. Testing demonstrates that localizing errors from the obstacle classifier are not significant. Obstacles do not appear to shift as the vehicle gets closer; they only get smaller as they are better localized. Therefore, an obstacle being added to the map for a second time should be contiguous with at least some part of the previous record of that obstacle.
  • a blob-based hysteresis module 234 will check the location of a newly reported obstacle and remove any previous obstacle blob at that position, before filling in the new size information.
  • the algorithm involved may initially recursively search the 24 neighboring pixels of a new obstacle pixel for non-zero values. If any are found, they are set to zero and the surrounding 8 neighbors are recursively searched until the entire contiguous blob is removed.
  • This method acts as a blob tracker and therefore is only capable of eliminating an old obstacle to replace it with a new one at the same location. It maintains memory of all obstacles while simultaneously refining the size and location of an object as the vehicle approaches and more information is available.
  • FMCW radars measure distance to a target using the backscatter returns' time- of-flight, ⁇ ; . This is accomplished by modulating the frequency of the transmitted signal in a sawtooth wave, as graphically illustrated in Figure 9. In this way, the frequency of the transmission is indexed by time (Equation 1, below). When the signal returns to the radar, an FFT is performed to extract the frequency. With the frequency of the signal known, the time of transmission can be determined. Using the speed of light, the distance of a backscattering object can be calculated from this time-of-flight (Equation 2, below).
  • the present invention in accordance with at least one presently preferred embodiment, indeed improves significantly upon conventional arrangements and affords obstacle detection and evasion, as well as reliable radar image updating, that contribute to much more efficient and effective operation of an autonomous vehicle.
  • the intensity of radar backscatter returns is generally a poor indicator of danger to a vehicle.
  • Methods as broadly contemplated herein provide favorable counterexamples.
  • An image of backscatter intensities is filtered with various image processing techniques then is thresholded as if it has become an image of risks or confidences.
  • Derivation research investigates discriminant functions that allow arbitrary numbers of classes and can use separability and discriminability as confidences instead of a filtered version of intensity.
  • the approaches discussed and contemplated herein in accordance with at least one embodiment of the present invention can be embodied as an add-on sensor to an already operable autonomous vehicle. As such, it might not be as viable if used as a primary or stand-alone sensor in an unstructured environment. Radar techniques to detect road edges [10] [15] or terrain quality would fill these gaps and may allow a radar-only, all-weather autonomous platform.

Abstract

An arrangement for obstacle detection in autonomous vehicles wherein two significant data manipulations are employed in order to provide a more accurate read of potential obstacles and thus contribute to more efficient and effective operation of an autonomous vehicle. A first data manipulation involves distinguishing between those potential obstacles that are surrounded by significant background scatter in a radar diagram and those that are not, wherein the latter are more likely to represent binary obstacles that are to be avoided. A second data manipulation involves updating a radar image to the extent possible as an object comes into closer range. Preferably, the first aforementioned data manipulation may be performed via context filtering, while the second aforementioned data manipulation may be performed via blob-based hysteresis.

Description

OBSTACLE DETECTION ARRANGEMENTS IN AND FOR AUTONOMOUS VEHICLES
CROSS-REFERENCE TO RELATED APPLICATION
[1] This application claims the benefit under 35 U.S.C. §1 19(e) of the earlier filing date of U.S. Provisional Application Serial Number 60/812,693 filed on June 9, 2006, which is hereby incorporated by reference.
BACKGROUND OF THE INVENTION
FIELD OF THE INVENTION
[ 2 ] The present invention relates generally to methods, systems, and apparatus for the autonomous navigation of terrain by a robot, and in particular to arrangements and processes for discerning and detecting obstacles to be avoided.
DESCRIPTION OF THE BACKGROUND
13 ] Herebelow, numerals presented in brackets - [ ] - refer to the list of references found towards the close of the instant disclosure.
[4 ] Autonomous (or, "self-guided" or "robotic") vehicles (e.g., cars, trucks, tanks,
"Humvees", other military vehicles) have been in development for several years, and continued refinements and improvements have lent great promise to a large number of military and non-military applications.
[ 5 ] One perennial challenge addressed in the development of autonomous vehicles lies in the mechanics of self-guiding and navigating and, more particularly, in the avoidance of obstacles, or the detection of obstacles and prompting of corrective action (e.g., swerving). Obstacles, as such, can take on a variety of forms, some lethal and some not. Those obstacles which are to be avoided at all costs are termed "binary obstacles". In the case of military applications, such obstacles could be in the form of a tank trap, a tank or vehicle barrier, telephone poles, large boulders, or other sizeable items that would readily compromise or inhibit a sufficiently free and smooth passage of the vehicle. In civilian applications, and especially in the context of smaller vehicles, binary obstacles would clearly include those of a scale just mentioned, but could also include smaller items such as cars, bicycles, pedestrians, animals and relatively small objects that yet could cause problems if struck or run over.
[ 63 Over the years, millimeter wave radar has emerged as a technology well- matched to outdoor vehicle navigation. It sees through dust and rain, doesn't depend on lighting, senses over a useful range, and can be cheap to mass- produce. Car manufacturers have successfully used radar for adaptive cruise control (ACC) and now offer them as options on luxury models and trucks [1] [2] [3]. Adaptations for autonomous vehicle navigation through unstructured terrain, however, have been much less successful for a variety of less-publicized weaknesses associated with radar [4].
[7 ] Radar is particularly good at detecting binary obstacles in the road, so this approach leaves the problem of identifying road edges and rough areas to other sensors, like LIDAR. LIDAR can thus address terrain challenges rather well, but leaves some concern about detecting all binary obstacles at the ranges sufficient to ensure the vehicle's avoidance thereof.
[ 8 ] Challenges of obstacle detection and avoidance certainly vary; a two-part problem thus arises by way of finding the rough parts of the terrain that should be avoided but may not be catastrophic, and finding binary obstacles that cannot be hit at any cost. 19 ] Terrain can be identified with an estimate of the risk or cost associated with its traversal, while obstacles that must be avoided are assigned maximum cost and termed binary obstacles, because they either exist or don't exist. Some binary obstacles are indigenous, like telephone poles, fence posts, cattle gaps, and rocks; others might be spontaneously introduced by people, like traffic barriers and other vehicles and steel hedgehog-style tank traps. The challenge for sensors is to identify these obstacles consistently at long ranges with low numbers of false positives.
[ 10 ] Prior attempts at radar sensing have faced several major hurdles. The low angular resolution, typically 1° to 2°, prevents shape identification of small obstacles. Only minimum data is observed, such as polarization, phase shift, and intensity of backscatter returns. Methods using electromagnetic effects like polarization to discriminate between soft and hard or horizontal and vertical objects can be confused in an object-rich environment like a desert road [5][6]. This leaves the intensity of backscatter returns, binned by range (linear distance from the radar antenna to an object) and azimuth (horizontal rotational angle of the antenna, from 0 to 360 degrees) as a sole, and usually inadequate, identifier.
[11] Several previous efforts to address this problem have used fixed thresholding
[7] [8] or constant false alarm rate (CFAR) thresholding [9] on the backscatter intensity data. It has been found that such methods are of marginal benefit at best on well-maintained highways and wholly insufficient for off-highway driving. The use of radar in autonomous vehicles to sense the environment has thus been generally limited to very structured environments like container storage areas at port facilities [10] or identifying clear obstacles on open, level ground [H]. For mainstream civilian use, thresholds are generally set at the size of a motorcyclist, or the smallest obstacle of concern on a highway, while hazardous desert passages present many dangers with smaller radar cross- sections that are not readily detected or addressed with conventional equipment.
[ 12 ] There is also the challenge of adequately discerning benign obstacles that do not need to be averted. While many obstacles have surfaces that reflect energy away from the radar antenna, returning very little backscatter, objects that pose little risk to a vehicle, such as brush, gentle inclines, and small rocks can have very large radar cross-sections and then return false positives. An insignificant object like a small rock, pothole, or bush may even have greater intensity than a guardrail, telephone pole, or fence post. Thus, the intensity of backscatter returns is a poor direct measure of the risk posed by an object.
[13] In view of the foregoing, a major need has been recognized in connection with implementing an arrangement for providing obstacle detection in autonomous vehicles that overcomes the shortcomings and disadvantages of prior efforts.
SUMMARY OF THE INVENTION
[ 14 ] In accordance with at least one presently preferred embodiment of the present invention, there is broadly contemplated herein an arrangement for obstacle detection in autonomous vehicles wherein two significant data manipulations are employed in order to provide a more accurate read of potential obstacles and thus contribute to more efficient and effective operation of an autonomous vehicle. A first data manipulation involves distinguishing between those potential obstacles that are surrounded by significant background scatter in a radar diagram and those that are not, wherein the latter are more likely to represent binary obstacles that are to be avoided. A second data manipulation involves updating a radar image to the extent possible as an object comes into closer range. [ 15 ] Preferably, the first aforementioned data manipulation may be performed via context filtering, while the second aforementioned data manipulation may be performed via blob-based hysteresis.
[16] Generally, there is broadly contemplated in accordance with at least one presently preferred embodiment of the present invention, a method of providing obstacle detection in an autonomous vehicle, the method comprising the steps of: obtaining a radar diagram; discerning at least one prospective obstacle in the radar diagram; ascertaining background scatter about the at least one prospective obstacle; classifying the at least one prospective obstacle in relation to the ascertained background scatter; and refining the radar diagram and reevaluating the at least one prospective obstacle; the reevaluating comprising repeating the steps of ascertaining and classifying.
[ 17 ] Further, there is broadly contemplated herein, in accordance with at least one presently preferred embodiment of the present invention, a system for providing obstacle detection in an autonomous vehicle, the system comprising: an arrangement for discerning at least one prospective obstacle in a radar diagram; an arrangement for ascertaining background scatter about the at least one prospective obstacle; an arrangement for classifying the at least one prospective obstacle in relation to the ascertained background scatter; and an arrangement for refining the radar diagram and reevaluating the at least one prospective obstacle; the refining and reevaluating arrangement acting to prompt a repeat of ascertaining background scatter about the at least one prospective obstacle and classifying the at least one prospective obstacle in relation to the ascertained background scatter.
[18] The novel features which are considered characteristic of the present invention are set forth herebelow. The invention itself, however, both as to its construction and its method of operation, together with additional objects and advantages thereof, will be best understood from the following description of the specific embodiments when read and understood in connection with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[19] The patent or application file contains at least one drawing executed in color.
Copies of this patent or patent application publication with color drawing(s) will be provided by the U.S. Patent and Trademark Office upon request and payment of the necessary fee.
[20] For the present invention to be clearly understood and readily practiced, the present invention will be described in conjunction with the following figures, wherein like reference characters designate the same or similar elements, which figures are incorporated into and constitute a part of the specification, wherein:
[21] Figure 1 shows a schematic of an overall architecture of navigation software in which at least one presently preferred embodiment of the present invention may be employed;
[22 ] Figure 2 schematically illustrates a processing pathway of a radar obstacle detection method.
[23] Figure 3 illustrates returns from an exemplary 180 degree radar sweep to a 75m range.
[24] Figure 4 shows the application of an energy filter to radar data.
[25] Figure 5 shows backscatter returns from a 30 gallon plastic trash can.
[26] Figure 6 graphically illustrates a kernel mask that may be employed during context filtering.
[27] Figure 7 graphically provides a side-by-side comparison of unprocessed and context-filtered radar data from a desert site. [28] Figure 8 provides a side-by-side comparison of a successive images of an obstacle refined by blob-based hysteresis.
[29] Figure 9 graphically illustrates time indexing in a FMCW radar.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS OF THE
PRESENT INVENTION
[30] It is to be understood that the figures and descriptions of the present invention have been simplified to illustrate elements that are relevant for a clear understanding of the invention, while eliminating, for purposes of clarity, other elements that may be well known. The detailed description will be provided herebelow with reference to the attached drawings.
[31] Hereby fully incorporated by reference, as if set forth in their entirety herein, are the copending and commonly assigned U.S. Patent Applications filed on even date herewith entitled "SOFTWARE ARCHITECTURE FOR HIGH-SPEED TRAVERSAL OF PRESCRIBED ROUTES" (inventors William Whittaker, Kevin Peterson, Chris Urmson) and "SYSTEM AND METHOD FOR AUTONOMOUSLY CONVOYING VEHICLES" (inventors Chris Urmson, William Whittaker, Kevin Peterson). These related applications disclose systems, arrangements and processes in the realm of autonomous vehicles that may be freely incorporable with one or more embodiments of the present invention and/or represent one or more contextual environments in which at least one embodiment of the present invention may be employed. These related applications may also readily be relied upon for a better understanding of basic technological concepts relating to the embodiments of the present invention.
[32] In the following description of embodiments of the present invention, the term
"autonomous" is used to indicate operation which is completely automatic or substantially automatic, that is, without significant human involvement in the operation. An autonomous vehicle will generally be unmanned, that is without a human pilot, or co-pilot. However, an autonomous vehicle may be driven or otherwise operated automatically, and have one or more human passengers. An autonomous vehicle may be adapted to operate under human control in a non- autonomous mode of operation.
[33] As used herein, "vehicle" refers to any self-propelled conveyance. In at least one embodiment, the description of the present invention will be undertaken with respect to vehicles that are automobiles. However, the use of that exemplary vehicle and environment in the description should not be construed as limiting. Indeed, the methods, systems, and apparatuses of the present invention may be implemented in a variety of circumstances. For example, the embodiments of the present invention may be useful for farming equipment, earth moving equipment, seaborne vehicles, and other vehicles that need to autonomously generate a path to navigate an environment.
[34] Fig. 1 shows a schematic of an overall architecture of navigation software in which at least one presently preferred embodiment of the present invention may be employed. A further appreciation of specific components forming such an architecture, as may be employed as an illustrative yet non-restrictive environment for at least one presently preferred embodiment of the present invention, may be gleaned from "SOFTWARE ARCHITECTURE FOR HIGHSPEED TRAVERSAL OF PRESCRIBED ROUTES", supra. As such, a radar 202 and binary detection arrangement (comprising, preferably, a pipeline 210 and radar module 230 as discussed herebelow) can preferably be integrated into a navigation architecture as shown in Fig. 1 and advantageously provide radar- based obstacle detection in such a context in a manner that can be more fully appreciated herebelow.
[35] Fig. 2 broadly illustrates a processing pathway of a radar obstacle detection arrangement 200, and associated method, in accordance with a presently preferred embodiment of the present invention. Reference to Fig. 2 will continue to be made throughout the instant disclosure as needed.
[36] Most generally, in an autonomous vehicle in accordance with at least one presently preferred embodiment of the present invention, a radar arrangement 202 transmits and receives radar energy (207) in a general sweep (to be better appreciated further below) and as such will rebound from one obstacle 208 after another. Data is then binned (209) by range and azimuth and fed to a "radar pipeline" 210, also to be better appreciated further below, that context-filters data in accordance with at least one particularly preferred embodiment of the present invention before proceeding (229) to a radar module 230. At radar module 230, data is preferably transformed from polar coordinates to rectangular coordinates (relative to the vehicle in question) before undergoing, in accordance with at least one particular preferred embodiment of the present invention, an updating and refinement (to the extent possible in view of time and range constraints) via blob-based hysteresis before being transmitted (238) to a remainder of a general navigation system (such as that discussed and illustrated herein with respect to Fig. 1.
[37] By way of some general considerations of relevance to at least one embodiment of the present invention, it is to be noted that when working with radar the energy of a 3D wave emitted from a point source decays as 1/R2. Radiation emitted by the antenna, reflected by a target, then returned to the receiver decays by a factor of 1/R . This means that an object at close range will have a much greater backscatter than the same object at far range. The radar antenna can compensate for this internally by multiplying by a regression-fitted R function so it reports range-invariant intensities. An object at close range will thus have the same intensity output value as it does at far range. While this solves several problems, it also increases noise at the greater ranges.
[38] The radar indicated at 202 can be embodied by essentially any suitable equipment; a very good illustrative and non-restrictive example would be a Navtech DSC2000 77 GHz Frequency Modulating Continuous Wave (FMCW) radar. An advantage of such equipment is that it provides the capability of applying (206) a Fast Fourier Transform (FFT) to data received by an antenna 204, whereupon the data can be binned by an onboard DSP, thus permitting the data to become available over Ethernet. In accordance with an illustrative and non-restrictive example, and as will be appreciated in discussions of working examples herein, the data output from such a radar 202 is expressed as intensity of backscatter in bins measuring 1.2 degrees in azimuth by 0.25 m in range. Such a radar provides a vertical beam width of 4 degrees with a scan rate (i.e., rotational velocity) of 360 degrees in 0.4 seconds.
[39] Preferably, data output from radar 208 is in the form of individual packets each containing a radar vector which is an 800-member vector of discretized intensities. By way of an illustrative and non-restrictive example (and as observed with an antenna from the Navtech radar mentioned above), values can range from 0 (minimum intensity) to 144 (maximum intensity) and be indexed by range in 0.25 m increments from 0 to 200 meters. Each radar vector also preferably is recorded with an azimuth at 0.1 degree precision. A full 360 degree sweep can thus include about 310 radar vectors at 1.2 degree separation. Because, as is normally the case, antenna 204 is spun at an imprecisely controlled rate and the samples are timed at 1 ms separation, the exact number of radar vectors in a sweep is not necessarily guaranteed. Additionally, the azimuth direction of radar vectors can of course vary slightly between sweeps, so the unique azimuth of any given radar vector should preferably be recorded.
[40] Inasmuch as objects behind the vehicle need not be considered in most practical applications, the half of the radar field of view facing "backward" can readily be eliminated, only those objects in the 180 degree arc from "straight left" to "straight right" in front of the sensor need be analyzed. The antenna 204 can thus be scanned at a steady rate, with about 0.2 seconds between when the left side data and right side data are recorded during a single sweep. Preferably, radar vectors will be individually time-stamped with their arrival times to ensure that proper vehicle poses are retrievable. Inasmuch as a noticeable increase in FFT noise at 75 meters has been observed, which at any rate represents a still considerable range within which to adequately navigate and detect and avoid obstacles at speeds of up to about 20 meters per second, it is certainly conceivable to consider ranges solely of less than 75 meters, whereby radar returns will more or less be noise-free.
[41] After retrieval from the radar antenna 204, radar vectors are preferably collected into a radar image, which is a wrapper for a matrix that indexes intensity by azimuth and range. This radar image can be expressed as a 180 degree view of radar backscatter intensities, an example of which is shown in Figure 3. More particularly, shown in Figure 3 is an exemplary 180 degree radar sweep to a 75m range with the Navtech equipment discussed above. Green represents areas of low backscatter, while red areas darken in proportion to the strength of backscatter returns. As will be appreciated herebelow, supported radar image functions include the ability to form windowed iterators and writing to image file or over Ethernet.
[42] Referring again to Fig. 2, the aforementioned radar images preferably pass through a software pipeline 210 that performs context-filtering operations on the data, the result of which will be more fully appreciated herebelow. As such, the pipeline 210 can preferably contain several software classes, including a reader 212, first filter 214, branch 216, followed in parallel by (a) a second filter 218 and a writer 220 to file (222) and (b) a third filter 224 and a writer 226 to Ethernet. A regulator 228 governing writers 220/226 is also preferably included. Overall, pipeline 210 is preferably configured at runtime with a custom scripting language.
[43] Reader 212 will preferably receive radar vectors from the radar 202 over
Ethernet and form the radar images. Writers preferably transmit data to file, shared memory, or over Ethernet; here, writers 220/226 are shown as writing to file and transmitting to Ethernet, respectively. In between reader 212 and writers 220/226 are configurable filters 214/218/224 as shown that can be ordered via the scripting language. All of these classes derive from the Pipe class, which contains a pointer to the previous Pipe in the Pipeline 210 (Figure 2). Branching is also supported (216), allowing multiple filtering methods on the same data or multiple output formats. Of all the class types- Reader, Filter, Writer, Branch, Regulator- only the reader 212 need be radar hardware specific. Thus, if there is a need to support a different radar antenna (e.g., a different azimuth-sweeping radar antenna), only the reader 212 would need to be modified. First filter 214 will preferably undertake the "context filtering" as broadly understood herein (and as described in more detail herebelow in accordance with at least one preferred embodiment) while second filter 218 and/or third filter 224 can undertake secondary filtering operations such as additional thresholding (e.g., to further increase the likelihood of avoiding false positives).
[44 ] To ensure that processing occurs in real time without overwhelming pipeline 210 with a "logjam" of several radar sweeps at the same time, regulator 229 can preferably apply a scheme to avoid or obviate such a contingency. In such a scheme, for instance, a single radar can be passed "backward" (i.e., towards reader 212) from any "final" element of the pipeline 210 (e.g., either of the writers 220/226). Here, the filters 214/218/224 would not act on the data represented by the image. When that radar image reaches the beginning of the series, the reader 212 can fill it with radar vectors and send it back "forward" through the pipeline 210 (i.e., towards writers 220/226), whereupon the filters 214/218/224 would actually act on the data.
[45] A pipeline 210 in accordance with at least one embodiment of the present invention will preferably afford some flexibility such that, e.g., filters can be dropped in and out or be reconfigured at runtime. The radar antenna 202 can also be replaced with required changes isolated to only one sub-class (i.e., reader class). Finally, the implementation of the radar image wrapper class can be completely reworked without affecting the filter processes. The radar image can be defined as a matrix of Cartesian coordinates, a lookup table of Cartesian coordinates for a polar coordinate storage structure, and directly as a polar coordinate structure. The pipe classes need not be changed to support such modifications.
[46] The radar method up to this point is modular, self-contained, and can operate freestanding. Its output (shown here on the output from writer 226) includes the location of obstacles indexed by azimuth and range and marked with data collection time stamps. To transform these values into a real world location of an obstacle, however, requires knowledge of the position antenna 204 at the time of each data collection. On most robots, this information is available to the sensor from the vehicle's estimation of position, orientation in 6 DOF (Degrees of Freedom). To utilize these resources without losing the modularity of the Pipeline approach, an additional radar module (230) process is preferably added that functionally follows pipeline 210.
[47] The radar module 230 preferably references the vehicle pose history, and therefore is not stand-alone. It receives input from the end of the pipeline 210 over Ethernet. This input data includes the azimuth and range (relative to the sensor at a collection timestamp) of location bins containing obstacles. These obstacles preferably are reported with binary confidence and are not further classified. Rather, radar module 230 is preferably configured to convert (232) the data from polar coordinates (azimuth range) relative to the sensor to a latitude/longitude location on the Earth's surface. The output 233 of conversion 232 thus preferably takes the form of a rectangular map, e.g., 100 meters by 100 meters, centered on the vehicle at an arbitrary time.
[48] As such, each obstacle pixel preferably ends up being converted to Cartesian coordinates, then transformed (236) by a calibration file and the vehicle pose to determine its location in the map. If an object has not already been reported at that location, it is added, and the map is forwarded (238) to the robot's navigation and planning algorithms (see Fig. 1). Prior to the mapping step (236), an updating of data via blob-based hysteresis (234) preferably takes place, to be better understood herebelow.
[49] The disclosure now continues with a more detailed discussion of context filtering and blob-based hysteresis, and the advantages of both as compared with other conceivable implementations. Reference should continue to be made to Fig. 2, in addition to other Figures mentioned herebelow.
[50] Generally speaking, a significant advantage enjoyed in accordance with at least one embodiment of the present invention is the rendering of a priori assumptions that are intuitive after posing radar data in image format. Existing filtering and classification methods can essentially be borrowed from camera-based image processing to identify obstacles representing a significant risk to a vehicle while not reporting false positives that are actually traversable. However, radar still reports little except range to objects, providing very few discriminable features.
[51] With 2D intensity information alone, there is little that is inherently different between the backscatter returns from a rut and those from a telephone pole. The former presents no problem to a large vehicle while driving into the latter would be disastrous. In short, while there is very little noise in a radar image, the signal to clutter ratio is extremely low in unstructured environments. Clutter produces high backscatter returns but is not dangerous to a HMMWV. An important challenge is thus to remove clutter and avoid false positives.
[52] In analogous sensing modalities, additional information is gleaned to eliminate clutter. LIDAR produces detailed geometry from which shape and roughness can be extracted. Stereovision and visual segmentation allow separation of objects that stick up sharply from the smooth road. Available radar antennas have too low a resolution for shape identification and have too great a vertical beam width to produce height maps.
[53] Radar images from a 77GHz antenna have several important characteristics.
Because the data is collected in polar format, the lateral resolution is higher close to the antenna than farther away. Roads, smooth building walls, and other planar surfaces reflect very little energy toward the radar unless their normal vector points back at the antenna. Internal angles, like the corner between two walls and a ceiling reflect strongly. Rough surfaces like grass, brush, brick walls, and plastics are less directionally dependent and produce moderate backscatter returns, which can create false positives.
[54] There are as many exceptions as rules, however. A street gutter is a non-oblique angle in only two directions, so it may return very little energy, while a drain in the gutter could become visible. A road may have few returns until it goes uphill and faces the antenna a little more directly. Especially with rough surfaces and grasses, this undesired ground return can be a significant source of false positives.
[55] Fixed thresholding is very common in previous research on radar navigation.
The advantage to this approach is that data can be processed instantaneously as it arrives from the antenna, rather than being formed into a 2D image. Thresholding represents an effort to connect intensity of backscatter to the risk associated with hitting an object. High intensity means more danger, and vice versa. Unfortunately, these properties are actually poorly related.
[56] Many objects have strong returns in the 77 GHz range but do not post a significant obstacle to automotive vehicles. Brush, grass, small ruts, gradual inclines, and other features show strongly with high intensity backscatter returns but are easily traversable by automotive vehicles. Conversely, many potentially dangerous metal objects are only visible in backscatter returns from certain angles. Specific "stealth" examples are highway signs and hedgehog tank traps that return very low intensities from most angles but can be very dangerous. [57] While fixed threshold methods have led to reports of success in structured environments, extended testing in off-highway conditions revealed a large number of false positives caused by vegetation, rough roads, and gentle hills. Setting the threshold high enough to avoid most false positives also meant only the largest metal objects with internal angles (like automobiles) are reported, and even recognition of these objects is not perfect.
[58] Energy filtering, on the other hand, is effected by convolving a rectangular kernel with the image. At every pixel, the intensities within the kernel were summed and if they pass a threshold, the pixel was classified as an obstacle. The energy filter works very well at detecting road edges like gutters and berms (common on dirt roads) and larger obstacles like cars and buildings. Unfortunately, it also has a lot of false positives.
[59] Large areas of low intensity, which typically include grass and uphill road sections, tend to be falsely reported as obstacles using this filter. While a car or building wall will typically be splotchy- strong returns mixing with near zero returns from angled surfaces or shadows- even mown lawn grass may produce steady returns over its whole area. For this reason, measuring energy in a region was a poor discriminator of obstacles in the scope of this research.
[60] Figure 4 shows the application of an energy filter to radar data, and highlights the disadvantages of this approach. The image on the left shows unprocessed radar data, where red corresponds to high backscatter returns. The right image is the same data processed with an energy filter. Green is safe to drive, red represents identified obstacles, white is missed obstacles, and black is false positives. Most of the black region (false positives) is mown grass, which is easily traversed by a HMMWV. [61] The failure of more straightforward methods suggests making use of more sophisticated models of a priori data. Since a sensor in accordance with at least one preferred embodiment of the present invention can support an already navigable vehicle, it might be possible to turn off the radar in situations where the previous methods are known to be untrustworthy. Then, obstacles will only be reported in areas of high confidence, reducing correct detections, but potentially reducing false positives to acceptable levels.
[62] Data collection in the Nevada desert has indicated that native vegetation tends to occur in clusters, rather than small, isolated stands. For instance, it is likely that an area either has a lot of sagebrush growing close together, or only possesses low grass and roadway. Most desert terrain either contains a very large or very small amount of clutter, without much of a middle ground.
[63] Observations indicated that energy and thresholding methods worked well in these areas of low clutter, so if they could be identified, the results from this filtering would be acceptable. An algorithm to take advantage of this characteristic should devalue confidences in regions containing large amounts of clutter.
[64] The global clutter filter creates a histogram of all values in the radar image. It then selects the intensity at a percentile selected as a parameter and subtracts that intensity from all values in the image. This was optimized by experimentation at the 80th percentile. If an image is 80% empty of backscatter, a common occurrence in low-clutter regions, this filter will have no effect on the raw data. In areas where more than 20% of the image contains significant backscatter, all returns are decreased. This effectively increases the burden of proof for the next stage of filtering. [65] Combining the global clutter filter with the energy or threshold filters is sufficient to identify many obstacles with few false positives. This method is robust to areas of high clutter and works well at identifying obstacles in otherwise clear areas. Because it only identifies obstacles in low-density environments, however, it misses true obstacles in clutter-filled regions.
[66] Local clutter filtering is another way to reduce confidence in the presence of clutter, but considers a reduced scope. A window centered on a pixel produces a histogram of pixel intensities within that window. The intensity at a particular percentile is subtracted from the pixel at the window's center. Therefore, this is the same algorithm as the global clutter filter, but is applied only on the area immediately surrounding a pixel. This approach produces limited success.
[67] Since the antenna is a physical device with a Gaussian distribution of beam intensity in the azimuth direction, the beam width is not discrete. In fact, the cited 1.2° beam width is the half-power width. This means that a strongly reflecting object, even if it is small enough to fit into one azimuth bin, will "bleed" into the surrounding azimuth bins. Similarly, since the range measurement is a binned result from a continuous FFT, a reflecting object will also bleed intensity into the surrounding range bins. Therefore, a single point object like a fencepost or barrel will actually show intensity in at least 9 bins, and the edges of all objects will be fuzzy.
[68] This phenomenon is clearly evident in Fig. 5, which shows the backscatter returns from a 30 gallon plastic trash can. The shape of the radar beam results in "bleeding" from the object into surrounding pixels causing a fuzzy appearance. A pixel with intensity of any real consequence is always surrounded by several pixels of non-zero intensity because of this bleeding effect. This overpowers the local clutter filter, because the bleeding is often the most significant source of non-zero intensity (clutter) in the histogram. The need to assemble and sort histograms at each pixel is also computationally intensive and difficult to manage in the real time required for high-speed driving.
[69] Local clutter filtering is desirable, but the fuzzy edges of intensity blobs prevent its implementation as described. However, the robot only requires radar to detect obstacles in the road, not those off in the vegetation. Most forms of clutter, like vegetation, road inclines, and rough surfaces appear in bunches, while most manmade obstacles like fence posts, and telephone poles are isolated and surrounded by road or clear dirt. Even larger objects like Jersey barriers and buildings don't normally reflect back to the transmitter except at breaks like connections between concrete sections or windows. These obstacles all follow a pattern of non-backscattering bins surrounding a small set of high-intensity pixels.
[70] A context filter, as implemented and employed in accordance with at least one presently preferred embodiment of the present invention, eliminates objects surrounded by clutter and recognizes that most real obstacles are small and surrounded by intensities very close to zero. (In accordance with an illustrative embodiment of the present invention, the context filter can be employed as a "first filter" indicated at 214 in Fig. 2). As shown in Fig. 6, it may preferably use two kernels of different radii centered on the same pixel (Figure 6). The inner kernel (here, nine pixels shaded in black) is "positive" space, while the outer annulus (the remainder, or here the thirty-six pixels not shaded in black) surrounding the inner kernel is "negative" space. Intensities within the positive space are summed like an energy filter, while intensities from the negative space are subtracted. The total is then normalized by the number of pixels in the inner kernel:
Figure imgf000022_0001
[72] where S is the intensity, |/| is the number of inner pixels, and O is the set of outer pixels. If S is greater than zero, the center pixel is set to S, otherwise it is set to zero.
[73] This filter distinguishes small objects surrounded by clear space and attenuates objects in close proximity with other objects. As a final step, a fixed threshold is preferably enforced to further bias the classifier away from false positives; with relation to the illustrative layout shown in Fig. 2 this could, for instance, be undertaken by second filter 218 and/or third filter 224 or, in an embodiment that does not involve branching as shown in Fig. 2, by essentially any filter that is downstream of first filter 214.
[74] With the filter implemented in this way, there can still be false positives on very smooth driving surfaces. Some minor rocks or rough patches slip through because the surrounding asphalt returns virtually no backscatter to the antenna. Only considering center pixels with intensity greater than an initial threshold eliminates the smooth asphalt false positives. Because so much of an image is blank, this also eliminates the significant majority of the processing requirements by not processing pixels that clearly don't contain obstacles.
[75] Figure 7 graphically illustrates unprocessed and context filtered radar data from a desert scene in Nevada. This comparison demonstrates how obstacles of interest are preserved by this filtering while clutter is eliminated. As shown in Fig. 7, the context filter is extremely successful at detecting the obstacles in the context and scope of this research. It has very low rates of false positives because it automatically becomes less sensitive in high clutter areas. By its design, this filter uses the context of an obstacle, not just its shape or intensity; the former is not recognizable by the low resolution beam and the latter is poorly correlated with obstacle danger. An area with no radar backscatter return isn't always clear, because angled surfaces can be stealth objects or the vertically narrow beam may be aiming above obstacles. Receiving even significant backscatter may mean nothing because it could come from grass, a gentle incline, or rough road surface. A backscattering object surrounded immediately by zero-intensity bins, however, almost always means a significant obstacle in the road that threatens the autonomous vehicle.
[76] Calibration parameters define the coordinate transformation between the radar and the vehicle. While the translation of the antenna origin can be physically measured from the vehicle origin, the algorithm is more sensitive to rotation, which occasionally needs recalibration. The Sandstorm vehicle (see, e.g., U.S. Provisional Application Serial No. 60/812,593, supra) poses problems because its vehicle coordinate origin is relative to its electronics enclosure, which is independently suspended from the vehicle chassis. Since the radar antenna is mounted on the chassis, any change to the rest position of the E-box (electronic box, or box containing electronic components) requires a recalibration of the radar. This is accomplished manually through trial and error, usually by driving up to a set of identifiable obstacles and correcting for any that are incorrectly localized.
[77] It is more difficult to tune the parameters dictating the context filter properties.
Using a learning algorithm is difficult because hand-classifying the required number of objects is not feasible. Finding fully representative training data is also a daunting task. Therefore, these parameters were also developed through experience. [78] The two threshold parameters, exercised before and after the kernel convolution, were initiated at values as low as possible to maximize the number of correct detections. If too many false positives were observed, these values were increased.
[79] Setting the radii of the two kernels is a more complicated problem. The inner radius dictates the maximum size of the obstacles that will be reported, while the outer radius determines how much clear space is required around an obstacle. They interact, however, such that the ratio of negative (outer) space to positive (inner) space also has a strong effect. If the ratio is 1 : 1 and the positive space is at a higher intensity than the outer space, an obstacle is reported. If the ratio is greater than 1:1, the obstacle must be more intense than the surrounding space to result in a reported obstacle.
[80] Testing has shown that maximizing the outer radius relative to the inner radius maximizes the correct classification rates. Increasing the outer radius increases the amount of space required between obstacles, so there is a limit to how far this can be taken. Azimuth separations of 6° and range separations of 2.25m were enforced, while objects of 3° width and 0.75m depth were optimally preserved. The actual object may be a different size, however, because of bleeding or the fact that most objects don't reflect for their whole length, like a building only returning backscatter at irregularities like windows and doors.
[81] The radar pipeline 210 preferably receives these parameters at runtime from the custom script language that controls it. In addition to calibration and filter settings, several other parameters are dictated by the hardware, like the angular width of azimuth bins. Radar devices may have slightly different scan rates and require individually tuned values. Since these values are passed at runtime, there is flexibility to change hardware during testing. [82] Turning now to the updating of images via blob-based hysteresis, navigation software typically polls perception routines like the radar module 230 for maps at a higher rate than the radar obstacle classifier refreshes, so persistence of obstacles is desirable. Also, an obstacle is not always visible in every sweep, instead appearing or disappearing as the vehicle approaches the obstacle. Because of this, a method of maintaining memory of obstacles is very desirable. This cannot be a rigid algorithm recording the location of all previous obstacles, however, because their position and size is refined as the vehicle gets closer.
[83 ] At long range, the angular resolution of the radar corresponds to several pixels in the rectangular space of the planner due to magnification. At a range of 50 meters, a single fencepost may appear as a several-pixel- wide object of 2 meters width or more. As the vehicle approaches the post, the reported obstacle width will narrow to the correct location. This can be appreciated from a working example graphically illustrated in Figure 8. In the left image of Fig. 8 the robot is about 30m from a magnified fence post. In the right image, the robot has approached to about 10m and the object's location and shape is refined. Therefore, the memory should preferably have some flexibility to clear previously reported obstacles in the case of new, better information.
[84] In one conceivable implementation, physics-based hysteresis takes advantage of the known physics of the context filter. Any obstacle that survives the context filter is surrounded by empty space by definition. Therefore, when an obstacle is reported, any pre-existing obstacle within a certain radius of it can be removed from memory. If a prior obstacle was reported somewhere and a new one appears very close to it, the algorithm assumes the old obstacle was misplaced or its size was overrepresented and uses the more recent information. [85] Because the area around the filter is expressed in polar coordinates, the clearing of this area must also be described by polar coordinates. With the 5 azimuth and 9 range pixels used in testing that means 45 "empty" pixels must be transformed from polar coordinates in the sensor's frame to Cartesian coordinates in the vehicle's frame for every obstacle-containing pixel. This is a non-trivial increase in the complexity of the transformation processing.
[86] Unfortunately, physics-based hysteresis sometimes produces an aliasing effect.
When the polar point is transformed into the vehicle reference frame, it is binned into the 0.25m resolution of the map. At this stage, information is lost, and two objects may appear slightly closer or slightly further apart than they are in reality. Because of this, two obstacles that are just far enough apart to be identified can incorrectly clear each other in the map using this hysteresis. This was first observed in a scene setup with several small boxes spaced about 1.5m apart, close to the minimum range separation required to consider isolated obstacles. Some boxes were correctly identified but were later removed, creating the appearance of a clear path where there was none.
[87 ] A blob-based hysteresis method, as broadly contemplated herein in accordance with at least one presently preferred embodiment of the present invention, and that operates solely in the Cartesian space of the obstacle map in memory, solves the aliasing problem. Testing demonstrates that localizing errors from the obstacle classifier are not significant. Obstacles do not appear to shift as the vehicle gets closer; they only get smaller as they are better localized. Therefore, an obstacle being added to the map for a second time should be contiguous with at least some part of the previous record of that obstacle.
[88] Preferably, a blob-based hysteresis module 234 will check the location of a newly reported obstacle and remove any previous obstacle blob at that position, before filling in the new size information. Preferably, the algorithm involved may initially recursively search the 24 neighboring pixels of a new obstacle pixel for non-zero values. If any are found, they are set to zero and the surrounding 8 neighbors are recursively searched until the entire contiguous blob is removed.
[89] This method acts as a blob tracker and therefore is only capable of eliminating an old obstacle to replace it with a new one at the same location. It maintains memory of all obstacles while simultaneously refining the size and location of an object as the vehicle approaches and more information is available.
[90] The disclosure will now turn to a brief discussion of challenges with Doppler shifting and how such challenges might be addressed.
[91] FMCW radars measure distance to a target using the backscatter returns' time- of-flight, Δ; . This is accomplished by modulating the frequency of the transmitted signal in a sawtooth wave, as graphically illustrated in Figure 9. In this way, the frequency of the transmission is indexed by time (Equation 1, below). When the signal returns to the radar, an FFT is performed to extract the frequency. With the frequency of the signal known, the time of transmission can be determined. Using the speed of light, the distance of a backscattering object can be calculated from this time-of-flight (Equation 2, below).
Figure imgf000027_0001
[93] d = — c (2)
[94] With a moving target or moving platform, however, backscattered signals are
Doppler shifted. This means the frequency of a reflected signal is not the same as it was when transmitted, so the time indexing is incorrect. At the highest speeds driven by the robots during testing (about 20 m/s), this error in location (about 1.5 m) is enough to prevent obstacle avoidance and is certainly enough to keep a hysteresis algorithm from working properly. Fortunately, this Doppler shift behavior is a well understood problem (Equation 3) depending onλ , the wavelength, and v , the closing velocity.
[95] Λ = — Λ « (3)
" λ 16.5GHz
196] dcomcted = d_d +76.5 *lm**v (4)
[ 97 ] If the velocity of the object toward the radar antenna is known, the Doppler shift can be corrected manually using Equation 4. In the testing described in this paper, all objects in the environment are static, so the only consideration is the speed of the robot. This speed is available to the radar module 230 from the vehicle pose information.
[98] The Doppler shift is a significant problem that must be overcome to use FMCW radar in non-static environments. Two potential methods are increasing the rate of frequency modulation to limit the shift and using obstacle tracking to calculate the obstacles' velocity. Tracking would require an antenna with higher refresh rates and more processing power, however, so it is not apparent what the solution to this problem will be.
[99] It will be appreciated from the foregoing that the present invention, in accordance with at least one presently preferred embodiment, indeed improves significantly upon conventional arrangements and affords obstacle detection and evasion, as well as reliable radar image updating, that contribute to much more efficient and effective operation of an autonomous vehicle. In brief recapitulation, the intensity of radar backscatter returns is generally a poor indicator of danger to a vehicle. Methods as broadly contemplated herein provide favorable counterexamples. An image of backscatter intensities is filtered with various image processing techniques then is thresholded as if it has become an image of risks or confidences. Derivation research investigates discriminant functions that allow arbitrary numbers of classes and can use separability and discriminability as confidences instead of a filtered version of intensity.
[100] Preferably, the approaches discussed and contemplated herein in accordance with at least one embodiment of the present invention can be embodied as an add-on sensor to an already operable autonomous vehicle. As such, it might not be as viable if used as a primary or stand-alone sensor in an unstructured environment. Radar techniques to detect road edges [10] [15] or terrain quality would fill these gaps and may allow a radar-only, all-weather autonomous platform.
[101] Without further analysis, the foregoing will so fully reveal the gist of the present invention and its embodiments that others can, by applying current knowledge, readily adapt it for various applications without omitting features that, from the standpoint of prior art, fairly constitute characteristics of the generic or specific aspects of the present invention and its embodiments.
[102] If not otherwise stated herein, it may be assumed that all components and/or processes described heretofore may, if appropriate, be considered to be interchangeable with similar components and/or processes disclosed elsewhere in the specification, unless an express indication is made to the contrary.
[ 103 ] If not otherwise stated herein, it may be assumed that all components and/or processes described heretofore may, if appropriate, be considered to be interchangeable with similar components and/or processes disclosed elsewhere in the specification, unless an express indication is made to the contrary. [104] If not otherwise stated herein, any and all patents, patent publications, articles and other printed publications discussed or mentioned herein are hereby incorporated by reference as if set forth in their entirety herein.
[105] It should be appreciated that the apparatus and method of the present invention may be configured and conducted as appropriate for any context at hand. The embodiments described above are to be considered in all respects only as illustrative and not restrictive. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
References
[I] Jones, W., "Keeping Cars from Crashing", IEEE Spectrum, 2001, vol. 38 issue 9, pp. 40- 45.
[2] WoIl, J., "VORAD Collision Warning Radar", IEEE Intl. Radar Conference, 1995, pp. 369-372.
[3] Gern, A., Franke, U., Levi, P., "Advanced Lane Recognition- Fusing Vision and Radar", Proceedings of the IEEE Intelligent Vehicle Symposium, 2000, pp. 45-51.
[4] Wanielik, G., Appenrodt, N., Neef, H., Schneider, R., Wenger, J., "Polarimetric Millimeter Wave Imaging Radar and Traffic Scene Interpretation", IEEE Colloquium on Automotive Radar and Navigation Techniques, 1998, pp. 4/1-4/7.
[5] Clark, S., Dissanayake, G., "Simultaneous localisation and map building using millimetre wave radar to extract natural features", Proceedings of IEEE International Conference on Robotics and Automation, 1999, vol. 2, pp. 1316-1321.
[6] Currie, N., Brown, C, Principles and Applications of Millimeter- Wave Radar, Artech House, Boston, 1987.
[7] Foessel, A., Chheda, S., and Apostolopoulos, D. "Short-range millimeter-wave radar perception in a polar environment." Proceedings of the International Conference on Field and Service Robotics, 1999, pp. 133-138.
[8] Kaliyaperumal, K., Lakshmanan, S., Kluge, K., "An algorithm for detecting roads and obstacles in radar images", IEEE Transactions on Vehicle Technology, 2001, vol. 50, issue l, pp. 170-182.
[9] Ferri, M., Galati, G., Naldi, M., Patrizi, E., "CFAR techniques for millimetre-wave miniradar", CIE International Conference of Radar, 1996, pp. 262-265.
[10] Clark, S., Durrant-Whyte, H., "Autonomous land vehicle navigation using millimeter wave radar", Proceedings of IEEE International Conference on Robotics and Automation, 1998, vol. 4, pp. 3697-3702.
[I I] Ruff, T., "Application of Radar to Detect Pedestrian Workers Near Mining Equipment", Applied Occupational and Environmental Hygiene, 2001, vol. 16, no. 8, pp. 798-808.
[12] Urmson, C, et al, "A Robust Approach to High-Speed Navigation for Unrehearsed Desert Terrain", Journal of Field Robotics, accepted for publication.
[13] Koon, P., "Evaluation of Autonomous Ground Vehicle Skills", master's thesis, tech. report CMU-RI-TR-06-13, Robotics Institute, Carnegie Mellon University, March, 2006. [14] Urmson, C, "Navigation Regimes for Off-Road Autonomy", doctoral dissertation, tech. report CMU-RI-TR-05-23, Robotics Institute, Carnegie Mellon University, May, 2005.
[15] Nikolova, M., Hero, A., "Segmentation of a Road from a Vehicle-Mounted Radar and Accuracy of the Estimation", Proceedings of IEEE Intelligent Vehicles Symposium, 2000, pp. 284-289.

Claims

1. A method of providing obstacle detection in an autonomous vehicle, said method comprising the steps of: obtaining a radar diagram; discerning at least one prospective obstacle in the radar diagram; ascertaining background scatter about the at least one prospective obstacle; classifying the at least one prospective obstacle in relation to the ascertained background scatter; and refining the radar diagram and reevaluating the at least one prospective obstacle; said reevaluating comprising repeating said steps of ascertaining and classifying.
2. The method according to Claim 1, wherein said classifying step comprises applying a context-based filter to data corresponding to the at least one prospective obstacle.
3. The method according to Claim 2, wherein said step of applying a context- based filter comprises applying a kernel filter.
4. The method according to Claim 3, wherein said step of applying a kernel filter comprises: choosing at least one pixel from the radar diagram corresponding to a discerned prospective obstacle; applying a first mathematical function to the at least one chosen pixel; and applying a second mathematical function to at least one pixel disposed adjacent to the at least one chosen pixel; and relating the first mathematical function and the second mathematical function towards classifying the at least one prospective obstacle.
5. The method according to Claim 4, wherein said step of applying a second mathematical function comprises applying a second mathematical function to a plurality of pixels disposed about a periphery of the at least one chosen pixel.
6. The method according to Claim 4, wherein: said step of applying a first mathematical function comprises deriving a first aggregate intensity, corresponding to the at least one chosen pixel; said step of applying a second mathematical function comprises deriving a second aggregate intensity, corresponding to the at least one pixel disposed adjacent to the at least one chosen pixel; said relating step comprising subtracting the second aggregate intensity from the first aggregate intensity.
7. The method according to Claim 6, wherein said relating step comprises normalizing, relative to a number of pixels in the at least one chosen pixel, the first aggregate intensity subtracted by the second aggregate intensity, to yield a normalized net intensity.
8. The method according to Claim 7, wherein said classifying step further comprises classifying a discerned prospective obstacle as a binary obstacle if the normalized net intensity is greater than a predetermined threshold value.
9. The method according to Claim 4, wherein the at least one chosen pixel corresponds to a maximum size for a prospective obstacle to be classified as a binary obstacle.
10. The method according to Claim 9, wherein the at least one pixel disposed adjacent to the at least one chosen pixel corresponds to a desired extent of clear space adjacent a binary obstacle.
11. The method according to Claim 1 , wherein said discerning step comprises labeling at least one discerned obstacle with polar radar coordinates.
12. The method according to Claim 11, wherein said refining comprises transforming at least a portion of the radar diagram from polar coordinates to rectangular coordinates.
13. The method according to Claim 12, wherein said transforming step comprises accessing a vehicle pose history
14. The method according to Claim 1, wherein said discerning step comprises time-stamping at least one discerned obstacle.
15. The method according to Claim 1, wherein said reevaluating step further comprises applying hysteresis to data corresponding to the at least one prospective obstacle.
16. The method according to Claim 15, wherein said step of applying hysteresis comprises evaluating, at different timepoints, bunched radar data corresponding to the at least one prospective obstacle.
17. The method according to Claim 16, wherein said evaluating step comprises: evaluating, at a first timepoint, a first group of bunched radar data corresponding to the at least one prospective obstacle; and evaluating, at a second timepoint, a second group of bunched radar data corresponding to the at least one prospective obstacle; the second group of bunched radar data being contiguous with respect to the first group of bunched radar data relative to a predetermined reference map.
18. The method according to Claim 17, wherein said evaluating step further comprises: replacing the first group of bunched radar data with the second group of bunched radar data; and storing the first group of bunched radar data in a history.
19. A system for providing obstacle detection in an autonomous vehicle, said system comprising: an arrangement for discerning at least one prospective obstacle in a radar diagram; an arrangement for ascertaining background scatter about the at least one prospective obstacle; an arrangement for classifying the at least one prospective obstacle in relation to the ascertained background scatter; and an arrangement for refining the radar diagram and reevaluating the at least one prospective obstacle; said refining and reevaluating arrangement acting to prompt a repeat of ascertaining background scatter about the at least one prospective obstacle and classifying the at least one prospective obstacle in relation to the ascertained background scatter.
20. The system according to Claim 19, wherein said classifying arrangement is acts to apply a context-based filter to data corresponding to the at least one prospective obstacle.
21. The system according to Claim 20, wherein said classifying arrangement acts to apply a kernel filter to data corresponding to the at least one prospective obstacle.
22. The system according to Claim 21, wherein said classifying arrangement acts to: choose at least one pixel from the radar diagram corresponding to a discerned prospective obstacle; apply a first mathematical function to the at least one chosen pixel; and apply a second mathematical function to at least one pixel disposed adjacent to the at least one chosen pixel; and relate the first mathematical function and the second mathematical function towards classifying the at least one prospective obstacle.
23. The system according to Claim 22, wherein said classifying arrangement acts to apply a second mathematical function to a plurality of pixels disposed about a periphery of the at least one chosen pixel.
24. The system according to Claim 22, wherein: the first mathematical function yields a first aggregate intensity, corresponding to the at least one chosen pixel; the second mathematical function yields a second aggregate intensity, corresponding to the at least one pixel disposed adjacent to the at least one chosen pixel; said classifying arrangement acts to subtract the second aggregate intensity from the first aggregate intensity.
25. The system according to Claim 24, wherein said classifying arrangement further acts to normalize, relative to a number of pixels in the at least one chosen pixel, the first aggregate intensity subtracted by the second aggregate intensity, to yield a normalized net intensity.
26. The system according to Claim 25, wherein said classifying arrangement further acts to classify a discerned prospective obstacle as a binary obstacle if the normalized net intensity is greater than a predetermined threshold value.
27. The system according to Claim 22, wherein the at least one chosen pixel corresponds to a maximum size for a prospective obstacle to be classified as a binary obstacle.
28. The system according to Claim 27, wherein the at least one pixel disposed adjacent to the at least one chosen pixel corresponds to a desired extent of clear space adjacent a binary obstacle.
29. The system according to Claim 19, wherein said discerning arrangement acts to label at least one discerned obstacle with polar radar coordinates.
30. The system according to Claim 29, wherein said refining and reevaluating arrangement acts to transform at least a portion of the radar diagram from polar coordinates to rectangular coordinates.
31. The system according to Claim 30, wherein said transforming said refining and reevaluating arrangement further acts to access a vehicle pose history
32. The system according to Claim 19, wherein said discerning arrangement acts to time-stamp at least one discerned obstacle.
33. The system according to Claim 19, wherein said refining and reevaluating arrangement further acts to apply hysteresis to data corresponding to the at least one prospective obstacle.
34. The system according to Claim 33, wherein said refining and reevaluating arrangement acts to evaluate, at different timepoints, bunched radar data corresponding to the at least one prospective obstacle.
35. The system according to Claim 34, wherein said refining and reevaluating arrangement acts to: evaluate, at a first timepoint, a first group of bunched radar data corresponding to the at least one prospective obstacle; and evaluate, at a second timepoint, a second group of bunched radar data corresponding to the at least one prospective obstacle; the second group of bunched radar data being contiguous with respect to the first group of bunched radar data relative to a predetermined reference map.
36. The system according to Claim 35, wherein said refining and reevaluating arrangement further acts to: replace the first group of bunched radar data with the second group of bunched radar data; and store the first group of bunched radar data in a history.
PCT/US2007/070918 2006-06-09 2007-06-11 Obstacle detection arrangements in and for autonomous vehicles WO2008070205A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US81269306P 2006-06-09 2006-06-09
US60/812,693 2006-06-09

Publications (2)

Publication Number Publication Date
WO2008070205A2 true WO2008070205A2 (en) 2008-06-12
WO2008070205A3 WO2008070205A3 (en) 2008-08-28

Family

ID=38802360

Family Applications (3)

Application Number Title Priority Date Filing Date
PCT/US2007/070918 WO2008070205A2 (en) 2006-06-09 2007-06-11 Obstacle detection arrangements in and for autonomous vehicles
PCT/US2007/070920 WO2007143757A2 (en) 2006-06-09 2007-06-11 Software architecture for high-speed traversal of prescribed routes
PCT/US2007/070919 WO2007143756A2 (en) 2006-06-09 2007-06-11 System and method for autonomously convoying vehicles

Family Applications After (2)

Application Number Title Priority Date Filing Date
PCT/US2007/070920 WO2007143757A2 (en) 2006-06-09 2007-06-11 Software architecture for high-speed traversal of prescribed routes
PCT/US2007/070919 WO2007143756A2 (en) 2006-06-09 2007-06-11 System and method for autonomously convoying vehicles

Country Status (2)

Country Link
US (3) US20100026555A1 (en)
WO (3) WO2008070205A2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107562057A (en) * 2017-09-07 2018-01-09 南京昱晟机器人科技有限公司 A kind of intelligent robot navigation control method
TWI743705B (en) * 2019-03-15 2021-10-21 日商山葉發動機股份有限公司 Vehicles traveling on an established route

Families Citing this family (236)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10338580B2 (en) * 2014-10-22 2019-07-02 Ge Global Sourcing Llc System and method for determining vehicle orientation in a vehicle consist
US20060095171A1 (en) * 2004-11-02 2006-05-04 Whittaker William L Methods, devices and systems for high-speed autonomous vehicle and high-speed autonomous vehicle
US8437900B2 (en) * 2007-01-30 2013-05-07 Komatsu Ltd. Control device for guided travel of unmanned vehicle
US8019514B2 (en) * 2007-02-28 2011-09-13 Caterpillar Inc. Automated rollover prevention system
US8606512B1 (en) 2007-05-10 2013-12-10 Allstate Insurance Company Route risk mitigation
US9932033B2 (en) 2007-05-10 2018-04-03 Allstate Insurance Company Route risk mitigation
US10157422B2 (en) 2007-05-10 2018-12-18 Allstate Insurance Company Road segment safety rating
US10096038B2 (en) 2007-05-10 2018-10-09 Allstate Insurance Company Road segment safety rating system
US20090088916A1 (en) * 2007-09-28 2009-04-02 Honeywell International Inc. Method and system for automatic path planning and obstacle/collision avoidance of autonomous vehicles
US7979174B2 (en) * 2007-09-28 2011-07-12 Honeywell International Inc. Automatic planning and regulation of the speed of autonomous vehicles
JP4978494B2 (en) * 2008-02-07 2012-07-18 トヨタ自動車株式会社 Autonomous mobile body and control method thereof
US8160765B2 (en) * 2008-03-03 2012-04-17 Cnh America Llc Method and system for coordinated vehicle control with wireless communication
IL192601A (en) * 2008-07-03 2014-07-31 Elta Systems Ltd Sensing/emitting apparatus, system and method
US8543331B2 (en) * 2008-07-03 2013-09-24 Hewlett-Packard Development Company, L.P. Apparatus, and associated method, for planning and displaying a route path
US20100053593A1 (en) * 2008-08-26 2010-03-04 Honeywell International Inc. Apparatus, systems, and methods for rotating a lidar device to map objects in an environment in three dimensions
US8121749B1 (en) 2008-09-25 2012-02-21 Honeywell International Inc. System for integrating dynamically observed and static information for route planning in a graph based planner
US20100082179A1 (en) * 2008-09-29 2010-04-01 David Kronenberg Methods for Linking Motor Vehicles to Reduce Aerodynamic Drag and Improve Fuel Economy
US8930058B1 (en) * 2008-10-20 2015-01-06 The United States Of America As Represented By The Secretary Of The Navy System and method for controlling a vehicle traveling along a path
IL200921A (en) * 2009-09-14 2016-05-31 Israel Aerospace Ind Ltd Infantry robotic porter system and methods useful in conjunction therewith
KR101314588B1 (en) * 2009-10-26 2013-10-07 한국전자통신연구원 Method and apparatus for producing map of artificial mark, method and apparatus for measuring position of mobile object by using same
WO2011064821A1 (en) * 2009-11-27 2011-06-03 トヨタ自動車株式会社 Autonomous moving object and control method
US8635015B2 (en) * 2009-12-17 2014-01-21 Deere & Company Enhanced visual landmark for localization
US20110153338A1 (en) * 2009-12-17 2011-06-23 Noel Wayne Anderson System and method for deploying portable landmarks
US8224516B2 (en) * 2009-12-17 2012-07-17 Deere & Company System and method for area coverage using sector decomposition
US8818711B2 (en) * 2009-12-18 2014-08-26 Empire Technology Development Llc 3D path analysis for environmental modeling
US8868325B2 (en) * 2010-04-05 2014-10-21 Toyota Jidosha Kabushiki Kaisha Collision judgment apparatus for vehicle
US9129523B2 (en) 2013-05-22 2015-09-08 Jaybridge Robotics, Inc. Method and system for obstacle detection for vehicles using planar sensor data
EP2423052B1 (en) * 2010-08-25 2015-01-28 Frankfurt University of Applied Sciences Method and device for detecting people
US8793036B2 (en) * 2010-09-22 2014-07-29 The Boeing Company Trackless transit system with adaptive vehicles
US8509982B2 (en) 2010-10-05 2013-08-13 Google Inc. Zone driving
US9187095B2 (en) 2010-10-12 2015-11-17 Volvo Lastvagnar Ab Method and arrangement for entering a preceding vehicle autonomous following mode
US20120109421A1 (en) * 2010-11-03 2012-05-03 Kenneth Scarola Traffic congestion reduction system
US8442790B2 (en) * 2010-12-03 2013-05-14 Qbotix, Inc. Robotic heliostat calibration system and method
KR101732902B1 (en) * 2010-12-27 2017-05-24 삼성전자주식회사 Path planning apparatus of robot and method thereof
DE102011010262B4 (en) 2011-01-27 2013-05-16 Carl Zeiss Meditec Ag Optical observation device with at least two each having a partial beam path having optical transmission channels
US8496078B2 (en) 2011-01-29 2013-07-30 GM Global Technology Operations LLC Semi-autonomous vehicle providing cargo space
US8627908B2 (en) 2011-01-29 2014-01-14 GM Global Technology Operations LLC Semi-autonomous vehicle providing an auxiliary power supply
RU2552960C2 (en) 2011-02-18 2015-06-10 СиЭнЭйч АМЕРИКА ЭлЭлСи System and method of trajectory control of vehicle used in harvester
JP5503578B2 (en) * 2011-03-10 2014-05-28 パナソニック株式会社 Object detection apparatus and object detection method
US20130006482A1 (en) * 2011-06-30 2013-01-03 Ramadev Burigsay Hukkeri Guidance system for a mobile machine
US20170242443A1 (en) 2015-11-02 2017-08-24 Peloton Technology, Inc. Gap measurement for vehicle convoying
US11334092B2 (en) 2011-07-06 2022-05-17 Peloton Technology, Inc. Devices, systems, and methods for transmitting vehicle data
US9582006B2 (en) 2011-07-06 2017-02-28 Peloton Technology, Inc. Systems and methods for semi-autonomous convoying of vehicles
WO2014145918A1 (en) 2013-03-15 2014-09-18 Peloton Technology, Inc. Vehicle platooning systems and methods
US10520952B1 (en) 2011-07-06 2019-12-31 Peloton Technology, Inc. Devices, systems, and methods for transmitting vehicle data
WO2018039134A1 (en) 2016-08-22 2018-03-01 Peloton Technology, Inc. Automated connected vehicle control system architecture
US10520581B2 (en) 2011-07-06 2019-12-31 Peloton Technology, Inc. Sensor fusion for autonomous or partially autonomous vehicle control
JP5472248B2 (en) * 2011-09-27 2014-04-16 株式会社デンソー Convoy travel device
JP2013073360A (en) * 2011-09-27 2013-04-22 Denso Corp Platoon driving device
US8510029B2 (en) 2011-10-07 2013-08-13 Southwest Research Institute Waypoint splining for autonomous vehicle following
WO2013062401A1 (en) * 2011-10-24 2013-05-02 Dawson Yahya Ratnam A machine vision based obstacle detection system and a method thereof
US8649962B2 (en) 2011-12-19 2014-02-11 International Business Machines Corporation Planning a route for a convoy of automobiles
US9041589B2 (en) * 2012-04-04 2015-05-26 Caterpillar Inc. Systems and methods for determining a radar device coverage region
US8718861B1 (en) 2012-04-11 2014-05-06 Google Inc. Determining when to drive autonomously
US9026367B2 (en) * 2012-06-27 2015-05-05 Microsoft Technology Licensing, Llc Dynamic destination navigation system
US9633436B2 (en) 2012-07-26 2017-04-25 Infosys Limited Systems and methods for multi-dimensional object detection
US10678259B1 (en) * 2012-09-13 2020-06-09 Waymo Llc Use of a reference image to detect a road obstacle
US9423498B1 (en) * 2012-09-25 2016-08-23 Google Inc. Use of motion data in the processing of automotive radar image processing
US9633564B2 (en) 2012-09-27 2017-04-25 Google Inc. Determining changes in a driving environment based on vehicle behavior
US9720412B1 (en) * 2012-09-27 2017-08-01 Waymo Llc Modifying the behavior of an autonomous vehicle using context based parameter switching
US8949016B1 (en) * 2012-09-28 2015-02-03 Google Inc. Systems and methods for determining whether a driving environment has changed
US9097800B1 (en) * 2012-10-11 2015-08-04 Google Inc. Solid object detection system using laser and radar sensor fusion
JP5673646B2 (en) * 2012-10-11 2015-02-18 株式会社デンソー Peripheral vehicle recognition device
US8949024B2 (en) * 2012-10-25 2015-02-03 Massachusetts Institute Of Technology Vehicle localization using surface penetrating radar
US9310213B2 (en) * 2012-11-08 2016-04-12 Apple Inc. Obtaining updated navigation information for road trips
EP2746833A1 (en) 2012-12-18 2014-06-25 Volvo Car Corporation Vehicle adaptation to automatic driver independent control mode
US10053120B2 (en) * 2012-12-28 2018-08-21 General Electric Company Vehicle convoy control system and method
US10262542B2 (en) * 2012-12-28 2019-04-16 General Electric Company Vehicle convoy control system and method
US9142063B2 (en) 2013-02-15 2015-09-22 Caterpillar Inc. Positioning system utilizing enhanced perception-based localization
US10222462B2 (en) * 2013-02-27 2019-03-05 Waymo Llc Adaptive algorithms for interrogating the viewable scene of an automotive radar
US8930122B2 (en) * 2013-03-15 2015-01-06 GM Global Technology Operations LLC Methods and systems for associating vehicles en route to a common destination
US20180210463A1 (en) 2013-03-15 2018-07-26 Peloton Technology, Inc. System and method for implementing pre-cognition braking and/or avoiding or mitigation risks among platooning vehicles
US11294396B2 (en) * 2013-03-15 2022-04-05 Peloton Technology, Inc. System and method for implementing pre-cognition braking and/or avoiding or mitigation risks among platooning vehicles
JP5737316B2 (en) * 2013-04-17 2015-06-17 株式会社デンソー Convoy travel system
US9147353B1 (en) 2013-05-29 2015-09-29 Allstate Insurance Company Driving analysis using vehicle-to-vehicle communication
US9857472B2 (en) * 2013-07-02 2018-01-02 Electronics And Telecommunications Research Institute Laser radar system for obtaining a 3D image
JP6217278B2 (en) * 2013-09-24 2017-10-25 株式会社デンソー Convoy travel control device
CN103530606B (en) * 2013-09-30 2016-06-29 中国农业大学 A kind of farm machinery navigation path extraction method under weeds environment
SE537603C2 (en) * 2013-09-30 2015-07-21 Scania Cv Ab Method and system for handling obstacles for vehicle trains
SE537618C2 (en) * 2013-09-30 2015-08-04 Scania Cv Ab Method and system for common driving strategy for vehicle trains
US9141112B1 (en) 2013-10-16 2015-09-22 Allstate Insurance Company Caravan management
US10692149B1 (en) 2013-12-06 2020-06-23 Allstate Insurance Company Event based insurance model
US10240930B2 (en) 2013-12-10 2019-03-26 SZ DJI Technology Co., Ltd. Sensor fusion
US9091558B2 (en) * 2013-12-23 2015-07-28 Automotive Research & Testing Center Autonomous driver assistance system and autonomous driving method thereof
WO2015152984A2 (en) 2014-01-22 2015-10-08 Polaris Sensor Technologies, Inc. Polarization imaging for facial recognition enhancement system and method
US9589195B2 (en) 2014-01-22 2017-03-07 Polaris Sensor Technologies, Inc. Polarization-based mapping and perception method and system
US10096067B1 (en) 2014-01-24 2018-10-09 Allstate Insurance Company Reward system related to a vehicle-to-vehicle communication system
US9355423B1 (en) 2014-01-24 2016-05-31 Allstate Insurance Company Reward system related to a vehicle-to-vehicle communication system
US9390451B1 (en) 2014-01-24 2016-07-12 Allstate Insurance Company Insurance system related to a vehicle-to-vehicle communication system
US9940676B1 (en) 2014-02-19 2018-04-10 Allstate Insurance Company Insurance system for analysis of autonomous driving
US10783586B1 (en) 2014-02-19 2020-09-22 Allstate Insurance Company Determining a property of an insurance policy based on the density of vehicles
US10803525B1 (en) 2014-02-19 2020-10-13 Allstate Insurance Company Determining a property of an insurance policy based on the autonomous features of a vehicle
US10783587B1 (en) 2014-02-19 2020-09-22 Allstate Insurance Company Determining a driver score based on the driver's response to autonomous features of a vehicle
US10796369B1 (en) 2014-02-19 2020-10-06 Allstate Insurance Company Determining a property of an insurance policy based on the level of autonomy of a vehicle
US9529364B2 (en) 2014-03-24 2016-12-27 Cnh Industrial America Llc System for coordinating agricultural vehicle control for loading a truck
US9766628B1 (en) * 2014-04-04 2017-09-19 Waymo Llc Vision-based object detection using a polar grid
US9304515B2 (en) * 2014-04-24 2016-04-05 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Regional operation modes for autonomous vehicles
US10114348B2 (en) 2014-05-12 2018-10-30 Deere & Company Communication system for closed loop control of a worksite
US9772625B2 (en) 2014-05-12 2017-09-26 Deere & Company Model referenced management and control of a worksite
US9475422B2 (en) * 2014-05-22 2016-10-25 Applied Invention, Llc Communication between autonomous vehicle and external observers
CN104049634B (en) * 2014-07-02 2017-02-01 燕山大学 Intelligent body fuzzy dynamic obstacle avoidance method based on Camshift algorithm
KR102329444B1 (en) * 2014-07-04 2021-11-24 주식회사 만도모빌리티솔루션즈 Control system of vehicle and method thereof
WO2016013996A1 (en) 2014-07-25 2016-01-28 Okan Üni̇versitesi̇ A close range vehicle following system which can provide vehicle distances and course by using various variables.
WO2016076936A2 (en) * 2014-08-26 2016-05-19 Polaris Sensor Technologies, Inc. Polarization-based mapping and perception method and system
AU2015347259A1 (en) * 2014-08-26 2017-03-30 Polaris Sensor Technologies, Inc. Polarization-based mapping and perception method and system
US9296411B2 (en) 2014-08-26 2016-03-29 Cnh Industrial America Llc Method and system for controlling a vehicle to a moving point
US9321461B1 (en) 2014-08-29 2016-04-26 Google Inc. Change detection using curve alignment
US9997077B2 (en) * 2014-09-04 2018-06-12 Honda Motor Co., Ltd. Vehicle operation assistance
CN105980950B (en) 2014-09-05 2019-05-28 深圳市大疆创新科技有限公司 The speed control of unmanned vehicle
CN110174903B (en) 2014-09-05 2023-05-09 深圳市大疆创新科技有限公司 System and method for controlling a movable object within an environment
WO2016033796A1 (en) 2014-09-05 2016-03-10 SZ DJI Technology Co., Ltd. Context-based flight mode selection
US9248834B1 (en) 2014-10-02 2016-02-02 Google Inc. Predicting trajectories of objects based on contextual information
WO2016100088A1 (en) * 2014-12-18 2016-06-23 Agco Corporation Method of path planning for autoguidance
CN104540093A (en) * 2015-01-21 2015-04-22 郑豪 Directional constant-distance type tracking system based on Bluetooth wireless technology
WO2016126316A1 (en) * 2015-02-06 2016-08-11 Delphi Technologies, Inc. Autonomous guidance system
WO2016126321A1 (en) 2015-02-06 2016-08-11 Delphi Technologies, Inc. Method and apparatus for controlling an autonomous vehicle
WO2016126317A1 (en) 2015-02-06 2016-08-11 Delphi Technologies, Inc. Method of automatically controlling an autonomous vehicle based on electronic messages from roadside infrastructure of other vehicles
JP6372384B2 (en) * 2015-02-09 2018-08-15 株式会社デンソー Vehicle-to-vehicle management device and vehicle-to-vehicle management method
CN104599588B (en) * 2015-02-13 2017-06-23 中国北方车辆研究所 A kind of computational methods of the current cost of grating map
US9625582B2 (en) * 2015-03-25 2017-04-18 Google Inc. Vehicle with multiple light detection and ranging devices (LIDARs)
DE102015106575A1 (en) * 2015-04-29 2016-11-03 Knorr-Bremse Systeme für Nutzfahrzeuge GmbH Method and device for regulating the speed of a vehicle
BR102016008666B1 (en) 2015-05-12 2022-10-11 Autonomous Solutions, Inc. CONTROL SYSTEM FOR A BASE STATION, METHOD FOR CONTROLLING AN AGRICULTURAL VEHICLE AND AUTONOMOUS AGRICULTURAL SYSTEM
US9547309B2 (en) 2015-05-13 2017-01-17 Uber Technologies, Inc. Selecting vehicle type for providing transport
WO2016183525A1 (en) * 2015-05-13 2016-11-17 Uber Technologies, Inc. Autonomous vehicle operated with guide assistance
US9494439B1 (en) 2015-05-13 2016-11-15 Uber Technologies, Inc. Autonomous vehicle operated with guide assistance of human driven vehicles
US10345809B2 (en) 2015-05-13 2019-07-09 Uber Technologies, Inc. Providing remote assistance to an autonomous vehicle
US10131362B1 (en) * 2015-06-23 2018-11-20 United Services Automobile Association (Usaa) Automobile detection system
DE102015213743B4 (en) * 2015-07-21 2021-10-28 Volkswagen Aktiengesellschaft Method and system for the automatic control of at least one following vehicle with a scout vehicle
KR101962889B1 (en) * 2015-07-27 2019-03-28 한국전자통신연구원 Robot motion data providing apparatus using a robot to work and method therefor
US9618938B2 (en) * 2015-07-31 2017-04-11 Ford Global Technologies, Llc Field-based torque steering control
US11137255B2 (en) * 2015-08-03 2021-10-05 Tomtom Global Content B.V. Methods and systems for generating and using localization reference data
US10712748B2 (en) 2015-08-26 2020-07-14 Peloton Technology, Inc. Devices, systems, and methods for generating travel forecasts for vehicle pairing
IL241403A0 (en) 2015-09-09 2016-05-31 Elbit Systems Land & C4I Ltd Open terrain navigation system and methods
EP3350554A4 (en) * 2015-09-18 2019-06-12 Slantrange, Inc. Systems and methods for determining statistics of plant populations based on overhead optical measurements
US10139828B2 (en) 2015-09-24 2018-11-27 Uber Technologies, Inc. Autonomous vehicle operated with safety augmentation
US9764470B2 (en) * 2015-10-05 2017-09-19 X Development Llc Selective deployment of robots to perform mapping
US9881219B2 (en) 2015-10-07 2018-01-30 Ford Global Technologies, Llc Self-recognition of autonomous vehicles in mirrored or reflective surfaces
US9632509B1 (en) 2015-11-10 2017-04-25 Dronomy Ltd. Operating a UAV with a narrow obstacle-sensor field-of-view
CA3005147C (en) 2015-11-20 2022-07-19 Uber Technologies, Inc. Controlling autonomous vehicles in connection with transport services
DE102015225241A1 (en) * 2015-12-15 2017-06-22 Volkswagen Aktiengesellschaft Method and system for automatically controlling a following vehicle with a fore vehicle
US9632507B1 (en) * 2016-01-29 2017-04-25 Meritor Wabco Vehicle Control Systems System and method for adjusting vehicle platoon distances based on predicted external perturbations
US10801846B2 (en) * 2016-01-29 2020-10-13 Komatsu Ltd. Work machine management system, work machine, and work machine management method
US10269075B2 (en) 2016-02-02 2019-04-23 Allstate Insurance Company Subjective route risk mapping and mitigation
US9864377B2 (en) * 2016-04-01 2018-01-09 Locus Robotics Corporation Navigation using planned robot travel paths
US10152891B2 (en) * 2016-05-02 2018-12-11 Cnh Industrial America Llc System for avoiding collisions between autonomous vehicles conducting agricultural operations
US10241514B2 (en) 2016-05-11 2019-03-26 Brain Corporation Systems and methods for initializing a robot to autonomously travel a trained route
JP6817337B2 (en) 2016-05-27 2021-01-20 ユーエーティーシー, エルエルシー Facilitating passenger boarding for self-driving cars
WO2017210200A1 (en) 2016-05-31 2017-12-07 Peloton Technology, Inc. Platoon controller state machine
US9987752B2 (en) 2016-06-10 2018-06-05 Brain Corporation Systems and methods for automatic detection of spills
US10282849B2 (en) 2016-06-17 2019-05-07 Brain Corporation Systems and methods for predictive/reconstructive visual object tracker
FR3053948B1 (en) * 2016-07-12 2018-07-20 Peugeot Citroen Automobiles Sa METHOD FOR ASSISTING A DRIVER OF A VEHICLE BASED ON INFORMATION PROVIDED BY A PILOT VEHICLE, AND DEVICE THEREFOR
US11216006B2 (en) * 2016-07-20 2022-01-04 Singapore University Of Technology And Design Robot and method for localizing a robot
US10471904B2 (en) 2016-08-08 2019-11-12 Toyota Motor Engineering & Manufacturing North America, Inc. Systems and methods for adjusting the position of sensors of an automated vehicle
US10369998B2 (en) 2016-08-22 2019-08-06 Peloton Technology, Inc. Dynamic gap control for automated driving
JP6610466B2 (en) * 2016-08-23 2019-11-27 株式会社デンソー Vehicle control system
US10108194B1 (en) 2016-09-02 2018-10-23 X Development Llc Object placement verification
US10274331B2 (en) 2016-09-16 2019-04-30 Polaris Industries Inc. Device and method for improving route planning computing devices
CN106383515A (en) * 2016-09-21 2017-02-08 哈尔滨理工大学 Wheel-type moving robot obstacle-avoiding control system based on multi-sensor information fusion
US10379540B2 (en) * 2016-10-17 2019-08-13 Waymo Llc Light detection and ranging (LIDAR) device having multiple receivers
US10274325B2 (en) 2016-11-01 2019-04-30 Brain Corporation Systems and methods for robotic mapping
US10001780B2 (en) * 2016-11-02 2018-06-19 Brain Corporation Systems and methods for dynamic route planning in autonomous navigation
US10528055B2 (en) 2016-11-03 2020-01-07 Ford Global Technologies, Llc Road sign recognition
SG10201609375XA (en) * 2016-11-09 2018-06-28 Cyclect Electrical Eng Pte Ltd Vehicle, system and method for remote convoying
US10723018B2 (en) 2016-11-28 2020-07-28 Brain Corporation Systems and methods for remote operating and/or monitoring of a robot
US10482767B2 (en) * 2016-12-30 2019-11-19 Bendix Commercial Vehicle Systems Llc Detection of extra-platoon vehicle intermediate or adjacent to platoon member vehicles
DE112017006531T5 (en) * 2017-01-25 2019-09-26 Ford Global Technologies, Llc REMOTE CONTROLLED VIRTUAL REALITY PARKING SERVICE
JP6837690B2 (en) 2017-01-27 2021-03-03 マサチューセッツ インスティテュート オブ テクノロジー Vehicle locating method and system using surface penetration radar
US20180217603A1 (en) * 2017-01-31 2018-08-02 GM Global Technology Operations LLC Efficient situational awareness from perception streams in autonomous driving systems
US10852730B2 (en) 2017-02-08 2020-12-01 Brain Corporation Systems and methods for robotic mobile platforms
DE102017202551A1 (en) * 2017-02-16 2018-08-16 Robert Bosch Gmbh Method and apparatus for providing a signal for operating at least two vehicles
IL250762B (en) 2017-02-23 2020-09-30 Appelman Dina Method of navigating an unmanned vehicle and system thereof
US11142203B2 (en) * 2017-02-27 2021-10-12 Ford Global Technologies, Llc Cooperative vehicle navigation
US10124688B2 (en) * 2017-03-08 2018-11-13 Toyota Research Institute, Inc. Systems and methods for rendezvousing with an autonomous modular vehicle to provide energy
US10293485B2 (en) * 2017-03-30 2019-05-21 Brain Corporation Systems and methods for robotic path planning
EP3396306B1 (en) * 2017-04-26 2019-11-27 Mitutoyo Corporation Method and system for calculating a height map of a surface of an object from an image stack in scanning optical 2.5d profiling of the surface by an optical system
CN107330921A (en) * 2017-06-28 2017-11-07 京东方科技集团股份有限公司 A kind of line-up device and its queuing control method
US20190016315A1 (en) * 2017-07-12 2019-01-17 Aptiv Technologies Limited Automated braking system
WO2019018337A1 (en) 2017-07-20 2019-01-24 Walmart Apollo, Llc Task management of autonomous product delivery vehicles
US10538239B2 (en) 2017-07-27 2020-01-21 International Business Machines Corporation Adapting driving based on a transported cargo
IL255050B (en) * 2017-10-16 2022-03-01 Israel Aerospace Ind Ltd Control over an autonomic vehicle
CN107817800A (en) * 2017-11-03 2018-03-20 北京奇虎科技有限公司 The collision processing method of robot and robot, electronic equipment
US10967862B2 (en) 2017-11-07 2021-04-06 Uatc, Llc Road anomaly detection for autonomous vehicle
KR102472768B1 (en) * 2017-11-23 2022-12-01 삼성전자주식회사 Method and apparatus for detecting object for autonomous vehicle
US10684134B2 (en) * 2017-12-15 2020-06-16 Waymo Llc Using prediction models for scene difficulty in vehicle routing
US11237877B2 (en) * 2017-12-27 2022-02-01 Intel Corporation Robot swarm propagation using virtual partitions
US10921823B2 (en) 2017-12-28 2021-02-16 Bendix Commercial Vehicle Systems Llc Sensor-based anti-hacking prevention in platooning vehicles
US20190204845A1 (en) 2017-12-29 2019-07-04 Waymo Llc Sensor integration for large autonomous vehicles
IL257428B (en) * 2018-02-08 2022-04-01 Israel Aerospace Ind Ltd Excavation by way of an unmanned vehicle
CN108460112B (en) * 2018-02-09 2021-07-06 上海思岚科技有限公司 Map storage method and system
CN108482368B (en) * 2018-03-28 2020-06-23 成都博士信智能科技发展有限公司 Unmanned vehicle anti-collision control method and device based on sand table
JP6989429B2 (en) * 2018-03-28 2022-01-05 株式会社東芝 The platooning operation system and the platooning operation method
US10816984B2 (en) * 2018-04-13 2020-10-27 Baidu Usa Llc Automatic data labelling for autonomous driving vehicles
US10908609B2 (en) * 2018-04-30 2021-02-02 Toyota Research Institute, Inc. Apparatus and method for autonomous driving
KR102528317B1 (en) * 2018-06-08 2023-05-03 탈레스 캐나다 아이엔씨 Controllers, systems and methods for vehicle control
US11669108B2 (en) * 2018-07-07 2023-06-06 Peloton Technology, Inc. Control of automated following in vehicle convoys
US10899323B2 (en) 2018-07-08 2021-01-26 Peloton Technology, Inc. Devices, systems, and methods for vehicle braking
EP3823795A4 (en) 2018-07-16 2022-04-06 Brain Corporation Systems and methods for optimizing route planning for tight turns for robotic apparatuses
US11204605B1 (en) * 2018-08-03 2021-12-21 GM Global Technology Operations LLC Autonomous vehicle controlled based upon a LIDAR data segmentation system
US11022693B1 (en) 2018-08-03 2021-06-01 GM Global Technology Operations LLC Autonomous vehicle controlled based upon a lidar data segmentation system
US10884411B1 (en) 2018-08-03 2021-01-05 GM Global Technology Operations LLC Autonomous vehicle controlled based upon a lidar data segmentation system and an aligned heightmap
WO2020044325A1 (en) * 2018-08-30 2020-03-05 Israel Aerospace Industries Ltd. Method of navigating a vehicle and system thereof
CN109062221A (en) * 2018-09-03 2018-12-21 成都市新筑路桥机械股份有限公司 A kind of intelligently marshalling Vehicular system and its control method
USD882426S1 (en) 2018-09-17 2020-04-28 Waymo Llc Integrated sensor assembly
CN109582032B (en) * 2018-10-11 2021-10-12 天津大学 Multi-rotor unmanned aerial vehicle rapid real-time obstacle avoidance path selection method in complex environment
US10762791B2 (en) 2018-10-29 2020-09-01 Peloton Technology, Inc. Systems and methods for managing communications between vehicles
US11536845B2 (en) 2018-10-31 2022-12-27 Waymo Llc LIDAR systems with multi-faceted mirrors
JP7049585B2 (en) * 2018-11-01 2022-04-07 トヨタ自動車株式会社 Leading mobility, follow-up mobility, and group driving control systems
WO2020122953A1 (en) * 2018-12-14 2020-06-18 Hewlett-Packard Development Company, L.P. Mobile autonomous fleet control
KR20200084423A (en) 2018-12-24 2020-07-13 삼성전자주식회사 Method and apparatus for generating local motion based on machine learning
FR3091614B1 (en) * 2019-01-04 2023-09-01 Transdev Group Electronic device and method for monitoring a scene around a motor vehicle, motor vehicle, transport system and associated computer program
CN109579849B (en) * 2019-01-14 2020-09-29 浙江大华技术股份有限公司 Robot positioning method, robot positioning device, robot and computer storage medium
CN109871420B (en) * 2019-01-16 2022-03-29 深圳乐动机器人有限公司 Map generation and partition method and device and terminal equipment
CN109901575A (en) * 2019-02-20 2019-06-18 百度在线网络技术(北京)有限公司 Vehicle routing plan adjustment method, device, equipment and computer-readable medium
US11947041B2 (en) 2019-03-05 2024-04-02 Analog Devices, Inc. Coded optical transmission for optical detection
US11427196B2 (en) 2019-04-15 2022-08-30 Peloton Technology, Inc. Systems and methods for managing tractor-trailers
US11169540B2 (en) * 2019-05-08 2021-11-09 Robotic Research, Llc Autonomous convoys maneuvering “deformable” terrain and “deformable” obstacles
US20210031760A1 (en) * 2019-07-31 2021-02-04 Nissan North America, Inc. Contingency Planning and Safety Assurance
JP7346997B2 (en) * 2019-08-21 2023-09-20 オムロン株式会社 Robot control device, robot control method, and program
JP2022547580A (en) * 2019-09-13 2022-11-14 ウェーブセンス, インコーポレイテッド Improved navigation and positioning using surface-seeking radar and deep learning
US11414002B2 (en) 2019-09-25 2022-08-16 The Boeing Company Systems, methods, and apparatus for high-traffic density air transportation
US11586222B2 (en) * 2019-09-25 2023-02-21 The Boeing Company Systems, methods, and apparatus for high-traffic density transportation pathways
CN110838228B (en) * 2019-10-18 2021-07-02 东南大学 Intelligent interactive driving system and device for commercial truck fleet
CN111006666B (en) * 2019-11-21 2021-10-29 深圳市优必选科技股份有限公司 Robot path planning method and device, storage medium and robot
US11741336B2 (en) * 2019-12-19 2023-08-29 Google Llc Generating and/or using training instances that include previously captured robot vision data and drivability labels
USD953176S1 (en) 2020-02-24 2022-05-31 Waymo Llc Sensor housing assembly
EP4114165A4 (en) * 2020-03-02 2024-04-03 Raven Ind Inc Guidance systems and methods
CN111397622B (en) * 2020-03-26 2022-04-26 江苏大学 Intelligent automobile local path planning method based on improved A-algorithm and Morphin algorithm
JP7075436B2 (en) * 2020-04-06 2022-05-25 ヤンマーパワーテクノロジー株式会社 Work vehicle control system
CN111338361A (en) * 2020-05-22 2020-06-26 浙江远传信息技术股份有限公司 Obstacle avoidance method, device, equipment and medium for low-speed unmanned vehicle
CN111813089B (en) * 2020-07-16 2021-11-23 北京润科通用技术有限公司 Simulation verification method, device and system for aircraft obstacle avoidance algorithm
US11884291B2 (en) 2020-08-03 2024-01-30 Waymo Llc Assigning vehicles for transportation services
CN112099493B (en) * 2020-08-31 2021-11-19 西安交通大学 Autonomous mobile robot trajectory planning method, system and equipment
US20220111859A1 (en) * 2020-10-12 2022-04-14 Ford Global Technologies, Llc Adaptive perception by vehicle sensors
US11709260B2 (en) * 2021-04-30 2023-07-25 Zoox, Inc. Data driven resolution function derivation
WO2022266061A1 (en) * 2021-06-14 2022-12-22 Robotic Research Opco, Llc Systems and methods for an autonomous convoy with leader vehicle
WO2023028274A1 (en) * 2021-08-25 2023-03-02 Cyngn, Inc. System and method of large-scale autonomous driving validation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040153217A1 (en) * 2001-04-12 2004-08-05 Bernhard Mattes Method for preventing collisions involving motor vehicles
US20040178943A1 (en) * 2002-12-29 2004-09-16 Haim Niv Obstacle and terrain avoidance sensor
US20040249571A1 (en) * 2001-05-07 2004-12-09 Blesener James L. Autonomous vehicle collision/crossing warning system
US20050278098A1 (en) * 1994-05-23 2005-12-15 Automotive Technologies International, Inc. Vehicular impact reactive system and method

Family Cites Families (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US626988A (en) * 1899-06-13 douglas
US5648901A (en) * 1990-02-05 1997-07-15 Caterpillar Inc. System and method for generating paths in an autonomous vehicle
GB9317983D0 (en) * 1993-08-28 1993-10-13 Lucas Ind Plc A driver assistance system for a vehicle
US5950967A (en) * 1997-08-15 1999-09-14 Westinghouse Air Brake Company Enhanced distributed power
JPH11144185A (en) * 1997-09-03 1999-05-28 Honda Motor Co Ltd Automatic drive control guidance system
US6223110B1 (en) * 1997-12-19 2001-04-24 Carnegie Mellon University Software architecture for autonomous earthmoving machinery
EP0973044B1 (en) * 1998-07-13 2006-08-09 Oerlikon Contraves Ag Method for tracking moving objects using specific characteristics
US6259988B1 (en) * 1998-07-20 2001-07-10 Lockheed Martin Corporation Real-time mission adaptable route planner
US6823249B2 (en) * 1999-03-19 2004-11-23 Agco Limited Tractor with monitoring system
JP3537705B2 (en) * 1999-05-31 2004-06-14 本田技研工業株式会社 Automatic following system
JP3791249B2 (en) * 1999-07-12 2006-06-28 株式会社日立製作所 Mobile device
JP2001222316A (en) * 2000-02-09 2001-08-17 Sony Corp System and method for managing robot
US6668216B2 (en) * 2000-05-19 2003-12-23 Tc (Bermuda) License, Ltd. Method, apparatus and system for wireless data collection and communication for interconnected mobile systems, such as for railways
US6445983B1 (en) * 2000-07-07 2002-09-03 Case Corporation Sensor-fusion navigator for automated guidance of off-road vehicles
US20020070849A1 (en) * 2000-12-07 2002-06-13 Teicher Martin H. Signaling system for vehicles travelling in a convoy
JP4159794B2 (en) * 2001-05-02 2008-10-01 本田技研工業株式会社 Image processing apparatus and method
ATE510247T1 (en) * 2001-06-12 2011-06-15 Irobot Corp METHOD AND SYSTEM FOR MULTI-MODAL COVERING FOR AN AUTONOMOUS ROBOT
US6640164B1 (en) * 2001-08-28 2003-10-28 Itt Manufacturing Enterprises, Inc. Methods and systems for remote control of self-propelled vehicles
GB0126497D0 (en) * 2001-11-03 2002-01-02 Dyson Ltd An autonomous machine
US6917893B2 (en) * 2002-03-14 2005-07-12 Activmedia Robotics, Llc Spatial data collection apparatus and method
US6829568B2 (en) * 2002-04-26 2004-12-07 Simon Justin Julier Method and apparatus for fusing signals with partially known independent error components
US6963795B2 (en) * 2002-07-16 2005-11-08 Honeywell Interntaional Inc. Vehicle position keeping system
WO2004016400A2 (en) * 2002-08-16 2004-02-26 Evolution Robotics, Inc. Systems and methods for the automated sensing of motion in a mobile robot using visual data
US7054716B2 (en) * 2002-09-06 2006-05-30 Royal Appliance Mfg. Co. Sentry robot system
US6728607B1 (en) * 2002-10-03 2004-04-27 Deere & Company Method and system for determining an energy-efficient path of a machine
US7015831B2 (en) * 2002-12-17 2006-03-21 Evolution Robotics, Inc. Systems and methods for incrementally updating a pose of a mobile device calculated by visual simultaneous localization and mapping techniques
JP3972366B2 (en) * 2003-09-26 2007-09-05 マツダ株式会社 Vehicle information providing device
US7272474B1 (en) * 2004-03-31 2007-09-18 Carnegie Mellon University Method and system for estimating navigability of terrain
US20060095171A1 (en) * 2004-11-02 2006-05-04 Whittaker William L Methods, devices and systems for high-speed autonomous vehicle and high-speed autonomous vehicle
JP4983088B2 (en) * 2005-08-03 2012-07-25 株式会社デンソー Map data generation device and information guide device
WO2008013568A2 (en) * 2005-12-30 2008-01-31 Irobot Corporation Autonomous mobile robot
US7620477B2 (en) * 2006-07-05 2009-11-17 Battelle Energy Alliance, Llc Robotic intelligence kernel
US7587260B2 (en) * 2006-07-05 2009-09-08 Battelle Energy Alliance, Llc Autonomous navigation system and method
US7584020B2 (en) * 2006-07-05 2009-09-01 Battelle Energy Alliance, Llc Occupancy change detection system and method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050278098A1 (en) * 1994-05-23 2005-12-15 Automotive Technologies International, Inc. Vehicular impact reactive system and method
US20040153217A1 (en) * 2001-04-12 2004-08-05 Bernhard Mattes Method for preventing collisions involving motor vehicles
US20040249571A1 (en) * 2001-05-07 2004-12-09 Blesener James L. Autonomous vehicle collision/crossing warning system
US20040178943A1 (en) * 2002-12-29 2004-09-16 Haim Niv Obstacle and terrain avoidance sensor

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107562057A (en) * 2017-09-07 2018-01-09 南京昱晟机器人科技有限公司 A kind of intelligent robot navigation control method
TWI743705B (en) * 2019-03-15 2021-10-21 日商山葉發動機股份有限公司 Vehicles traveling on an established route

Also Published As

Publication number Publication date
WO2007143756A3 (en) 2008-10-30
US20080059007A1 (en) 2008-03-06
WO2007143756A2 (en) 2007-12-13
US20100026555A1 (en) 2010-02-04
WO2007143757A2 (en) 2007-12-13
US20080059015A1 (en) 2008-03-06
WO2008070205A3 (en) 2008-08-28

Similar Documents

Publication Publication Date Title
US20100026555A1 (en) Obstacle detection arrangements in and for autonomous vehicles
Nissimov et al. Obstacle detection in a greenhouse environment using the Kinect sensor
US20200217950A1 (en) Resolution of elevation ambiguity in one-dimensional radar processing
US10203409B2 (en) Method and device for the localization of a vehicle from a fixed reference map
CN101966846B (en) Travel's clear path detection method for motor vehicle involving object deteciting and enhancing
Ilas Electronic sensing technologies for autonomous ground vehicles: A review
US11475678B2 (en) Lane marker detection and lane instance recognition
US20150336575A1 (en) Collision avoidance with static targets in narrow spaces
WO2017139432A1 (en) Ultra wide band radar localization
Levinson Automatic laser calibration, mapping, and localization for autonomous vehicles
Reymann et al. Improving LiDAR point cloud classification using intensities and multiple echoes
US11842430B2 (en) Methods and systems for ground segmentation using graph-cuts
EP4016115A1 (en) Vehicle localization based on radar detections
Chang et al. Concealment and obstacle detection for autonomous driving
Vandapel et al. Experimental results in using aerial ladar data for mobile robot navigation
CN112784679A (en) Vehicle obstacle avoidance method and device
Reina et al. Traversability analysis for off-road vehicles using stereo and radar data
Hong et al. An intelligent world model for autonomous off-road driving
Meis et al. Radar image acquisition and interpretation for automotive applications
EP4016129A1 (en) Radar reference map generation
EP4027169A2 (en) Radar reference map generation
CN116086467A (en) Target detection and positioning system based on unmanned integrated card under port connection scene
CN114120275A (en) Automatic driving obstacle detection and recognition method and device, electronic equipment and storage medium
Wu Data processing algorithms and applications of LiDAR-enhanced connected infrastructure sensing
Johnston Off-highway obstacle detection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07870981

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: RU

122 Ep: pct application non-entry in european phase

Ref document number: 07870981

Country of ref document: EP

Kind code of ref document: A2