US20050216182A1 - Vehicle routing and path planning - Google Patents
Vehicle routing and path planning Download PDFInfo
- Publication number
- US20050216182A1 US20050216182A1 US10/877,814 US87781404A US2005216182A1 US 20050216182 A1 US20050216182 A1 US 20050216182A1 US 87781404 A US87781404 A US 87781404A US 2005216182 A1 US2005216182 A1 US 2005216182A1
- Authority
- US
- United States
- Prior art keywords
- path
- heuristic
- mobile agent
- mission
- criterion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
Definitions
- Al artificial intelligence
- classical planning, hierarchical-task-network planning, and case-based planning use symbolic planning based on logic and reasoning.
- the problem of interest is essentially numeric, and hence less suited for reasoning about goals and sub-goals.
- Al planning techniques may be applied to a higher-level strategic planning problem—i.e., how to decide what the mission goals are—their utility is diminished in the context of tactical planning problems wherein the mission goals are already known.
- DAMN Distributed Architecture for Mobile Navigation
- DAMN includes behaviors such as “road following,” “seeking the next navigation goal,” “obstacle avoidance,” and “avoid hazards.” Each behavior provides a vote on the next direction to take.
- a command arbiter decides upon the best direction, which is then acted upon by the UGV.
- DAMN also includes a global navigator to determine a full path to a goal position.
- DAMN uses a D* (dynamic A*) search algorithm. This approach, however, does not accommodate as many criteria and as much information, at the deliberative planning level, that are generally at play, when determining mission assignments and paths that are not fooled by local gradients.
- the systems and methods described herein are generally directed at, among other things, embodying local as well as global criteria in operation planning (including path planning and routing), yielding global solutions more efficiently and reliably, handling a myriad of constraints typically present in various mobile agent operation planning contexts of interest, placing a fair emphasis on balancing workload distribution among members of a fleet of mobile agents, often dynamically, and having sufficient flexibility to incorporate new criteria into, remove existing criteria from, or reorder priorities in, an operation plan design process.
- the systems and methods disclosed herein employ a domain-specific multi-objective optimization algorithm, such as, without limitation, a context-influenced genetic algorithm, to dynamically determine paths for one or more mobile agents to accomplish one or more mission goals; a path may include instructions on traversing at least a portion thereof.
- a domain-specific multi-objective optimization algorithm such as, without limitation, a context-influenced genetic algorithm
- the invention includes a method of determining a path having an ordered set of waypoints to be visited by a mobile agent to accomplish a mission.
- the method includes (a) producing candidate paths using a multi-objective optimization algorithm, subject to a path production heuristic; (b) selecting a path from the candidate paths, subject to a path selection heuristic; (c) instructing the mobile agent to move according to the selected path; (d) modifying a maintained subset of the candidate paths to produce a new candidate path using the algorithm and subject to the path production heuristic; (e) designating either the currently-selected path or the new candidate path as the newly-selected path, subject to the path selection heuristic; and (f) instructing the mobile agent to move according to the newly-selected path.
- the method includes, after step (f), repeating steps (d)-(f) at least once prior to the mobile agent accomplishing the mission.
- the multi-objective optimization algorithm includes an evolutionary algorithm, such as, for example and without limitation, a genetic algorithm.
- modifying the maintained subset of candidate paths is performed continually (e.g., in the background), even after instruction has been issued, as in step (c), to the mobile agent to commence movement according to the selected path.
- This at least in part, is to produce, with time, an improved path to be selected for the mobile agent, and to dynamically respond to stimuli in the environment or to time-sensitive mission requirements.
- the subset of candidate paths to be maintained satisfies a diversity criterion, thereby increasing a likelihood of approaching a globally optimal, and not merely a locally optimal, solution.
- the path selection heuristic and the path production heuristic are mutually independent; according to one particular practice, this independence hold laterally across a given iteration of the steps (d)-(f), or longitudinally across different iterations.
- production of one candidate path is independent of production of another candidate path, whether in the same iteration/generation or across different iterations/generations.
- one or more of the path production heuristic and the path selection heuristic may be time dependent.
- the heuristics may be influenced by environmental characteristics and/or evolving mission requirements.
- modifying the candidate paths includes a combination of adding, deleting, and reordering waypoints along a path belonging to the candidate paths.
- the invention includes a method of determining paths for a fleet of mobile agents to accomplish missions, every path having an ordered set of waypoints to be visited by a corresponding mobile agent.
- the method includes: (a) producing candidate path sets using a multi-objective optimization algorithm, subject to a path production heuristic; (b) selecting a path set from the candidate path sets, wherein every mobile agent has an associated path belonging to the selected path set, subject to a path selection heuristic; (c) instructing a first subset of the mobile agents to move according to paths respectively associated with the first subset; (d) modifying a maintained subset of the candidate path sets to produce a new candidate path set using the algorithm, subject to the path production heuristic; (e) designating either the selected path set or the new candidate path set as the newly-selected path set, subject to the path selection heuristic; and (f) instructing a second subset of the mobile agents to move according to paths belonging to the newly-selected path set, respectively associated with the second subset.
- the method includes, after step (f), repeating steps (d)-(f) at least once prior to the mobile agents accomplishing the missions.
- the multi-objective optimization algorithm includes an evolutionary algorithm, such as, for example, a genetic algorithm. Methods of evolutionary algorithms in general, and genetic algorithms in particular, are described in “Multi-Objective Optimization Using Evolutionary Algorithms,” by Kalyanmoy Deb, John Wiley & Sons, 2001, ISBN: 047187339X.
- modifying the candidate path sets includes a combination of adding, deleting, and reordering waypoints along a path belonging to the candidate paths. Additionally, the modifying may include trading of one or more waypoints between two or more of the mobile agents, as needed or as preferred.
- the path selection heuristic and the path production heuristic are mutually independent.
- the path selection heuristic at one iteration is independent of the path selection heuristic at another iteration.
- the same is true for the path production heuristic.
- One or both of the path selection heuristic and the path production heuristic may be time dependent.
- production of candidate path sets is performed continually, possibly in the background; this is at least partially to dynamically update path sets so an improved path set is selected for the fleet of mobile agents.
- the dynamic updating may at least in part be influenced by environmental influences or evolving mission requirements.
- FIG. 1 depicts an inferred path set having two paths to be traversed by two respective mobile agents from their respective reference locations.
- FIG. 2 ( a ) depicts a representative block diagram of an illustrative embodiment of a method of determining a path having an ordered set of waypoints to be visited by a mobile agent to accomplish a mission.
- FIG. 2 ( b ) depicts an embodiment of a continually-running operation planning algorithm that includes evolution and improvement.
- FIG. 2 ( c ) depicts an illustrative continual interaction of the operation planning algorithm with the real-world environment.
- FIG. 3 depicts a genome of a path set having two chromosomes representative of two respective sequences of waypoints to be visited by two respective mobile agents.
- FIG. 4 depicts an illustrative greedy evaluation of the duration and mission success associated with a chromosomal representation of a path associated with a mobile agent.
- FIG. 5 depicts an embodiment of a tactical behavior planning algorithm in the context of an unmanned ground vehicle (UGV) navigation system.
- UUV unmanned ground vehicle
- FIG. 6 depicts an embodiment of a communication network of multiple UGVs and a human operator over which collaborative planning occurs using distributed processing and partial information sharing.
- FIG. 7 ( a ) depicts exemplary evolved paths with no known enemies using genetic operators.
- FIG. 7 ( b ) depicts exemplary evolved paths with no known enemies using genetic operators and tactical advocates.
- FIG. 8 ( a ) depicts exemplary evolved paths with two known enemies using genetic operators.
- FIG. 8 ( b ) depicts exemplary evolved paths with two known enemies using genetic operators and tactical advocates.
- FIG. 9 depicts the fitness of the best individual path by generation for each of four experimental conditions.
- FIGS. 10 ( a )- 10 ( c ) depict a sequence of three plans illustrating, respectively, an execution plan before, a new plan substantially immediately upon, and a new plan shortly after discovery of a new enemy.
- the mobile agent includes a vehicle, which may be manned or unmanned.
- the vehicle may include a sea, ground, air, or space vehicle, or an amphibious vehicle capable of movement in, and across a boundary of, two or more terrain types (e.g., a sea-ground amphibious vehicle, an amphibious craft capable of traveling in and beyond a planet's atmosphere, etc.).
- the sea vehicle may be capable of movement on an aquatic surface region, subsurface region, or both.
- a ground vehicle may be capable of movement underground, on a ground surface, or both.
- the mobile agent includes a human (e.g., a soldier, a rescue worker, or another mobile personnel unit), a robot or robotic component (e.g., a robotic arm), a computer-generated agent, an animal (e.g., monkey, dog, dolphin, bird, or other trained or trainable animal), or any other agent trained, trainable, configured, or configurable to receive and follow movement instruction.
- a human e.g., a soldier, a rescue worker, or another mobile personnel unit
- a robot or robotic component e.g., a robotic arm
- a computer-generated agent e.g., an animal (e.g., monkey, dog, dolphin, bird, or other trained or trainable animal), or any other agent trained, trainable, configured, or configurable to receive and follow movement instruction.
- an animal e.g., monkey, dog, dolphin, bird, or other trained or trainable animal
- the systems and methods described herein determine a path for a fleet of mobile agents—wherein the fleet may include a combination
- the systems and methods described herein are directed at solving a combination of routing and path planning problems.
- the systems and methods disclosed herein solve the two problems jointly and adapt the solution dynamically to a changing environment and a time-dependent mission goal landscape.
- the planning system In response to an environment and a set of dynamically-changing mission requirements, it is desirable for the planning system to perform replanning of both reactive (local) and deliberative (global) varieties.
- reactive replanning are when a UGV avoids an obstacle or turns to run away from an enemy.
- deliberative replanning is when a UGV discovers a previously-unknown enemy and modifies its path, typically its entire remaining path, to circumvent the enemy and remain hidden en route to its next mission goal.
- Another, more complex, example of deliberative replanning is when a first UGV, upon discovering an enemy and determining that reaching a next mission goal in time is difficult, entails unacceptable risk, or is no longer feasible, trades goals with a second UGV having a substantially unobstructed path to the first UGV's next mission goal.
- the systems and methods described herein treat the operation planning problem as a multi-objective optimization problem to determine an operation plan for multiple UGVs and achieve multiple mission goals while satisfying multiple criteria, such as tactical criteria, as best as possible based on known environmental and tactical situation knowledge available.
- An operation plan includes a set of paths, one path for each UGV, wherein each path includes a sequence of navigation waypoints.
- the path also includes instructions (generally real-time instructions) for local movement between a pair of the waypoints.
- a mission goal includes a geographic location or area to be visited by the UGV; optionally, the mission goal also has an associated temporal constraint, generally expressed as a time window of arrival at, and/or departure from, the area.
- the UGV may be assigned zero or more mission goals.
- a tactical criterion is defined, at least in part, as a property of the operation plan that is desirable for a particular state of the environment, such as, without limitation, enemy avoidance, hazard avoidance, stealth, or rapid achievement of mission goals.
- a path includes an ordered set of waypoints to be visited by a mobile agent.
- FIG. 1 illustrates geographic locations of the waypoints and the inferred directed path along the respective paths for each of the UGVs 101 and 102 , starting at the UGV's respective current location.
- Waypoints A-C are mission points belonging to the path to be traversed by the UGV 101
- the waypoints D-E are mission points belonging to the path to be traversed by the UGV 102 .
- the waypoint 110 (and other hollow circles shown in the figure) denotes a route point
- the waypoint 120 and other solid circles shown in the figure) denotes a travel point.
- a mission point is a waypoint associated with a predetermined mission goal at a specific geographic location; the sequence of mission points along the path assigned to a UGV determines the order in which the UGV accomplishes its assigned missions.
- a sequence of route points determines the general route followed by the vehicle between two mission points.
- a route point may be manipulated—i.e., traded between UGVs, removed from the path, added to the path, or reordered—by the systems and methods described herein, as part of replanning and continual path and/or operation plan improvement.
- a sequence of travel points is used to specify a detailed route followed between two route points. Note that UGV paths may intersect, as shown by 130 in FIG. 1 .
- FIG. 2 ( a ) depicts a block diagram representation 200 of an exemplary method of determining a path having an ordered set of waypoints to be visited by a mobile agent to accomplish a mission.
- the method Based on an optional initial/seed population 202 of paths, the method generates 203 a set of candidate paths based at least partially on a path production heuristic 204 and a multi-objective optimization algorithm 208 .
- the initial/seed population 202 is randomly produced.
- the multi-objective optimization algorithm 208 includes an evolutionary algorithm, such as, for example and without limitation, a genetic algorithm.
- one path is selected 205 for the mobile agent, the selection being at least partially based on a path selection heuristic 206 .
- the method further includes instructing 207 the mobile agent to move according to the selected path.
- the method 200 includes continually and/or iteratively modifying 209 the candidate paths to produce new candidate paths 203 , again, at least partially according to the path production heuristic 204 and the multi-objective optimization algorithm 208 .
- the modification may include a random modification of existing candidate paths to produce new candidate paths.
- a modifier is employed that advocates a path modification based at least partially on a tactical criterion.
- the entire remaining portion of a first mobile agent's path may be recomputed, and this may include addition, deletion, reordering, or trading (with a second mobile agent, if one exists) of waypoints along the first mobile agent's previously-assigned path.
- One or more of these modifications may be independent of a previously chosen or currently in-effect path.
- the method compares a maintained subset 211 of the candidate paths and the previously-selected path 205 , to select a new path 213 subject to the path selection heuristic 206 , and instructs 215 the mobile agent to move according to the newly-selected path.
- determination of what subset of the candidate paths to maintain from one iteration to another is at least partially based on a diversity criterion; a diverse path population ensures that the iteration evolves toward a global solution and does not gravitate to a mere local optimum.
- Selection of the new path may be triggered by any of a number of stimuli. For example, appearance of an enemy proximal to the previously-selected path 205 may trigger selection of the new path so the mobile agent being affected may circumvent the enemy.
- the path production heuristic 204 and the path selection heuristic 206 are mutually independent of each other. Additionally, one or both of the path production heuristic 204 and the path selection heuristic 206 at a particular iteration may be independent of one or more respective counterparts at previous or future iterations.
- One or more of the path production heuristic 204 and the path selection heuristic 206 may be time-dependent; one example of where this is the case is when heuristics are altered due to changing environmental conditions or mission requirements.
- production or selection of a particular candidate path is mutually independent of the production or selection of another candidate path, whether in the same iteration or a different iteration.
- a domain-specific path selection heuristic 206 captures a rule-of-thumb for evaluating the relative worth of a given path (in particular, to enable comparisons with alternative paths).
- This heuristic generally reflects one or more planning constraints, and in particular, the relative importance or interactions between those constraints.
- the path selection rule-of-thumb may capture “expert knowledge” applied by human planners.
- a weighted sum of a set of heuristic computations is employed to form a single criterion for path selection.
- the path production heuristic 204 and the path selection heuristic 206 employed by the systems and methods described herein are generally referred to as Advocates and Critics for Tactical Behaviors (ACTB).
- multiple advocates 252 are used to generate 251 a number of new plans 256 (or improve 251 existing plans) based at least partially on a set of seed/initial or existing plans.
- the new plans 256 are evaluated 253 by a number of critics 254 .
- the results of the evaluations by the critics 254 are combined to produce a measure/score 255 of the quality of each plan, and by which the plans are ranked 258 .
- a subset of the plans are then chosen 257 for further manipulation in a subsequent cycle 250 .
- the choice of plan depends on the path selection heuristic, whereas in another embodiment the choice of plan depends on the path production heuristic that rewards diversity in the candidate path population.
- the planning cycle 250 proceeds in a continual manner (possibly independently of other cycles), and interacts with the world in a continual manner. As better plans are evolved, they may influence the current plan being executed. As new discoveries are made or changes occur in the world, the planning process is informed so that the advocates 252 and the critics 254 may incorporate new information into their processing.
- an advantage of the continual, cyclic nature 250 of the ACTB system is two-fold.
- the set of new plans 256 improves overall, at least partially based on more recent knowledge.
- the critics 254 evaluate the plans based at least partially on the most recent knowledge available.
- the advocates 252 and the critics 254 interact, albeit typically indirectly.
- every advocate is associated with a specific critic, but the relationship need not be tightly linked. This indicates that certain tactical behaviors are associated with certain tactical objectives. For example, in an exemplary embodiment, a road-following advocate tends to generate plans that better satisfy a duration critic.
- distinct advocates drive the planning system in distinct directions.
- the road-following advocate may promote changes opposite to, or substantially distinct from, those promoted by an enemy avoidance advocate. While advocates are typically, though not always, employed and applied independently, this does not pose an issue, because the critics will resolve the best changes or combinations of changes over time.
- distinct critics drive the system in distinct directions.
- the duration critic may penalize plans that a surveillance critic rewards (e.g., a plan that sends UGVs to the tops of hills generally takes longer to execute than a plan that sends the UGVs over flat terrain).
- This competition is indicative of the variety of, at times even conflicting, tactical objectives desirable for a mission.
- the overall needs of the mission are taken into account, and the relative importance of each tactical objective is accommodated by the systems and methods described herein.
- the systems and methods disclosed herein employ, in one embodiment, a mechanism to combine the evaluations of the critics and resolve these issues; the design of this mechanism's functionality can generally be relevant to the overall performance of the ACTB system.
- Three exemplary tactical advocates include mission-allocation advocate, avoid-untraversable advocate, and a road-following advocate. It is understood, however, that these three types are illustrative only and should not be considered limiting in any way. Other applications may have fewer or more advocate types, or, generally, may employ different advocate types.
- the mission-allocation advocate exploits knowledge of the mission goals and their requirements to allocate mission goals to the UGVs. Typically, it operates in two modes, though other numbers and types of modes are conceivable, depending on the context.
- a first exemplary mode is selected if a mission goal remains outstanding, that is, it has not yet been assigned to any UGV within a given path genome.
- the mission-allocation advocate assigns the outstanding mission goal to a randomly-selected UGV by inserting a corresponding mission point into the path of the randomly-selected UGV, next to an existing route point or mission point closest to the outstanding mission point along the UGV's path. The order of the new mission point is determined by the existing information in the UGV's path.
- a second exemplary mode is used when all missions have been assigned within a plan.
- the mission-allocation advocate randomly removes a sequence of one or more mission points—as well as all intermediate waypoints—from a path, and inserts that sequence before or after a randomly-chosen mission point in a randomly-chosen path in the plan.
- the missions may be inserted within the same UGV's path, thereby performing an effective re-ordering of the mission goals, or in the path of a different UGV, thereby performing a switch of mission goals from a first UGV to a second UGV.
- the new ordering of missions is randomly determined.
- the avoid-untraversable advocate exploits terrain knowledge and a model of the movement capabilities of the UGVs to determine routes that do not have waypoints in untraversable terrain. For example, and without limitation, rivers and lakes may be untraversable for particular types of UGVs.
- the advocate identifies waypoints belonging to paths that lie in untraversable terrain, and then randomly selects one of these waypoints and moves it to a location on a traversable terrain. The new location is selected by searching on an arc towards the “traversable-predecessor” of the selected untraversable-waypoint. Any intervening waypoints are eliminated (since they necessarily would have been untraversable).
- the road-following advocate exploits knowledge of the road network to determine a path segment between a pair of mission or route points making maximal use of roads.
- roads are represented symbolically in a geospatial information development toolkit (e.g., OpenmapTM, BBN Technologies, Cambridge, Mass., USA).
- the road-following advocate randomly chooses two mission or route points on a randomly-chosen path.
- the road-following advocate determines the closest road point to each selected waypoint, and obtains the shortest road path between those road points.
- This road path is represented as a sequence of travel points with route points at the each of its termini. The new sequence replaces the path between the original selection points.
- a segment of road may be highly curved or irregular, and as such it may require a large number of points to specify that segment in a piece-wise linear manner. If the travel points were included as possible points of selection by the other advocates, the process of selecting waypoints for adaptation would be overwhelmed by the large number of travel points. For example, an advocate may spend the majority of its time moving road points, and thus be highly ineffective at optimizing the route between mission goals.
- the road lookups of the road-following advocate seem to serve a purpose similar to the shortest-path lookups of existing planning techniques.
- the road lookup is limited to identifying only small road segments, and it has little or no impact upon the exploration of cross-country paths by other advocates.
- an embodiment of the ACTB systems and methods described herein uses multiple tactical advocates to determine routes, and creates routes by making a number of small changes at random locations in the path. This enables the systems and methods described herein to explore a wide variety of routes between the two mission goals and adapt that route according to multiple tactical criteria. For example, some segments may result in a poor fitness according to one tactical critic; eventual removal or modification of those segments by tactical advocates may produce an improved path according to that critic.
- a typical embodiment of the ACTB systems and methods described herein employs several types of tactical critics, all of which return path evaluations that are greater than or equal to 0, where lower numbers indicate better plans. It is understood, however, that these critic types, described below, as well as their typical output ranges, are illustrative only, and should not be construed as limiting in any way. Other applications may have fewer or more types of tactical critics, or, generally, may employ different critic types, possibly having alternative output value ranges.
- Three exemplary types of critics include a traversability critic, a safety critic, and a stealth critic.
- the traversability critic exploits terrain knowledge and a model of the movement capabilities of the UGVs to identify portions of the path that wander into untraversable terrain. According to one practice, the traversability critic returns a penalty value proportional to distance traveled on untraversable terrain over all paths; it is worth noting that in the embodiment described, a path wandering into untraversable terrain is allowed, but is penalized accordingly.
- the safety critic exploits knowledge of the known enemy locations and a model of enemy capabilities to evaluate whether a given plan places one or more UGVs in danger by placing them too close to a known enemy. According to one practice, the safety critic returns a penalty value proportional to the distance traveled within danger range of the enemy, over all candidate paths; it is worth noting that in the embodiment described, a path that places a corresponding UGV dangerously close to the enemy location is allowed, but is penalized accordingly.
- the stealth critic exploits knowledge of the known enemy locations and line-of-sight computations to evaluate whether a given plan puts one or more UGVs at risk by placing them in the line of sight of a known enemy.
- the line of sight is computed using a geospatial information development toolkit and a model of the surveillance capabilities of the enemy.
- the advocate returns a penalty proportional to the distance traveled within surveillance range of the enemy; it is worth noting that in the embodiment described, a path that places a corresponding UGV within the enemy's surveillance range (e.g., direct line of sight, radar observation, sonar observation, etc.) is allowed, but is penalized accordingly.
- critic types employed by one or more embodiments of the systems and methods described herein include a mission-success critic, a total-duration critic, and a max-duration critic, but as mentioned earlier, these exemplary critic types should not be construed as limiting the scope of the systems and methods described herein.
- the ACTB systems and methods described herein facilitate performing continual operation planning within a dynamic environment, in which UGVs move, and wherein knowledge of the environment and tactical situation is time dependent.
- the systems and methods described herein include a simulation-based system in which a deliberative planning process 261 explicitly interacts with a simulated world environment 262 in a continual cycle 263 .
- the operation planning process employs an ACTB-influenced genetic algorithm to evolve multiple notional operation plans 261 for a predetermined number of generations. After a genetic run, the best plan is adopted as the prevailing execution plan, even if the plan was randomly generated at a particular iteration/run.
- the execution plan is communicated to the UGVs, which typically use a simple (non-reactive) execution model to visit their assigned waypoints.
- world events such as the discovery of a new enemy location, may occur. These events trigger the operation planning process to evolve a new plan that incorporates the new tactical situation.
- the execution process may be suspended at regular intervals, and the deliberative process executed to explore further improvements to the prevailing operational plan. Path populations associated with the genetic algorithm are persistent across genetic runs.
- the ACTB-based systems and methods described herein may be programmed in Java and the simulation environment may use a geographic system to represent terrain information, provide basic functionality for making geographic inquiries, and provide a graphical interface.
- Alternatives to the programming language Java and the geographic system OpenMapTM may be employed without departing from the scope of the systems and methods described herein.
- the systems and methods described herein typically employ an evolutionary algorithm to search for a good path for a mobile agent.
- Evolutionary algorithms are well-suited for complex, multi-objective optimization problems.
- evolutionary algorithms offer an additional advantage of being easily tailored to a particular domain for improved performance.
- the systems and methods described herein take advantage of this through use of tactical advocates and tactical critics.
- the advocates include domain-specific mutations that modify a plan based on knowledge of good tactics.
- the critics compute various evaluation metrics corresponding to various criteria indicative of a good plan.
- the structure of the systems and methods disclosed herein makes it easy to add advocates and critics and hence to incorporate additional domain knowledge; similarly, it is straightforward to remove an advocate or critic.
- the ACTB genetic algorithm employed in an embodiment of the systems and methods described herein is based at least partially on the notion that potentially significant improvements to an overall plan are possible by applying a succession of small, goal-directed changes. According to one practice, these goal-directed changes are made using domain-specific genetic operators, termed tactical advocates, as already mentioned.
- the improvements include global changes to one or more paths (in particular, to the remaining portions thereof) assigned to the UGVs.
- ACTB genetically mutated genetic mutation and crossover operators
- the tactical advocates continue to make novel plans rather than continue to rehash old ones.
- three mutation operators are used, each of which performs a mutation on a randomly-chosen chromosome within the genome. The number and nature of the mutation operators is understood to vary based on context, so the three illustrative types described herein should not be considered limiting in any way.
- an insert-waypoint mutation operator randomly selects a waypoint on the chromosome and inserts a single waypoint before or after that point.
- the geographic location of the new waypoint is typically a small random distance in a random direction from a line connecting its neighbors.
- a remove-section mutation operator randomly selects two waypoints in the chromosome and removes them and waypoints in between.
- a nudge-waypoint mutation operator randomly selects a waypoint from the chromosome and modifies its geographic location slightly in a random direction.
- two genetic crossover operators are employed. It is understood that other contexts and applications may include genetic crossover operators distinct from, and fewer or larger in number than, the crossovers described herein.
- the path-crossover operator is applied to a single genome parent, and performs variable-length one-point crossover between two randomly-chosen chromosomes within the genome.
- plan-crossover operator is applied to two genome parents, and performs variable-length one-point crossover between a randomly-chosen chromosome in one parent and a randomly-chosen chromosome in the other.
- the ACTB genetic algorithm accommodates the multi-objective nature of the operation planning problem by using multiple, distinct evaluation components to determine fitness of candidate paths.
- a tactical critic represents a domain-specific evaluation component that computes a single term of a fitness function.
- Each tactical critic typically evaluates how well a given operation plan satisfies a tactical criterion.
- a critic for safety may evaluate a plan to determine how much danger the UGVs are placed in, due to traveling too close to a known enemy.
- outputs of multiple critics are combined using a weighted sum to form a single fitness value.
- the weights associated with the critics reflect the tactical priorities of the operation.
- the ACTB genetic algorithm accommodates a constraint-based nature of the operation planning problem by allowing ostensibly “illegal” individual paths into the population of candidate paths or path sets; an illegal path includes a path that violates at least one problem constraint.
- a path set includes, as the name implies, a set of paths; every mobile agent has an associated path belonging to the path set.
- the ACTB genetic algorithm-based systems and methods described herein employ fitness values to reflect the magnitude of a violation.
- a tactical critic evaluates a candidate operation plan (i.e., candidate path or path set) against a tactical criterion, it assigns a penalty if the plan violates that criterion.
- a critic assigned the task of determining whether a candidate path is traversable may still accept a path that crosses water (an untraversable terrain), but will assign a high penalty to the path.
- critics typically assign a penalty proportional to the degree of the violation.
- the distance to be “traveled” in water at least in part determines a magnitude or severity of the penalty imposed.
- a genome is defined as a set of n chromosomes, wherein each chromosome defines the path for one of the UGVs as a variable-length sequence of geographic locations, or waypoints, to be visited by the UGV for a remainder of the operation plan.
- each chromosome defines a piece-wise linear directed path.
- the first waypoint in each path represents the next waypoint to be visited by the corresponding UGV.
- the first segment of the path is understood to be the straight line between the UGV's current location and the first waypoint in the path.
- an exemplary genetic representation includes three types of waypoints, each representing a different conceptual aspect of a path. It is understood, however, that not each of the three types of waypoints is required in all embodiments.
- a route point may be genetically manipulated, e.g, mutated.
- a notable feature of travel points is that they generally are not available for selection as points of genetic manipulation by the systems and methods described herein. Rather, they are used to incorporate specific path segments between two consecutive mission points or route points. The relative benefits of these segments may then be evaluated through a genetic search.
- route-points are used by a road-following advocate to represent complex road segments.
- FIG. 3 illustrates a sample genome having two chromosomes 300 and 350 , respectively corresponding to a first UGV and a second UGV.
- Waypoints A-C are mission points belonging to the chromosome 300
- the waypoints D-E are mission points belonging to the chromosome 350 .
- each of the route points 301 - 305 and 311 - 313 is represented by a hollow circle
- each of the travel points 341 - 345 (and others not labeled in the figure) is represented as a solid circle.
- the three types of waypoints are used to enable genetic manipulations targeted to different levels of planning. For example, manipulation of mission points and their order performs the task of scheduling missions, whereas manipulation of route points performs the task of route planning. In the context of a UGV platform, little to no distinction is required between the three types of waypoints, except that travel points typically may not be genetically mutated or otherwise manipulated.
- the deliberative UGV planning problem shares some characteristics with a classic optimization problem, called the vehicle routing problem with time windows (VRPTW). In both problems, multiple vehicles are to move in such a way as to arrive at particular locations during predetermined time intervals.
- the UGV planning problem has nontrivial additional complications.
- One is that the paths between locations are generally not well-defined, and a task of the planning algorithm is to find a good path over a combination of roads and off-road terrain.
- a second complication is that the deliberative UGV planning entails a greater number of criteria to be considered for determining a good plan.
- the operation planning systems and methods described herein are well-suited to solve the VRPTW.
- the problem involves predetermined time windows during which the mobile agents—UGVs in our illustrative example—are to arrive at, and/or depart from, a subset of the waypoints.
- the systems and methods described herein employ, in one aspect, a greedy algorithm providing a unique interpretation of a path with respect to time, allowing the details of the path to be varied quite significantly.
- a greedy algorithm providing a unique interpretation of a path with respect to time, allowing the details of the path to be varied quite significantly. This is unlike a traditional “shortest-paths” lookup, wherein a set of fixed routes between key locations is stored; it also is unlike “dynamic” shortest-paths lookup, wherein the shortest routes between key locations are occasionally recomputed to accommodate new information.
- the greedy algorithm provides a suitable framework for evaluating a path's duration as well as the success or failure of accomplishing the missions as part of the constrained operation planning problem.
- This problem may be defined more concretely as follows: given one or more mobile agents operating within a complex environment (e.g., urban environment, terrain with multiple features and terrain types, multiple known enemy locations, etc.), given one or more mission goals involving visiting specific locations within that environment, and given a set of general and mission-specific constraints, both hard and soft, imposed on prospective routes (e.g., the route must avoid enemies, the missions must be achieved as soon as possible, etc.), the constrained deliberative planning problem includes determining a route for each mobile agent such that over all mobile agents in the fleet, missions are successfully accomplished while satisfying substantially all hard constraints and soft constraints as best as possible.
- the greedy algorithm employed by the systems and methods described herein assumes that a mobile agent will travel between the waypoints along its assigned path at a speed less than or equal to a maximum speed allowable or feasible for the terrain over which it is traveling.
- the algorithm also assumes that each of a subset of the waypoints represents an attempt to fulfill a respective mission, subject to a specified time window.
- An aspect of the greedy algorithm includes the notion that if a vehicle, traveling at its maximum speed, arrives at a mission point earlier than a prescribed time window, it may wait or move sufficiently slowly to ensure a timely arrival and/or departure.
- a mission is considered accomplished as long as a vehicle arrives to accomplish the mission at any point between the start and end time. This provides a mechanism for stepping through the path and evaluating the times at which the missions are accomplished.
- the greedy algorithm may be employed within an evolutionary algorithm to solve the deliberative planning problem, when a route may be evaluated, typically dynamically, against multiple criteria other than simple travel time. That is to say, there is no a priori “best” route between a pair of mission locations; this variability can have an impact on the success of the deliberative planning effort. This is the case, for example, in planning for one or more UGVs whose routes are not only to be sufficiently quick, but must also enable the UGV to satisfy a variety of tactical constraints, such as avoiding enemies.
- An issue when evaluating such a plan for deliberative planning includes a tradeoff between how to schedule the times of the missions versus the computation of the travel times when routes are not static in quality.
- Existing approaches are generally capable of performing one task or the other, but not both. If scheduling is performed, then a fixed routing lookup is assumed. If routing is performed, then the times on the missions are assumed to be secondary, and the focus is shifted to path planning.
- each mission goal is defined as a geographic location with an “earliest” time of arrival and a “latest” time of arrival; additionally or alternatively, an earliest time of departure or latest time of departure may be specified.
- the duration of travel between each successive pair of waypoints is computed—for example, by assuming that a mobile agent, such as a UGV, travels at its established maximum speed for each terrain type; alternatively, the UGV may be assumed to travel at an established average speed for each terrain type.
- the greedy algorithm steps through all waypoints on the chromosome in order, evaluates a duration of the path segments, and assesses mission success as it proceeds.
- the algorithm assumes that if a UGV arrives at a mission point too early, it is possible for that UGV to arrive on time (e.g., through the simple mechanism of driving more slowly or by waiting a prescribed amount at an appropriate venue).
- a notion of a “minimum wait time” for the UGV at each mission point is employed to model completion of the missions. It is assumed that the UGV will wait for the minimum possible amount of time at each of its assigned mission points to satisfy the time window constraint.
- the greedy algorithm assumes a model of UGV movement speed over different terrain types, and evaluates a travel time between successive mission points based at least partially on distances and terrain traversed over the (piece-wise linear) path between them.
- each UGV is assumed to travel as fast as possible between mission points (though, in an alternative embodiment, an average speed is considered), and then wait as little as needed (if early) to meet a prescribed time window requirement (i.e., the greedy choice).
- a prescribed time window requirement i.e., the greedy choice.
- the mission-success critic evaluates how well a given plan comes to accomplishing mission goals, and returns a penalty proportional to the number of failed goals and a corresponding degree of failure.
- the total-duration critic evaluates how long each UGV takes to execute its chromosome, and returns the sum of the durations of all chromosomes.
- the max-duration critic evaluates how long each UGV takes to execute its chromosome, and returns the longest duration over all chromosomes.
- FIG. 4 illustrates a sample evaluation of the duration and mission success of a path chromosome.
- Table 1 provides a summary snapshot of FIG. 4 .
- the systems and methods described herein assume that it would wait the minimum amount of time required to meet the “earliest” mission time window, and then move on; the arrival time window for the mission point A is 20 minutes for ‘A’, the interval [12:10 and 12:30]). At each subsequent mission point this same computation occurs. If the earliest possible arrival time lies within the prescribed time window, then the minimum wait time is 0 minutes (e.g., ‘C’).
- the greedy algorithm employed by the systems and methods described herein provides, in a typical embodiment, a unique way of interpreting an inherently spatial property (namely, travel time along a route) within the context of an inherently temporal constraint (namely, to accomplish the missions during prescribed time windows).
- this provides a mechanism for exploring variations both in the routes (“space”) and in the order of the missions (“time”), with little, if any, loss of capability.
- space the routes
- time the order of the missions
- a plan may be evolved that correctly fulfills the mission time requirements while obeying the necessity of travel and multiple constraints upon that travel.
- FIG. 5 depicts a prototype application 500 of an embodiment of the ACTB systems and methods described herein, directed at the problem of deliberative planning of UGV routes.
- the figure illustrates an example of where the application fits into a larger on-vehicle processing system.
- the vehicle perception subsystem 502 processes and analyzes the sensor data 504 to produce an evolving description of a region that at least partially surrounds the vehicle. This description is reconciled 505 ( a )- 505 ( b ) with a priori topographic information and ongoing situation assessment 506 , and intelligence 508 that is streamed to, collected by, or otherwise communicated to the vehicle. Similarly, and optionally, inputs and parameter settings are received by the vehicle from the operator 510 .
- the adaptive tactical behaviors box 516 encompasses what is shown in FIG. 2 ( a ).
- the systems and methods described herein produce a sequence of vehicle commands 512 of waypoints expected to be followed by the vehicle.
- commands sensor commands are issued 514 to the vehicle, instructing the vehicle to manipulate one or more sensors or data collection instruments.
- the command 514 may include instructions to the vehicle to turn on an onboard camera and point it to a particular target. These waypoints are turned over to a navigation subsystem of the systems and methods described herein.
- a property of an effective, automated replanning system and method for UGVs includes a capability for each UGV to cooperate with other UGVs in developing effective global solutions, but to accomplish this task without requiring the full knowledge of all known intelligence across all UGVs.
- a centralized solution while it may be capable of developing highly-optimized solutions over all mission objectives, is impractical, because it requires an ability to maintain complete centralized knowledge of all environmental information, as well as update every UGV essentially at all times.
- a distributed system with full knowledge communicated to every UGV, while it may allow processing requirements to be distributed and enable each UGV to make globally-optimized decisions, is also impractical, because it requires essentially full communication and rapid update of newly-discovered knowledge.
- a mobile agent such as a vehicle
- vehicles exchange sensor-derived information and may receive updated intelligence or directives from one or more human operators.
- FIG. 6 illustrates a possible communication network of multiple UGVs, possibly including a human operator, over which collaborative planning may occur. The exemplary system and method depicted by FIG. 6 illustrates that the deliberative planning may occur with sharing of partial information, not necessarily complete information, among the vehicles.
- An aspect of the systems and methods described herein includes effectively planning locally while achieving path solutions that are optimized to some degree at a global level; as mentioned earlier, the entirety of the remainder of a path for a mobile agent may change as a result of the continual updating of the population of candidate paths, which may occur in part from an unexpected local stimulus, such as an unpredicted appearance of an enemy.
- Each UGV may execute its own directed evolutionary system, and independently evolve new candidates for the selected team plan based upon knowledge available to that UGV.
- Each UGV may periodically transmit its best plan, or sets of plans, to other UGVs with which it communicates. The UGVs may then perform an information handshake to agree on changes to be incorporated. For example, in such an embodiment, plans requiring two UGVs to swap objectives may be implemented if those UGVs are in communication.
- the niche-based approaches to genetic algorithms are suitable for providing valuable insights into the problem. In a niche-based approach, different niche populations are maintained and optimized on information locally available to that niche. In one embodiment, genetic operators occasionally transmit information between niches, thereby providing the capability for the information in one local search to affect another local search, even if indirectly.
- UGVs Another property of an effective automated replanning system for UGVs is a capability to develop plans wherein multiple UGVs demonstrate cooperative group behaviors that enable them to solve their missions more effectively. For example, and without limitation, some UGVs may provide reconnaissance support for others. UGVs may cooperate to maintain effective communication networks.
- ACTB systems and methods described herein provide both types of solutions.
- a formation advocate may be specialized for a certain known formation and promote its use.
- advocates for generic cooperative behaviors, such as scouting and rear support may promote emergence of novel cooperative responses.
- An effective replanning system is typically robust to planning under unknown environments.
- a UGV may venture into terrain that is poorly understood.
- basic capabilities of the perception and navigation system generally enable a UGV to adjust to most environments, it is desirable for the planning system to recognize and accommodate limitations of those systems. For example, if a UGV is incapable of traversing a terrain at least a portion of which is above a certain grade, then identifying and planning routes around such terrain can be important tactical behaviors. Interaction between scouting behaviors and planning effectiveness is desirable.
- the ACTB systems and methods described herein facilitate rapid prototyping and study of a variety of scouting behaviors.
- a realistic replanning system or method it is desirable for a realistic replanning system or method to be capable of having human planners control the planning process. Moreover, it is desirable for such a system or method to be capable of controlling the planning to varying degrees under variable circumstances. While the systems and methods described herein allow the planning process to be completely automated, they may be influenced indirectly by providing new intelligence information. In a typical practical setting, a human operator will be able to take direct control, with automated planning taking a back seat. The human operator may provide a specific plan (or portion of a plan) designed to remain fixed. In an ACTB system or method, the human may adjust the priorities of the planning system to reflect the most recent mission priorities through the selection of critics and their weights. For example, and without limitation, while safety and enemy avoidance may be the overriding factors under normal circumstances in a particular exemplary embodiment, some missions may require extreme risk to be acceptable in order to reach a mission objective.
- test scenarios have been developed wherein multiple UGVs cooperate to solve complementary and competing mission goals while both minimizing mission completion time and minimizing risk to mission success.
- One such exemplary scenario incorporates an actual terrain that the U.S. Army uses as a test bed for some of its UGVs.
- the systems and methods described herein demonstrate that continually-improved plans may be generated quickly, both before and during plan execution, in response to changes in the tactical situation or other environmental stimuli or time-dependent mission demands.
- the ACTB systems and methods described herein were tested under four conditions to demonstrate the effectiveness of the tactical critics for multi-objective optimization and examine the search capabilities of ACTB when using tactical advocates in conjunction with traditional genetic operators over using traditional genetic operators alone.
- the experiment examined the basic tactical route planning capabilities of the ACTB systems and methods described herein. Time scheduling aspects of the problem were minimized by widening the mission time windows. However, path duration was still a prized factor (i.e., do all the missions as soon as possible).
- Critic weights were selected to assign a very high penalty to untraversable portions of the routes and to missed missions, a moderate penalty to exposure to the enemy (i.e., completing the mission is more important than avoiding the enemy), and a small penalty to path duration; the penalty for maximum-duration was weighted twice as strongly as total-duration, to encourage a more balanced distribution of mission goals among UGVs.
- FIGS. 7-8 Final plans developed in the four experimental conditions are illustrated in FIGS. 7-8 .
- the systems and methods described herein were employed to examine basic routing in a simple situation with no known enemies.
- the systems and methods described herein were employed to examine tactical routing in a situation with two known enemies.
- three friendly UGVs shown as 701 - 703 in FIG. 7 ( a )
- each UGV's path is indicated by a different line thickness; for example, the path 730 is associated with the UGV 702 .
- Off-road terrain 740 is shown in white (background), rivers and other waterways are shown in light gray (e.g., 710 ), roads are shown in dark gray (e.g., 720 ), and missions points are depicted by the circles (e.g., 750 ). Intersections of road with water are considered to have bridges. Travel on-road is assumed to be roughly 10 times faster than travel off-road. A similar convention is used in subsequent FIGS. 7 ( b ), 8 ( a )- 8 ( b ), and 10 ( a )- 10 ( c ), even though these figures are not specifically labeled as such. These figures show that in all conditions, the system was able to evolve plans that were traversable, met all mission goals and distributed mission goals among all three UGVs. In both enemy conditions, the evolved paths avoided the known enemies.
- Constraint 1 Avoid Exposed Areas: generally, it is a good idea to avoid areas that are more visible and have a higher risk of exposure to the enemy.
- Heuristic 1 For a given location, compute how exposed the location is based on the number of points it is in line-of-sight with, within a radial distance, e.g., 30 meters.
- Constraint 2 Avoid Lethal Areas: generally, it is a good idea to avoid traveling too close to an area occupied by an enemy. Heuristic 2(a): If the UGV is within the enemy's firing range, then it is in danger, otherwise not. Heuristic 2(b): Danger rating is proportional to distance from the enemy.
- Constraint 3 Engage enemy: generally, it is a good idea to attack and destroy an enemy. Heuristic 3(a): If the UGV is within firing range of the enemy, then it is engaged, otherwise not. Heuristic 3(b): Engagement score is inversely proportional to distance of the UGV from the enemy.
- Constraint 4 Duration: generally, it is a good idea to reach destinations as quickly as possible.
- Heuristic 4(a) distance duration is proportional to the length of the path.
- Heuristic 4(b) travel time duration is based at least partially on the average speed of the UGV over each terrain type encountered along the path.
- Constraint 5 Avoid an Unknown: generally, it is a good idea to travel across terrain that has been recently surveilled.
- Heuristic 5(a) for a given location, compute how many points within a certain radius (e.g., 20 meters) have been visited previously by the UGV or by another friendly mobile agent.
- Heuristic 5(b) Time-Decayed Observation: for a given location, compute how many points within a certain radius (e.g., 20 meters) have been visited recently by the UGV or by another friendly mobile agent.
- FIG. 9 illustrates fitness of the best individual in each generation for each of four experimental conditions. Each condition was run for 30,000 generations to ensure convergence, but all conditions had converged by 7,000 generations. The range of fitness values was very large due to high penalty values assigned by the tactical critics, and the results are graphed on a logarithmic scale to emphasize the changes over evolution. A fitness value under 1,000 indicates no penalty except for duration of path.
- the ACTB systems and methods described herein clearly demonstrated better plans when using advocates, as illustrated by the better (i.e., lower) fitness values overall and by the rapid achievement of a plan with no major penalties around 1,000 generations as opposed to over 4,000 generations for traditional conditions.
- the final plan generated in the advocate conditions completed all missions in roughly half the time of the plans generated in the traditional conditions. As illustrated in FIGS. 7 ( b ) and 8 ( b ), this improvement is substantially due to improved road following.
- FIG. 10 illustrates a plan under execution immediately before discovery of an enemy.
- FIG. 10 ( b ) illustrates the re-planning activity initiated upon the discovery (after a few generations).
- FIG. 10 ( c ) illustrates the new plan generated after 300 generations and passed to the two UGVs (shown in the top left of each figure) for execution. Note that after replanning, all UGV paths avoid the area surrounding the enemy.
- the systems and methods described herein constitute an approach to deliberative planning for coordinating UGVs.
- This approach is based at least partially on representing the planning problem as a multi-objective optimization problem, which in an exemplary embodiment is solved using a genetic algorithm to dynamically search for a good path or path set.
- Multiple evaluation components called tactical critics
- tactical advocates enable the evolution of plans satisfying multiple tactical constraints.
- Domain-specific operators called tactical advocates, greatly speed the search process yielding rapid plan turnaround.
- ACTB systems and methods described herein maintain good plans in response to such changes, and are flexible enough to having advocates and critics developed and introduced for additional tactical behaviors, as well as accepting new mechanisms for distributing the evolutionary algorithm to make ACTB amenable to implementation within UGV platforms directly.
- the systems and methods described herein may be applied in non-military contexts.
- the systems and methods described herein may be employed to provide operation planning for a fleet of courier delivery vehicles in a metropolitan city.
- Domain-specific path selection and production heuristics may be used in this context. For example, avoiding high-traffic areas, avoiding safety-compromised intersections or low-throughput streets and highways may be used to at least partially guide the operation planning of the deliveries.
- time window constraints may be used to schedule delivery of packages during desirable or mandatory times intervals.
Abstract
Description
- This application incorporates by reference in entirety, and claims priority to and benefit of, U.S. Provisional Patent Application No. 60/555,778, filed on 24 Mar. 2004.
- Technologies for the control of unmanned ground vehicles (UGVs) attempting to perform local path navigation while traversing unknown, off-road terrains permit simple longer-range path planning, such as navigation between human-specified waypoints. However, these technologies have yet to develop automated plan generation strategies toward achieving higher-level mission goals (e.g., reconnaissance, surveillance, and target acquisition) in light of changing environmental conditions, evolving mission requirements, and a desire or need to coordinate movement of multiple vehicles.
- Approaches that have been used to address path planning and routing have included traditional artificial intelligence (Al) algorithms. For example, classical planning, hierarchical-task-network planning, and case-based planning use symbolic planning based on logic and reasoning. However, the problem of interest is essentially numeric, and hence less suited for reasoning about goals and sub-goals. Although Al planning techniques may be applied to a higher-level strategic planning problem—i.e., how to decide what the mission goals are—their utility is diminished in the context of tactical planning problems wherein the mission goals are already known.
- Other approaches, such as coordinated robot planning, focus on collision avoidance as a primary criterion for path planning. In many contexts, there is so much space compared to the number of vehicles that the probability of collision is slim. Yet other multi-robot planning algorithms are primarily concerned with formations and moving of vehicles in unison, and not on balanced workload distribution. Investigations into coordinating robot behavior by dividing the workload have generally been reactive (local) rather than deliberative (global), losing the benefits of planning ahead for multiple goals. Furthermore, when assigning goals, path planning is treated as a separate problem, thus substantially ignoring an enemy or obstacle between a vehicle/robot and a nearby goal point.
- One investigation into controlling UGVs is the Distributed Architecture for Mobile Navigation (DAMN), carried out at Carnegie Mellon University. DAMN includes behaviors such as “road following,” “seeking the next navigation goal,” “obstacle avoidance,” and “avoid hazards.” Each behavior provides a vote on the next direction to take. A command arbiter decides upon the best direction, which is then acted upon by the UGV. DAMN also includes a global navigator to determine a full path to a goal position. However, DAMN uses a D* (dynamic A*) search algorithm. This approach, however, does not accommodate as many criteria and as much information, at the deliberative planning level, that are generally at play, when determining mission assignments and paths that are not fooled by local gradients.
- There is a need for improved operation planning systems and methods for one or more mobile agents that incorporate any combination of deliberative and reactive planning strategies. The systems and methods described herein are generally directed at, among other things, embodying local as well as global criteria in operation planning (including path planning and routing), yielding global solutions more efficiently and reliably, handling a myriad of constraints typically present in various mobile agent operation planning contexts of interest, placing a fair emphasis on balancing workload distribution among members of a fleet of mobile agents, often dynamically, and having sufficient flexibility to incorporate new criteria into, remove existing criteria from, or reorder priorities in, an operation plan design process. In one embodiment, the systems and methods disclosed herein employ a domain-specific multi-objective optimization algorithm, such as, without limitation, a context-influenced genetic algorithm, to dynamically determine paths for one or more mobile agents to accomplish one or more mission goals; a path may include instructions on traversing at least a portion thereof.
- According to one aspect, the invention includes a method of determining a path having an ordered set of waypoints to be visited by a mobile agent to accomplish a mission. The method includes (a) producing candidate paths using a multi-objective optimization algorithm, subject to a path production heuristic; (b) selecting a path from the candidate paths, subject to a path selection heuristic; (c) instructing the mobile agent to move according to the selected path; (d) modifying a maintained subset of the candidate paths to produce a new candidate path using the algorithm and subject to the path production heuristic; (e) designating either the currently-selected path or the new candidate path as the newly-selected path, subject to the path selection heuristic; and (f) instructing the mobile agent to move according to the newly-selected path.
- In one particular embodiment, the method includes, after step (f), repeating steps (d)-(f) at least once prior to the mobile agent accomplishing the mission. In another embodiment, the multi-objective optimization algorithm includes an evolutionary algorithm, such as, for example and without limitation, a genetic algorithm.
- According to one practice, modifying the maintained subset of candidate paths is performed continually (e.g., in the background), even after instruction has been issued, as in step (c), to the mobile agent to commence movement according to the selected path. This, at least in part, is to produce, with time, an improved path to be selected for the mobile agent, and to dynamically respond to stimuli in the environment or to time-sensitive mission requirements.
- According to one practice, the subset of candidate paths to be maintained satisfies a diversity criterion, thereby increasing a likelihood of approaching a globally optimal, and not merely a locally optimal, solution. According to one embodiment, the path selection heuristic and the path production heuristic are mutually independent; according to one particular practice, this independence hold laterally across a given iteration of the steps (d)-(f), or longitudinally across different iterations. In an illustrative embodiment, production of one candidate path is independent of production of another candidate path, whether in the same iteration/generation or across different iterations/generations. Additionally, one or more of the path production heuristic and the path selection heuristic may be time dependent. The heuristics may be influenced by environmental characteristics and/or evolving mission requirements. According to various embodiments, modifying the candidate paths includes a combination of adding, deleting, and reordering waypoints along a path belonging to the candidate paths.
- The systems and methods described herein are suitable for operation planning applications involving a fleet of mobile agents. According to one aspect, the invention includes a method of determining paths for a fleet of mobile agents to accomplish missions, every path having an ordered set of waypoints to be visited by a corresponding mobile agent. The method includes: (a) producing candidate path sets using a multi-objective optimization algorithm, subject to a path production heuristic; (b) selecting a path set from the candidate path sets, wherein every mobile agent has an associated path belonging to the selected path set, subject to a path selection heuristic; (c) instructing a first subset of the mobile agents to move according to paths respectively associated with the first subset; (d) modifying a maintained subset of the candidate path sets to produce a new candidate path set using the algorithm, subject to the path production heuristic; (e) designating either the selected path set or the new candidate path set as the newly-selected path set, subject to the path selection heuristic; and (f) instructing a second subset of the mobile agents to move according to paths belonging to the newly-selected path set, respectively associated with the second subset.
- In one particular practice according to this embodiment, the method includes, after step (f), repeating steps (d)-(f) at least once prior to the mobile agents accomplishing the missions. In one embodiment, the multi-objective optimization algorithm includes an evolutionary algorithm, such as, for example, a genetic algorithm. Methods of evolutionary algorithms in general, and genetic algorithms in particular, are described in “Multi-Objective Optimization Using Evolutionary Algorithms,” by Kalyanmoy Deb, John Wiley & Sons, 2001, ISBN: 047187339X.
- According to various embodiments, modifying the candidate path sets includes a combination of adding, deleting, and reordering waypoints along a path belonging to the candidate paths. Additionally, the modifying may include trading of one or more waypoints between two or more of the mobile agents, as needed or as preferred.
- According to one practice, the path selection heuristic and the path production heuristic are mutually independent. In one particular embodiment, the path selection heuristic at one iteration is independent of the path selection heuristic at another iteration. In another embodiment, the same is true for the path production heuristic. One or both of the path selection heuristic and the path production heuristic may be time dependent. In yet another embodiment, production of candidate path sets is performed continually, possibly in the background; this is at least partially to dynamically update path sets so an improved path set is selected for the fleet of mobile agents. The dynamic updating may at least in part be influenced by environmental influences or evolving mission requirements.
- Further features and advantages of the invention will be apparent from the following description of illustrative embodiments, and from the claims.
- The following figures depict certain illustrative embodiments of the invention in which like reference numerals refer to like elements. These depicted embodiments are to be understood as illustrative of the invention and not as limiting in any way.
-
FIG. 1 depicts an inferred path set having two paths to be traversed by two respective mobile agents from their respective reference locations. -
FIG. 2 (a) depicts a representative block diagram of an illustrative embodiment of a method of determining a path having an ordered set of waypoints to be visited by a mobile agent to accomplish a mission. -
FIG. 2 (b) depicts an embodiment of a continually-running operation planning algorithm that includes evolution and improvement. -
FIG. 2 (c) depicts an illustrative continual interaction of the operation planning algorithm with the real-world environment. -
FIG. 3 depicts a genome of a path set having two chromosomes representative of two respective sequences of waypoints to be visited by two respective mobile agents. -
FIG. 4 depicts an illustrative greedy evaluation of the duration and mission success associated with a chromosomal representation of a path associated with a mobile agent. -
FIG. 5 depicts an embodiment of a tactical behavior planning algorithm in the context of an unmanned ground vehicle (UGV) navigation system. -
FIG. 6 depicts an embodiment of a communication network of multiple UGVs and a human operator over which collaborative planning occurs using distributed processing and partial information sharing. -
FIG. 7 (a) depicts exemplary evolved paths with no known enemies using genetic operators. -
FIG. 7 (b) depicts exemplary evolved paths with no known enemies using genetic operators and tactical advocates. -
FIG. 8 (a) depicts exemplary evolved paths with two known enemies using genetic operators. -
FIG. 8 (b) depicts exemplary evolved paths with two known enemies using genetic operators and tactical advocates. -
FIG. 9 depicts the fitness of the best individual path by generation for each of four experimental conditions. - FIGS. 10(a)-10(c) depict a sequence of three plans illustrating, respectively, an execution plan before, a new plan substantially immediately upon, and a new plan shortly after discovery of a new enemy.
- To provide an overall understanding, certain illustrative practices and embodiments will now be described, including a system and method for determining a path having an ordered set of waypoints to be visited by a mobile agent to accomplish a mission. In a typical embodiment, the mobile agent includes a vehicle, which may be manned or unmanned. For example, and without limitation, the vehicle may include a sea, ground, air, or space vehicle, or an amphibious vehicle capable of movement in, and across a boundary of, two or more terrain types (e.g., a sea-ground amphibious vehicle, an amphibious craft capable of traveling in and beyond a planet's atmosphere, etc.). The sea vehicle may be capable of movement on an aquatic surface region, subsurface region, or both. Analogously, a ground vehicle may be capable of movement underground, on a ground surface, or both.
- In an alternative embodiment, the mobile agent includes a human (e.g., a soldier, a rescue worker, or another mobile personnel unit), a robot or robotic component (e.g., a robotic arm), a computer-generated agent, an animal (e.g., monkey, dog, dolphin, bird, or other trained or trainable animal), or any other agent trained, trainable, configured, or configurable to receive and follow movement instruction. In a typical embodiment, the systems and methods described herein determine a path for a fleet of mobile agents—wherein the fleet may include a combination of the agent types described above or their equivalents—to accomplish a set of missions.
- The systems and methods disclosed herein are described primarily in the context of unmanned ground vehicles (UGVs). However, it is understood that these systems and methods may be adapted, modified, and applied in other applications, and that such other additions, modifications, and uses will not depart from the scope hereof.
- In one aspect, the systems and methods described herein are directed at solving a combination of routing and path planning problems. The systems and methods disclosed herein solve the two problems jointly and adapt the solution dynamically to a changing environment and a time-dependent mission goal landscape.
- In response to an environment and a set of dynamically-changing mission requirements, it is desirable for the planning system to perform replanning of both reactive (local) and deliberative (global) varieties. Examples of reactive replanning are when a UGV avoids an obstacle or turns to run away from an enemy. An example of deliberative replanning is when a UGV discovers a previously-unknown enemy and modifies its path, typically its entire remaining path, to circumvent the enemy and remain hidden en route to its next mission goal. Another, more complex, example of deliberative replanning is when a first UGV, upon discovering an enemy and determining that reaching a next mission goal in time is difficult, entails unacceptable risk, or is no longer feasible, trades goals with a second UGV having a substantially unobstructed path to the first UGV's next mission goal.
- In one aspect, the systems and methods described herein treat the operation planning problem as a multi-objective optimization problem to determine an operation plan for multiple UGVs and achieve multiple mission goals while satisfying multiple criteria, such as tactical criteria, as best as possible based on known environmental and tactical situation knowledge available.
- An operation plan includes a set of paths, one path for each UGV, wherein each path includes a sequence of navigation waypoints. In a typical embodiment, the path also includes instructions (generally real-time instructions) for local movement between a pair of the waypoints.
- A mission goal includes a geographic location or area to be visited by the UGV; optionally, the mission goal also has an associated temporal constraint, generally expressed as a time window of arrival at, and/or departure from, the area. The UGV may be assigned zero or more mission goals.
- A tactical criterion is defined, at least in part, as a property of the operation plan that is desirable for a particular state of the environment, such as, without limitation, enemy avoidance, hazard avoidance, stealth, or rapid achievement of mission goals.
- As defined herein, a path includes an ordered set of waypoints to be visited by a mobile agent.
FIG. 1 illustrates geographic locations of the waypoints and the inferred directed path along the respective paths for each of theUGVs UGV 101 and the waypoints D-E are mission points belonging to the path to be traversed by theUGV 102. The waypoint 110 (and other hollow circles shown in the figure) denotes a route point, whereas the waypoint 120 (and other solid circles shown in the figure) denotes a travel point. - A mission point is a waypoint associated with a predetermined mission goal at a specific geographic location; the sequence of mission points along the path assigned to a UGV determines the order in which the UGV accomplishes its assigned missions. A sequence of route points determines the general route followed by the vehicle between two mission points. A route point may be manipulated—i.e., traded between UGVs, removed from the path, added to the path, or reordered—by the systems and methods described herein, as part of replanning and continual path and/or operation plan improvement. A sequence of travel points is used to specify a detailed route followed between two route points. Note that UGV paths may intersect, as shown by 130 in
FIG. 1 . -
FIG. 2 (a) depicts ablock diagram representation 200 of an exemplary method of determining a path having an ordered set of waypoints to be visited by a mobile agent to accomplish a mission. Based on an optional initial/seed population 202 of paths, the method generates 203 a set of candidate paths based at least partially on apath production heuristic 204 and amulti-objective optimization algorithm 208. According to one practice, the initial/seed population 202 is randomly produced. In a typical embodiment, themulti-objective optimization algorithm 208 includes an evolutionary algorithm, such as, for example and without limitation, a genetic algorithm. - From the candidate paths, one path is selected 205 for the mobile agent, the selection being at least partially based on a
path selection heuristic 206. In this embodiment, the method further includes instructing 207 the mobile agent to move according to the selected path. - Optionally, the
method 200 includes continually and/or iteratively modifying 209 the candidate paths to producenew candidate paths 203, again, at least partially according to thepath production heuristic 204 and themulti-objective optimization algorithm 208. In one embodiment, the modification may include a random modification of existing candidate paths to produce new candidate paths. In an alternative embodiment, a modifier is employed that advocates a path modification based at least partially on a tactical criterion. In the modification executed by the systems and methods described herein, the entire remaining portion of a first mobile agent's path may be recomputed, and this may include addition, deletion, reordering, or trading (with a second mobile agent, if one exists) of waypoints along the first mobile agent's previously-assigned path. One or more of these modifications (addition, deletion, reordering, and trading of waypoints) may be independent of a previously chosen or currently in-effect path. - According to one embodiment, the method compares a maintained
subset 211 of the candidate paths and the previously-selectedpath 205, to select anew path 213 subject to thepath selection heuristic 206, and instructs 215 the mobile agent to move according to the newly-selected path. According to one practice, determination of what subset of the candidate paths to maintain from one iteration to another is at least partially based on a diversity criterion; a diverse path population ensures that the iteration evolves toward a global solution and does not gravitate to a mere local optimum. - Selection of the new path may be triggered by any of a number of stimuli. For example, appearance of an enemy proximal to the previously-selected
path 205 may trigger selection of the new path so the mobile agent being affected may circumvent the enemy. In an exemplary embodiment, thepath production heuristic 204 and the path selection heuristic 206 are mutually independent of each other. Additionally, one or both of thepath production heuristic 204 and the path selection heuristic 206 at a particular iteration may be independent of one or more respective counterparts at previous or future iterations. One or more of thepath production heuristic 204 and the path selection heuristic 206 may be time-dependent; one example of where this is the case is when heuristics are altered due to changing environmental conditions or mission requirements. In yet another illustrative embodiment, production or selection of a particular candidate path is mutually independent of the production or selection of another candidate path, whether in the same iteration or a different iteration. - In one aspect, a domain-specific
path selection heuristic 206 captures a rule-of-thumb for evaluating the relative worth of a given path (in particular, to enable comparisons with alternative paths). This heuristic generally reflects one or more planning constraints, and in particular, the relative importance or interactions between those constraints. Analogously to the path production heuristic, the path selection rule-of-thumb may capture “expert knowledge” applied by human planners. In an exemplary embodiment, a weighted sum of a set of heuristic computations is employed to form a single criterion for path selection. - In one embodiment, the
path production heuristic 204 and the path selection heuristic 206 employed by the systems and methods described herein are generally referred to as Advocates and Critics for Tactical Behaviors (ACTB). As shown inFIG. 2 (b)—which coarsely depicts an exemplary embodiment of themethod 200 ofFIG. 2 (a) involving advocates and critics—theadvocates 252 and thecritics 254 interact in acycle 250 that is exposed to, and influenced by, the “world” defined by the problem environment. Within theexemplary planning cycle 250 of an embodiment of the ACTB system and/or method,multiple advocates 252 are used to generate 251 a number of new plans 256 (or improve 251 existing plans) based at least partially on a set of seed/initial or existing plans. Thenew plans 256 are evaluated 253 by a number ofcritics 254. The results of the evaluations by thecritics 254 are combined to produce a measure/score 255 of the quality of each plan, and by which the plans are ranked 258. A subset of the plans are then chosen 257 for further manipulation in asubsequent cycle 250. In one embodiment, the choice of plan depends on the path selection heuristic, whereas in another embodiment the choice of plan depends on the path production heuristic that rewards diversity in the candidate path population. According to one practice, theplanning cycle 250 proceeds in a continual manner (possibly independently of other cycles), and interacts with the world in a continual manner. As better plans are evolved, they may influence the current plan being executed. As new discoveries are made or changes occur in the world, the planning process is informed so that theadvocates 252 and thecritics 254 may incorporate new information into their processing. - According to one aspect, an advantage of the continual,
cyclic nature 250 of the ACTB system is two-fold. First, at each iteration, the set ofnew plans 256 improves overall, at least partially based on more recent knowledge. Second, at each iteration thecritics 254 evaluate the plans based at least partially on the most recent knowledge available. Thus, a change to the tactical situation is detected and incorporated by the systems and methods described herein; subsequent modifications by theadvocates 252 are rewarded based at least partially on the new situation. - Within the
ACTB system 250, theadvocates 252 and thecritics 254 interact, albeit typically indirectly. According to one embodiment, every advocate is associated with a specific critic, but the relationship need not be tightly linked. This indicates that certain tactical behaviors are associated with certain tactical objectives. For example, in an exemplary embodiment, a road-following advocate tends to generate plans that better satisfy a duration critic. - Furthermore, in one embodiment, distinct advocates drive the planning system in distinct directions. For example, the road-following advocate may promote changes opposite to, or substantially distinct from, those promoted by an enemy avoidance advocate. While advocates are typically, though not always, employed and applied independently, this does not pose an issue, because the critics will resolve the best changes or combinations of changes over time.
- Analogously, in one embodiment, distinct critics drive the system in distinct directions. For example, the duration critic may penalize plans that a surveillance critic rewards (e.g., a plan that sends UGVs to the tops of hills generally takes longer to execute than a plan that sends the UGVs over flat terrain). This competition is indicative of the variety of, at times even conflicting, tactical objectives desirable for a mission. The overall needs of the mission are taken into account, and the relative importance of each tactical objective is accommodated by the systems and methods described herein. Furthermore, the systems and methods disclosed herein employ, in one embodiment, a mechanism to combine the evaluations of the critics and resolve these issues; the design of this mechanism's functionality can generally be relevant to the overall performance of the ACTB system.
- Three exemplary tactical advocates include mission-allocation advocate, avoid-untraversable advocate, and a road-following advocate. It is understood, however, that these three types are illustrative only and should not be considered limiting in any way. Other applications may have fewer or more advocate types, or, generally, may employ different advocate types.
- The mission-allocation advocate exploits knowledge of the mission goals and their requirements to allocate mission goals to the UGVs. Typically, it operates in two modes, though other numbers and types of modes are conceivable, depending on the context. A first exemplary mode is selected if a mission goal remains outstanding, that is, it has not yet been assigned to any UGV within a given path genome. In this mode, and according to one embodiment, the mission-allocation advocate assigns the outstanding mission goal to a randomly-selected UGV by inserting a corresponding mission point into the path of the randomly-selected UGV, next to an existing route point or mission point closest to the outstanding mission point along the UGV's path. The order of the new mission point is determined by the existing information in the UGV's path.
- A second exemplary mode is used when all missions have been assigned within a plan. According to the embodiment, the mission-allocation advocate randomly removes a sequence of one or more mission points—as well as all intermediate waypoints—from a path, and inserts that sequence before or after a randomly-chosen mission point in a randomly-chosen path in the plan. Thus, the missions may be inserted within the same UGV's path, thereby performing an effective re-ordering of the mission goals, or in the path of a different UGV, thereby performing a switch of mission goals from a first UGV to a second UGV. In this offspring path set, the new ordering of missions is randomly determined.
- The avoid-untraversable advocate exploits terrain knowledge and a model of the movement capabilities of the UGVs to determine routes that do not have waypoints in untraversable terrain. For example, and without limitation, rivers and lakes may be untraversable for particular types of UGVs. The advocate identifies waypoints belonging to paths that lie in untraversable terrain, and then randomly selects one of these waypoints and moves it to a location on a traversable terrain. The new location is selected by searching on an arc towards the “traversable-predecessor” of the selected untraversable-waypoint. Any intervening waypoints are eliminated (since they necessarily would have been untraversable).
- The road-following advocate exploits knowledge of the road network to determine a path segment between a pair of mission or route points making maximal use of roads. In one particular practice, roads are represented symbolically in a geospatial information development toolkit (e.g., Openmap™, BBN Technologies, Cambridge, Mass., USA). The road-following advocate randomly chooses two mission or route points on a randomly-chosen path. Using deterministic routines that query the road representation, the road-following advocate determines the closest road point to each selected waypoint, and obtains the shortest road path between those road points. This road path is represented as a sequence of travel points with route points at the each of its termini. The new sequence replaces the path between the original selection points.
- There are contexts wherein using travel points is preferable to using route points. A segment of road may be highly curved or irregular, and as such it may require a large number of points to specify that segment in a piece-wise linear manner. If the travel points were included as possible points of selection by the other advocates, the process of selecting waypoints for adaptation would be overwhelmed by the large number of travel points. For example, an advocate may spend the majority of its time moving road points, and thus be highly ineffective at optimizing the route between mission goals.
- At first glance, the road lookups of the road-following advocate seem to serve a purpose similar to the shortest-path lookups of existing planning techniques. However, the road lookup is limited to identifying only small road segments, and it has little or no impact upon the exploration of cross-country paths by other advocates. Rather than simply looking up a shortest path between two mission goals, which may be a tactically poor choice, an embodiment of the ACTB systems and methods described herein uses multiple tactical advocates to determine routes, and creates routes by making a number of small changes at random locations in the path. This enables the systems and methods described herein to explore a wide variety of routes between the two mission goals and adapt that route according to multiple tactical criteria. For example, some segments may result in a poor fitness according to one tactical critic; eventual removal or modification of those segments by tactical advocates may produce an improved path according to that critic.
- A typical embodiment of the ACTB systems and methods described herein employs several types of tactical critics, all of which return path evaluations that are greater than or equal to 0, where lower numbers indicate better plans. It is understood, however, that these critic types, described below, as well as their typical output ranges, are illustrative only, and should not be construed as limiting in any way. Other applications may have fewer or more types of tactical critics, or, generally, may employ different critic types, possibly having alternative output value ranges.
- Three exemplary types of critics include a traversability critic, a safety critic, and a stealth critic. The traversability critic exploits terrain knowledge and a model of the movement capabilities of the UGVs to identify portions of the path that wander into untraversable terrain. According to one practice, the traversability critic returns a penalty value proportional to distance traveled on untraversable terrain over all paths; it is worth noting that in the embodiment described, a path wandering into untraversable terrain is allowed, but is penalized accordingly.
- The safety critic exploits knowledge of the known enemy locations and a model of enemy capabilities to evaluate whether a given plan places one or more UGVs in danger by placing them too close to a known enemy. According to one practice, the safety critic returns a penalty value proportional to the distance traveled within danger range of the enemy, over all candidate paths; it is worth noting that in the embodiment described, a path that places a corresponding UGV dangerously close to the enemy location is allowed, but is penalized accordingly.
- The stealth critic exploits knowledge of the known enemy locations and line-of-sight computations to evaluate whether a given plan puts one or more UGVs at risk by placing them in the line of sight of a known enemy. According to one practice, the line of sight is computed using a geospatial information development toolkit and a model of the surveillance capabilities of the enemy. The advocate returns a penalty proportional to the distance traveled within surveillance range of the enemy; it is worth noting that in the embodiment described, a path that places a corresponding UGV within the enemy's surveillance range (e.g., direct line of sight, radar observation, sonar observation, etc.) is allowed, but is penalized accordingly.
- Other critic types employed by one or more embodiments of the systems and methods described herein include a mission-success critic, a total-duration critic, and a max-duration critic, but as mentioned earlier, these exemplary critic types should not be construed as limiting the scope of the systems and methods described herein.
- In an illustrative embodiment, the ACTB systems and methods described herein facilitate performing continual operation planning within a dynamic environment, in which UGVs move, and wherein knowledge of the environment and tactical situation is time dependent. As illustrated in
FIG. 2 (c), in one aspect, the systems and methods described herein include a simulation-based system in which adeliberative planning process 261 explicitly interacts with asimulated world environment 262 in acontinual cycle 263. - In one embodiment, the operation planning process employs an ACTB-influenced genetic algorithm to evolve multiple notional operation plans 261 for a predetermined number of generations. After a genetic run, the best plan is adopted as the prevailing execution plan, even if the plan was randomly generated at a particular iteration/run. The execution plan is communicated to the UGVs, which typically use a simple (non-reactive) execution model to visit their assigned waypoints. As execution proceeds, world events, such as the discovery of a new enemy location, may occur. These events trigger the operation planning process to evolve a new plan that incorporates the new tactical situation. Additionally, in an illustrative embodiment, the execution process may be suspended at regular intervals, and the deliberative process executed to explore further improvements to the prevailing operational plan. Path populations associated with the genetic algorithm are persistent across genetic runs.
- From an implementation point of view, the ACTB-based systems and methods described herein may be programmed in Java and the simulation environment may use a geographic system to represent terrain information, provide basic functionality for making geographic inquiries, and provide a graphical interface. Alternatives to the programming language Java and the geographic system OpenMap™ may be employed without departing from the scope of the systems and methods described herein.
- The systems and methods described herein typically employ an evolutionary algorithm to search for a good path for a mobile agent. Evolutionary algorithms are well-suited for complex, multi-objective optimization problems. In addition to their ability to search efficiently through large and complex spaces, evolutionary algorithms offer an additional advantage of being easily tailored to a particular domain for improved performance.
- The systems and methods described herein take advantage of this through use of tactical advocates and tactical critics. The advocates include domain-specific mutations that modify a plan based on knowledge of good tactics. The critics compute various evaluation metrics corresponding to various criteria indicative of a good plan. The structure of the systems and methods disclosed herein makes it easy to add advocates and critics and hence to incorporate additional domain knowledge; similarly, it is straightforward to remove an advocate or critic.
- The ACTB genetic algorithm employed in an embodiment of the systems and methods described herein is based at least partially on the notion that potentially significant improvements to an overall plan are possible by applying a succession of small, goal-directed changes. According to one practice, these goal-directed changes are made using domain-specific genetic operators, termed tactical advocates, as already mentioned. In various illustrative embodiments, the improvements include global changes to one or more paths (in particular, to the remaining portions thereof) assigned to the UGVs.
- In a typical embodiment, several traditional genetic mutation and crossover operators are used by ACTB to maintain sufficient diversity in the path population. In this embodiment, the tactical advocates continue to make novel plans rather than continue to rehash old ones. According to one practice, three mutation operators are used, each of which performs a mutation on a randomly-chosen chromosome within the genome. The number and nature of the mutation operators is understood to vary based on context, so the three illustrative types described herein should not be considered limiting in any way. As mentioned earlier, typically, only mission points and route points are manipulated (i.e., genetically mutated), not travel points.
- In one embodiment, an insert-waypoint mutation operator randomly selects a waypoint on the chromosome and inserts a single waypoint before or after that point. The geographic location of the new waypoint is typically a small random distance in a random direction from a line connecting its neighbors.
- A remove-section mutation operator randomly selects two waypoints in the chromosome and removes them and waypoints in between.
- A nudge-waypoint mutation operator randomly selects a waypoint from the chromosome and modifies its geographic location slightly in a random direction.
- In one exemplary embodiment of the systems and methods described herein, two genetic crossover operators are employed. It is understood that other contexts and applications may include genetic crossover operators distinct from, and fewer or larger in number than, the crossovers described herein.
- The path-crossover operator is applied to a single genome parent, and performs variable-length one-point crossover between two randomly-chosen chromosomes within the genome.
- The plan-crossover operator is applied to two genome parents, and performs variable-length one-point crossover between a randomly-chosen chromosome in one parent and a randomly-chosen chromosome in the other.
- In an alternative embodiment, in addition to the tactical advocates, traditional domain-generic operators are also used to augment path search capabilities of the systems and methods described herein and/or to maintain path population diversity.
- In one aspect, the ACTB genetic algorithm accommodates the multi-objective nature of the operation planning problem by using multiple, distinct evaluation components to determine fitness of candidate paths. Specifically, according to one practice, a tactical critic represents a domain-specific evaluation component that computes a single term of a fitness function. Each tactical critic typically evaluates how well a given operation plan satisfies a tactical criterion. For example, and without limitation, a critic for safety may evaluate a plan to determine how much danger the UGVs are placed in, due to traveling too close to a known enemy. In a typical embodiment, outputs of multiple critics are combined using a weighted sum to form a single fitness value. In a military context, for example, the weights associated with the critics reflect the tactical priorities of the operation.
- The ACTB genetic algorithm accommodates a constraint-based nature of the operation planning problem by allowing ostensibly “illegal” individual paths into the population of candidate paths or path sets; an illegal path includes a path that violates at least one problem constraint. A path set includes, as the name implies, a set of paths; every mobile agent has an associated path belonging to the path set.
- Additionally, the ACTB genetic algorithm-based systems and methods described herein employ fitness values to reflect the magnitude of a violation. Specifically, when a tactical critic evaluates a candidate operation plan (i.e., candidate path or path set) against a tactical criterion, it assigns a penalty if the plan violates that criterion. For example, in an illustrative embodiment, a critic assigned the task of determining whether a candidate path is traversable may still accept a path that crosses water (an untraversable terrain), but will assign a high penalty to the path. To enable a relative judgment amongst “illegal” plans, critics typically assign a penalty proportional to the degree of the violation. According to one exemplary practice, the distance to be “traveled” in water at least in part determines a magnitude or severity of the penalty imposed.
- In an embodiment involving n UGVs, where n≧1 (more typically, n≧2), a genome is defined as a set of n chromosomes, wherein each chromosome defines the path for one of the UGVs as a variable-length sequence of geographic locations, or waypoints, to be visited by the UGV for a remainder of the operation plan. According to one practice, to evaluate a fitness of the genome, a successive pair of waypoints is assumed to be connected with a straight line. Therefore, according to the practice, each chromosome defines a piece-wise linear directed path. The first waypoint in each path represents the next waypoint to be visited by the corresponding UGV. Typically, the first segment of the path is understood to be the straight line between the UGV's current location and the first waypoint in the path.
- To enable effective genetic manipulations, an exemplary genetic representation includes three types of waypoints, each representing a different conceptual aspect of a path. It is understood, however, that not each of the three types of waypoints is required in all embodiments.
- In the embodiment wherein the systems and methods described herein employ a genetic algorithm, a route point may be genetically manipulated, e.g, mutated.
- A notable feature of travel points is that they generally are not available for selection as points of genetic manipulation by the systems and methods described herein. Rather, they are used to incorporate specific path segments between two consecutive mission points or route points. The relative benefits of these segments may then be evaluated through a genetic search. In an exemplary embodiment, route-points are used by a road-following advocate to represent complex road segments.
-
FIG. 3 illustrates a sample genome having twochromosomes chromosome 300 and the waypoints D-E are mission points belonging to thechromosome 350. As inFIG. 1 , each of the route points 301-305 and 311-313 is represented by a hollow circle, and each of the travel points 341-345 (and others not labeled in the figure) is represented as a solid circle. - The three types of waypoints are used to enable genetic manipulations targeted to different levels of planning. For example, manipulation of mission points and their order performs the task of scheduling missions, whereas manipulation of route points performs the task of route planning. In the context of a UGV platform, little to no distinction is required between the three types of waypoints, except that travel points typically may not be genetically mutated or otherwise manipulated.
- The deliberative UGV planning problem shares some characteristics with a classic optimization problem, called the vehicle routing problem with time windows (VRPTW). In both problems, multiple vehicles are to move in such a way as to arrive at particular locations during predetermined time intervals. However, the UGV planning problem has nontrivial additional complications. One is that the paths between locations are generally not well-defined, and a task of the planning algorithm is to find a good path over a combination of roads and off-road terrain. A second complication is that the deliberative UGV planning entails a greater number of criteria to be considered for determining a good plan.
- The operation planning systems and methods described herein are well-suited to solve the VRPTW. In particular, the problem involves predetermined time windows during which the mobile agents—UGVs in our illustrative example—are to arrive at, and/or depart from, a subset of the waypoints.
- To solve the problem involving time window constraints, the systems and methods described herein employ, in one aspect, a greedy algorithm providing a unique interpretation of a path with respect to time, allowing the details of the path to be varied quite significantly. This is unlike a traditional “shortest-paths” lookup, wherein a set of fixed routes between key locations is stored; it also is unlike “dynamic” shortest-paths lookup, wherein the shortest routes between key locations are occasionally recomputed to accommodate new information.
- The greedy algorithm provides a suitable framework for evaluating a path's duration as well as the success or failure of accomplishing the missions as part of the constrained operation planning problem. This problem may be defined more concretely as follows: given one or more mobile agents operating within a complex environment (e.g., urban environment, terrain with multiple features and terrain types, multiple known enemy locations, etc.), given one or more mission goals involving visiting specific locations within that environment, and given a set of general and mission-specific constraints, both hard and soft, imposed on prospective routes (e.g., the route must avoid enemies, the missions must be achieved as soon as possible, etc.), the constrained deliberative planning problem includes determining a route for each mobile agent such that over all mobile agents in the fleet, missions are successfully accomplished while satisfying substantially all hard constraints and soft constraints as best as possible.
- According to one practice, the greedy algorithm employed by the systems and methods described herein assumes that a mobile agent will travel between the waypoints along its assigned path at a speed less than or equal to a maximum speed allowable or feasible for the terrain over which it is traveling. The algorithm also assumes that each of a subset of the waypoints represents an attempt to fulfill a respective mission, subject to a specified time window. An aspect of the greedy algorithm includes the notion that if a vehicle, traveling at its maximum speed, arrives at a mission point earlier than a prescribed time window, it may wait or move sufficiently slowly to ensure a timely arrival and/or departure. A mission is considered accomplished as long as a vehicle arrives to accomplish the mission at any point between the start and end time. This provides a mechanism for stepping through the path and evaluating the times at which the missions are accomplished.
- In a larger context, the greedy algorithm may be employed within an evolutionary algorithm to solve the deliberative planning problem, when a route may be evaluated, typically dynamically, against multiple criteria other than simple travel time. That is to say, there is no a priori “best” route between a pair of mission locations; this variability can have an impact on the success of the deliberative planning effort. This is the case, for example, in planning for one or more UGVs whose routes are not only to be sufficiently quick, but must also enable the UGV to satisfy a variety of tactical constraints, such as avoiding enemies.
- An issue when evaluating such a plan for deliberative planning includes a tradeoff between how to schedule the times of the missions versus the computation of the travel times when routes are not static in quality. Existing approaches are generally capable of performing one task or the other, but not both. If scheduling is performed, then a fixed routing lookup is assumed. If routing is performed, then the times on the missions are assumed to be secondary, and the focus is shifted to path planning.
- According to one practice, the greedy algorithm interpretation of the execution of a path chromosome is as follows. Each mission goal is defined as a geographic location with an “earliest” time of arrival and a “latest” time of arrival; additionally or alternatively, an earliest time of departure or latest time of departure may be specified. The duration of travel between each successive pair of waypoints is computed—for example, by assuming that a mobile agent, such as a UGV, travels at its established maximum speed for each terrain type; alternatively, the UGV may be assumed to travel at an established average speed for each terrain type. Using the current time in the simulated world and the current location of the UGV as a starting point, the greedy algorithm steps through all waypoints on the chromosome in order, evaluates a duration of the path segments, and assesses mission success as it proceeds. In one aspect, the algorithm assumes that if a UGV arrives at a mission point too early, it is possible for that UGV to arrive on time (e.g., through the simple mechanism of driving more slowly or by waiting a prescribed amount at an appropriate venue).
- In a greedy algorithm implementation of the systems and methods described herein, a notion of a “minimum wait time” for the UGV at each mission point is employed to model completion of the missions. It is assumed that the UGV will wait for the minimum possible amount of time at each of its assigned mission points to satisfy the time window constraint.
- According to one practice, the greedy algorithm assumes a model of UGV movement speed over different terrain types, and evaluates a travel time between successive mission points based at least partially on distances and terrain traversed over the (piece-wise linear) path between them. Typically, each UGV is assumed to travel as fast as possible between mission points (though, in an alternative embodiment, an average speed is considered), and then wait as little as needed (if early) to meet a prescribed time window requirement (i.e., the greedy choice). Thus, no special representation of time windows is required in the genome. The mission-success critic evaluates how well a given plan comes to accomplishing mission goals, and returns a penalty proportional to the number of failed goals and a corresponding degree of failure. The total-duration critic evaluates how long each UGV takes to execute its chromosome, and returns the sum of the durations of all chromosomes. The max-duration critic evaluates how long each UGV takes to execute its chromosome, and returns the longest duration over all chromosomes.
-
FIG. 4 illustrates a sample evaluation of the duration and mission success of a path chromosome. Table 1 provides a summary snapshot ofFIG. 4 . For example, and without limitation, if a UGV traveling at its maximum allowable or feasible terrain speed over a set of route and travel points arrives at the first mission point on its assigned path too early, the systems and methods described herein assume that it would wait the minimum amount of time required to meet the “earliest” mission time window, and then move on; the arrival time window for the mission point A is 20 minutes for ‘A’, the interval [12:10 and 12:30]). At each subsequent mission point this same computation occurs. If the earliest possible arrival time lies within the prescribed time window, then the minimum wait time is 0 minutes (e.g., ‘C’). If it is impossible for the UGV to arrive at a mission before or during the prescribed time window (i.e., it is late), then the amount of tardiness is recorded and the mission is flagged as having failed (e.g., 5 minutes late for ‘D’). However, according to one practice, in a greedy fashion, the systems and methods described herein continue to evaluate the path and assume that, though late, the UGV went as quickly as possible to a subsequent mission point. For example, though late for mission D, the greedy choice is to depart as soon as possible (i.e., 1:45, indicated in bold in Table 1). Thus, being late for one mission (e.g., ‘D’) does not preclude success at a subsequent mission (e.g., ‘B’).TABLE 1 Mission A: Mission C: Mission D: Mission B: Earliest arrival: Earliest arrival: Earliest arrival: Earliest arrival: 12:00 + 0:10 = 12:30 + 0:15 = 12:45 + 1:00 = 1:45 + 0:10 = 12:10 12:45 1:45 1:55 → Wait time = → Wait time = → Wait time = → Wait time = 0:20 0:00 −0:05 0:00 (EARLY) (SUCCESS) (FAILURE) (SUCCESS) - The greedy algorithm employed by the systems and methods described herein provides, in a typical embodiment, a unique way of interpreting an inherently spatial property (namely, travel time along a route) within the context of an inherently temporal constraint (namely, to accomplish the missions during prescribed time windows).
- In conjunction with an evolutionary algorithm, this provides a mechanism for exploring variations both in the routes (“space”) and in the order of the missions (“time”), with little, if any, loss of capability. Over multiple generations and various sequences of waypoints, a plan may be evolved that correctly fulfills the mission time requirements while obeying the necessity of travel and multiple constraints upon that travel.
-
FIG. 5 depicts aprototype application 500 of an embodiment of the ACTB systems and methods described herein, directed at the problem of deliberative planning of UGV routes. The figure illustrates an example of where the application fits into a larger on-vehicle processing system. Thevehicle perception subsystem 502 processes and analyzes thesensor data 504 to produce an evolving description of a region that at least partially surrounds the vehicle. This description is reconciled 505(a)-505(b) with a priori topographic information andongoing situation assessment 506, andintelligence 508 that is streamed to, collected by, or otherwise communicated to the vehicle. Similarly, and optionally, inputs and parameter settings are received by the vehicle from the operator 510. All this information—the scene description, the situation awareness, the intelligence, and the operator settings/inputs—forms the input to a typical embodiment of the ACTB systems and methods described herein, which continually monitor and review this data and formulate aplan 511, chosen from a pool of adaptively maintainedtactical behaviors 516, for the vehicle's movement. In a sense, the adaptivetactical behaviors box 516 encompasses what is shown inFIG. 2 (a). Specifically, the systems and methods described herein produce a sequence of vehicle commands 512 of waypoints expected to be followed by the vehicle. Optionally, commands sensor commands are issued 514 to the vehicle, instructing the vehicle to manipulate one or more sensors or data collection instruments. For example, and without limitation, thecommand 514 may include instructions to the vehicle to turn on an onboard camera and point it to a particular target. These waypoints are turned over to a navigation subsystem of the systems and methods described herein. - A property of an effective, automated replanning system and method for UGVs includes a capability for each UGV to cooperate with other UGVs in developing effective global solutions, but to accomplish this task without requiring the full knowledge of all known intelligence across all UGVs. A centralized solution, while it may be capable of developing highly-optimized solutions over all mission objectives, is impractical, because it requires an ability to maintain complete centralized knowledge of all environmental information, as well as update every UGV essentially at all times. A distributed system with full knowledge communicated to every UGV, while it may allow processing requirements to be distributed and enable each UGV to make globally-optimized decisions, is also impractical, because it requires essentially full communication and rapid update of newly-discovered knowledge.
- In an exemplary realistic solution, a mobile agent, such as a vehicle, is capable of local processing and vehicles are assumed to be interconnected with respect to communications. In the exemplary solution, vehicles exchange sensor-derived information and may receive updated intelligence or directives from one or more human operators.
FIG. 6 illustrates a possible communication network of multiple UGVs, possibly including a human operator, over which collaborative planning may occur. The exemplary system and method depicted byFIG. 6 illustrates that the deliberative planning may occur with sharing of partial information, not necessarily complete information, among the vehicles. - An aspect of the systems and methods described herein includes effectively planning locally while achieving path solutions that are optimized to some degree at a global level; as mentioned earlier, the entirety of the remainder of a path for a mobile agent may change as a result of the continual updating of the population of candidate paths, which may occur in part from an unexpected local stimulus, such as an unpredicted appearance of an enemy.
- For an ACTB system and/or method, the continual replanning nature of the approach provides valuable benefits. Each UGV may execute its own directed evolutionary system, and independently evolve new candidates for the selected team plan based upon knowledge available to that UGV. Each UGV may periodically transmit its best plan, or sets of plans, to other UGVs with which it communicates. The UGVs may then perform an information handshake to agree on changes to be incorporated. For example, in such an embodiment, plans requiring two UGVs to swap objectives may be implemented if those UGVs are in communication. The niche-based approaches to genetic algorithms are suitable for providing valuable insights into the problem. In a niche-based approach, different niche populations are maintained and optimized on information locally available to that niche. In one embodiment, genetic operators occasionally transmit information between niches, thereby providing the capability for the information in one local search to affect another local search, even if indirectly.
- Another property of an effective automated replanning system for UGVs is a capability to develop plans wherein multiple UGVs demonstrate cooperative group behaviors that enable them to solve their missions more effectively. For example, and without limitation, some UGVs may provide reconnaissance support for others. UGVs may cooperate to maintain effective communication networks.
- In a well understood domain, use of pre-determined formations is an effective solution to this problem. However, in a poorly-understood and/or highly-dynamic environment, the most effective cooperative behaviors will generally not be known a priori, and have to be discovered automatically by the replanning system and/or method. The ACTB systems and methods described herein provide both types of solutions. One the one hand, a formation advocate may be specialized for a certain known formation and promote its use. One the other hand, advocates for generic cooperative behaviors, such as scouting and rear support, may promote emergence of novel cooperative responses.
- An effective replanning system is typically robust to planning under unknown environments. In general, a UGV may venture into terrain that is poorly understood. Although basic capabilities of the perception and navigation system generally enable a UGV to adjust to most environments, it is desirable for the planning system to recognize and accommodate limitations of those systems. For example, if a UGV is incapable of traversing a terrain at least a portion of which is above a certain grade, then identifying and planning routes around such terrain can be important tactical behaviors. Interaction between scouting behaviors and planning effectiveness is desirable. The ACTB systems and methods described herein facilitate rapid prototyping and study of a variety of scouting behaviors.
- In one aspect, it is desirable for a realistic replanning system or method to be capable of having human planners control the planning process. Moreover, it is desirable for such a system or method to be capable of controlling the planning to varying degrees under variable circumstances. While the systems and methods described herein allow the planning process to be completely automated, they may be influenced indirectly by providing new intelligence information. In a typical practical setting, a human operator will be able to take direct control, with automated planning taking a back seat. The human operator may provide a specific plan (or portion of a plan) designed to remain fixed. In an ACTB system or method, the human may adjust the priorities of the planning system to reflect the most recent mission priorities through the selection of critics and their weights. For example, and without limitation, while safety and enemy avoidance may be the overriding factors under normal circumstances in a particular exemplary embodiment, some missions may require extreme risk to be acceptable in order to reach a mission objective.
- Experimental Results
- To validate the approach employed by the systems and methods described herein, test scenarios have been developed wherein multiple UGVs cooperate to solve complementary and competing mission goals while both minimizing mission completion time and minimizing risk to mission success. One such exemplary scenario incorporates an actual terrain that the U.S. Army uses as a test bed for some of its UGVs. The systems and methods described herein demonstrate that continually-improved plans may be generated quickly, both before and during plan execution, in response to changes in the tactical situation or other environmental stimuli or time-dependent mission demands.
- The ACTB systems and methods described herein were tested under four conditions to demonstrate the effectiveness of the tactical critics for multi-objective optimization and examine the search capabilities of ACTB when using tactical advocates in conjunction with traditional genetic operators over using traditional genetic operators alone. The experiment examined the basic tactical route planning capabilities of the ACTB systems and methods described herein. Time scheduling aspects of the problem were minimized by widening the mission time windows. However, path duration was still a prized factor (i.e., do all the missions as soon as possible).
- In all conditions, a steady-state genetic algorithm was run using a fixed population size of 50, fitness-proportional selection was used, and offspring competed with all members of the population. Most advocates and genetic operators were applied with the same likelihood of selection (i.e., 1.0). To encourage the system to explore complex paths, insert-waypoint was applied with twice the likelihood of the above (i.e., 2.0), and remove-section with half the likelihood (i.e., 0.5). Critic weights were selected to assign a very high penalty to untraversable portions of the routes and to missed missions, a moderate penalty to exposure to the enemy (i.e., completing the mission is more important than avoiding the enemy), and a small penalty to path duration; the penalty for maximum-duration was weighted twice as strongly as total-duration, to encourage a more balanced distribution of mission goals among UGVs.
- Final plans developed in the four experimental conditions are illustrated in
FIGS. 7-8 . In the first pair of conditions (FIGS. 7(a)-7(b)), the systems and methods described herein were employed to examine basic routing in a simple situation with no known enemies. In the second pair of conditions (FIGS. 8(a)-8(b)), the systems and methods described herein were employed to examine tactical routing in a situation with two known enemies. In the figures, three friendly UGVs (shown as 701-703 inFIG. 7 (a)) are located in the top left, and each UGV's path is indicated by a different line thickness; for example, thepath 730 is associated with theUGV 702. Off-road terrain 740 is shown in white (background), rivers and other waterways are shown in light gray (e.g., 710), roads are shown in dark gray (e.g., 720), and missions points are depicted by the circles (e.g., 750). Intersections of road with water are considered to have bridges. Travel on-road is assumed to be roughly 10 times faster than travel off-road. A similar convention is used in subsequent FIGS. 7(b), 8(a)-8(b), and 10(a)-10(c), even though these figures are not specifically labeled as such. These figures show that in all conditions, the system was able to evolve plans that were traversable, met all mission goals and distributed mission goals among all three UGVs. In both enemy conditions, the evolved paths avoided the known enemies. - Several prospective planning responses that are potentially useful upon discovery of an enemy near the UGV can be employed. These responses may capture “expert knowledge” used by human planners (and/or by human soldiers in a field). As such, several exemplary heuristics may be defined as follows: (a) Attack: head directly at the enemy and engage in battle; (b) Avoid: change the path, even if slightly, to avoid going too close to the enemy; (c) Hide: change path so that the UGV goes behind some cover that will block it from the enemy's line of sight; (d) Evasive Maneuvers: plan a jagged path to avoid being an easy target for enemy fire; (e) Retreat and Circumvent: retreat as fast as possible and plan a wide circuit around the enemy enabling the UGV to continue toward its mission; and (f) Surveil: change path to minimize risk of detection by the enemy, but still keep the enemy in sight.
- It is understood that these sample heuristics are illustrative and should not be construed as limiting in any way. Alternative or additional heuristics may be developed for other situations faced by the UGVs.
- Furthermore, in planning a path for a UGV, it may be desirable to capture several different prospective path evaluation constraints/heuristics. As such, listed below are illustrative exemplary constraint/heuristic sets used, in some combination, by the systems and methods described herein:
- Constraint 1: Avoid Exposed Areas: generally, it is a good idea to avoid areas that are more visible and have a higher risk of exposure to the enemy. Heuristic 1: For a given location, compute how exposed the location is based on the number of points it is in line-of-sight with, within a radial distance, e.g., 30 meters.
- Constraint 2: Avoid Lethal Areas: generally, it is a good idea to avoid traveling too close to an area occupied by an enemy. Heuristic 2(a): If the UGV is within the enemy's firing range, then it is in danger, otherwise not. Heuristic 2(b): Danger rating is proportional to distance from the enemy.
- Constraint 3: Engage enemy: generally, it is a good idea to attack and destroy an enemy. Heuristic 3(a): If the UGV is within firing range of the enemy, then it is engaged, otherwise not. Heuristic 3(b): Engagement score is inversely proportional to distance of the UGV from the enemy.
- Constraint 4: Duration: generally, it is a good idea to reach destinations as quickly as possible. Heuristic 4(a): distance duration is proportional to the length of the path. Heuristic 4(b): travel time duration is based at least partially on the average speed of the UGV over each terrain type encountered along the path.
- Constraint 5: Avoid an Unknown: generally, it is a good idea to travel across terrain that has been recently surveilled. Heuristic 5(a): for a given location, compute how many points within a certain radius (e.g., 20 meters) have been visited previously by the UGV or by another friendly mobile agent. Heuristic 5(b): Time-Decayed Observation: for a given location, compute how many points within a certain radius (e.g., 20 meters) have been visited recently by the UGV or by another friendly mobile agent.
- It is understood that these sample heuristics are illustrative and should not be construed as limiting in any way. Alternative or additional heuristics may be developed. For example, and without limitation, these constraint/heuristic sets may be combined into a combined path selection heuristic, as in the following illustrative examples.
- Combined Heuristic 1: Get the Job Done and Lower the Risk: High emphasis on (1), (2a), (5). Low emphasis on (4b).
- Combined Heuristic 2: Patrol, and Tolerate High Risk: high emphasis on (3a) and (5). Low emphasis on (1) and (4a).
-
FIG. 9 illustrates fitness of the best individual in each generation for each of four experimental conditions. Each condition was run for 30,000 generations to ensure convergence, but all conditions had converged by 7,000 generations. The range of fitness values was very large due to high penalty values assigned by the tactical critics, and the results are graphed on a logarithmic scale to emphasize the changes over evolution. A fitness value under 1,000 indicates no penalty except for duration of path. The ACTB systems and methods described herein clearly demonstrated better plans when using advocates, as illustrated by the better (i.e., lower) fitness values overall and by the rapid achievement of a plan with no major penalties around 1,000 generations as opposed to over 4,000 generations for traditional conditions. The final plan generated in the advocate conditions completed all missions in roughly half the time of the plans generated in the traditional conditions. As illustrated in FIGS. 7(b) and 8(b), this improvement is substantially due to improved road following. - The systems and methods described herein were also tested in a dynamic simulation mode, as described earlier, wherein new enemies may be detected as the UGVs execute an operation plan. The ACTB system has demonstrated the capability for rapid and effective replanning in response to these changes in the tactical situation, as illustrated in
FIG. 10 .FIG. 10 (a) illustrates a plan under execution immediately before discovery of an enemy.FIG. 10 (b) illustrates the re-planning activity initiated upon the discovery (after a few generations).FIG. 10 (c) illustrates the new plan generated after 300 generations and passed to the two UGVs (shown in the top left of each figure) for execution. Note that after replanning, all UGV paths avoid the area surrounding the enemy. - The systems and methods described herein, in various embodiments, constitute an approach to deliberative planning for coordinating UGVs. This approach is based at least partially on representing the planning problem as a multi-objective optimization problem, which in an exemplary embodiment is solved using a genetic algorithm to dynamically search for a good path or path set. Multiple evaluation components, called tactical critics, enable the evolution of plans satisfying multiple tactical constraints. Domain-specific operators, called tactical advocates, greatly speed the search process yielding rapid plan turnaround. By continually searching for improvements to the plan, the systems and methods described herein ensure that the plan adapts to changes in the tactical situation. The ACTB systems and methods described herein maintain good plans in response to such changes, and are flexible enough to having advocates and critics developed and introduced for additional tactical behaviors, as well as accepting new mechanisms for distributing the evolutionary algorithm to make ACTB amenable to implementation within UGV platforms directly.
- The contents of all references, including patents and published patent applications, cited throughout this application are hereby incorporated by reference in entirety.
- The systems and methods described herein may be applied in non-military contexts. For example, and without limitation, the systems and methods described herein may be employed to provide operation planning for a fleet of courier delivery vehicles in a metropolitan city. Domain-specific path selection and production heuristics may be used in this context. For example, avoiding high-traffic areas, avoiding safety-compromised intersections or low-throughput streets and highways may be used to at least partially guide the operation planning of the deliveries. Similarly, time window constraints may be used to schedule delivery of packages during desirable or mandatory times intervals.
- Many other equivalents to the specific embodiments of the invention and the specific methods and practices associated with the systems and methods described herein exist. Accordingly, the invention is not to be limited to the embodiments, methods, and practices disclosed herein, but is to be understood from the following claims, which are to be interpreted as broadly as allowed under the law.
Claims (76)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/877,814 US20050216182A1 (en) | 2004-03-24 | 2004-06-24 | Vehicle routing and path planning |
US12/206,296 US7693653B2 (en) | 2004-03-24 | 2008-09-08 | Vehicle routing and path planning |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US55577804P | 2004-03-24 | 2004-03-24 | |
US10/877,814 US20050216182A1 (en) | 2004-03-24 | 2004-06-24 | Vehicle routing and path planning |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/206,296 Division US7693653B2 (en) | 2004-03-24 | 2008-09-08 | Vehicle routing and path planning |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050216182A1 true US20050216182A1 (en) | 2005-09-29 |
Family
ID=34991167
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/877,814 Abandoned US20050216182A1 (en) | 2004-03-24 | 2004-06-24 | Vehicle routing and path planning |
US12/206,296 Expired - Fee Related US7693653B2 (en) | 2004-03-24 | 2008-09-08 | Vehicle routing and path planning |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/206,296 Expired - Fee Related US7693653B2 (en) | 2004-03-24 | 2008-09-08 | Vehicle routing and path planning |
Country Status (1)
Country | Link |
---|---|
US (2) | US20050216182A1 (en) |
Cited By (108)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050075784A1 (en) * | 2003-10-07 | 2005-04-07 | Gray Sarah Ann | Modular path planner |
US20060047421A1 (en) * | 2004-08-25 | 2006-03-02 | Microsoft Corporation | Computing point-to-point shortest paths from external memory |
US20060184292A1 (en) * | 2005-02-16 | 2006-08-17 | Lockheed Martin Corporation | Mission planning system for vehicles with varying levels of autonomy |
US20070067095A1 (en) * | 2005-07-01 | 2007-03-22 | Honeywell International Inc. | Method and apparatus for resolving ambiguous waypoints |
US20070073897A1 (en) * | 2005-06-21 | 2007-03-29 | Mehdi Sharifzadeh | Optimal sequenced route query operation and device |
US20070094165A1 (en) * | 2005-10-26 | 2007-04-26 | Motorola, Inc. | Path determination facilitation method |
US20070179690A1 (en) * | 2006-02-01 | 2007-08-02 | Stewart Brian G | Variable path automated guided vehicle |
US20070208548A1 (en) * | 2006-03-03 | 2007-09-06 | Solido Design Automation Inc. | Modeling of systems using canonical form functions and symbolic regression |
US20070288156A1 (en) * | 2006-05-17 | 2007-12-13 | The Boeing Company | Route search planner |
US20080010044A1 (en) * | 2006-07-05 | 2008-01-10 | Ruetsch Gregory R | Using interval techniques to solve a parametric multi-objective optimization problem |
US20080027590A1 (en) * | 2006-07-14 | 2008-01-31 | Emilie Phillips | Autonomous behaviors for a remote vehicle |
US20080086241A1 (en) * | 2006-10-06 | 2008-04-10 | Irobot Corporation | Autonomous Behaviors for a Remove Vehicle |
US20080091341A1 (en) * | 2006-06-27 | 2008-04-17 | Microsoft Corporation | Route monetization |
US20080096628A1 (en) * | 2006-10-23 | 2008-04-24 | Zbigniew Czyzewski | Security devices for implementing hand-held wagering |
US20080162242A1 (en) * | 2006-12-27 | 2008-07-03 | Verizon Services Organization Inc. | Dispatching Prioritized Jobs At Multiple Locations To Workers |
US20080215512A1 (en) * | 2006-09-12 | 2008-09-04 | New York University | System, method, and computer-accessible medium for providing a multi-objective evolutionary optimization of agent-based models |
US20080270331A1 (en) * | 2007-04-26 | 2008-10-30 | Darrin Taylor | Method and system for solving an optimization problem with dynamic constraints |
US20090037033A1 (en) * | 2007-05-14 | 2009-02-05 | Emilie Phillips | Autonomous Behaviors for a Remote Vehicle |
US20090088916A1 (en) * | 2007-09-28 | 2009-04-02 | Honeywell International Inc. | Method and system for automatic path planning and obstacle/collision avoidance of autonomous vehicles |
US20090105935A1 (en) * | 2007-10-17 | 2009-04-23 | Lockheed Martin Corporation | Hybrid heuristic national airspace flight path optimization |
US20090112843A1 (en) * | 2007-10-29 | 2009-04-30 | International Business Machines Corporation | System and method for providing differentiated service levels for search index |
US20090112645A1 (en) * | 2007-10-25 | 2009-04-30 | Lockheed Martin Corporation | Multi objective national airspace collaborative optimization |
US20090157311A1 (en) * | 2007-12-14 | 2009-06-18 | Microsoft Corporation | Federated route production |
US20090175219A1 (en) * | 2006-05-22 | 2009-07-09 | Augustin Chaintreau | Representation of a Delay Path in Mobile Networks |
WO2009105288A3 (en) * | 2008-02-19 | 2009-10-15 | Microsoft Corporation | Route reward augmentation |
US20090319112A1 (en) * | 2007-09-28 | 2009-12-24 | Honeywell International Inc. | Automatic planning and regulation of the speed of autonomous vehicles |
US20100030467A1 (en) * | 2005-10-31 | 2010-02-04 | Honeywell International Inc. | System and method for performing 4-dimensional navigation |
US7664094B1 (en) * | 2001-12-10 | 2010-02-16 | At&T Corp. | System for utilizing genetic algorithm to provide constraint-based routing of packets in a communication network |
US20100057358A1 (en) * | 2008-08-28 | 2010-03-04 | TeleType Co., Inc. | Portable gps map device for commercial vehicle industry |
US7734387B1 (en) * | 2006-03-31 | 2010-06-08 | Rockwell Collins, Inc. | Motion planner for unmanned ground vehicles traversing at high speeds in partially known environments |
US20110029234A1 (en) * | 2009-07-29 | 2011-02-03 | Lockheed Martin Corporation | Threat Analysis Toolkit |
US20110106339A1 (en) * | 2006-07-14 | 2011-05-05 | Emilie Phillips | Autonomous Behaviors for a Remote Vehicle |
US20110153338A1 (en) * | 2009-12-17 | 2011-06-23 | Noel Wayne Anderson | System and method for deploying portable landmarks |
US20110153072A1 (en) * | 2009-12-17 | 2011-06-23 | Noel Wayne Anderson | Enhanced visual landmark for localization |
US20110153136A1 (en) * | 2009-12-17 | 2011-06-23 | Noel Wayne Anderson | System and method for area coverage using sector decomposition |
US20110172850A1 (en) * | 2009-09-14 | 2011-07-14 | Israel Aerospace Industries Ltd. | Infantry robotic porter system and methods useful in conjunction therewith |
US20110178704A1 (en) * | 2008-12-08 | 2011-07-21 | Micro-Star International Co., Ltd. | Navigation device capable of adding path to navigation plan independently |
US8060297B2 (en) | 2007-12-14 | 2011-11-15 | Microsoft Corporation | Route transfer between devices |
US8090532B2 (en) | 2007-12-14 | 2012-01-03 | Microsoft Corporation | Pedestrian route production |
US8121749B1 (en) | 2008-09-25 | 2012-02-21 | Honeywell International Inc. | System for integrating dynamically observed and static information for route planning in a graph based planner |
US20120158280A1 (en) * | 2008-01-14 | 2012-06-21 | Ravenscroft Donald L | Computing route plans for routing around obstacles having spatial and temporal dimensions |
WO2012097200A1 (en) * | 2011-01-12 | 2012-07-19 | Cummins Intellectual Property, Inc. | System and method of determining freight/load distribution for multiple vehicles |
CN102611119A (en) * | 2012-03-14 | 2012-07-25 | 华北电力大学 | Multi-target reactive power optimization method for electric system |
US20120192138A1 (en) * | 2011-01-24 | 2012-07-26 | Microsoft Corporation | Graph partitioning with natural cuts |
US20120290148A1 (en) * | 2009-11-24 | 2012-11-15 | Chinagps Co., Ltd (Shenzhen) | Method and system for dispatching vehicle |
CN102820662A (en) * | 2012-08-17 | 2012-12-12 | 华北电力大学 | Distributed power source contained power system multi-target reactive-power optimization method |
US8346391B1 (en) * | 2006-12-28 | 2013-01-01 | Science Applications International Corporation | Methods and systems for an autonomous robotic platform |
US8392117B2 (en) | 2009-05-22 | 2013-03-05 | Toyota Motor Engineering & Manufacturing North America, Inc. | Using topological structure for path planning in semi-structured environments |
US8442765B1 (en) | 2012-11-08 | 2013-05-14 | Honeywell International Inc. | Shared state selection and data exchange for collaborative navigation using conditionally independent parallel filters |
US8473198B2 (en) | 2007-12-14 | 2013-06-25 | Microsoft Corporation | Additional content based on intended travel destination |
US20130339266A1 (en) * | 2012-06-15 | 2013-12-19 | Telogis, Inc. | Vehicle fleet routing system |
US20140032049A1 (en) * | 2012-07-24 | 2014-01-30 | GM Global Technology Operations LLC | Steering Assist in Driver Initiated Collision Avoidance Maneuver |
US8660338B2 (en) | 2011-03-22 | 2014-02-25 | Honeywell International Inc. | Wide baseline feature matching using collobrative navigation and digital terrain elevation data constraints |
US8718925B2 (en) | 2006-06-27 | 2014-05-06 | Microsoft Corporation | Collaborative route planning for generating personalized and context-sensitive routing recommendations |
US8760397B2 (en) | 2007-04-24 | 2014-06-24 | Irobot Corporation | Control system for a remote vehicle |
US8793065B2 (en) | 2008-02-19 | 2014-07-29 | Microsoft Corporation | Route-based activity planner |
US20140222278A1 (en) * | 2011-08-25 | 2014-08-07 | Nissan Motor Co., Ltd. | Autonomous driving control system for vehicle |
US20140244114A1 (en) * | 2011-10-03 | 2014-08-28 | Toyota Jidosha Kabushiki Kaisha | Driving assistance system for vehicle |
US8868323B2 (en) | 2011-03-22 | 2014-10-21 | Honeywell International Inc. | Collaborative navigation using conditional updates |
TWI486807B (en) * | 2013-04-16 | 2015-06-01 | Papago Inc | Judgment System of Vehicle Moving State and Its Judgment |
US9104752B2 (en) | 2012-11-16 | 2015-08-11 | Honeywell International Inc. | Data sharing among conditionally independent parallel filters |
US9283674B2 (en) | 2014-01-07 | 2016-03-15 | Irobot Corporation | Remotely operating a mobile robot |
CN105929843A (en) * | 2016-04-22 | 2016-09-07 | 天津城建大学 | Robot path planning method based on improved ant colony algorithm |
US20160364248A1 (en) * | 2013-08-30 | 2016-12-15 | Insitu, Inc. | Vehicle user interface adaptation |
WO2017004419A1 (en) | 2015-06-30 | 2017-01-05 | Unmanned Innovations, Inc. | Systems and methods for multi-mode unmanned vehicle mission planning and control |
CN106370175A (en) * | 2016-08-18 | 2017-02-01 | 深圳市沃特沃德股份有限公司 | Pet navigation method and device |
US9669904B2 (en) | 2011-05-12 | 2017-06-06 | Unmanned Innovations, Inc. | Systems and methods for multi-mode unmanned vehicle mission planning and control |
US20170193434A1 (en) * | 2015-11-09 | 2017-07-06 | Simbe Robotics, Inc | Method for tracking stock level within a store |
US20170341235A1 (en) * | 2016-05-27 | 2017-11-30 | General Electric Company | Control System And Method For Robotic Motion Planning And Control |
EP3287864A1 (en) * | 2016-08-26 | 2018-02-28 | Thales | Method for aided-piloting an aircraft, associated computer program product and device for aided-piloting |
US9933781B1 (en) | 2016-11-23 | 2018-04-03 | Denso International America, Inc. | Data-driven planning for automated driving |
US9945677B1 (en) * | 2015-07-23 | 2018-04-17 | X Development Llc | Automated lane and route network discovery for robotic actors |
US10074279B1 (en) * | 2017-03-07 | 2018-09-11 | Denso International America, Inc. | Inference-aware motion planning |
CN109144067A (en) * | 2018-09-17 | 2019-01-04 | 长安大学 | A kind of Intelligent cleaning robot and its paths planning method |
CN109211255A (en) * | 2017-07-06 | 2019-01-15 | 罗伯特·博世有限公司 | For the method for the motor vehicle programme path with automotive vehicle system |
CN109269518A (en) * | 2018-08-31 | 2019-01-25 | 北京航空航天大学 | A kind of movable fixture confined space path generating method based on intelligent body |
US20190086220A1 (en) * | 2017-09-21 | 2019-03-21 | Whp Workflow Solutions, Inc. | Dynamic target coverage using mobile assets |
WO2019075008A1 (en) * | 2017-10-10 | 2019-04-18 | The Government Of The United States Of America, As Represented By The Secretary Of The Navy | Method for identifying optimal vehicle paths when energy is a key metric or constraint |
US10311385B2 (en) | 2012-06-15 | 2019-06-04 | Verizon Patent And Licensing Inc. | Vehicle fleet routing system |
US10331131B2 (en) | 2011-05-12 | 2019-06-25 | Unmanned Innovations, Inc. | Systems and methods for payload integration and control in a multi-mode unmanned vehicle |
US10452072B2 (en) * | 2017-05-25 | 2019-10-22 | Ford Global Technologies, Llc | Methods and apparatuses for vehicle wading safety |
CN110377019A (en) * | 2018-09-05 | 2019-10-25 | 天津京东深拓机器人科技有限公司 | A kind of route generating method and device |
US20190370746A1 (en) * | 2018-06-05 | 2019-12-05 | Signal Ocean Ltd | Carrier path prediction based on dynamic input data |
CN110749333A (en) * | 2019-11-07 | 2020-02-04 | 中南大学 | Unmanned vehicle motion planning method based on multi-objective optimization |
US10558492B2 (en) | 2016-10-07 | 2020-02-11 | Raytheon Company | System and method for predicting the performance of a radar |
CN111157002A (en) * | 2019-12-30 | 2020-05-15 | 西安电子科技大学 | Aircraft 3D path planning method based on multi-agent evolutionary algorithm |
US10656645B1 (en) * | 2017-10-10 | 2020-05-19 | Uatc, Llc | Determining autonomous vehicle routes |
CN111464935A (en) * | 2014-06-18 | 2020-07-28 | 维里逊专利及许可公司 | Transmitting method, service platform, providing method and tracking method |
CN111615082A (en) * | 2020-05-13 | 2020-09-01 | 长安大学 | Heterogeneous automatic driving vehicle formation calculation task collaborative execution method |
CN111708369A (en) * | 2020-07-17 | 2020-09-25 | 武汉科技大学 | Route planning method for transformer substation inspection robot |
US10809734B2 (en) | 2019-03-13 | 2020-10-20 | Mobile Industrial Robots A/S | Route planning in an autonomous device |
CN111967692A (en) * | 2020-09-07 | 2020-11-20 | 中国人民解放军国防科技大学 | Search and rescue resource scheme generation method based on search theory and related equipment |
CN112013829A (en) * | 2020-08-12 | 2020-12-01 | 西安工业大学 | Multi-UAV/UGV (unmanned aerial vehicle/user generated Unit) cooperative long-term operation path planning method based on multi-objective optimization |
CN112085288A (en) * | 2020-09-16 | 2020-12-15 | 浙江财经大学 | Logistics distribution route recommendation method and system based on two-stage optimization |
CN112269382A (en) * | 2020-10-21 | 2021-01-26 | 桂林电子科技大学 | Robot multi-target path planning method |
US10921132B2 (en) | 2017-07-26 | 2021-02-16 | Motorola Solutions, Inc. | Method to determine search area for a person of interest based upon past route information from searchers |
US20210348926A1 (en) * | 2018-10-23 | 2021-11-11 | Offshore Navigation Limited | An apparatus for determining an optimal route of a maritime ship |
CN113885527A (en) * | 2021-11-02 | 2022-01-04 | 中国矿业大学 | Method for planning medicine taking path by utilizing ant colony of niche |
US11257141B2 (en) | 2018-06-20 | 2022-02-22 | Simbe Robotics, Inc. | Method for managing click and delivery shopping events |
US11287272B2 (en) | 2018-11-19 | 2022-03-29 | International Business Machines Corporation | Combined route planning and opportunistic searching in variable cost environments |
US11341454B2 (en) | 2016-05-19 | 2022-05-24 | Simbe Robotics, Inc. | Method for tracking placement of products on shelves in a store |
US20220245585A1 (en) * | 2021-01-29 | 2022-08-04 | Walmart Apollo, Llc | System and method for generating last-mile delivery routes |
US11454973B2 (en) * | 2018-01-11 | 2022-09-27 | Uatc, Llc | Mapped driving paths for autonomous vehicle |
GB2608272A (en) * | 2018-07-05 | 2022-12-28 | Qinetiq Ltd | Route determination |
US11592299B2 (en) | 2020-03-19 | 2023-02-28 | Mobile Industrial Robots A/S | Using static scores to control vehicle operations |
WO2023056308A1 (en) * | 2021-09-29 | 2023-04-06 | Johnson Controls Tyco IP Holdings LLP | Systems and methods for randomized autonomous robot security applications |
US20230171154A1 (en) * | 2020-04-17 | 2023-06-01 | Telefonaktiebolaget Lm Ericsson (Publ) | Network node and method for handling operations in a communications network |
US11953901B2 (en) * | 2020-05-05 | 2024-04-09 | Uatc, Llc | Determining autonomous vehicle routes |
Families Citing this family (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009007965A2 (en) * | 2007-07-09 | 2009-01-15 | Technion Research & Development Foundation Ltd | Routing methods for multiple geographical entities |
US20090296990A1 (en) * | 2008-05-30 | 2009-12-03 | United Parcel Service Of America, Inc. | Evaluating driver walk distances and building types using overhead imagery |
AU2010290950B2 (en) * | 2009-09-07 | 2015-02-05 | Bae Systems Plc | Path determination |
US9064222B2 (en) * | 2010-05-14 | 2015-06-23 | The Boeing Company | Real time mission planning |
KR101732902B1 (en) * | 2010-12-27 | 2017-05-24 | 삼성전자주식회사 | Path planning apparatus of robot and method thereof |
JP5772332B2 (en) * | 2011-07-20 | 2015-09-02 | 富士通株式会社 | Program, method and apparatus for determining a tour route |
US8589013B2 (en) | 2011-10-25 | 2013-11-19 | Jaybridge Robotics, Inc. | Method and system for dynamically positioning a vehicle relative to another vehicle in motion |
WO2013070123A1 (en) * | 2011-11-08 | 2013-05-16 | Saab Ab | Route planning system and method for minimizing exposure to threats |
US10552581B2 (en) | 2011-12-30 | 2020-02-04 | Elwha Llc | Evidence-based healthcare information management protocols |
US10679309B2 (en) | 2011-12-30 | 2020-06-09 | Elwha Llc | Evidence-based healthcare information management protocols |
US10340034B2 (en) | 2011-12-30 | 2019-07-02 | Elwha Llc | Evidence-based healthcare information management protocols |
US10559380B2 (en) | 2011-12-30 | 2020-02-11 | Elwha Llc | Evidence-based healthcare information management protocols |
US20130173298A1 (en) | 2011-12-30 | 2013-07-04 | Elwha LLC, a limited liability company of State of Delaware | Evidence-based healthcare information management protocols |
US10528913B2 (en) | 2011-12-30 | 2020-01-07 | Elwha Llc | Evidence-based healthcare information management protocols |
US10475142B2 (en) | 2011-12-30 | 2019-11-12 | Elwha Llc | Evidence-based healthcare information management protocols |
US8566027B2 (en) | 2012-03-14 | 2013-10-22 | Lockheed Martin Corporation | Route re-planning using enemy force lethality projection |
US10346784B1 (en) | 2012-07-27 | 2019-07-09 | Google Llc | Near-term delivery system performance simulation |
GB2505464B (en) * | 2012-08-31 | 2019-12-18 | Bae Systems Plc | Route planning |
US8843303B1 (en) * | 2012-12-17 | 2014-09-23 | Rockwell Collins, Inc. | Risk-aware contingency flight re-planner system and related method |
US9188986B2 (en) | 2013-10-01 | 2015-11-17 | Jaybridge Robotics, Inc. | Computer-implemented method and system for dynamically positioning a vehicle relative to another vehicle in motion for on-the-fly offloading operations |
US10956855B1 (en) * | 2015-08-16 | 2021-03-23 | Palidian Incorporated | Integrated multi-location scheduling, routing, and task management |
US10296862B1 (en) * | 2016-05-12 | 2019-05-21 | Northrop Grumman Systems Corporation | Time interval assessment of mission plan quality dynamic asset allocation |
CN107390690A (en) * | 2017-07-28 | 2017-11-24 | 江西理工大学 | Merge the robot path planning method of chaos and elite backward learning |
CN107831790A (en) * | 2017-09-21 | 2018-03-23 | 南京航空航天大学 | A kind of Alliance Establishment method of the isomery unmanned plane collaboratively searching strike based on multi-objective genetic algorithm |
US11747825B2 (en) | 2018-10-12 | 2023-09-05 | Boston Dynamics, Inc. | Autonomous map traversal with waypoint matching |
CN109799820B (en) * | 2019-01-22 | 2020-12-22 | 智慧航海(青岛)科技有限公司 | Unmanned ship local path planning method based on comparative random road map method |
US10467562B1 (en) * | 2019-02-18 | 2019-11-05 | Coupang, Corp. | Systems and methods for computerized balanced delivery route assignment |
US10467563B1 (en) * | 2019-02-18 | 2019-11-05 | Coupang, Corp. | Systems and methods for computerized balanced delivery route pre-assignment |
BR112021024241A2 (en) | 2019-06-20 | 2022-04-12 | Monsanto Technology Llc | Systems and methods for routing combine/harvesters |
EP4010659A1 (en) | 2019-08-06 | 2022-06-15 | Boston Dynamics, Inc. | Intermediate waypoint generator |
US11619945B2 (en) | 2019-09-30 | 2023-04-04 | GM Cruise Holdings LLC. | Map prior layer |
US11529961B2 (en) * | 2019-09-30 | 2022-12-20 | GM Cruise Holdings LLC. | Tracking object path in map prior layer |
CN111141289A (en) * | 2019-12-17 | 2020-05-12 | 佛山科学技术学院 | Early warning navigation method and system for bath chair |
US10809080B2 (en) | 2020-03-23 | 2020-10-20 | Alipay Labs (singapore) Pte. Ltd. | System and method for determining routing by learned selective optimization |
US11443258B2 (en) * | 2020-11-26 | 2022-09-13 | Shopify Inc. | Real-time order delivery coordination between multiple merchants |
CN115167418B (en) * | 2022-07-04 | 2023-06-27 | 未岚大陆(北京)科技有限公司 | Transfer path generation method, apparatus, electronic device, and computer storage medium |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7184886B1 (en) * | 2001-12-21 | 2007-02-27 | Garmin Ltd. | Navigation system, method and device with detour algorithm |
US7239962B2 (en) * | 2003-02-21 | 2007-07-03 | Sony Corporation | Method and apparatus for a routing agent |
US7363126B1 (en) * | 2002-08-22 | 2008-04-22 | United Parcel Service Of America | Core area territory planning for optimizing driver familiarity and route flexibility |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5961571A (en) * | 1994-12-27 | 1999-10-05 | Siemens Corporated Research, Inc | Method and apparatus for automatically tracking the location of vehicles |
KR100235240B1 (en) * | 1995-10-31 | 1999-12-15 | 모리 하루오 | Navigation apparatus |
-
2004
- 2004-06-24 US US10/877,814 patent/US20050216182A1/en not_active Abandoned
-
2008
- 2008-09-08 US US12/206,296 patent/US7693653B2/en not_active Expired - Fee Related
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7184886B1 (en) * | 2001-12-21 | 2007-02-27 | Garmin Ltd. | Navigation system, method and device with detour algorithm |
US7363126B1 (en) * | 2002-08-22 | 2008-04-22 | United Parcel Service Of America | Core area territory planning for optimizing driver familiarity and route flexibility |
US7239962B2 (en) * | 2003-02-21 | 2007-07-03 | Sony Corporation | Method and apparatus for a routing agent |
Cited By (165)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7664094B1 (en) * | 2001-12-10 | 2010-02-16 | At&T Corp. | System for utilizing genetic algorithm to provide constraint-based routing of packets in a communication network |
US8064432B2 (en) | 2001-12-10 | 2011-11-22 | At&T Intellectual Property Ii, L.P. | System for utilizing genetic algorithm to provide constraint-based routing of packets in a communication network |
US7110881B2 (en) * | 2003-10-07 | 2006-09-19 | Deere & Company | Modular path planner |
US20050075784A1 (en) * | 2003-10-07 | 2005-04-07 | Gray Sarah Ann | Modular path planner |
US20060047421A1 (en) * | 2004-08-25 | 2006-03-02 | Microsoft Corporation | Computing point-to-point shortest paths from external memory |
US20060184292A1 (en) * | 2005-02-16 | 2006-08-17 | Lockheed Martin Corporation | Mission planning system for vehicles with varying levels of autonomy |
US20070073897A1 (en) * | 2005-06-21 | 2007-03-29 | Mehdi Sharifzadeh | Optimal sequenced route query operation and device |
US20070067095A1 (en) * | 2005-07-01 | 2007-03-22 | Honeywell International Inc. | Method and apparatus for resolving ambiguous waypoints |
WO2007050427A2 (en) * | 2005-10-26 | 2007-05-03 | Motorola Inc. | Path determination facilitation method |
US20070094165A1 (en) * | 2005-10-26 | 2007-04-26 | Motorola, Inc. | Path determination facilitation method |
US7461035B2 (en) * | 2005-10-26 | 2008-12-02 | Motorola, Inc. | Path determination facilitation method |
WO2007050427A3 (en) * | 2005-10-26 | 2009-04-16 | Motorola Inc | Path determination facilitation method |
US7756632B2 (en) * | 2005-10-31 | 2010-07-13 | Honeywell International Inc. | System and method for performing 4-dimensional navigation |
US20100030467A1 (en) * | 2005-10-31 | 2010-02-04 | Honeywell International Inc. | System and method for performing 4-dimensional navigation |
US20070179690A1 (en) * | 2006-02-01 | 2007-08-02 | Stewart Brian G | Variable path automated guided vehicle |
US7991521B2 (en) | 2006-02-01 | 2011-08-02 | Jervis B. Webb Company | Variable path automated guided vehicle |
WO2007090181A3 (en) * | 2006-02-01 | 2008-05-02 | Webb Co Jervis B | Variable path automated guided vehicle |
US20070208548A1 (en) * | 2006-03-03 | 2007-09-06 | Solido Design Automation Inc. | Modeling of systems using canonical form functions and symbolic regression |
US8332188B2 (en) * | 2006-03-03 | 2012-12-11 | Solido Design Automation Inc. | Modeling of systems using canonical form functions and symbolic regression |
US7734387B1 (en) * | 2006-03-31 | 2010-06-08 | Rockwell Collins, Inc. | Motion planner for unmanned ground vehicles traversing at high speeds in partially known environments |
US9791860B2 (en) | 2006-05-12 | 2017-10-17 | Irobot Defense Holdings Inc. | Autonomous behaviors for a remote vehicle |
US20100274487A1 (en) * | 2006-05-17 | 2010-10-28 | Neff Michael G | Route search planner |
US20070288156A1 (en) * | 2006-05-17 | 2007-12-13 | The Boeing Company | Route search planner |
US9127913B2 (en) * | 2006-05-17 | 2015-09-08 | The Boeing Company | Route search planner |
US20090175219A1 (en) * | 2006-05-22 | 2009-07-09 | Augustin Chaintreau | Representation of a Delay Path in Mobile Networks |
US8509084B2 (en) * | 2006-05-22 | 2013-08-13 | Thomson Licensing | Representation of a delay path in mobile networks |
US20080091341A1 (en) * | 2006-06-27 | 2008-04-17 | Microsoft Corporation | Route monetization |
US8718925B2 (en) | 2006-06-27 | 2014-05-06 | Microsoft Corporation | Collaborative route planning for generating personalized and context-sensitive routing recommendations |
US8793066B2 (en) | 2006-06-27 | 2014-07-29 | Microsoft Corporation | Route monetization |
US20080010044A1 (en) * | 2006-07-05 | 2008-01-10 | Ruetsch Gregory R | Using interval techniques to solve a parametric multi-objective optimization problem |
US7664622B2 (en) * | 2006-07-05 | 2010-02-16 | Sun Microsystems, Inc. | Using interval techniques to solve a parametric multi-objective optimization problem |
US20080027590A1 (en) * | 2006-07-14 | 2008-01-31 | Emilie Phillips | Autonomous behaviors for a remote vehicle |
US8108092B2 (en) | 2006-07-14 | 2012-01-31 | Irobot Corporation | Autonomous behaviors for a remote vehicle |
US8577517B2 (en) | 2006-07-14 | 2013-11-05 | Irobot Corporation | Autonomous behaviors for a remote vehicle |
US20110106339A1 (en) * | 2006-07-14 | 2011-05-05 | Emilie Phillips | Autonomous Behaviors for a Remote Vehicle |
US8326469B2 (en) | 2006-07-14 | 2012-12-04 | Irobot Corporation | Autonomous behaviors for a remote vehicle |
US8396611B2 (en) | 2006-07-14 | 2013-03-12 | Irobot Corporation | Autonomous behaviors for a remote vehicle |
US20080215512A1 (en) * | 2006-09-12 | 2008-09-04 | New York University | System, method, and computer-accessible medium for providing a multi-objective evolutionary optimization of agent-based models |
US8015127B2 (en) * | 2006-09-12 | 2011-09-06 | New York University | System, method, and computer-accessible medium for providing a multi-objective evolutionary optimization of agent-based models |
US20080086241A1 (en) * | 2006-10-06 | 2008-04-10 | Irobot Corporation | Autonomous Behaviors for a Remove Vehicle |
US8843244B2 (en) * | 2006-10-06 | 2014-09-23 | Irobot Corporation | Autonomous behaviors for a remove vehicle |
US20080096628A1 (en) * | 2006-10-23 | 2008-04-24 | Zbigniew Czyzewski | Security devices for implementing hand-held wagering |
US20080162242A1 (en) * | 2006-12-27 | 2008-07-03 | Verizon Services Organization Inc. | Dispatching Prioritized Jobs At Multiple Locations To Workers |
US8799046B2 (en) * | 2006-12-27 | 2014-08-05 | Verizon Patent And Licensing Inc. | Dispatching prioritized jobs at multiple locations to workers |
US8346391B1 (en) * | 2006-12-28 | 2013-01-01 | Science Applications International Corporation | Methods and systems for an autonomous robotic platform |
US8682485B2 (en) | 2006-12-28 | 2014-03-25 | Leidos, Inc. | Methods and systems for an autonomous robotic platform |
US8760397B2 (en) | 2007-04-24 | 2014-06-24 | Irobot Corporation | Control system for a remote vehicle |
US9195256B2 (en) | 2007-04-24 | 2015-11-24 | Irobot Corporation | Control system for a remote vehicle |
US20080270331A1 (en) * | 2007-04-26 | 2008-10-30 | Darrin Taylor | Method and system for solving an optimization problem with dynamic constraints |
US8069127B2 (en) | 2007-04-26 | 2011-11-29 | 21 Ct, Inc. | Method and system for solving an optimization problem with dynamic constraints |
US8255092B2 (en) | 2007-05-14 | 2012-08-28 | Irobot Corporation | Autonomous behaviors for a remote vehicle |
US8447440B2 (en) | 2007-05-14 | 2013-05-21 | iRobot Coporation | Autonomous behaviors for a remote vehicle |
US20090037033A1 (en) * | 2007-05-14 | 2009-02-05 | Emilie Phillips | Autonomous Behaviors for a Remote Vehicle |
US7979174B2 (en) | 2007-09-28 | 2011-07-12 | Honeywell International Inc. | Automatic planning and regulation of the speed of autonomous vehicles |
US20090088916A1 (en) * | 2007-09-28 | 2009-04-02 | Honeywell International Inc. | Method and system for automatic path planning and obstacle/collision avoidance of autonomous vehicles |
US20090319112A1 (en) * | 2007-09-28 | 2009-12-24 | Honeywell International Inc. | Automatic planning and regulation of the speed of autonomous vehicles |
US8185298B2 (en) | 2007-10-17 | 2012-05-22 | Lockheed Martin Corporation | Hybrid heuristic national airspace flight path optimization |
US20090105935A1 (en) * | 2007-10-17 | 2009-04-23 | Lockheed Martin Corporation | Hybrid heuristic national airspace flight path optimization |
US20090112645A1 (en) * | 2007-10-25 | 2009-04-30 | Lockheed Martin Corporation | Multi objective national airspace collaborative optimization |
US20090112843A1 (en) * | 2007-10-29 | 2009-04-30 | International Business Machines Corporation | System and method for providing differentiated service levels for search index |
US8428859B2 (en) | 2007-12-14 | 2013-04-23 | Microsoft Corporation | Federated route production |
US20090157311A1 (en) * | 2007-12-14 | 2009-06-18 | Microsoft Corporation | Federated route production |
US8473198B2 (en) | 2007-12-14 | 2013-06-25 | Microsoft Corporation | Additional content based on intended travel destination |
US8060297B2 (en) | 2007-12-14 | 2011-11-15 | Microsoft Corporation | Route transfer between devices |
US8090532B2 (en) | 2007-12-14 | 2012-01-03 | Microsoft Corporation | Pedestrian route production |
US9513125B2 (en) * | 2008-01-14 | 2016-12-06 | The Boeing Company | Computing route plans for routing around obstacles having spatial and temporal dimensions |
US20120158280A1 (en) * | 2008-01-14 | 2012-06-21 | Ravenscroft Donald L | Computing route plans for routing around obstacles having spatial and temporal dimensions |
WO2009105288A3 (en) * | 2008-02-19 | 2009-10-15 | Microsoft Corporation | Route reward augmentation |
US8793065B2 (en) | 2008-02-19 | 2014-07-29 | Microsoft Corporation | Route-based activity planner |
US20100057358A1 (en) * | 2008-08-28 | 2010-03-04 | TeleType Co., Inc. | Portable gps map device for commercial vehicle industry |
US8121749B1 (en) | 2008-09-25 | 2012-02-21 | Honeywell International Inc. | System for integrating dynamically observed and static information for route planning in a graph based planner |
US20110178704A1 (en) * | 2008-12-08 | 2011-07-21 | Micro-Star International Co., Ltd. | Navigation device capable of adding path to navigation plan independently |
US8392117B2 (en) | 2009-05-22 | 2013-03-05 | Toyota Motor Engineering & Manufacturing North America, Inc. | Using topological structure for path planning in semi-structured environments |
US20110029234A1 (en) * | 2009-07-29 | 2011-02-03 | Lockheed Martin Corporation | Threat Analysis Toolkit |
US9115996B2 (en) * | 2009-07-29 | 2015-08-25 | Lockheed Martin Corporation | Threat analysis toolkit |
US20110172850A1 (en) * | 2009-09-14 | 2011-07-14 | Israel Aerospace Industries Ltd. | Infantry robotic porter system and methods useful in conjunction therewith |
US8774981B2 (en) * | 2009-09-14 | 2014-07-08 | Israel Aerospace Industries Ltd. | Infantry robotic porter system and methods useful in conjunction therewith |
US20120290148A1 (en) * | 2009-11-24 | 2012-11-15 | Chinagps Co., Ltd (Shenzhen) | Method and system for dispatching vehicle |
US8706411B2 (en) * | 2009-11-24 | 2014-04-22 | Chinagps Co., Ltd. (Shenzhen) | Method and system for dispatching vehicle |
US20110153072A1 (en) * | 2009-12-17 | 2011-06-23 | Noel Wayne Anderson | Enhanced visual landmark for localization |
US20110153136A1 (en) * | 2009-12-17 | 2011-06-23 | Noel Wayne Anderson | System and method for area coverage using sector decomposition |
US8224516B2 (en) * | 2009-12-17 | 2012-07-17 | Deere & Company | System and method for area coverage using sector decomposition |
US8666554B2 (en) | 2009-12-17 | 2014-03-04 | Deere & Company | System and method for area coverage using sector decomposition |
US8635015B2 (en) | 2009-12-17 | 2014-01-21 | Deere & Company | Enhanced visual landmark for localization |
US20110153338A1 (en) * | 2009-12-17 | 2011-06-23 | Noel Wayne Anderson | System and method for deploying portable landmarks |
US8989946B2 (en) | 2009-12-17 | 2015-03-24 | Deere & Company | System and method for area coverage using sector decomposition |
US9235817B2 (en) | 2011-01-12 | 2016-01-12 | Cummins Intellectual Property, Inc. | System and method of determining freight/load distribution for multiple vehicles |
WO2012097200A1 (en) * | 2011-01-12 | 2012-07-19 | Cummins Intellectual Property, Inc. | System and method of determining freight/load distribution for multiple vehicles |
US10579953B2 (en) | 2011-01-12 | 2020-03-03 | Cummins Intellectual Property, Inc. | System and method of determining freight/load distribution for multiple vehicles |
US8738559B2 (en) * | 2011-01-24 | 2014-05-27 | Microsoft Corporation | Graph partitioning with natural cuts |
US20120192138A1 (en) * | 2011-01-24 | 2012-07-26 | Microsoft Corporation | Graph partitioning with natural cuts |
US8868323B2 (en) | 2011-03-22 | 2014-10-21 | Honeywell International Inc. | Collaborative navigation using conditional updates |
US8660338B2 (en) | 2011-03-22 | 2014-02-25 | Honeywell International Inc. | Wide baseline feature matching using collobrative navigation and digital terrain elevation data constraints |
US9669904B2 (en) | 2011-05-12 | 2017-06-06 | Unmanned Innovations, Inc. | Systems and methods for multi-mode unmanned vehicle mission planning and control |
US10331131B2 (en) | 2011-05-12 | 2019-06-25 | Unmanned Innovations, Inc. | Systems and methods for payload integration and control in a multi-mode unmanned vehicle |
US20140222278A1 (en) * | 2011-08-25 | 2014-08-07 | Nissan Motor Co., Ltd. | Autonomous driving control system for vehicle |
US9182761B2 (en) * | 2011-08-25 | 2015-11-10 | Nissan Motor Co., Ltd. | Autonomous driving control system for vehicle |
US20140244114A1 (en) * | 2011-10-03 | 2014-08-28 | Toyota Jidosha Kabushiki Kaisha | Driving assistance system for vehicle |
US9205864B2 (en) * | 2011-10-03 | 2015-12-08 | Toyota Jidosha Kabushiki Kaisha | Driving assistance system for vehicle |
CN102611119A (en) * | 2012-03-14 | 2012-07-25 | 华北电力大学 | Multi-target reactive power optimization method for electric system |
US10311385B2 (en) | 2012-06-15 | 2019-06-04 | Verizon Patent And Licensing Inc. | Vehicle fleet routing system |
US20130339266A1 (en) * | 2012-06-15 | 2013-12-19 | Telogis, Inc. | Vehicle fleet routing system |
US10528062B2 (en) * | 2012-06-15 | 2020-01-07 | Verizon Patent And Licensing Inc. | Computerized vehicle control system for fleet routing |
US20190251485A1 (en) * | 2012-06-15 | 2019-08-15 | Verizon Patent And Licensing Inc. | Vehicle fleet routing system |
US10664770B2 (en) * | 2012-06-15 | 2020-05-26 | Verizon Patent And Licensing Inc. | Vehicle fleet routing system |
US20170277198A1 (en) * | 2012-06-15 | 2017-09-28 | Telogis, Inc. | Computerized vehicle control system for fleet routing |
US20140032049A1 (en) * | 2012-07-24 | 2014-01-30 | GM Global Technology Operations LLC | Steering Assist in Driver Initiated Collision Avoidance Maneuver |
US8849515B2 (en) * | 2012-07-24 | 2014-09-30 | GM Global Technology Operations LLC | Steering assist in driver initiated collision avoidance maneuver |
CN102820662A (en) * | 2012-08-17 | 2012-12-12 | 华北电力大学 | Distributed power source contained power system multi-target reactive-power optimization method |
US8442765B1 (en) | 2012-11-08 | 2013-05-14 | Honeywell International Inc. | Shared state selection and data exchange for collaborative navigation using conditionally independent parallel filters |
US9104752B2 (en) | 2012-11-16 | 2015-08-11 | Honeywell International Inc. | Data sharing among conditionally independent parallel filters |
TWI486807B (en) * | 2013-04-16 | 2015-06-01 | Papago Inc | Judgment System of Vehicle Moving State and Its Judgment |
US9996364B2 (en) * | 2013-08-30 | 2018-06-12 | Insitu, Inc. | Vehicle user interface adaptation |
US20160364248A1 (en) * | 2013-08-30 | 2016-12-15 | Insitu, Inc. | Vehicle user interface adaptation |
US9592604B2 (en) | 2014-01-07 | 2017-03-14 | Irobot Defense Holdings, Inc. | Remotely operating a mobile robot |
US9789612B2 (en) | 2014-01-07 | 2017-10-17 | Irobot Defense Holdings, Inc. | Remotely operating a mobile robot |
US9283674B2 (en) | 2014-01-07 | 2016-03-15 | Irobot Corporation | Remotely operating a mobile robot |
CN111464935A (en) * | 2014-06-18 | 2020-07-28 | 维里逊专利及许可公司 | Transmitting method, service platform, providing method and tracking method |
WO2017004419A1 (en) | 2015-06-30 | 2017-01-05 | Unmanned Innovations, Inc. | Systems and methods for multi-mode unmanned vehicle mission planning and control |
US9945677B1 (en) * | 2015-07-23 | 2018-04-17 | X Development Llc | Automated lane and route network discovery for robotic actors |
US20170193434A1 (en) * | 2015-11-09 | 2017-07-06 | Simbe Robotics, Inc | Method for tracking stock level within a store |
US11276034B2 (en) * | 2015-11-09 | 2022-03-15 | Simbe Robotics, Inc. | Method for tracking stock level within a store |
US10607182B2 (en) * | 2015-11-09 | 2020-03-31 | Simbe Robotics, Inc. | Method for tracking stock level within a store |
CN105929843A (en) * | 2016-04-22 | 2016-09-07 | 天津城建大学 | Robot path planning method based on improved ant colony algorithm |
US11341454B2 (en) | 2016-05-19 | 2022-05-24 | Simbe Robotics, Inc. | Method for tracking placement of products on shelves in a store |
US20170341235A1 (en) * | 2016-05-27 | 2017-11-30 | General Electric Company | Control System And Method For Robotic Motion Planning And Control |
CN106370175A (en) * | 2016-08-18 | 2017-02-01 | 深圳市沃特沃德股份有限公司 | Pet navigation method and device |
EP3287864A1 (en) * | 2016-08-26 | 2018-02-28 | Thales | Method for aided-piloting an aircraft, associated computer program product and device for aided-piloting |
US10558492B2 (en) | 2016-10-07 | 2020-02-11 | Raytheon Company | System and method for predicting the performance of a radar |
US9933781B1 (en) | 2016-11-23 | 2018-04-03 | Denso International America, Inc. | Data-driven planning for automated driving |
US10074279B1 (en) * | 2017-03-07 | 2018-09-11 | Denso International America, Inc. | Inference-aware motion planning |
US10452072B2 (en) * | 2017-05-25 | 2019-10-22 | Ford Global Technologies, Llc | Methods and apparatuses for vehicle wading safety |
CN109211255A (en) * | 2017-07-06 | 2019-01-15 | 罗伯特·博世有限公司 | For the method for the motor vehicle programme path with automotive vehicle system |
US10921132B2 (en) | 2017-07-26 | 2021-02-16 | Motorola Solutions, Inc. | Method to determine search area for a person of interest based upon past route information from searchers |
US20190086220A1 (en) * | 2017-09-21 | 2019-03-21 | Whp Workflow Solutions, Inc. | Dynamic target coverage using mobile assets |
US10794712B2 (en) * | 2017-09-21 | 2020-10-06 | Getac Technology Corporation | Dynamic target coverage using mobile assets |
US10656645B1 (en) * | 2017-10-10 | 2020-05-19 | Uatc, Llc | Determining autonomous vehicle routes |
US11029168B2 (en) * | 2017-10-10 | 2021-06-08 | The Government Of The United States Of America, As Represented By The Secretary Of The Navy | Method for identifying optimal vehicle paths when energy is a key metric or constraint |
WO2019075008A1 (en) * | 2017-10-10 | 2019-04-18 | The Government Of The United States Of America, As Represented By The Secretary Of The Navy | Method for identifying optimal vehicle paths when energy is a key metric or constraint |
US11454973B2 (en) * | 2018-01-11 | 2022-09-27 | Uatc, Llc | Mapped driving paths for autonomous vehicle |
US10977605B2 (en) * | 2018-06-05 | 2021-04-13 | Signal Ocean Ltd | Carrier path prediction based on dynamic input data |
US20190370746A1 (en) * | 2018-06-05 | 2019-12-05 | Signal Ocean Ltd | Carrier path prediction based on dynamic input data |
US11257141B2 (en) | 2018-06-20 | 2022-02-22 | Simbe Robotics, Inc. | Method for managing click and delivery shopping events |
GB2608272A (en) * | 2018-07-05 | 2022-12-28 | Qinetiq Ltd | Route determination |
GB2608272B (en) * | 2018-07-05 | 2023-06-14 | Qinetiq Ltd | Route determination |
CN109269518A (en) * | 2018-08-31 | 2019-01-25 | 北京航空航天大学 | A kind of movable fixture confined space path generating method based on intelligent body |
CN110377019A (en) * | 2018-09-05 | 2019-10-25 | 天津京东深拓机器人科技有限公司 | A kind of route generating method and device |
CN109144067A (en) * | 2018-09-17 | 2019-01-04 | 长安大学 | A kind of Intelligent cleaning robot and its paths planning method |
US20210348926A1 (en) * | 2018-10-23 | 2021-11-11 | Offshore Navigation Limited | An apparatus for determining an optimal route of a maritime ship |
US11287272B2 (en) | 2018-11-19 | 2022-03-29 | International Business Machines Corporation | Combined route planning and opportunistic searching in variable cost environments |
US10809734B2 (en) | 2019-03-13 | 2020-10-20 | Mobile Industrial Robots A/S | Route planning in an autonomous device |
CN110749333A (en) * | 2019-11-07 | 2020-02-04 | 中南大学 | Unmanned vehicle motion planning method based on multi-objective optimization |
CN111157002A (en) * | 2019-12-30 | 2020-05-15 | 西安电子科技大学 | Aircraft 3D path planning method based on multi-agent evolutionary algorithm |
US11592299B2 (en) | 2020-03-19 | 2023-02-28 | Mobile Industrial Robots A/S | Using static scores to control vehicle operations |
US20230171154A1 (en) * | 2020-04-17 | 2023-06-01 | Telefonaktiebolaget Lm Ericsson (Publ) | Network node and method for handling operations in a communications network |
US11953901B2 (en) * | 2020-05-05 | 2024-04-09 | Uatc, Llc | Determining autonomous vehicle routes |
CN111615082A (en) * | 2020-05-13 | 2020-09-01 | 长安大学 | Heterogeneous automatic driving vehicle formation calculation task collaborative execution method |
CN111708369A (en) * | 2020-07-17 | 2020-09-25 | 武汉科技大学 | Route planning method for transformer substation inspection robot |
CN112013829A (en) * | 2020-08-12 | 2020-12-01 | 西安工业大学 | Multi-UAV/UGV (unmanned aerial vehicle/user generated Unit) cooperative long-term operation path planning method based on multi-objective optimization |
CN111967692A (en) * | 2020-09-07 | 2020-11-20 | 中国人民解放军国防科技大学 | Search and rescue resource scheme generation method based on search theory and related equipment |
CN112085288A (en) * | 2020-09-16 | 2020-12-15 | 浙江财经大学 | Logistics distribution route recommendation method and system based on two-stage optimization |
CN112269382A (en) * | 2020-10-21 | 2021-01-26 | 桂林电子科技大学 | Robot multi-target path planning method |
US20220245585A1 (en) * | 2021-01-29 | 2022-08-04 | Walmart Apollo, Llc | System and method for generating last-mile delivery routes |
WO2023056308A1 (en) * | 2021-09-29 | 2023-04-06 | Johnson Controls Tyco IP Holdings LLP | Systems and methods for randomized autonomous robot security applications |
CN113885527A (en) * | 2021-11-02 | 2022-01-04 | 中国矿业大学 | Method for planning medicine taking path by utilizing ant colony of niche |
Also Published As
Publication number | Publication date |
---|---|
US7693653B2 (en) | 2010-04-06 |
US20090125225A1 (en) | 2009-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7693653B2 (en) | Vehicle routing and path planning | |
US8924069B1 (en) | Artificial immune system approach for airborne vehicle maneuvering | |
Xia et al. | Neural inverse reinforcement learning in autonomous navigation | |
US20040073341A1 (en) | Real-time route and sensor planning system with variable mission objectives | |
US6718261B2 (en) | Architecture for real-time maintenance of distributed mission plans | |
WO2003071465A2 (en) | Architecture for automatic evaluation of team reconnaissance and surveillance plans | |
CN109656264A (en) | For being generated to the method implemented by computer and system in the path 3D in landing site for aircraft | |
Valiente et al. | Robustness and adaptability of reinforcement learning-based cooperative autonomous driving in mixed-autonomy traffic | |
Bernardini et al. | Autonomous search and tracking via temporal planning | |
Li et al. | Multi-agent path finding with prioritized communication learning | |
CN111813144A (en) | Multi-unmanned aerial vehicle collaborative route planning method based on improved flocks of sheep algorithm | |
Shiroshita et al. | Behaviorally diverse traffic simulation via reinforcement learning | |
US7647232B2 (en) | Real-time team coordination system for reconnaissance and surveillance missions | |
Hussain et al. | Evolution-based deliberative planning for cooperating unmanned ground vehicles in a dynamic environment | |
Dai et al. | Work-in-progress: Leveraging the selfless driving model to reduce vehicular network congestion | |
Hamed et al. | Hybrid formation control for multi-robot hunters based on multi-agent deep deterministic policy gradient | |
Alazzam et al. | Solving multiple traveling salesmen problem using discrete pigeon inspired optimizer | |
Collins et al. | A UAV routing and sensor control optimization algorithm for target search | |
Goyal | Multi-agent deep reinforcement learning for robocup rescue simulator | |
Zhang et al. | Route planning for unmanned air vehicles with multiple missions using an evolutionary algorithm | |
Zhang et al. | Cat: Closed-loop adversarial training for safe end-to-end driving | |
Lucas et al. | Automatic vehicle navigation with bandwidth constraints | |
Stouch et al. | Coevolving collection plans for UAS constellations | |
Ruiz et al. | Prize-collecting traveling salesman problem: a reinforcement learning approach | |
Hussain et al. | Evolution-Based Deliberative Planning for Cooperating UGV's in a Dynamic Environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BBNT SOLUTIONS LLC, MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HUSSAIN, TALIB S.;ESTRADA, RICHARD F.;LAZARUS, RICHARD B.;AND OTHERS;REEL/FRAME:015218/0331;SIGNING DATES FROM 20040809 TO 20040922 |
|
AS | Assignment |
Owner name: BBN TECHNOLOGIES CORP., MASSACHUSETTS Free format text: MERGER;ASSIGNOR:BBNT SOLUTIONS LLC;REEL/FRAME:017274/0318 Effective date: 20060103 Owner name: BBN TECHNOLOGIES CORP.,MASSACHUSETTS Free format text: MERGER;ASSIGNOR:BBNT SOLUTIONS LLC;REEL/FRAME:017274/0318 Effective date: 20060103 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: BANK OF AMERICA, N.A., MASSACHUSETTS Free format text: INTELLECTUAL PROPERTY SECURITY AGREEMENT SUPPLEMENT;ASSIGNOR:BBN TECHNOLOGIES CORP.;REEL/FRAME:021926/0017 Effective date: 20081124 Owner name: BANK OF AMERICA, N.A.,MASSACHUSETTS Free format text: INTELLECTUAL PROPERTY SECURITY AGREEMENT SUPPLEMENT;ASSIGNOR:BBN TECHNOLOGIES CORP.;REEL/FRAME:021926/0017 Effective date: 20081124 |
|
AS | Assignment |
Owner name: BBN TECHNOLOGIES CORP. (AS SUCCESSOR BY MERGER TO Free format text: RELEASE OF SECURITY INTEREST;ASSIGNOR:BANK OF AMERICA, N.A. (SUCCESSOR BY MERGER TO FLEET NATIONAL BANK);REEL/FRAME:023427/0436 Effective date: 20091026 |