|Publication number||US7386389 B2|
|Application number||US 11/437,807|
|Publication date||Jun 10, 2008|
|Filing date||May 22, 2006|
|Priority date||Nov 20, 2003|
|Also published as||DE10354322A1, EP1685546A1, US20060271269, WO2005052883A1|
|Publication number||11437807, 437807, US 7386389 B2, US 7386389B2, US-B2-7386389, US7386389 B2, US7386389B2|
|Inventors||Uwe Stolle, Christian Salzmann|
|Original Assignee||Beyerische Motoren Werke Aktiengesellschaft|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (21), Non-Patent Citations (3), Referenced by (5), Classifications (9), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application claims priority under 35 U.S.C. § 120 to International Patent Application No. PCT/EP2004/010704 filed Sep. 22, 2004, and under 35 U.S.C. § 119 to German Patent Application No. 103 54 322.8 filed Nov. 20, 2003, the entire disclosure of these documents is herein expressly incorporated by reference.
The present invention relates to a method for determining the driving situation of a motor vehicle and a corresponding system.
Due to the growing volume of information made available in a motor vehicle and the associated demands on the driver, some well-directed relief is required when the burden is high due to the traffic situation. The present invention provides a method and a system for relieving the driver.
One aspect of the inventive method is the use of data provided in the motor vehicle, representing the value of at least one state variable of the vehicle. This data may be made available via the data bus of the vehicle, for example, for implementation of the inventive method. In a first step, a data record providing the history of the at least one state variable is supplied. In a second step, a neural network is made available by a suitably programmed computer in the motor vehicle. The neural network of the inventive method can have at least one input layer and one output layer, each layer having a plurality of perceptrons. In a third step, the respective value of the at least one state variable of the respective point in time, can be standardized to the range of 0 to 1, like all the other values, is sent to one perceptron of the neural network, the current driving situation then being output by the perceptrons of the output layer of the neural network after being trained.
A perceptron is a mathematical function (software function) formed by software, calculating from input values an output value that is relayed to various perceptrons. The input values are weighted and the output value of the perceptron is a mapping function of the weighted input values according to the software function.
Exemplary embodiments of the present invention are explained in greater detail below.
An exemplary neural network is a sigmoid network that can have three layers. Each perceptron in this case is formed by the sigmoid function, which is essentially known. Sigmoid networks are advantageously characterized in that the output value of the perceptron, i.e., the software function is largely linear to the output value, which simplifies further processing.
The prevailing driving state of the vehicle can be ascertained, i.e., determined, from a chronological sequence of driving situations that have been detected. A prevailing driving state is assigned to a chronological sequence of driving situations that have been detected on the basis of at least one assignment specification. Instead of the early driving state, a new driving state can be determined only when the new driving state has already been ascertained, i.e., determined, repeatedly within an interval of time that has elapsed.
A different situation can be assigned to each perceptron of the output layer and/or its output signal. The maximum output signal of all the output signals of the perceptrons of the output layer indicates the current driving situation of the motor vehicle.
The output signal, preferably the signal peak of the first perceptron of the output layer of the neural network can indicate a “stop and go” driving situation. The output signal, such as a signal peak, of the second perceptron of the output layer of the neural network can be defined by the “city traffic” driving situation. The output signal, can be a signal peak of the third perceptron of the output layer of the neural network stands for the “cruising” driving situation. The output signal, can be the signal peak, of the fourth perceptron of the output layer of the neural network stands for the “sporty” driving situation. In summary, the input layer of the neural network, i.e., the corresponding computer program in this embodiment of the present invention, is chronologically supplied with a data record having state variables of the motor vehicle and the driving situation is determined chronologically on the basis of the maximum output signal of all perceptrons of the output layer. It is self-evident that the driving situation thereby determined may also be grouped into a larger or smaller number of classes (“stop and go,” etc.).
In an exemplary embodiment of the present invention, the driving situation is considered to be nonspecific when the difference between the value of the maximum output signal of all perceptrons of the output layer and the value of the next smaller output signal of the respective perceptron is less than 20%. The same thing is also true alternatively or additionally in another exemplary embodiment when the value of the greatest output signal of all perceptrons of the output layer is smaller than 10% of its maximum value. For these optional inventive measures it is possible to take into account only driving situations that have been determined with sufficient certainty. This is true in particular of determination of driving state based on the driving situation ascertained.
In another exemplary embodiment of the present invention, the extent of the information to be relayed to the driver of the vehicle is based on the driving state ascertained. With the inventive method and/or system, information with a high deflection such as a telephone call can be withheld from the driver temporarily during a driving state which makes high demands on the driver, e.g., rapid driving on the autobahn, i.e., highway, and/or it is saved for later display, e.g., in a less demanding driving situation.
In an inventive embodiment and/or an inventive man-machine interface, the driver is able to select which information and/or which totality of information in a driving situation of the first class such as “stop and go” is to be displayed and/or output for him in a second class, e.g., “city driving,” etc.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.
The present invention is explained in greater detail below with reference to figures, in which
For future vehicle generations, it is important for the use of the MMI systems (MMI=man-machine interface) and the vehicle electronics to be adjusted adaptively to the needs of the driver. This may mean, for example, that the quantity of information supplied is reduced in the MMI of the vehicle in dynamic driving situations (e.g., sporty driving or autobahn driving), and more information may be made available to the driver in relaxed or even stationary driving situations (e.g., stop and go).
An important factor here is the driving situation such as fast, concentrated driving on a high-speed road or autobahn, searching driving in an inner city area or relaxed “cruising” on a rural road. This driving situation is to be recognized by a software method—preferably on the basis of data already available in the vehicle. Ideally it should be possible to ascertain the driving situation with data available in the vehicle anyway, e.g., data, messages and/or telegrams on the CAN bus of the vehicle. One problem here is that the driving situation may be perceived very subjectively. Furthermore, the driving situation cannot be determined as a point on the basis of a single point in time, but instead a period of time of a certain length in the past must be taken into account. One prerequisite for this is autonomous detection of the driving situation by the vehicle itself.
The inventive method is executed on an inventive software system which recognizes the prevailing driving situation on the basis of data made available anyway via the electronic vehicle bus system (e.g., CAN bus). The optional exclusive use of data available anyway has the advantage that no additional sensors or control units are required—even a GPS system is not absolutely necessary.
One aspect of the present invention lies in the “estimate” of the similarity of the real driving situation to the driving situations defined above because there is virtually never an exact correspondence. Another aspect is that the driving situations are situational models which should be evaluated over a certain period of time to allow reliable detection. If only the current data, in particular the CAN data, at the particular point in time were to be used (e.g., every 500 ms), the driving situation might change very 500 ms, which is not desirable. For example, stopping at a stop light or briefly stopping would already be classified and “stop and go,” despite the fact that it is perhaps only a short interruption in the “cruising” driving situation. The inventive method and/or system are therefore provided with a certain “inertia.”
The inventive method and/or system is/are preferably implemented by a software solution and/or a programmed sequence control forming a neural sigmoid network (e.g., a processor executing machine-readable code). After a continuous supply of values describing the state variable of the vehicle, the sigmoid network outputs a classification of the driving situation determined on the basis of the values. In other words, the neural network, i.e., the sigmoid network supplies a “similarity value,” which indicates the similarity of the supplied data, i.e., values to stored data, i.e., values responding to the aforementioned driving situations and determined as part of training of the sigmoid network in the aforementioned driving situations. With a high similarity, the driving situation assigned to the corresponding stored values is assumed to be currently accurate. In this exemplary embodiment, CAN messages that conform to a standard from the standardized CAN bus inside the vehicle are used as the input variables into the sigmoid network. These CAN messages are preferably chronologically discretized and normalized. Additionally or alternatively, data of another data bus provided in the vehicle may of course also be used to determine the driving situation and/or the driving state, if necessary.
At regular intervals, e.g., intervals of approximately 500 ms to approximately 2000 ms, a software component picks up defined types of messages from the CAN bus and processes them accordingly. In doing so, the following known data telegrams of the CAN bus are analyzed:
It is self-evident that more or fewer data telegrams, i.e., data may be used for calculation of the driving situation and the driving state if expedient or necessary.
To permit an easier understanding of the present invention, first the principles of the inventive method and a brief outline of a neural network of the present invention will be given, followed by detailed explanation of the inventive method and/or system.
The inventive method is roughly divided into two parts.
In the first step the inventive neural network, in particular a sigmoid network, is parametrized. In parameterization, the neural network is trained and the transitions and parameters of the “inertia” of the system are adjusted.
In a second step, the instantaneous driving situation and the current driving situation in the vehicle are calculated. The second step is divided as follows:
Neural networks are known methods and/or systems from informatics which are used, e.g., in image recognition or voice recognition. A neural network consists of a quantity of so-called perceptrons. A perceptron is a software function having a quantity of input values and calculating an output value from them, said output value being relayed as input to various perceptrons. The output value of a perceptron is the result of an imaging function of the weighted input values (inputs) according to the following function, preferably the sigmoid function:
where “inputs” refers to the quantity of weighted input edges.
A neural network consists of n layers, which in turn already consist of perceptrons, a perceptron of the n-th layer has all the perceptron outputs of the (n−1)-th layer as input values. A three-layer sigmoid network is preferably used with the inventive method and/or system.
An important aspect of neural networks is that they are “trainable” and can thus be adapted to the concrete object. In “training,” the network is adjusted through examples by the known so-called “backpropagation” method so that it classifies the new input values like “training values.”
In training by the “backpropagation” method, a signal pattern to be recognized is applied to the input layer of the sigmoid network. The perceptrons, i.e., the corresponding software components, perform a calculation according to the sigmoid function. The result of the calculation is output in the form of a signal pattern by the output layer of the sigmoid network. Parameterization of the weights of the individual perceptrons is performed by using known learning methods (algorithms) in which examples of input patterns are created and corresponding output patterns are preselected. The algorithm then adjusts the weights so that the preselected output patterns are calculated, i.e., formed for the preselected input patterns.
Example according to the present invention: the input pattern is a set of CAN messages corresponding to the “city traffic” driving situation. The weights are adjusted so that the output pattern of all the perceptrons of the output layer yields signal level 0, only perceptron number 2 yields signal level 1. Signal level 1 on perceptron 2 is assumed to be representative for the “city traffic” driving situation. After conclusion of training, the sigmoid network according to the present invention classifies real driving situations in a manner similar to that used in the examples of training. It is sufficient here if the real driving situations and/or the data patterns are similar to the trained driving situations and/or their data patterns. A complete correspondence is advantageously not necessary for mostly reliable and correct classification.
The inventive method and/or system provided in a vehicle is/are described in greater detail below.
In a first step, the driving situation is determined at regular intervals on the basis of the instantaneous relevant CAN data and a certain data history of this data by the three-layered neural network, preferably a gradient network or a sigmoid network. For example, the current driving situation may be determined every 0.5 second using the speed and acceleration data at the current point in time t and the data at points in time t−2 seconds and t−4 seconds.
Then in a second step, the driving state is determined on the basis of the current driving situation and the past driving situation as detected. The present recognized driving situation, the past recognized driving situations and the instantaneous driving situation all play a role in determining the driving state.
This will now be illustrated on the basis of an example. If the current driving state is “city traffic” and “stop and go” is recognized repeatedly as the driving situation, then the driving situation is set at “stop and go,” i.e., this is regarded as a given, only after the “stop and go” driving situation has been recognized, for example, eight times in a row. Otherwise, in city traffic, a single traffic stop light would result in the driving state being set at “stop and go.” However, if the vehicle is in the “cruising” driving state, then it is preferable here to set “stop and go” as the driving state after recognizing the “stop and go” driving state, for example, three times in a row.
As shown in the flow chart 100 in
The sigmoid network is trained to the driving situations considered expedient by the backpropagation and/or gradient descent method. In other words, a certain driving situation is assigned to certain (typical) input data into the sigmoid network and the weighting in the sigmoid network is adjusted so that the output perceptron that stands for a certain driving situation corresponding to the respective driving situation is assigned a maximum value.
A driving state is assigned to a history of driving situations by the state machine 5 according to the table in
The steps in the computation method for ascertaining the driving situation and the driving state are explained in greater detail below on the basis of exemplary data:
1. Picking up the relative telegrams on the CAN bus in a certain interval—e.g., every 0.5 to 2 seconds.
2. Processing the CAN data to yield a result vector having the parameters:
Then all data is normalized to have a value between 0 and 1.
3. Input of the vectors into the sigmoid network. Calculation of the current driving situation on the basis of the current and last two data records—i.e., an evaluation based on speed, steering performance and shifting in the last 6 seconds. The following outputs of the output layer of the sigmoid network are processed:
4. The current driving situation which is calculated every two seconds determines the driving state as follows:
This method may be employed with various parameters (history, CAN data, etc.). Therefore, a general description of the method will be presented here again.
This method depends on the amount of n input parameters, the width of the time window δ and the length of the history included, k·δ. This method is thus roughly divided into the following steps:
1. Picking up, discretizing and processing the s input parameters:
2. Calculating the input vector with the help of the three-layer sigmoid network:
An input signal vector of the defined form from 1.d is fed in discrete intervals δ into the sigmoid network. The data is shifted here through the input vector (“sliding window”).
3. Calculation of the driving situation:
4. Analysis and time delay due to state machine:
The current driving situation, which is calculated periodically with regard to δ, now has the following influence on the driving state ascertained.
5. The method begins again at step 1.
It is clear from the preceding discussion that the inventive method and/or system permit(s) a reliable determination of the driving situation and the driving state. In other words, the actual driving mode is used as input, not the trip environment (e.g., “highway”) and the driving mode presumed to be associated with it. Systems based on position determination (when “on the highway” is determined (via GPS), then the driving situation is considered as being “driving fast”) are less accurate because the actual driving mode may be quite different. With the method and system described here, it is possible to determine the driving situation regardless of the driving environment. For example, if the driver exhibits a driving performance on a country road like that on the highway, the “highway” driving situation will be recognized regardless of the actual conditions.
The inventive method and/or system may be advantageously—but need not be—implemented on the basis of the standard CAN data present in the vehicle anyway or other data that is inexpensively available in any vehicle without requiring special equipment such as a navigation system with GPS.
The inventive method and/or system is easily parametrizable and can thus be adapted flexibly to altered boundary conditions (e.g., new type of vehicle). Furthermore, the hardware requirements are very low (CPU, memory use) and no additional sensors are required. Therefore, it can be made available to drivers at a low cost from this standpoint as well.
The inventive method and/or system for recognition of the driving situation and/or driving state may be used for adaptation of any suitably adaptable vehicle functions and/or vehicle systems depending on the driving situation and/or driving state. Examples include:
Another possible application is operation of man-machine interfaces as a function of driving situation and/or driving state to relieve the burden on the driver; for example, the quantity of information forwarded to the driver may be based on the current driving state. Of course there are many other applications that take into account the current driving state.
The foregoing disclosure has been set forth merely to illustrate the invention and is not intended to be limiting. Since modifications of the disclosed embodiments incorporating the spirit and substance of the invention may occur to persons skilled in the art, the invention should be construed to include everything within the scope of the appended claims and equivalents thereof.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5410477||Mar 19, 1992||Apr 25, 1995||Hitachi, Ltd.||Control system for an automotive vehicle having apparatus for predicting the driving environment of the vehicle|
|US5522015||Feb 10, 1995||May 28, 1996||Honda Giken Kogyo Kabushiki Kaisha||Neural network and learning method for linearly unseparable patterns|
|US5566072||Aug 10, 1994||Oct 15, 1996||Mitsubishi Jidosha Kogyo Kabushiki Kaisha||Method and apparatus for estimating a road traffic condition and method and apparatus for controlling a vehicle running characteristic|
|US5751915 *||Aug 31, 1993||May 12, 1998||Werbos; Paul J.||Elastic fuzzy logic system|
|US5821860||May 20, 1997||Oct 13, 1998||Honda Giken Kogyo Kabushiki Kaisha||Driving condition-monitoring apparatus for automotive vehicles|
|US5924085 *||May 23, 1997||Jul 13, 1999||Werbos; Paul J.||Stochastic encoder/decoder/predictor|
|US5991675||Mar 25, 1994||Nov 23, 1999||Honda Giken Kogyo Kabushiki Kaisha||Vehicle control system based on estimation of the driving skill of a vehicle operator|
|US6208981||Jan 26, 1998||Mar 27, 2001||Siemens Aktiengesellschaft||Circuit configuration for controlling a running-gear or drive system in a motor vehicle|
|US6424956 *||Mar 18, 1999||Jul 23, 2002||Paul J. Werbos||Stochastic encoder/decoder/predictor|
|US6882992 *||Sep 1, 2000||Apr 19, 2005||Paul J. Werbos||Neural networks for intelligent control|
|US7062333 *||Feb 5, 2002||Jun 13, 2006||Yamaha Hatsudoki Kabushiki Kaisha||Optimal solution search device, device for controlling controlled object by optimizing algorithm, and optimal solution search program|
|US20020128751||Jan 19, 2002||Sep 12, 2002||Johan Engstrom||System and method for real-time recognition of driving patters|
|DE4209150A1||Mar 20, 1992||Sep 24, 1992||Hitachi Ltd||Unified control system for motor vehicle - uses forecasting unit for travelling conditions and ambient coeffts. receiving signals from manual setters and sensors|
|DE4419317A1||Jun 1, 1994||Dec 8, 1994||Honda Motor Co Ltd||System for controlling a vehicle on the basis of an estimation of the driving experience of a vehicle driver|
|DE4425957C1||Jul 21, 1994||Mar 14, 1996||Daimler Benz Ag||Motor vehicle cruise control device|
|DE10010681A1||Mar 4, 2000||Sep 6, 2001||Heinz J Theuerkauf||Simulating signal from electronic sensor in motor vehicle using virtual sensor in vehicle control device, based on neural network model|
|DE19523111A1||Jun 26, 1995||Jan 2, 1997||Daimler Benz Ag||Regulation of distance between motor vehicles, between vehicle behind and vehicle in front|
|DE19527323A1||Jul 26, 1995||Jan 30, 1997||Siemens Ag||Schaltungsanordnung zum Steuern einer Einrichtung in einem Kraftfahrzeug|
|DE19720626A1||May 16, 1997||Dec 11, 1997||Honda Motor Co Ltd||Monitoring device for position of vehicle driving controls for determining deterioration of driver performance|
|DE19818239A1||Apr 23, 1998||Oct 28, 1999||Volkswagen Ag||Vorrichtung zur Einschlafwarnung eines Kraftfahrzeugführers|
|EP0521729A2||Jul 3, 1992||Jan 7, 1993||Honda Giken Kogyo Kabushiki Kaisha||Neural network and learning method therefor|
|1||German Search Report for 103 54 322.8 dated Jun. 14, 2004.|
|2||International Search Report for PCT/EP2004/010704 dated Dec. 21, 2004.|
|3||L.A. Klein: "Sensor Technologies and Data Requirements for ITS", 2001, Artech House, London, XP002309262.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8924138 *||Dec 7, 2011||Dec 30, 2014||Vnomics Corp.||System and method for measuring and reducing vehicle fuel waste|
|US9116233 *||Jul 10, 2012||Aug 25, 2015||Broadcom Corporation||Power mode control for sensors|
|US20100292894 *||Nov 18, 2010||Gm Global Technology Operations, Inc.||Method and apparatus for controlling an active vehicle subsystem|
|US20120143449 *||Dec 7, 2011||Jun 7, 2012||Vnomics Corp.||System and method for measuring and reducing vehicle fuel waste|
|US20140019044 *||Jul 10, 2012||Jan 16, 2014||Broadcom Corporation||Power Mode Control for Sensors|
|U.S. Classification||701/106, 701/115|
|International Classification||G06F19/00, G08G1/01, G05B13/02|
|Cooperative Classification||G05B13/027, G08G1/0104|
|European Classification||G08G1/01B, G05B13/02C1|
|Aug 1, 2006||AS||Assignment|
Owner name: BAYERISCHE MOTOREN WERKE AKTIENGESELLSCHAFT, GERMA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:STOLLE, UWE;SALZMANN, CHRISTIAN;REEL/FRAME:018143/0615;SIGNING DATES FROM 20060522 TO 20060710
|Nov 17, 2011||FPAY||Fee payment|
Year of fee payment: 4
|Nov 30, 2015||FPAY||Fee payment|
Year of fee payment: 8