Publication number | US5761386 A |

Publication type | Grant |

Application number | US 08/876,722 |

Publication date | Jun 2, 1998 |

Filing date | Jun 16, 1997 |

Priority date | Apr 5, 1996 |

Fee status | Paid |

Publication number | 08876722, 876722, US 5761386 A, US 5761386A, US-A-5761386, US5761386 A, US5761386A |

Inventors | Stephen Robert Lawrence, C. Lee Giles |

Original Assignee | Nec Research Institute, Inc. |

Export Citation | BiBTeX, EndNote, RefMan |

Patent Citations (13), Non-Patent Citations (68), Referenced by (67), Classifications (18), Legal Events (4) | |

External Links: USPTO, USPTO Assignment, Espacenet | |

US 5761386 A

Abstract

A method and apparatus for the prediction of time series data, specifically, the prediction of a foreign currency exchange rate. The method disclosed transforms the time series data into a difference of a series, compresses the transformed data using a log transformation, converts the compressed data into symbols, and subsequently trains one or more neural networks on the symbols such that a prediction is generated. Alternative embodiments demonstrate the conversion by a self-organizing map and training by a recurrent neural network.

Claims(5)

1. A system for predicting foreign exchange rate time series data comprising:

a means for preprocessing which receives the foreign exchange rate time series data and then transforms the foreign exchange rate data;

a means for symbolic conversion that produces a sequence of symbols from the transformed foreign exchange rate data, wherein said symbolic conversion means includes a self-organizing map neural network having a plurality of nodes, wherein each one of said nodes is an independent symbol and each one of said symbols is encoded according to a topological ordering of the nodes in the self organizing map;

a means for grammatical inference which predicts a given foreign exchange rate from the sequence of symbols produced by the self-organizing map neural network, wherein said grammatical inference means includes an Elman recurrent neural network having an input layer, an output layer and a hidden layer, each layer comprising one or more nodes, each node in the input layer being connected to each node in the hidden layer and each node in the hidden layer being connected to each node in the output layer as well as each node in the hidden layer such that said predicted foreign exchange rate is dependent upon both the sequence of symbols and a current state of said recurrent neural network;

a means for rule extraction which extracts, using an extraction method, one or more production rules from the grammatical inference means regarding the prediction of the given foreign exchange rate; and

a means for confidence estimation which estimates a confidence of the given foreign exchange rate wherein the confidence estimation means generates the confidence estimate of the prediction of the foreign exchange rate using outputs of the grammatical inference means according to y_{m} (y_{m} -y_{2m}) where y_{m} is a maximum and y_{2m} is a second maximum output of the recurrent neural network;

wherein the given foreign exchange rate predicted by the grammatical inference means, the production rules extracted by the rule extraction means and the confidence estimate of the foreign exchange rate estimated by the confidence estimation means are continuously generated and dependent upon a respective input and a current state of the recurrent neural network.

2. The prediction system according to claim 1 further comprising:

a means for rule interpretation which interprets the extracted rules.

3. The prediction system according to claim 1 wherein said foreign exchange rate data is nonstationary and said preprocessing module further comprises:

a means for differencing which reduces the nonstationarity of the foreign exchange rate data.

4. The prediction system according to claim 1 wherein said preprocessing module further comprises:

a means for compressing which produces a log compression of the foreign exchange rate data.

5. The prediction system according to claim 1 wherein said rule extraction means further comprises:

a means for quantizing which quantizes a state space of hidden nodes contained in the grammatical inference means such that a deterministic finite state automata corresponding to training data is obtained.

Description

This is a continuation of application Ser. No. 08/628,442 filed Apr. 5, 1996, now abandoned.

This invention relates generally to the field of time series data prediction, and in particular to a method and apparatus for predicting daily foreign exchange rates.

The volume of the foreign exchange market is staggering. In 1989, for example, about 650 billion dollars per day was exchanged. Of the currencies exchanged in the foreign exchange market, the most important are: the US dollar (which acts as a reference currency), the Japanese Yen, the British Pound, the German Mark, and the Swiss Franc. Given the importance and volume of foreign currency exchanged, many financial institutions are trying to predict foreign exchange rates using a variety of proprietary methods. See, for example, Jean Y. Lequarre, "Foreign Currency Dealing: A Brief Introduction", in Time Series Prediction: Forecasting the Future and Understanding the Past", A. S. Weigend and N. A. Gershenfeld, eds., Addison-Wesley, 1993.

Unfortunately, however, foreign exchange rate data exhibit very high noise, and significant non-stationarity thereby minimizing their predictability. Attempts to overcome the high noise inherent in foreign exchange rate data using an overparameterised model and/or techniques such as weight decay or weight elimination to control overfitting have been described by A. S. Weigend, B. A. Huberman, and D. E. Rumelhart in an article entitled "Predicting Sunspots and Exchange Rates with Connectionist Networks", which appeared in Nonlinear Modeling and Forecasting, SFI Studies in the Sciences of Complexity Proc., Vol. XII, Addison-Wesley, 1992, but have met with only limited success.

Neural network technology has been of great interest in recent years, and has recently been applied to the problem of predicting noisy time series data. For example, U.S. Pat. No. 5,444,819 (issued to Michiro Negishi on Aug. 22, 1995, hereinafter referred to as the '819 patent) teaches a system into which time series data are input and moving average values and their differences are generated. As taught by the patentees, a standard neural network is trained on the averages, differences, and a pattern class to predict the value of change.

A method and apparatus for retrieving signals embedded in noise and subsequently analyzing the signals was shown in U.S. Pat. No. 5,402,520 (issued to Bonnie S. Schnitta on Mar 6, 1992 and hereinafter referred to as the '520 patent). The apparatus disclosed in this patent includes an input device for receiving input signals having noise and at least one noise filter retrieves data signals embedded in the input signals.

Despite initial optimism for neural network technology, its effect to date on the prediction of noisy time series data such as foreign currency exchange data has been limited. Consequently, a continued need exists in the art for methods and apparatus which quickly, efficiently, and accurately allow prediction from noisy time series data and in particular, to predict daily foreign exchange rates.

The above problem is solved and an advance is made over the prior art in accordance with the principles of the invention wherein a prediction is made of a time series of data which oftentimes exhibits significant nonstationarity and noise.

Viewed from one aspect, the invention is directed to an apparatus for predicting time series data. The invention includes: a preprocessing module for transforming the data; a symbolic conversion module which produces a sequence of symbols from the transformed data; and a grammatical inference module which predicts a given quantity from the sequence of symbols. Enhancements to the apparatus include a confidence estimation module which estimates a confidence of the given quantity, a rule extraction module which extracts one or more rules, and a rule interpretation module which produces predictions using the extracted rules.

Viewed from another aspect, the invention is directed to a method for predicting time series data. The method disclosed transforms the time series data into a difference of a series, compresses the transformed data using a log transformation, converts the compressed data into symbols, and subsequently trains one or more neural networks on the symbols such that a prediction is generated. Alternative embodiments demonstrate the conversion by a self-organizing map and training by a recurrent neural network.

Other advantages and features will become apparent from the following drawings, detailed description, and claims.

The teachings of the present invention can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

FIG. 1 is a graph of exemplary time-series data;

FIG. 2 is a flow chart showing the processing of time-series data according to the teachings of the present invention;

FIG. 3 is an illustration of a self-organizing map;

FIG. 4 is an illustration of an Ellman neural network;

FIG. 5 is an illustration of a sample extracted automata; and

FIG. 6 is a block diagram of a system for predicting time-series data according to the teachings of the present invention.

A preferred embodiment of the invention will now be described while referring to the figures, several of which may be simultaneously referred to during the course of the following description. Before thoroughly describing the method and apparatus which is the subject of the present invention, however, a discussion of the theoretical background is helpful.

Particularly, the efficient market hypothesis (EMH) has found broad acceptance in the financial community and in its weak form, asserts that the price of an asset reflects all of the information that can be obtained from past prices of the asset, i.e., the movement of the price is unpredictable. See, for example, H. White, "Economic Prediction Using Neural Networks: The Case of IBM Daily Stock Returns," in the proceedings of the IEEE Conference on Neural Networks, pages 451-458.

The best prediction of a price is the current price when the actual price follows what is generally called a random walk. One argument in support of the EMH is that if any profit opportunities appear, they will be exploited immediately by traders, and thereby disappear. Conversely, arguments against the EMH include the fact that not all information reaches all traders simultaneously, and that traders with more sophisticated methods may be able to make better use of the available information.

Assuming the validity of the EMH, then a financial series can be modeled as the addition of a noise component at each step:

x(k+1)=x(k)+ε(k), (1)

where ε(k) is a zero mean Gaussian variable with variance σ and x(k) is the value of the time series at step k. The best estimation is:

x(k+1)=x(k) (2)

In other words, if the series is truly a random walk, then the best estimate for the next time period is equal to the current estimate. Now, if it is assumed that there is a predictable component to the series, then:

x(k+1)=x(k)+f(x(k),x(k-1), . . . ,x(k-n+1))+ε(k) (3)

may be used where ε(k) is a zero mean Gaussian variable with variance σ, and f(·) is a nonlinear function in its arguments. In this case, the best estimate is given by

x(k+1)=x(k)+f(x(k),x(k-1), . . . ,x(k-n+1)) (4)

Attempts at prediction using this model are problematic as the series often contains a trend. For example, and with reference to FIG. 1, a neural network trained on section A has no chance of generalizing to the test data in section B, because the model was not trained with data in this range. Those skilled in the art can readily appreciate that neural networks are good at interpolation, and not extrapolation.

A common solution to this is to use the model

δ(k+1)=f(δ(k),δ(k-1), . . . ,δ(k-n+1))+ν(k),(5)

where

δ(k+1)x(k+1)-x(k) (6)

and ν(k) is a zero mean Gaussian variable with variance σ. In this case, the best estimate is

δ(k+1)=f(δ(k),δ(k-1), . . . ,δ(k-n+1)).(7)

Note that the first order differences are being used instead of the raw time series.

With reference to FIG. 2, there is shown a block diagram of the system which is the subject of the present invention. Specifically, raw financial series 200 are received by differencing block 210, whose output is processed at log compression block 220 which compresses the dynamic range of the series and reduces the effect of any outlying data. Next, quantization/symbolic encoding is performed at block 230, and grammatical inference (see, for example, C. L. Giles et al, "Second-Order Recurrent Neural Networks for Grammatical Inference," 1991 IEEE INNS International Conference on Neural Networks, IEEE Press, Piscataway, N.J., 1992) at block 240 which results in change prediction output 250.

The above process starts with a set of raw time series values y(k), k=1, 2, . . . , N, and y(k) ε . These values denote the daily closing prices of a particular financial time series, however, they could be any time series. Next, the first difference of the series, y(k), is taken as follows:

δ(k)=y(k)-y(k-1), (8)

which results in δ(k), δ(k) ε , k=1, 2, . . . , N-1.

In order to compress the dynamic range of the series and reduce the effect of outliers, a log transformation of the data is used, namely:

x(k)=sign(δ(k))(log(|δ(k)|+1)),(9)

which results in x(k), k=1, 2, . . . , N-1, x(k) ε. However, for some time series this is not necessary. Other compression methods may be used for other time series.

Next, a delay embedding of this series is considered,

X(k,d_{1})=(x(k),x(k-1),x(k-2), . . . ,x(k-d_{1}+1)), (10)

where d_{1} is the delay embedding dimension and X(k,d_{1}) is a state vector.

This delay embedding forms the input to a self-organizing map (SOM). Consequently, the SOM input is the last d_{1} values of the log transformed delta time series. The output of the SOM is a topographical location of the winning node, where each node represents one symbol in the resulting grammatical inference problem.

As used in the present invention, the SOM can be represented by the equation

S(k)=g(X(k,d)), (11)

where S(k) ε 0, 1, 2, . . . n_{s} -1!, n_{s} is the number of symbols (nodes) for the SOM. By way of convenience, each node in the SOM is assigned an integer index ranging from 0 to the number of nodes minus 1.

Next, an Elman recurrent neural network is trained on the sequence of outputs from the SOM. Those skilled in the art can readily appreciate that many recurrent networks may be used. For the recurrent network:

O(k+1)=C^{T}z_{k}+c_{0}(12)

and

z_{k}=F_{n}.sbsb.h (Az_{k-1}+Bu_{k}+b), (13)

where C is a n_{h} ×n_{o} vector representing the weights from the hidden layer to the output nodes, n_{h} is the number of hidden nodes, n_{o} is the number of output nodes, c_{0} is a scalar, and z_{k}, z_{k} ε ^{n}.sbsp.h is an n_{h} ×1 vector which denotes the outputs of the hidden layer neurons.

Additionally, u_{k} is a d_{2} ×1 vector, where d_{2} is the embedding dimension used for the the input window of symbols that is presented to the SOM. As such, u_{k} may be represented by: ##EQU1## and A and B are matrices of appropriate dimensions which represent the feedback weights from the hidden nodes to the hidden nodes and the weights from the input layer to the hidden layer respectively. Similarly, F_{n}.sbsb.h is a n_{h} ×1 vector containing the sigmoid functions, b is a n_{h} ×1 vector that denotes the bias of each hidden layer neuron, O(k) is a n_{o} ×1 vector containing the outputs of the network, and n_{o} is 2.

Consequently,

O(k+1)=F_{1}(δ(k),δ(k-1),δ(k-2),δ(k-3),δ(k-4))(15)

or

O(k+1)=F_{2}(y(k),y(k-1),y(k-2),y(k-3),y(k-4),y(k-5)). (16)

As an example of how the recurrent neural network could be trained, the net could have two outputs whereby the first output is trained to predict the probability of a positive change in the series, and the second output is trained to predict the probability of a negative change.

The self-organizing map (SOM), described by Teuvo Kohonen in an article entitled "The Self-Organizing Map," which appeared in Proceedings of the IEEE, vol. 78, pp. 1464-1480, in 1980, is an unsupervised learning process which learns a distribution of a set of patterns without any class information. In particular, a pattern is projected from a possibly high dimensional input space to a position in the map, a low dimensional display space . Oftentimes, the display space is divided into a grid having a number of intersections wherein each intersection is represented in the network by a neuron. Information is encoded as the location of an activated neuron. The SOM, unlike most classification or clustering techniques, preserves the topological ordering of classes in the input space in the resulting display space . Similarity as measured using a metric in the input space , is preserved in the display space .

In operation, the SOM defines a mapping from an input space ^{n} onto a topologically ordered set of nodes, usually in a lower dimensional space . An example of such a two-dimensional SOM is shown in FIG. 3. A reference vector, m_{i} ≡ μ_{i1}, μ_{i2}, . . . , μ_{in} !^{T} ε^{n}, is assigned to each node in the SOM.

During training, each input x is compared to all the m_{i} obtaining the location of the closest match according to a particular metric. The input point is mapped to this location in the SOM.

Nodes in the SOM are updated according to the metric and their position in relation to the winning node. For example, nodes may be updated according to:

m_{i}(t+1)=m_{i}(t)+h_{ci}(t) x(t)-m_{i}(t)!, (17)

where t is the time during learning and h_{ci} (t) is the neighborhood function, a smoothing kernel which is maximum at m_{c}. Usually, h_{ci} (t)=h(∥r_{c} -r_{i} ∥,t), where r_{c} and r_{i} represent the locations of nodes in the SOM output space . r_{c} is the node with the closest weight vector to the input sample and r_{i} ranges over all nodes. h_{ci} (t) approaches 0 as ∥r_{c} -r_{i} ∥ increases and also as t approaches ∞. A widely applied neighborhood function is: ##EQU2## where α(t) is a scalar valued learning rate and σ(t) defines the width of the kernel. Those skilled in the art can readily appreciate that a SOM can be viewed as a non-linear projection of the probability density, p(x), of the input patterns x.

As described, the nodes in the display space encode the information contained in the input space ^{n}. Since there are M nodes in , this implies that the input pattern vectors x ε ^{n} are transformed to a set of M symbols, while preserving their original topological ordering in ^{n}. Thus, if the original input patterns are highly noisy, the quantization into the set of M symbols while preserving the original topological ordering can be understood as a form of filtering. The amount of filtering is controlled by M. If M is large, this implies there is little reduction in the noise content of the resulting symbols. On the other hand, if M is small, this implies that there is a "heavy" filtering effect, resulting in only a small number of symbols.

With reference now to FIG. 4, one example of a recurrent neural network which could be used for grammatical inference in the present invention is the Elman neural network. As is shown in FIG. 4, the Elman neural network has feedback from each of the hidden nodes to all of the hidden nodes. The set of M symbols from the output of the SOM are discretely encoded in a linear fashion into a single input for the recurrent neural network (e.g. if M=3, the single input is either -1, 0, or 1). Those skilled in the art will readily recognize that linear encoding is justified by the topographical order of the symbols. In order to facilitate the training of the recurrent network, an input window of is used, i.e. a number of the most recent symbols are presented to separate input neurons of the recurrent neural network.

According to the teachings of the present invention, non-stationarity of the signal is advantageously overcome by building models based on a short time period only. Specifically, models are trained using a certain number of data points. Separate models may be trained for prediction at various times.

The extraction of symbolic knowledge greatly aids the interpretability of the solution. The ordered triple of a discrete Markov process ({state; input→next-state}) can be extracted from a recurrent neural network and used to form an equivalent deterministic finite state automata (DFA). While this extraction can be done by clustering the activation values of the recurrent state neurons, automata extracted using this process can only recognize regular grammars. Note that as used herein, a regular grammar G is a 4-tuple G={S, N, T, P} where S is the start symbol, N and T are non-terminal and terminal symbols, respectively, and P represents productions of the form A→a or A→aB where A, B ε N and αε T.

A method which can be used for automata extraction in the present invention is the same as described C. L. Giles, et. al., in an article entitled "Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks", which appeared in Neural Computation, 4(3), pp. 393-495, 1992. However, other extraction methods can be used and are known in the art.

A sample extracted automata for the British Pound is shown in FIG. 5. The DFA has been minimized using minimization techniques known in the art.

Summarizing the DFA extraction method, the state z_{k} is first divided into q cells, i.e., z_{ki}, i=1, 2, . . . , n_{h} is divided into q segments. Hence the total number of quantized states is q^{n}.sbsp.h. By running the inputs u_{k}, k=1, 2, . . . , N through the network, the corresponding set of activations z_{k}, k=1, 2, . . . , N can be obtained. These activations are often clustered in the activation space and quantize to the same value.

For a given input sequence, the path through the DFA may be followed. For the example given in FIG. 5, ##EQU3## and the output is given by ##EQU4## where S_{i}, i=0, 1, 2 are the output symbols from the SOM. (In FIG. 12 S_{1} is the dotted line, S_{2} is the gray line, and S_{3}, is the solid line). n_{i}, i=1, 2, . . . , 9 are the nodes in the minimized DFA.

The DFA nodes, n_{i}, i=1, 2, . . . , 9 are not related to any physical entities. Nevertheless, the extracted DFA may be linked back to the original time series as follows:

1. Transforming the SOM input data sequence, X(k, d_{1}), into the SOM output symbols, giving, →; and

2. Following the path of these symbols through the DFA and determining if the current state corresponds to a positive or negative change.

Hence, for a given sequence of financial data, the transitions in the DFA may be determined.

A block diagram of one embodiment is shown in FIG. 6. Specifically, time series data 600 is received by pre-processing module 610, which transforms the input data. In a preferred embodiment, the pre-processing module may contain differencing module 611 which reduces any nonstationarity of the data, and compression module 613, which generates a log compression of the data.

Symbolic conversion module 620 which preferrably is performed with self-organizing map 621, produces a sequence of symbols from the transformed data. Grammatical inference module 630, having one or more recurrent neural networks 631 or employing other grammatical inference methods, predicts a given quantity from the sequence of symbols.

Alternatively, the prediction system may include confidence estimation module 650 which estimates a confidence of the given quantity using any of a variety of metrics.

Rule interpretation module 640, advantageously produces predictions using the extracted rules.

It should now be quite evident to those skilled in the art, that while the invention was shown and described in detail in the context of a preferred embodiment, and with various modifications thereto, a wide variety of other modifications can be made without departing from the scope of our inventive teachings.

Patent Citations

Cited Patent | Filing date | Publication date | Applicant | Title |
---|---|---|---|---|

US5109475 * | Aug 14, 1989 | Apr 28, 1992 | Hitachi, Ltd. | Method and a system for selection of time series data |

US5121467 * | Aug 3, 1990 | Jun 9, 1992 | E.I. Du Pont De Nemours & Co., Inc. | Neural network/expert system process control system and method |

US5140523 * | Dec 14, 1990 | Aug 18, 1992 | Ktaadn, Inc. | Neural network for predicting lightning |

US5247584 * | Jan 9, 1992 | Sep 21, 1993 | Bodenseewerk Geratetechnik Gmbh | Signal processing unit for classifying objects on the basis of signals from sensors |

US5303330 * | Dec 17, 1992 | Apr 12, 1994 | Bell Communications Research, Inc. | Hybrid multi-layer neural networks |

US5398300 * | Dec 30, 1991 | Mar 14, 1995 | Hnc, Inc. | Neural network having expert system functionality |

US5402520 * | Mar 6, 1992 | Mar 28, 1995 | Schnitta; Bonnie S. | Neural network method and apparatus for retrieving signals embedded in noise and analyzing the retrieved signals |

US5406481 * | Jun 28, 1994 | Apr 11, 1995 | Nippon Telegraph And Telephone Corporation | Rainfall, snowfall forecast apparatus and method |

US5444819 * | Apr 27, 1993 | Aug 22, 1995 | Mitsubishi Denki Kabushiki Kaisha | Economic phenomenon predicting and analyzing system using neural network |

US5455890 * | Sep 30, 1993 | Oct 3, 1995 | Motorola, Inc. | Method for structuring an expert system utilizing one or more neural networks |

US5461699 * | Oct 25, 1993 | Oct 24, 1995 | International Business Machines Corporation | Forecasting using a neural network and a statistical forecast |

US5479573 * | Jan 25, 1993 | Dec 26, 1995 | Pavilion Technologies, Inc. | Predictive network with learned preprocessing parameters |

US5490062 * | May 11, 1994 | Feb 6, 1996 | The Regents Of The University Of California | Real-time neural network earthquake profile predictor |

Non-Patent Citations

Reference | ||
---|---|---|

1 | A.S. Weigend, B.A. Huberman, and D.E. Rumelhart, "Predicting Sunspots and Exchange Rates with Connectionist Networks", Nonlinear Modeling and Forecasting, SFI Studies in the Sciences of Complexity Proc., vol. XII, Addison-Wesley, 1992. | |

2 | * | A.S. Weigend, B.A. Huberman, and D.E. Rumelhart, Predicting Sunspots and Exchange Rates with Connectionist Networks , Nonlinear Modeling and Forecasting, SFI Studies in the Sciences of Complexity Proc., vol. XII, Addison Wesley, 1992. |

3 | Aityan, "Stability of a threshold recall control self organizing recurrent neural network," IEEE 1992 Int'l conf. on neural networks v3, Dec. 1992. | |

4 | * | Aityan, Stability of a threshold recall control self organizing recurrent neural network, IEEE 1992 Int l conf. on neural networks v3, Dec. 1992. |

5 | Azoff, "Neural Network Time Series Forecasting of Financial Markets," pp. 21-90, John Wiley & Sons, Dec. 1994. | |

6 | * | Azoff, Neural Network Time Series Forecasting of Financial Markets, pp. 21 90, John Wiley & Sons, Dec. 1994. |

7 | Barr et al., "Using Neural Networks to Manage Investments," AI Expert, vol. 9 pp. 16-21, Feb. 1994. | |

8 | * | Barr et al., Using Neural Networks to Manage Investments, AI Expert, vol. 9 pp. 16 21, Feb. 1994. |

9 | C.L. Giles, et al, "Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks", Neural Computation, 4(3), pp. 393-495, 1992. | |

10 | * | C.L. Giles, et al, Learning and Extracting Finite State Automata with Second Order Recurrent Neural Networks , Neural Computation, 4(3), pp. 393 495, 1992. |

11 | C.L. Giles, et al., "Second-Order Recurrent Neural Networks for Grammatical Inference", 1991 IEEE INNS International Conference on Neural Networks IEEE Press, Piscataway, NJ 1992. | |

12 | * | C.L. Giles, et al., Second Order Recurrent Neural Networks for Grammatical Inference , 1991 IEEE INNS International Conference on Neural Networks IEEE Press, Piscataway, NJ 1992. |

13 | Carson, "Insulin Dosage Adjestment Using Time Series Analysis and Rule Based Reasoning," Proceedings of the 12th Triennial World Congress of the International Federation of Automatic Control, pp. 161-164, Jul. 1993. | |

14 | * | Carson, Insulin Dosage Adjestment Using Time Series Analysis and Rule Based Reasoning, Proceedings of the 12th Triennial World Congress of the International Federation of Automatic Control, pp. 161 164, Jul. 1993. |

15 | Chen, "Constructive learning of recurrent neural networks," Neural networks, IEEE 1993 intl conference, Dec. 1993. | |

16 | * | Chen, Constructive learning of recurrent neural networks, Neural networks, IEEE 1993 intl conference, Dec. 1993. |

17 | Collopy et al., "Rule Based Forecasting: Development and Validation of an Expert Systems Approach to Combininb Time Series Extrapolations," vol. 38 issue 10 pp. 1394-1414, Oct. 1992. | |

18 | * | Collopy et al., Rule Based Forecasting: Development and Validation of an Expert Systems Approach to Combininb Time Series Extrapolations, vol. 38 issue 10 pp. 1394 1414, Oct. 1992. |

19 | Fletcher, "Neural networks for learning grammars," IEE colloq. (1993) n92: grammatical inference, Dec. 1993. | |

20 | * | Fletcher, Neural networks for learning grammars, IEE colloq. (1993) n92: grammatical inference, Dec. 1993. |

21 | Giles, "Constructive learningn of recurrent neural networks: limitations of recurrent . . . ," IEEE transactions on neural networks, v6 n4, Jul. 1995. | |

22 | Giles, "Inserting rules into recurrent neural networks," IEE Neural networks for signal processing II, Dec. 1992. | |

23 | Giles, "Learning, extracting, inserting and verifying grammatical information in recurrent neural networks," IEE colloq.(1993) No. 92: grammatical inference, Dec. 1993. | |

24 | Giles, "Second order recurrent neural networks for grammatical inference," 1991 IEE intl conf (Seattle), Dec. 1991. | |

25 | * | Giles, Constructive learningn of recurrent neural networks: limitations of recurrent . . . , IEEE transactions on neural networks, v6 n4, Jul. 1995. |

26 | * | Giles, Inserting rules into recurrent neural networks, IEE Neural networks for signal processing II, Dec. 1992. |

27 | * | Giles, Learning, extracting, inserting and verifying grammatical information in recurrent neural networks, IEE colloq.(1993) No. 92: grammatical inference, Dec. 1993. |

28 | * | Giles, Second order recurrent neural networks for grammatical inference, 1991 IEE intl conf (Seattle), Dec. 1991. |

29 | Goudreau, "First order versus second order single layer recurrent neural networks," IEEE transactions on neural networks, v5 n3, May 1994. | |

30 | * | Goudreau, First order versus second order single layer recurrent neural networks, IEEE transactions on neural networks, v5 n3, May 1994. |

31 | H. White, "Economic Prediction Using Neural Networks: The Case of IBM Daily Stock Returns", Proceedings of the IEEE Conference on Neural Networks, pp. 451-458. | |

32 | * | H. White, Economic Prediction Using Neural Networks: The Case of IBM Daily Stock Returns , Proceedings of the IEEE Conference on Neural Networks, pp. 451 458. |

33 | * | Hoda S Abdel Aty Zohdy, Neural netwokrs for pattern discovery and optimization in signal processing and applications, Electrical and computer engineering 1995 Canadian conf, Dec. 1995. |

34 | Hoda S Abdel-Aty-Zohdy, "Neural netwokrs for pattern discovery and optimization in signal processing and applications," Electrical and computer engineering 1995 Canadian conf, Dec. 1995. | |

35 | Jean Y. Lequarre, "Foreign Currency Dealing: A Brief Introduction", in Time Series Prediction: Forecasting the Future and Understanding the Past, A.S. Weigend and N.A. Gershenfeld, eds., Addison-Wesley, 1993. | |

36 | * | Jean Y. Lequarre, Foreign Currency Dealing: A Brief Introduction , in Time Series Prediction: Forecasting the Future and Understanding the Past, A.S. Weigend and N.A. Gershenfeld, eds., Addison Wesley, 1993. |

37 | Lawrence, "Can recurrent neural networks learn natural language grammars?," IEE 1996 intl conf on neural network, Feb. 1996. | |

38 | * | Lawrence, Can recurrent neural networks learn natural language grammars , IEE 1996 intl conf on neural network, Feb. 1996. |

39 | * | Liszka Hackzell, Categorization of Fetal Heart Rate Patterns Using Neural Networks, IEEE Computers in Cardiology, 1994, pp. 97 100, Jul. 1994. |

40 | Liszka-Hackzell, "Categorization of Fetal Heart Rate Patterns Using Neural Networks," IEEE Computers in Cardiology, 1994, pp. 97-100, Jul. 1994. | |

41 | McDonnell, "Evolving Recurrent Perceptrons for Time-Series Modeling," IEEE Transactions on Neural Networks, vol. 5 issue 1, pp. 24-38, Jan. 1994. | |

42 | * | McDonnell, Evolving Recurrent Perceptrons for Time Series Modeling, IEEE Transactions on Neural Networks, vol. 5 issue 1, pp. 24 38, Jan. 1994. |

43 | Moon, "Self organized recurrent neural network for classifying spatio temporal data," IEEE Proceedings of the intl conf. on signal processing applications and technology, Dec. 1994. | |

44 | * | Moon, Self organized recurrent neural network for classifying spatio temporal data, IEEE Proceedings of the intl conf. on signal processing applications and technology, Dec. 1994. |

45 | Nie, "A Fuzzy Neural Approach to Time Series Prediction," IEEE Neural Networks, 1994 International Conference, pp. 3164-3169, Dec. 1994. | |

46 | * | Nie, A Fuzzy Neural Approach to Time Series Prediction, IEEE Neural Networks, 1994 International Conference, pp. 3164 3169, Dec. 1994. |

47 | Rao et al., "A Recurrent Neural Network for Nonlinear Time Series Prediction-A Comparative Study," IEEE Neural Networks for Signal Processing II, pp. 531-539, Dec. 1992. | |

48 | * | Rao et al., A Recurrent Neural Network for Nonlinear Time Series Prediction A Comparative Study, IEEE Neural Networks for Signal Processing II, pp. 531 539, Dec. 1992. |

49 | Refenes et al., "Managing Exchange Rate Prediction Strategies with Neural Networks," Techniques and Applications of Neural Networks, pp. 109-116, Dec. 1993. | |

50 | * | Refenes et al., Managing Exchange Rate Prediction Strategies with Neural Networks, Techniques and Applications of Neural Networks, pp. 109 116, Dec. 1993. |

51 | Schmidhuber, "Learning complex, extended sequences using the principle of history compression," Neural computation v4 n2, Dec. 1992. | |

52 | * | Schmidhuber, Learning complex, extended sequences using the principle of history compression, Neural computation v4 n2, Dec. 1992. |

53 | Schotles, "Unsupervised context learning in natural language processing," Neural Networks, 1991 IEEE intl conf (Seattle), Dec. 1991. | |

54 | * | Schotles, Unsupervised context learning in natural language processing, Neural Networks, 1991 IEEE intl conf (Seattle), Dec. 1991. |

55 | Stanley et al., "Predicting the Canadian Spot Exchange Rate with Neural Networks," Proceedings of the IEEE/IAFE Computational Intelligence for Financial Engineering, pp. 108-112, Apr. 1995. | |

56 | * | Stanley et al., Predicting the Canadian Spot Exchange Rate with Neural Networks, Proceedings of the IEEE/IAFE Computational Intelligence for Financial Engineering, pp. 108 112, Apr. 1995. |

57 | Teuvo Kohonen, "The Self-Organizing Map", Proceedings of the IEEE, vol.\78,pp.\1464-1481, in 1980. | |

58 | * | Teuvo Kohonen, The Self Organizing Map , Proceedings of the IEEE, vol.\78,pp.\1464 1481, in 1980. |

59 | Tino, "Learning and extracting initial mealy automata with a modular neural network model," Neural computation v7 n4, Dec. 1995. | |

60 | * | Tino, Learning and extracting initial mealy automata with a modular neural network model, Neural computation v7 n4, Dec. 1995. |

61 | Wang, "Anticipation based temporal pattern generation," IEEE transactions on systems, man and cybernetics, v25 n4, Apr. 1995. | |

62 | * | Wang, Anticipation based temporal pattern generation, IEEE transactions on systems, man and cybernetics, v25 n4, Apr. 1995. |

63 | Wurtz et al., "A Neural Decision Support System for Predicting Currency Exchange Rates," Proceedings of the Second International Conference on Artificial Applications on Wall Street, pp. 205-209, Dec. 1993. | |

64 | * | Wurtz et al., A Neural Decision Support System for Predicting Currency Exchange Rates, Proceedings of the Second International Conference on Artificial Applications on Wall Street, pp. 205 209, Dec. 1993. |

65 | Zeng, "Discrete recurrent neural networks for grammatical inference," IEEE transactions on neural networks, v5, n2, Mar. 1994. | |

66 | * | Zeng, Discrete recurrent neural networks for grammatical inference, IEEE transactions on neural networks, v5, n2, Mar. 1994. |

67 | Zhang, "A high order temporal neural network for word recognition," ICASSP '95: Acoustics, Speech & Signal processing IEEE v 5, Dec. 1995. | |

68 | * | Zhang, A high order temporal neural network for word recognition, ICASSP 95: Acoustics, Speech & Signal processing IEEE v 5, Dec. 1995. |

Referenced by

Citing Patent | Filing date | Publication date | Applicant | Title |
---|---|---|---|---|

US6269345 * | Dec 2, 1997 | Jul 31, 2001 | Jacques Riboud | Transfer system and method for transferring amounts in different local currencies between a plurality of local banking organization |

US6321216 * | Dec 2, 1997 | Nov 20, 2001 | Abb Patent Gmbh | Method for analysis and display of transient process events using Kohonen map |

US6493691 | Jul 1, 1999 | Dec 10, 2002 | Siemens Ag | Assembly of interconnected computing elements, method for computer-assisted determination of a dynamics which is the base of a dynamic process, and method for computer-assisted training of an assembly of interconnected elements |

US6532449 * | Sep 14, 1998 | Mar 11, 2003 | Ben Goertzel | Method of numerical times series prediction based on non-numerical time series |

US6594622 * | Nov 29, 2000 | Jul 15, 2003 | International Business Machines Corporation | System and method for extracting symbols from numeric time series for forecasting extreme events |

US6728691 | Mar 1, 2000 | Apr 27, 2004 | Siemens Aktiengesellschaft | System and method for training and using interconnected computation elements to determine a dynamic response on which a dynamic process is based |

US6735580 * | Aug 25, 2000 | May 11, 2004 | Westport Financial Llc | Artificial neural network based universal time series |

US6889165 | Jul 2, 2002 | May 3, 2005 | Battelle Memorial Institute | Application specific intelligent microsensors |

US6915283 * | Jul 2, 2001 | Jul 5, 2005 | Sony Corporation | Information processing apparatus and method, and recording medium |

US6941202 | Aug 19, 2003 | Sep 6, 2005 | Battelle Memorial Institute | Diagnostics/prognostics using wireless links |

US7072875 | Mar 25, 2005 | Jul 4, 2006 | Sony Corporation | Information processing apparatus and method, and recording medium |

US7082421 | Mar 25, 2005 | Jul 25, 2006 | Sony Corporation | Information processing apparatus and method, and recording medium |

US7089219 | Mar 25, 2005 | Aug 8, 2006 | Sony Corporation | Information processing apparatus and method, and recording medium |

US7174340 * | Aug 17, 2000 | Feb 6, 2007 | Oracle International Corporation | Interval-based adjustment data includes computing an adjustment value from the data for a pending adjustment in response to retrieval of an adjusted data value from a database |

US7194434 | Jun 14, 2001 | Mar 20, 2007 | Sergio Piccioli | Method for predictive determination of financial investment performance |

US7337142 * | Oct 27, 1999 | Feb 26, 2008 | Sun Microsystems, Inc. | Multiple exchange rate tracking in a financial transaction manager |

US7529703 * | Nov 18, 2004 | May 5, 2009 | Citigroup Global Markets, Inc. | Method and system for artificial neural networks to predict price movements in the financial markets |

US7536343 | Nov 26, 2004 | May 19, 2009 | Fx Alliance, Llc | Protocol-independent asset trading system and methods |

US7540408 | Jun 22, 2006 | Jun 2, 2009 | Hip Consult Inc. | Apparatus and method for facilitating money or value transfer |

US7634438 | Dec 23, 2005 | Dec 15, 2009 | Fx Alliance, Llc | Dynamic account mapping system for computerized asset trading |

US7672920 * | Jan 30, 2007 | Mar 2, 2010 | Sony Corporation | Apparatus and method for embedding recurrent neural networks into the nodes of a self-organizing map |

US7680718 | Jun 29, 2006 | Mar 16, 2010 | Itg Software Solutions, Inc. | System and method for generating real-time indicators in a trading list or portfolio |

US7761363 | Oct 7, 2004 | Jul 20, 2010 | Fx Alliance, Llc | Internal trade requirement order management and execution system |

US7783585 * | Apr 5, 2007 | Aug 24, 2010 | Sony Corporation | Data processing device, data processing method, and program |

US7865415 | Mar 4, 2004 | Jan 4, 2011 | Morgan Stanley | Intelligent simulation analysis method and system |

US7979330 | Dec 9, 2005 | Jul 12, 2011 | Andrew Cumming | Conditional rate modelling |

US8001033 | Mar 15, 2010 | Aug 16, 2011 | Itg Software Solutions, Inc. | System and method for generating real-time indicators in a trading list or portfolio |

US8019665 | Mar 24, 2003 | Sep 13, 2011 | Bloomberg L.P. | Variable pricing for and conditional availability of proposals for trading of financial interests |

US8458059 | Dec 13, 2000 | Jun 4, 2013 | Capguard.Com, Inc. | Apparatus and method for assessing market conditions |

US8521637 | Oct 8, 2009 | Aug 27, 2013 | Itg Software Solutions, Inc. | Systems, methods, and computer program products for providing real time analytic widgets in a financial trading system |

US20020013756 * | Jun 14, 2001 | Jan 31, 2002 | Sergio Piccioli | Method for predictive determination of financial investment performance |

US20020018127 * | Jul 2, 2001 | Feb 14, 2002 | Jun Tani | Information processing apparatus and method, and recording medium |

US20020073004 * | Dec 13, 2000 | Jun 13, 2002 | Capguard.Com, Inc. | Apparatus and method for assessing market conditions |

US20040193530 * | Aug 19, 2003 | Sep 30, 2004 | Andrew Hausman | Complementary trading of interests |

US20040213448 * | Apr 28, 2003 | Oct 28, 2004 | Asn Technology Corp. | Apparatus for recognizing counterfeit currency and method thereof |

US20040236667 * | Mar 4, 2004 | Nov 25, 2004 | Peter Cotton | Intelligent simulation analysis method and system |

US20050114257 * | Oct 7, 2004 | May 26, 2005 | Neill Penney | Internal trade requirement order management and execution system |

US20050114258 * | Oct 7, 2004 | May 26, 2005 | Neill Penney | Fix-enabled order management method and apparatus |

US20050119982 * | Jan 21, 2003 | Jun 2, 2005 | Masato Ito | Information processing apparatus and method |

US20050131790 * | Nov 18, 2004 | Jun 16, 2005 | Benzschawel Terry L. | Method and system for artificial neural networks to predict price movements in the financial markets |

US20050137960 * | Nov 26, 2004 | Jun 23, 2005 | Brann John E.T. | Protocol-independent asset trading system and methods |

US20050137961 * | Nov 26, 2004 | Jun 23, 2005 | Brann John E.T. | Latency-aware asset trading system |

US20050137962 * | Nov 26, 2004 | Jun 23, 2005 | Neill Penney | Quick-filling customer asset trading system |

US20050197983 * | Mar 25, 2005 | Sep 8, 2005 | Jun Tani | Information processing apparatus and method, and recording medium |

US20050197984 * | Mar 25, 2005 | Sep 8, 2005 | Jun Tani | Information processing apparatus and method, and recording medium |

US20050197985 * | Mar 25, 2005 | Sep 8, 2005 | Jun Tani | Information processing apparatus and method, and recording medium |

US20060015439 * | Jun 20, 2005 | Jan 19, 2006 | Brann John E T | Shareable quote streams |

US20060015440 * | Jun 20, 2005 | Jan 19, 2006 | Neill Penney | Dynamic liquidity management system |

US20060095292 * | Dec 9, 2005 | May 4, 2006 | Andrew Cumming | Conditional rate modelling |

US20060161497 * | Dec 23, 2005 | Jul 20, 2006 | Neill Penney | Dynamic account mapping system for computerized asset trading |

DE102007031643A1 * | Jul 6, 2007 | Jan 8, 2009 | Dirnstorfer, Stefan, Dr. | Verfahren zur Ableitung eines Zustandsraumes |

EP1627275A2 * | Mar 5, 2004 | Feb 22, 2006 | Morgan Stanley | Intelligent simulation analysis method and system |

WO2000008599A2 * | Jul 1, 1999 | Feb 17, 2000 | Neuneier Ralf | Assembly of interconnected computing elements, method for computer-assisted determination of a dynamics which is the base of a dynamic process, and method for computer-assisted training of an assembly of interconnected elements |

WO2000055809A2 * | Mar 1, 2000 | Sep 21, 2000 | Neuneier Ralf | Assembly of interconnected computing elements, method for computer-assisted determination of a dynamic which is the base of a dynamic process, and method for computer-assisted training of an assembly of interconnected elements |

WO2000062250A2 * | Apr 3, 2000 | Oct 19, 2000 | Neuneier Ralf | Assembly of interconnected computing elements, method for computer-assisted determination of a dynamic which is the base of a dynamic process, and method for computer-assisted training of an assembly of interconnected elements |

WO2001009698A2 * | Aug 1, 2000 | Feb 8, 2001 | Tradeworx Inc | System, method, and article of manufacture for estimating a probability with which a limit order will be filled |

WO2001009699A2 * | Aug 1, 2000 | Feb 8, 2001 | Tradeworx Inc | System, method, and article of manufacture for estimating a price of a limit order |

WO2001009700A2 * | Aug 1, 2000 | Feb 8, 2001 | Tradeworx Inc | System, method, and article of manufacture for estimating a time |

WO2001015079A1 * | Aug 25, 2000 | Mar 1, 2001 | Li Bin | An artificial neural network based universal time series |

WO2001057648A2 * | Jan 9, 2001 | Aug 9, 2001 | Neuneier Ralf | Configuration of interconnected arithmetic elements, and method for the computer-aided determination of a second state of a system in a first state space from a first state of the system in the first state space |

WO2001085018A2 * | May 7, 2001 | Nov 15, 2001 | Gustavo Deco | Method and device for the classification of serial values, computer-readable storage medium and computer programme element |

WO2002069176A1 * | Feb 21, 2001 | Sep 6, 2002 | Brett A Boyle | Offshootfund investment process |

WO2002089054A1 * | Apr 29, 2002 | Nov 7, 2002 | Managed Quantitative Advisors | System and method for valuation of companies |

WO2003083695A1 * | Mar 11, 2003 | Oct 9, 2003 | Roger Dingledine | Support vector machines for prediction and classification in supply chain management and other applications |

WO2004109566A1 * | Jun 10, 2004 | Dec 16, 2004 | Andrew Cumming | Conditional rate modelling |

WO2005050396A2 * | Nov 18, 2004 | Jun 2, 2005 | Citigroup Global Markets Inc | Method and system for artificial neural networks to predict price movements in the financial markets |

WO2006061320A2 * | Nov 22, 2005 | Jun 15, 2006 | Siemens Ag | Method, arrangement, computer program with program code means, and computer program product for determining a future state of a dynamic system |

Classifications

U.S. Classification | 706/20, 706/925, 705/35, 706/21, 706/30 |

International Classification | G06Q40/00, G06N3/04, G06N3/00, G06F15/18 |

Cooperative Classification | G06N3/049, G06Q40/00, G06K9/00523, G06Q40/04, Y10S706/925 |

European Classification | G06Q40/04, G06Q40/00, G06N3/04T, G06K9/00M2 |

Legal Events

Date | Code | Event | Description |
---|---|---|---|

Jan 4, 1999 | AS | Assignment | Owner name: NEC CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NEC RESEARCH INSTITUTE, INC.;REEL/FRAME:009678/0592 Effective date: 19981230 |

Nov 8, 2001 | FPAY | Fee payment | Year of fee payment: 4 |

Nov 4, 2005 | FPAY | Fee payment | Year of fee payment: 8 |

Nov 4, 2009 | FPAY | Fee payment | Year of fee payment: 12 |

Rotate