WO1997027525A2 - Non-algorithmically implemented artificial neural networks and components thereof - Google Patents
Non-algorithmically implemented artificial neural networks and components thereof Download PDFInfo
- Publication number
- WO1997027525A2 WO1997027525A2 PCT/US1997/000886 US9700886W WO9727525A2 WO 1997027525 A2 WO1997027525 A2 WO 1997027525A2 US 9700886 W US9700886 W US 9700886W WO 9727525 A2 WO9727525 A2 WO 9727525A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- neural network
- training
- data
- ofthe
- spreadsheet
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
- G06N3/105—Shells for specifying net layout
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
Definitions
- This invention relates generally to artificial neural networks and more particularly, to artificial neural networks implemented in a non-algorithmic fashion in a data space, such as a spreadsheet, so as to facilitate cascading of such artificial neural networks and so as to facilitate artificial neural networks capable of operating within the data space, including networks which move through the data space and self- train on data therewithin.
- a data space such as a spreadsheet
- the creativity machine paradigm involves progressively purturbing a first neural network having a predetermined knowledge domain such that the perturbed network continuously outputs a stream of concepts, and monitoring the outputs or stream of concepts with a second neural network which is trained to identify only useful concepts.
- the perturbations may be achieved by different means, including the introduction of noise to the network, or degradation ofthe network.
- the present application provides an excellent system for constructing such creativity machines, and further builds upon the creativity machine invention to achieve self training neural networks.
- ANNs new artificial neural networks
- ANNs new artificial neural networks
- these techniques require the skills of a neural network specialist who may spend many hours developing the training and/or implementation software for such algorithms.
- the new training data when new training data is obtained, the new training data must be manually appended to the preexisting set of training data and network training must be reinitiated, requiring additional man hours
- the generalization capacity ofthe network may be lowered
- a principal object ofthe present invention is to provide a user friendly system of implementing or simulating neural networks in which movement of such networks and cascading of such networks is facilitated
- Another object ofthe present invention is to provide self training artificial neural networks.
- a further object ofthe present invention is to provide artificial neural networks capable of analyzing data within a data space
- Yet another object ofthe present invention is to provide artificial neural networks which are mobile within a data space.
- Still another object ofthe present invention is to provide artificial neural networks which can be easily duplicated within a data space and which can be easily interconnected to facilitate the construction of more complex artificial neural network systems.
- neural network object includes artificial neural networks or combinations of artificial neural networks implemented within such a data space and having an associated set of properties and methods. These properties and methods may be inco ⁇ orated within a knowledge domain of each artificial neural network and may also be inco ⁇ orated in programs associated with the artificial neural networks.
- the data space or spreadsheet includes a plurality of cells and the spreadsheet application allows for association or interrelating of such cells through relative cell referencing. While use ofthe spreadsheet application Microsoft Excel is suggested herein, it is understood that other spreadsheet applications could be utilized, and it if further understood that new applications could be engineered for the pu ⁇ ose of creating a data space suitable for construction and operation of neural network objects as described herein. Moreover, while the various neural network objects described below may refer to programs being associated therewith, it is understood that in a data space where self referencing is permissible, such programs could be eliminated.
- the state of any given neuron may be evaluated by relative cell referencing and resident spreadsheet functions. Unlike traditional algorithmic network simulation, all neuron activations are simultaneously visible and randomly accessible within the data space simulation. More like a network of virtual, analog devices, this simulation may be considered quasi-parallel, with all neurons updated with each wave of data space calculation or renewal, where spreadsheet renewal is asynchronous with the feed forward algorithm.
- Neural network objects are mobile within the data space as provided by the spreadsheet application which typically includes resident commands for cutting and pasting groups of cells. Accordingly, movement of neural network objects is achieved by simultaneously cutting the information within the cell group or cell array comprising the neural network object from one location within the data space and pasting the same information to another location within the data space. Such movement may be accomplished manually or through programs associated with the neural network objects. Alternatively, neural network objects can be replicated using a copy command and moved elsewhere within the data space. Such neural network objects are advantageously implemented without requiring any underlying software based algorithm and are therefore extremely versatile and user friendly. Moreover, neural network objects are easily portable such as by saving or storing, on a computer readable storage medium such as a floppy disk, information operable to effect such neural network objects..
- neural network objects can be easily cascaded such that the outputs from one neural network object are applied as inputs to another neural network object.
- the compound or cascaded neural networks which result are transparent in operation and easily accessible for modification and repair. Accordingly, recurrences and all manner of neural network paradigms, including IAC, Boltzmann Machine, Harmonium, Hopfield nets, and self-organizing maps, may be readily implemented.
- neural network objects can be cascaded so as to simulate interconnected processes or hardware devices, wherein each neural network object is trained within a knowledge domain of a particular process or hardware device.
- this specification provides several examples of other neural network objects in order to demonstrate both their versatility and utility.
- One advantageous neural network object provides for the training of an artificial neural network. This self training artificial neural network object
- STANNO is a simple alternative to Adaptive Resonance Technology, disclosed in Ca ⁇ enter et al U.S. Patent No. 5,214,715, wherein complex algorithms are utilized to allow neural networks to flexibly adapt to new, emerging information.
- the STANNO requires no such complex algorithms.
- training an artificial neural network requires a set of training data, including multiple input vectors and associated output vectors, and includes various techniques such as backpropagation, involving repetitive application of input vectors to an input layer ofthe artificial neural network. With each application of an input vector, the actual output ofthe artificial neural network, obtained at the output layer, can be evaluated in light ofthe desired output so that the connection weights and/or biases ofthe artificial neural network can be adjusted.
- the self training artificial neural network object or STANNO may include imaging cells which allow the STANNO to observe or input data located within the data space utilizing the aforementioned relative cell referencing scheme
- the artificial neural network which is to be trained is itself part ofthe STANNO, and at least some ofthe imaging cells may be representative ofthe input layer ofthe artificial neural network.
- the remaining imaging cells can be used by the STANNO to compare the actual output ofthe artificial neural network with the desired output associated with each particular input vector.
- the STANNO also includes a training network which is configured to adjust the weights ofthe artificial neural network as determined by comparing the actual output ofthe artificial neural network with the desired output.
- the training network may include four associated modules to implement the backpropagation training regime
- the first module is configured to determine what the activation level of each artificial neural network neuron would be if the inputs thereto are increased by some infinitesimal amount.
- the second module determines the derivatives of neuron activations with respect to net input thereto.
- the third module determines error terms and the fourth module determines correction values from which the weights and biases ofthe artificial neural network can be adjusted.
- the STANNO may be operable to move within the data space such that with each movement thereof the artificial neural network is trained on an input vector and corresponding output vector within the data space.
- the STANNO may continuously move through and thereby continuously train the artificial neural network within the data space.
- the STANNO may also remain stationary while training the artificial neural network on data which is fed directly into the data space, such as data from known systems which may include known devices or processes.
- Such a data feed may take the form of a dynamic data exchange
- the STANNO is a network training a network with neither represented in algorithmic code.
- the artificial neural network may be copied from or moved from the STANNO and placed at another location within the data space or placed in an entirely different data space for operation.
- a device prototyping system is achievable.
- a prototyping neural network is constructed, wherein at least some ofthe neurons of the prototyping neural network are represented by component neural networks, each trained within a knowledge domain of a component which will be used to construct the device being prototyped.
- the finalized weighting values associated therewith can be used to determine how to interconnect the components in order to construct the prototyped device.
- a second neural network object acts as a data filtering artificial neural network object (DFANNO) whereby data within a data space can be monitored, analyzed, and manipulated in order to either locate novel data or to locate suspect data within the data space.
- DFANNO data filtering artificial neural network object
- the underlying theory is based on the use of an autoassociative neural network which is a network having a knowledge domain wherein input data vectors within the knowledge domain are mapped to themselves. Thus, if an input vector to the autoassociative neural network falls within the knowledge domain thereof, the result is an output vector therefrom which closely matches the input vector.
- the DFANNO is operable to determine whether or not the STANNO has already trained the artificial neural network on a given set of data, or data similar thereto.
- the STANNO has already trained the artificial neural network on the set of data, the artificial neural network is not trained on the given set of data, thereby reducing time wasted by retraining on redundant data. Conversely, if the DFANNO determines that the STANNO has not trained the artificial neural network on the data, the STANNO is permitted to train the artificial neural network on such data.
- the DFANNO may also operate as a separate entity within a data space.
- the DFANNO is operable to analyze data within the data space to determine if any ofthe data does not follow an overall pattem associated with the data, such as data which has been affected by noise or some other disturbance which may have occurred in the data gathering process.
- the DFANNO finds such data it is operable to either remove, delete, or relocate the data from the data space or to in some way tag the data as being suspect. Accordingly, the DFANNO is also an effective device for eliminating or calling attention to suspect data within a given data space.
- a third neural network object acts as a data scanning artificial neural network object (DSANNO) whereby various groupings of data within the data space are examined in attempt to find a set of data values having a predetermined relationship.
- the DSANNO may be stationary within the data space yet able to focus its attention to various groups of cells within the data space by taking advantage of relative cell referencing.
- the DSANNO includes a field positioning neural network which is operable to determine the position ofthe group of cells within the data space which will be analyzed by the DSANNO Through relative cell referencing, a set of imaging cells associated with the DSANNO is used to develop a working image ofthe group of cells which will be analyzed.
- a searching network is then utilized to view the working image from some perspective which is in turn analyzed by a detection network which determines if the set of data values making up the perspective meets the predetermined or desired relationship. Any set meeting the relationship can be tagged or possibly copied to another part ofthe data space.
- the DSANNO is thus useful as a tool for examining large databases for data strings having some desired relationship.
- Fig. 1 is an illustration of a traditional neural network neuron and the corresponding data space simulation thereof;
- Fig 1 A is a partial block diagram of a computer
- Fig. 2 illustrates a plurality of neural network objects in a system for simulating interconnected processes or hardware devices
- Fig. 3 is a block diagram illustration of a neural network object operable within a data space
- Fig. 4 is a high level flow chart for movement ofthe neural network object illustrated in Fig. 3;
- Fig. 5 is a high level flow chart providing the neural network object of Fig. 3 with the ability to act upon the data space;
- Fig. 6 is a block diagram illustration of a self training artificial neural network object which includes an artificial neural network and a training network;
- Fig. 7 is a flow chart illustration of traditional backpropagation neural network training
- Fig. 8 is a continuation ofthe flow chart of Fig. 7;
- Fig. 9 is a nodal illustration of an exemplary artificial neural network which forms part ofthe self training artificial neural network object of Fig. 6;
- Fig. 10 is a data space simulation or implementation ofthe artificial neural network illustrated in Fig. 8;
- Fig. 11 illustrates a first module ofthe training network associated with Fig. 6, the first module operable to determine activation levels when inputs are increased by some small amount;
- Fig. 12 illustrates a second module ofthe training network associated with Fig. 6, the second module operable to determine the derivative of neuron activations with respect to net inputs thereto;
- Fig. 13 illustrates a third module ofthe training network associated with Fig. 6, the third module operable to determine error terms
- Fig. 14 illustrates a fourth module ofthe training network associated with Fig. 6, the fourth module operable to determine weight update terms for the artificial neural network illustrated in Fig. 10;
- Fig. 15 illustrates the self training artificial neural network of Fig. 6 as it moves through and trains within the data space
- Fig. 16 is a Visual Basic program associated with the self training artificial neural network illustrated in Figs. 10-15;
- Fig. 17 illustrates a plurality of sets of training data;
- Figs. 18-21 illustrate various portions of an integrated self training artificial neural network object, wherein the training network is integrated with the artificial neural network being trained;
- Fig. 22 illustrates a subroutine associated with the integrated self training artificial neural network of Figs. 18-21;
- Fig, 23 illustrates a plurality of self training artificial neural network objects training simultaneously within a data space,
- Fig 24 illustrates a subroutine flow chart for implementing dynamic pruning in association with self training artificial neural network objects
- Fig 25 illustrates a subroutine flow chart for implementing dynamic addition of neurons in association with self training artificial neural network objects
- Fig 26 illustrates an exemplary untrained device prototyping neural network
- Fig 27 illustrates the device prototyping neural network of Fig 26 after training, including finalized weight values
- Fig 28 illustrates a data filtering artificial neural network object
- Fig 29 is a Visual Basic program associated with the data filtering artificial neural network object of Fig. 28,
- Fig 30 is a block diagram illustration of a data filtering artificial neural network object associated with a self training artificial neural network object, both objects moving together through a data space,
- Fig 31 is a block diagram illustration of a data scanning artificial neural network object, including a search network, a detection network, and a field positioning network,
- Fig 32 is a nodal illustration of an autoassociative neural network which forms the field positioning network of Fig 31 ,
- Fig 33 illustrates an exemplary viewing field ofthe data scanning artificial neural network object of Fig 31,
- Fig 34 is a nodal illustration of an autoassociative neural network which forms the search network of Fig 31, and Fig 35 is a nodal illustration of an exemplary detection network for the data scanning artificial neural network object of Fig 31
- number 10 in Fig 1 refers to a classical representation of a neural network neuron and number 12 refers to the implementation ofthe neuron 10 in a data space 14
- the illustrated data space 14 includes a plurality of columns 16 and a plurality of rows 18, each column 16 being identifiable by a letter at the top thereof and each row 18 being identifiable by a number located at the left hand side thereof
- the column and row combination results in a plurality of cells 20, each of which may be identified by a corresponding letter and number designation
- This data space 14 configuration is typical of spreadsheets within a spreadsheet application
- the data space implementation 12 of neuron 10 is the building block of neural network objects described herein, but deviations may be used which do not deviate from the spirit ofthe present invention
- the data space implementation 12 includes a first plurality of cells 22, in this case five (5) cells, each having an associated predetermined numeric value, whyroid w 2 , w 3 , w 4 , and ⁇ respectively
- the number of cells 22 will vary depending on the number of inputs to the neural network neuron 10
- a second plurality of cells 24 contain input values x,, x ⁇ x 3 , and x 4
- the plurality of cells 22 include four (4) corresponding weight values w,, w 2 , w 3 , and w 4 , and one bias value ⁇
- the terms weight or weighting value include bias values which are presumed to be associated with constant neuron inputs of one (1)
- the numeric value associated with each cell 22 may be randomly assigned while in a trained neural network the numeric values are determined by training
- a plurality of data space implemented neurons 12 may be used to construct artificial neural networks in accordance with the present invention.
- Such networks typically include both hidden layer and output layer neurons. Accordingly, in such networks, input values for a given neuron may be values associated with activation cells of another neuron within the neural network Utilizing such data space implemented neurons 12 advantageously facilitates construction of artificial neural networks without requiring any specialized algorithm implementing software
- a given artificial neural network is constructed or implemented in a spreadsheet or data space 14
- advantage may be taken of resident spreadsheet capabilities such as the ability to copy and paste a group of cells or to cut and paste a group of cells.
- artificial neural networks constructed in accordance with the present invention may be easily interconnected to construct increasingly complex artificial neural networks.
- One advantageous use for such artificial neural networks is in providing a system for simulating interconnected processes or interconnected devices such as electronic or mechanical devices.
- FIG. 2 Such a system is illustrated in Fig. 2 wherein two data spaces 30 and 32, which may be distinct but associated spreadsheets, such as spreadsheets associated in workbook form, are shown.
- various neural network objects 34, 36, 38, and 40 Located in data space 30 are various neural network objects 34, 36, 38, and 40, in which the cross-hatched regions represent cells associated with the operation of each.
- each neural network object 34, 36, 38, and 40 may be trained within the knowledge domain of some electrical component such as a resistor, capacitor, inductor, or transistor.
- the knowledge domain of any electrical component could be inco ⁇ orated into a neural network object within the data space 30.
- Such a system would be particularly useful when there is no existing mathematical model for the component's behavior. Having established a plurality of operable, neural network objects such as 34,
- various electronic circuit configurations can then be simulated by copying the neural network objects to the data space 32, as indicated by arrow 42 with respect to neural network object 40, so as to interconnect, through relative cell referencing, the neural network objects in the configuration ofthe electronic circuit to be simulated.
- providing a spreadsheet, or plurality of spreadsheets in workbook form, with multiple neural network objects, each trained to emulate a particular electronic device results in a system for simulating electronic circuits of numerous configurations.
- such a system is advantageously user friendly due to the graphical representation of each neural network object which allows a user to easily manipulate such objects as required for a particular application.
- neural network objects can be configured for numerous pu ⁇ oses. Some important aspects of such neural network objects is their ability to autonomously move within the data space, to operate on or alter data or other objects within the data space, and to self organize.
- Fig. 3 illustrates the block diagram configuration of a neural network object 44 which may be operable to move within the data space 14, alter or otherwise operate on data or other objects within the data space 14, and/or self organize.
- the neural network object 44 includes a first data space implemented artificial neural network 46 and also includes one or more imaging cells 48 which, through relative cell referencing, form a working image of a portion 50 ofthe data space 14.
- the imaging cells 48 are tantamount to a visual or receptive field in neurobiology.
- the image developed by the imaging cells 48 is then input to the artificial neural network 46, again through relative cell referencing.
- This first artificial neural network 46 may be trained within a known knowledge domain so as to process the input data and result in some desired output.
- the artificial neural network 46 could be trained to simulate the output of a known system, such as a materials manufacturing process or some hardware device, in response to a multi variable vector input thereto.
- the artificial neural network 46 may be an untrained network which is to be trained on the data referenced by the imaging cells 48.
- the neural network object 44 may also include other associated networks 51.
- the neural network object 44 may be operable, via a program associated therewith, to perform some task. Exemplary programming routines are illustrated in the high level flow charts of Figs. 4 and 5.
- the routine 52 of Fig. 4 could be utilized to cause the neural network object 44 to move, wherein the movement is dependent upon some information produced by the neural network object 44.
- step 56 Staring at 54, such info ⁇ nation would be obtained therefrom at step 56 and the movement would then be carried out by step 58, with the routine ending at 60.
- the routine 62 of Fig. 5 could be utilized to delete or otherwise alter the data located in the portion 50 ofthe data space 14, or to self organize such as by modifying the artificial neural network 46.
- the intended action ofthe neural network object 44 would be determined, starting at 64, from information obtained therefrom at step 66. The action would then be carried out at step 68, with the program ending at 70.
- Autonomy ofthe neural network object 44 is ensured by partitioning its internal function from any governing algorithm in a technique resembling encapsulation within object-oriented programming wherein class objects or different portions of a computer code conceal data and algorithms from each other, passing only restricted information between each other.
- the encapsulation feature allows for the portability ofthe class objects.
- the concept of encapsulation is extended to artificial neural networks wherein the activity between an algorithm and a neural network is segregated. Therefore, the neural network object 44, such as shown in Fig. 3, autonomously makes decisions based upon the imaged portion 50 ofthe data space 14 and the algorithm, 52 or 62, then effects those decisions.
- Various neural network objects can be constructed in accordance with the present invention to perform various functions or simulate known systems.
- a self training artificial neural network object or STANNO 72 which is operable to train an artificial neural network 74 is illustrated in Fig. 6.
- the STANNO 72 includes a plurality of imaging cells 76, the artificial neural network 74 which is to be trained, and a training network 78.
- the training network 78 includes four modules, 80, 82, 84, and 86 which are configured to implement backpropagation training ofthe artificial neural network 74.
- backpropagation training includes generating a table of random numbers corresponding to a starting set of weights at step 92. An input vector, p , is then input to the randomly set neural network at step 94.
- net input values to the hidden layer nodes or neurons are then calculated, wherein net pj h , the total input to the jth hidden (h) layer neuron is the sum ofthe products of all inputs, x ⁇ , and weights w ⁇ plus the bias term ⁇ j h as demonstrated by the equation of step 96.
- the outputs from the hidden layer are then calculated as demonstrated by the equation of step 98 where i pj represents the activation level of the jth hidden layer neuron as a function of its net input and f represents some functional relation such as a sigmoid, linear threshold function, or hyperbolic tangent.
- the net input values to each unit ofthe output layer are then calculated as demonstrated by the equation at step 100, wherein the superscript o refers to the output layer quantities.
- the outputs ofthe output layer nodes or neurons are then calculated as demonstrated by the equation at step 102.
- the flow chart 88 then continues at 104 in Fig. 8.
- the error terms for each ofthe output units and each ofthe hidden layer units are then calculated according to the equations of steps 106 and 108.
- the weights on the output layer are updated according to the equation of step 110, and the weights on the hidden layer are then updated according to the equation of step 112, wherein ⁇ represents the learning parameter.
- An error term Ep is then calculated according to the equation at step 114.
- a new input vector is then selected and training returns to step 94, as indicated by 116, with training continuing until the error Ep reaches some minimal value, as determined at step 118.
- the flow chart 88 ends at 120.
- the STANNO 72 of Fig. 6 utilizes the training network 78 to perform these operations in parallel fashion.
- the training network 78 includes first module 80 which is identical to the artificial neural network 74 except that it determines what the activation levels are when each ofthe inputs is increased by some infinitesimal amount, which may be represented by a value ⁇ of 0.01. It is understood that other values of ⁇ could also be utilized without departing from the scope ofthe present invention
- the second module 82 determines the derivatives of cell activations with respect to net input to those cells.
- the third module 84 utilizes the derivatives to determine the error terms corresponding to steps 106 and 108 of flow chart 88.
- the fourth module 86 determines weight updates, and the weights ofthe artificial neural network 74 and the first module 80 are then adjusted, as indicated by arrow 122, using the updates produced by the training network 78.
- training ofthe artificial neural network 74 is not carried out with algorithmic code, but rather by a network training a network.
- FIGs. 9 through 14 illustrate in greater detail the different portions ofthe STANNO 72 of Fig. 6.
- a traditional representation 124 ofthe artificial neural network 74 is illustrated in Fig. 9.
- a two input neuron, 126 and 128, one output neuron 130 feed forward neural network is depicted, including a hidden layer 132 having three neurons 134, 136, and 138.
- numerous artificial neural network configurations, including more complex artificial neural networks, could be trained as described herein.
- Fig. 10 illustrates a corresponding data space implementation ofthe artificial neural network 74. Also shown in Fig. 10 are the imaging cells 76. In relation to Fig. 9, the imaging cells Dl and El of Fig. 10 correspond to the input neurons 126 and 128 respectively, and activation cells F3, F4, and F5 relate to hidden layer neurons 134, 136 and 138 respectively. Cells D3 and E3 contain the weighting values and cell D4 contains the bias value for neuron 134. Similarly, cells D5, E5, and D6 contain the weight and bias values for neuron 136, while cells D7, E7, and D8 contain the weight and bias values for neuron 138.
- each activation cell F3, F4, and F5 represents the activation level of respective neuron 134, 136, and 138, and is determined by a transfer function which references, either directly or indirectly, the corresponding weight and bias value containing cells as well as the imaging cells Dl and El .
- Activation cell H3 of Fig. 10 corresponds to the output neuron 130 of Fig. 9 and cells G3, G4, G5, and G6 contain the weight and bias values for the neuron 130.
- the transfer function of activation cell H3 references, either directly or indirectly, each ofthe hidden layer activation cells F3, F4, and F5 as well as each ofthe weight and bias containing cells G3, G4, G5, and G6.
- cells F6, F7, F8, and H4 are not necessary for simulating operation ofthe artificial neural network 74. Rather, cells F6, F7, F8 and H4 are used to determine the net input to each ofthe neurons 134, 136, 138, and 130, respectively, in accordance with steps 96 and 100 of flow chart 88, see Fig. 7. These determined values are then utilized by the training network 78, see Fig. 6, as indicated below. Altematively, the SUMPRODUCT functions within cells F6, F7, F8, and H4 could be directly inco ⁇ orated in the respective transfer functions of cells F3, F4, F5, and H3.
- the first module 80 ofthe training network 78 is illustrated in Fig. 11. It is evident that, similar to Fig. 10, the first module 80 contains the data space implementation ofthe artificial neural network 74 illustrated in Fig. 9. However, during training, the inputs to the first module 80 are increased by some infinitesimal amount ⁇ , as indicated by cells D9 and E9, in order to determine the effect on the activation level of, as well as the net input to, each ofthe hidden layer neurons 134, 136, and 138 and the output neuron 130 ofthe artificial neural network 74. The values determined in the first module 80 are then utilized by the second module 82 which is illustrated in Fig. 12 and is operable to determine the derivative of cell activations, which represent neuron activations, with respect to net inputs thereto.
- the derivatives are approximated according to the equations in cells F18, F20, F22, and HI 8, which represent the difference in activation value over the difference in net input.
- cell F18 approximates the derivative ofthe hidden layer neuron 134, Fig. 9, with respect to the net input thereto by dividing the difference between the numeric value associated with cell Fl 1 and the numeric value associated with cell F3 by the difference between the numeric value associated with cell F14 and the numeric value associated with cell F6.
- Similar derivatives for the remaining hidden layer neurons 136 and 138 as well as the output neuron 130 are determined at cells F20, F22, and HI 8 respectively.
- Fig. 13 illustrates the third module 84 ofthe training network 78 wherein the error terms corresponding to steps 106 and 108 of flow chart 88 are determined.
- the error term ⁇ ⁇ ° is determined by multiplying the value associated with cell II by the value associated with cell H18, the value associated with cell II being the difference between the actual output ofthe artificial neural network 74 and the desired output and the value associated with cell HI 8 being the derivative value determined in the second module 82.
- the ⁇ pk ° term of cell H26 is then backpropagated to determine the error terms for the hidden layer neurons 134, 136, and 138 in each of cells F26, F28 and F30.
- the value of cells F18, G3 and H26 are multiplied together, the value associated with cell F18 being the derivative value determined in the second module 82 and the value associated with cell G3 being the weight term from hidden layer neuron 134 to output neuron 130 Similarly, in cells F28 and F30, the error terms for respective hidden layer neurons 136 and 138 are determined.
- weight update terms are determined.
- the weight update terms correspond to the ( ⁇ p ⁇ ) portion ofthe equation shown in step 110 of flow chart 88, where the learning parameter ⁇ has a value of one (1).
- the respective weight update terms for the weight values associated with cells G4 and G5 of Fig 10 are determined in cells G35 and G36 In cell G37, the weight update term for the bias value is determined, the i pj term being designated as one ( 1 ) as explicitly shown
- the weight update terms for the hidden layer weights and biases are also determined in the fourth module 86 These weight update terms correspond to the ( ⁇ pj ⁇ j ) portion ofthe equation shown in step 112 of flow chart 88, where ⁇ , the learning parameter, is again given a value of one (1).
- cell D34 determines the weight update term for cell D3 of Fig 10 by multiplying the numeric value associated with cell F26 by the numeric value associated with cell Dl, the value associated with cell F26 being the ⁇ pj h term determined in the third module 84 and the value associated with cell D 1 being the input value to the hidden layer neuron 134.
- cells E34, D35, D36, E36, D37, D38, E38 and D39 determine the weight update terms for each ofthe values in respective cells E3, D4, D5, E5, D6, D7, E7, and D8, of Fig 10
- the training network 78 determines all weight updates from observed errors, utilizing a parallel computation scheme built upon the backpropagation paradigm There are no algorithmic sequences of steps constituting the partial derivatives, error terms, and updates.
- the weight update terms determined in the fourth module 86 must then be added to their corresponding weight terms in the artificial neural network 74 and the first module 80.
- the STANNO 72 is operable to move to another location in order to train on another set of data within the data space 14
- the operation ofthe STANNO 72 is best shown in Fig 15 where the STANNO 72 is shown in block diagram form Multiple sets of training data may be located in columns A, B, and L ofthe data space, with columns A and B containing the inputs and column L containing the corresponding desired output
- the STANNO 72 is operable to move down one row and train on another set of data
- the STANNO 72 moves through and therefore trains on the training data, with the error or difference between actual output ofthe artificial neural network 74 and the desired output in column L decreasing accordingly, and displayed at cell 140
- Movement ofthe STANNO 72 and updating ofthe weight values ofthe artificial neural network 74 are achieved via software such as the Visual Basic program 142 shown in Fig. 16
- the program may be located in a separate spreadsheet, not shown, which is associated with the spreadsheet or data space 14 ofthe STANNO 72.
- program portion 144 the last training data point, lasti, and the Epoch value are recovered from the spreadsheet
- the program portion 146 randomly assigns initial weights between -8 and 8 to the weight cells ofthe artificial neural network 74.
- the x value corresponds to a row within the data space and the y value corresponds to a column with the data space.
- weights may be initialized by placing the spreadsheet function rand() within the appropriate cell and calling a calculate command.
- program portion 148 artificial neural network training takes place, with the Epoch value representing the number of times the STANNO 72 will be permitted to train on the training data, and the i value representing the number of rows or sets of data the STANNO 72 will be permitted to train on.
- the calculate term 150 triggers all calculations within the data space 14
- update lines 152 update the weight cells by adding to them the weight update values determined in the fourth module 86 ofthe training network 78.
- program portion 154 determines if the STANNO 72 has reached the end ofthe training data, as indicated by zero (0) values in the training input columns.
- Program portion 156 causes the STANNO 72 to move down one row within the data space 14
- Program portion 158 operates to move the STANNO 72 back up to the top ofthe training data.
- the movement ofthe STANNO 72 is accomplished by the copy and paste commands, which leave behind a diagnostic trail of network inputs and outputs. Cutting and pasting would erase this trail Training will be completed when the STANNO 72 has moved through the training data a predetermined number of times, which in this case is the upper limit ofthe Epoch value, or 1000 Alternatively, training could continue until the RMS error associated with the artificial neural network falls below some predetermined value
- STANNO 72 illustrated in Figs 6 and 10-15 along with associated program 142 is merely one configuration among many possibilities for self training neural network objects
- the important aspect ofthe invention being a network which trains a network
- Figs. 17 through 21 illustrate an altemative configuration for an integrated self training artificial neural network where the artificial neural network being trained and the associated training network are integrated with each other in the data space 14
- Figs 17 through 21 all refer to different portions ofthe same data space 14, and
- Fig 17 particularly illustrates columns A through S ofthe data space 14
- Columns B through S contain multiple sets of training data, one set per row, where the sets include nine (9) inputs 160, designated xpl through xp9, and nine (9) associated outputs 160, designated ypl through yp9
- the number of sets of training data is limited only by the maximum number of rows allowable in the data space 14
- Figs 18 through 21 illustrate portions thereof It is assumed that the artificial neural network being trained is a 9-9-9 network, having nine inputs, nine hidden layer neurons, and nine output layer neurons
- Fig 18 illustrates columns
- Figs 19 through 21 the configuration for two levels of neurons is illustrated, rows three (3) through twelve (12) representing the first level 164 and rows thirteen (13) through twenty-two (22) representing the second level 166 Seven more levels of neurons are included in a complete configuration, but, for ease of understanding, are not shown.
- Fig. 18 With reference to column U of Fig 19, it is seen that the values determined in Fig. 18 are utilized to normalize the training inputs.
- cells U3 through Ul 1 determine the normalization of each input, thus the cell combination U3 through Ul 1 represents the input vector.
- Cells U13 through U21 similarly represent the same input vector for the second level In column V, the delta value, 0.01 or -0.01, is added to the normalized inputs of column U.
- a sigmoid has a linear region around the value 0.5, it is desirous when adding the delta value to the normalized input to adjust the input towards the linear region.
- the function IF(U3 ⁇ 0.5, U3+0.01, U3-0.01), causes the positive delta value to be added to normalized inputs which are less than 0.5 and causes the negative delta value to be added to normalized inputs which are greater than 0.5.
- Similar values are used as indicated by the relative references of cells VI 3 through V21.
- the cells of column W contain the hidden layer weight values wji, where j represents the neuron level and i represents the input associated therewith, with biases given the designation q as shown in cells W12 and W22.
- the training based updated hidden layer weight values are determined in the cells of column X.
- the activation levels and derivatives of activation level with respect to net input thereto are determined for each hidden layer neuron level.
- the activation level and net input for the normalized inputs of column U are determined in cell Y3 and Y4, respectively, the activation level and net input for the delta adjusted inputs of column V are determined in cells Y5 and Y6, respectively, and the derivative value is determined in cell Y7.
- Corresponding values for the second level 166 are determined in cells Y13 through Y17.
- each of cells Y3, Y13, Y23, Y33, Y43, Y53, Y63, Y73, Y83 will contain the activation level of a hidden layer neuron.
- all activation levels, act j(xp) are relatively referenced such that, for example, the values associated with cells Z3 through Zl 1 represent an input vector to be applied to the output layer neurons Accordingly, in column AA, the delta value is added to the activation levels of column Z
- Column AB contains the output layer weight values wkj and the training based updated output layer weight values are determined in column AC Referring to Fig.
- Fig 22 illustrates a subroutine 168 which accomplishes this task
- the first line selects the cells of column X associated with the integrated self training artificial neural network and the second line copies those cells
- the third line selects the destination column for the copied material and the fourth and fifth lines operate to paste only the numeric values associated with the copied cells into the destination column.
- subroutine 168 operates to replace the output layer weight values of column AB with the updated weight values of column AC As compared to the update method illustrated in portion 152 of Fig. 16, the subroutine 168 is able to complete the weight updating much more quickly, advantageously increasing training speed.
- the integrated self training artificial neural network illustrated in Figs. 18 through 21 is capable of training in real time as training data flows through the data space 14.
- the data would flow through predetermined rows or columns and the integrated self training artificial neural network would remain stationary in the data space 14 while the training data moves relative thereto.
- the STANNO 72 illustrated in Figs 10 through 14 could also be utilized with such a dynamic data exchange.
- Another advantage of self training artificial neural networks is that multiple networks may be trained simultaneously, in parallel fashion, on the same, or different, sets of training data.
- multiple self training artificial neural network objects such as 170, 172, and 174, may be positioned within the data space 14 so as to train on the data flowing through the columns as indicated at 176.
- Each self trainer 170, 172, and 174 may also be configured to train on only some ofthe columns of data in order to result in trained networks having different knowledge domains.
- each self trainer could train on completely different sets of data, such as where STANNO 170 trains on the data flowing through the columns to the left and STANNO 172 trains on the data flowing through the columns to the right, or where multiple self training neural network objects train on distinct data within separate spreadsheets altogether.
- Such a parallel training scheme would be extremely difficult to implement using traditional algorithm based training.
- a subroutine 178 such as illustrated in Fig 24 may be provided, such as by embedding the subroutine 178 within the spreadsheet or data space
- the variable N may be a count of the number of sets of training data which have been operated upon and which is set to zero (0) at the beginning of training
- Tl may be a predetermined value which is chosen to represent a change in magnitude associated with the activation level ofthe neuron
- T2 may be a predetermined number which is chosen to represent a number of activation level changes of magnitude greater than Tl
- the subroutine 178 is run in association with each wave of spreadsheet calculation
- the subroutine starts at 180 and at step 182 the change in activation level, ⁇ .
- step 184 if the change in activation is greater than Tl, the variable TRANSITIONS is increased by one.
- the N count, or count of number of sets of training data is increased by one and at step 188 the N count is evaluated to see if it has reached a PREDETERMINED NUMBER. If N has not reached the PREDETERMINED NUMBER, the subroutine ends at 190 However, if the N count has reached the PREDETERMINED NUMBER, step 192 is reached and the N count is again set to zero At step 192 the TRANSITIONS variable is evaluated to see if it is less than the number T2, if not, the subroutine 178 ends at 190
- Tl and T2 can be chosen to reflect the fact that the neuron is not significantly involved in the training regime and can therefore be pruned out of the artificial neural network, while the PREDETERMINED NUMBER of step 188 can be chosen to reflect how often the neuron should be evaluated to see if it should be eliminated
- a subroutine 198 illustrated in Fig. 25, may be associated with the operation of a self training neural network object
- the subroutine 198 begins at 200 and at step 202, the RMS
- ERROR between actual outputs and desired or training outputs, determined after each set of training data is operated upon, is evaluated to determine if it exceeds a desired THRESHOLD ERROR, which is predetermined so as to be indicative of successful inco ⁇ oration ofthe desired knowledge domain within the artificial neural network. If the RMS ERROR has fallen below the THRESHOLD ERROR, the subroutine 198 ends at step 204. Conversely, if the RMS ERROR exceeds the THRESHOLD ERROR, step 206 is reached where N, the count of sets of training data, is evaluated to determine if it exceeds a THRESHOLD N number If N does not exceed the THRESHOLD N number, the subroutine 198 ends at step 204.
- step 208 if N exceeds the THRESHOLD N number, step 208 is reached.
- the THRESHOLD N number should be chosen so as to indicate that the training operation has continued long enough to determine that the artificial neural network being trained is not large enough, and that in order to train the artificial neural network to be able to achieve the desired THRESHOLD ERROR, the artificial neural network must be enlarged.
- a prototypical neuron with randomized weights is copied and added to the hidden layer.
- step 208 a prototypical neuron with randomized weights is copied and added to the hidden layer.
- the prototyping neural network 216 includes seven (7) hidden layer neurons 218, 220, 222, 224, 226, 228, and 230 respectively.
- Each hidden layer neuron is represented by a component neural network which is trained within a knowledge domain of one ofthe harmonic generating devices which will be used as components from which to construct the prototyped device.
- the output F( ⁇ ) may appear as 232.
- the prototyping neural network can be trained within the desired knowledge domain ofthe prototyped device, which is reflected in a conversion ofthe sinusoid 214 to a cyclic square pulse.
- Fig. 27 illustrates the resulting prototyping neural network 216 after training, including weight values. As seen, all hidden layer weights approach one. With regard to the output layer weights, the weights for neurons 218, 222, 226, and 230 approach zero, and thus no connection to the output is shown. However, the illustrated weights for neurons 220, 224, and 228 approach (2/ ⁇ ), (2/3 ⁇ ), and (2/5 ⁇ ) respectively, along with a bias value of (1/2).
- the resulting output ofthe prototyping neural network is F( ⁇ ) as shown in the equation 234 and the graph 236.
- the weights which result from training the device prototyping neural network can then be correlated to how the components should be interconnected in order to construct the prototyped device. In this exemplary case, it is evident that odd harmonic generating devices would be directly re ⁇
- This prototyping system can be utilized in conjunction with many types of components.
- the important aspect ofthe system is that if a neural network model for each component can be constructed, a prototyping neural network can be trained without requiring explicit knowledge ofthe functional relation between inputs and outputs ofthe components because the self training scheme is able to determine derivative values without knowing the functional relation
- traditional backpropagation algorithms require foreknowledge ofthe functional relation and its derivative
- DATA FILTERING/MONITORING Another neural network object which may be constructed is a data filtering neural network object or DFANNO 238 such as shown in Fig 28.
- the underlying theory ofthe DFANNO 238 is that of an autoassociative neural network 240
- the autoassociative neural network 240 is an artificial neural network which is trained to map inputs to themselves. Accordingly, an input vector within the knowledge domain ofthe autoassociative neural network 240 results in an output vector therefrom which closely matches the input vector
- the network 240 will produce at its output 244 another vector v' representative ofthe closest vector seen in the training data or generalized from the training data.
- Av lv'
- A represents the autoassociative neural network 240 and 1 is the unitary matrix with diagonal elements of 1.
- ⁇ represents the error or vector difference between input and output vectors ofthe autoassociative neural network 240
- the autoassociative neural network 240 Prior to operation, the autoassociative neural network 240 should be trained on a plurality of sets of control data. Each set of control data should be carefully selected so as to reflect the desired knowledge domain and so as to ensure that each set of control data has not been affected by systematic error or random noise.
- the DFANNO 238 moves through a data space 14 encountering different rows of data, such as 246, each representing an input vector thereto, an RMS error between each input vector and each output vector is determined as indicated at 248. If, for a given input vector, the error exceeds a predetermined level, the DFANNO 238 is then operable to perform some operation on the row 246 of data making up the input vector. For example, the row 246 of data may be deleted from the data space 14 entirely, relocated, or tagged as suspect.
- the DFANNO 238 is effective for moving through the data space 14, as indicated by arrow 250, and examining the data therein to find data which may have been caused by some systematic error or random noise introduced to the data or which occurred when the data was originally gathered.
- a Visual Basic program 252 which achieves these operations is illustrated in
- the calculate line 254 triggers all calculations within the data space 14.
- the For-Next Loop 256 is provided to determine if the DFANNO 238 has reached a point in the data space 14 where there is no more data, as indicated by all cells of a particular row being zero. If there is no more data the operation of DFANNO 238 is halted. Line 258 and portion 260 determine the operation the DFANNO 238 will take with respect to a particular row of data. In each of these lines cell (1,10) ofthe data space 14 represents a flag. If the flag is zero (0) the DFANNO 238 is in the data tag mode but if the flag is set to one (1) the DFANNO 238 is in the data destroy mode.
- the DFANNO 238 could be stationary within the data space 14 while data from some system or device to be monitored by the DFANNO 238 is fed into predetermined locations within the data space 14 through a dynamic data exchange, such that the DFANNO 238 operates on the data as it is fed through the data space 14.
- the DFANNO 238 could be operable to shut down the system or device.
- the DFANNO 238, either alone or in combination with other networks provides an effective system monitor.
- Data filtering artificial neural networks can also advantageously be used in association with self training artificial neural networks. Such an association is illustrated in Fig.
- a DFANNO 238 has been appended to an STANNO 72 such that the two neural network objects move with each other through the data space 14 as shown by arrow 264.
- the DFANNO 238 is operable to determine if the data at any given location is novel to the training ofthe STANNO 72.
- the error determined by the DFANNO 238 exceeds a predetermined level, the data is considered novel and the STANNO 72 trains on such data.
- the DFANNO 238 would be operable to cause the two associated neural network objects to move on to another set of data without allowing the STANNO 72 to train on the data, thereby reducing time wasted by retraining on redundant data.
- DATA SCANNING Fig. 31 illustrates a block diagram configuration of a data scanning artificial neural network object or DSANNO 266.
- the DSANNO 266 is stationary within the data space 14 but capable of directing its view to various groups of cells within the data space 14 utilizing relative cell referencing.
- the DSANNO 266 includes a search network 268, a detection network 270, and a field positioning network 272.
- the field positioning network 272 autonomously moves the viewing field 274 of the DSANNO 266 about the data.
- the graphical antenna 276 may be utilized as a guide to the human viewer as to where the DSANNO 266 is focusing its attention, however, the antenna 276 is not required for operation ofthe DSANNO 266.
- Viewing field 274 positioning is achieved utilizing an autoassociative neural network 278 in which the weights and biases are subjected to noise sources, as shown by arrow 280 in Fig. 32, so that the autoassociative neural network 278 imagines various possibilities within its training domain.
- the noise source may be random numbers applied to the weights and biases ofthe autoassociative neural network 278.
- the autoassociative neural network 278 used is trained on a table of (x, y) values having integer values associated with the cells containing the data.
- the autoassociative neural network 278 As the autoassociative neural network 278 is subjected to noise, it generates outputs reflecting the constraints within the training database, namely that it generate only integer values corresponding to data containing cells.
- the perturbed autoassociative neural network 278 is a random integer generator.
- a relatively smooth trajectory of viewing field 274 positions is generated because x and y coordinates are only gradually altered with each feedthrough cycle of this recurrent net.
- the net effect is this configuration is to produce continuous random movement ofthe viewing field 274 ofthe DSANNO 266, and is similar to the population-polling process used to govern human eye movement.
- a group of imaging cells 288, see Fig. 31, utilize relative cell referencing to develop a working image ofthe viewing field 274.
- the working image may then be communicated to the search network 268 as indicated by arrow 290.
- the imaging cells 288 ofthe DSANNO 266 are illustrated and include a 4x4 array of cells.
- the search network 268, illustrated in Fig. 34 is utilized to view the imaging cells 288 from a perspective such as that illustrated by the bold cells 292 of Fig. 33.
- the development of such a perspective is achieved utilizing an autoassociative neural network 294 which has been trained on numerous examples of data string configurations within the imaging cells 288.
- Noise 296 is then introduced to the network 294 such that the network 294 produces an imagined data string configuration at its output 298 which will be examined by the detection network 270 of Fig. 31.
- an exemplary detection network 270 is illustrated in Fig. 35.
- This detection network 270 is trained to output a one at 300 if the inputs applied at input layer 302 obey the search criteria.
- the training domain can be chosen as required for a particular application. For example, the training domain may output a one when the inputs thereto have some predetermined relationship. The output of a one then acts to enable the DSANNO 266 to perform some operation such as tagging the data string, copying the data string to another portion ofthe data space 14, or enabling a wave file 304, see Fig.
- data scanning neural network objects could include different viewing field configurations and could develop different data strings to be viewed by appropriate detection networks and DSANNO 266 is merely exemplary ofthe overall configuration. Accordingly, data scanning artificial neural network objects are useful for examining large databases for data strings having some predetermined, desired relationship, and then in some way identifying such data strings.
- the creativity machine paradigm involves progressively purturbing a first neural network, or imagination engine (IE), having a predetermined knowledge domain such that the perturbed network continuously outputs a stream of concepts, and monitoring the outputs or stream of concepts with a second neural network, or alert associative center (AAC), which is trained to identify only useful concepts
- IE imagination engine
- AAC alert associative center
- the perturbations may be achieved by different means, including the introduction of noise to the network, or degradation ofthe network.
- Such machines can be simulated within a data space in accordance with the present invention and also trained in as part of self training artificial neural network objects in accordance with the present invention
- the resident rand() function may be utilized to alter the weights ofthe IE in order to achieve perturbation
- relative cell referencing facilitates feeding the outputs ofthe IE to the inputs ofthe AAC.
- both the IE and the AAC are particularly applicable to training ofthe IE and the AAC of creativity machines because both the IE and the AAC will typically have at least some training data in common. At times it may be desirable to change the knowledge domain ofthe IE and or the AAC. For example, if a creativity machine is trained in coffee mug design, the IE is initially trained on known, produced coffee mug shapes and the AAC is trained to recognize a good coffee mug shape from a bad coffee mug shape Over time, the range of known, produced coffee mug shapes may increase, or, the public's perception of what a good coffee mug shape is may change Thus, in order to keep the creativity machine up to date, both the IE and the AAC may need to be trained on new data Utilizing the hereinbefore described training technique, both networks can be trained on new data without having to completely retrain either network on the data it had been trained on previously Further, because the techniques described herein allow multiple neural network to n simultaneously, a creativity machine, including an IE and an AAC could run while replica
- the data space cells utilized in simulating the neuron 10 need not be arranged as shown, but could be located in various portions ofthe data space.
- self training artificial neural networks it is understood that there are numerous configurations for achieving the underlying invention which is a network training another network.
- numerous programs could be associated with the self training artificial neural networks, as well as the data filtering and data scanning neural networks.
- routines could be included within individual cells ofthe same spreadsheet or data space in which the neural networks are constmcted. Accordingly, the spirit and scope ofthe invention are to be limited only by the terms ofthe appended claims.
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP9526939A JP2000504129A (en) | 1996-01-26 | 1997-01-17 | Artificial neural networks executed without algorithms and their components. |
CA002243120A CA2243120C (en) | 1996-01-26 | 1997-01-17 | Neural network based data examining system and method |
EP97906423A EP0892957A4 (en) | 1996-01-26 | 1997-01-17 | Non-algorithmically implemented artificial neural networks and components thereof |
AU21120/97A AU716593B2 (en) | 1996-01-26 | 1997-01-17 | Non-algorithmically implemented artificial neural networks and components thereof |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/592,767 US5845271A (en) | 1996-01-26 | 1996-01-26 | Non-algorithmically implemented artificial neural networks and components thereof |
US08/592,767 | 1996-01-26 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO1997027525A2 true WO1997027525A2 (en) | 1997-07-31 |
WO1997027525A3 WO1997027525A3 (en) | 1997-09-04 |
Family
ID=24371988
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1997/000886 WO1997027525A2 (en) | 1996-01-26 | 1997-01-17 | Non-algorithmically implemented artificial neural networks and components thereof |
Country Status (6)
Country | Link |
---|---|
US (4) | US5845271A (en) |
EP (1) | EP0892957A4 (en) |
JP (1) | JP2000504129A (en) |
AU (1) | AU716593B2 (en) |
CA (1) | CA2243120C (en) |
WO (1) | WO1997027525A2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU734922B2 (en) * | 1997-07-18 | 2001-06-28 | International Business Machines Corporation | Optimizer with neural network estimator |
CN111914984A (en) * | 2019-05-07 | 2020-11-10 | 财团法人成大研究发展基金会 | Artificial neural network normalization system for identification device |
Families Citing this family (132)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5659666A (en) * | 1994-10-13 | 1997-08-19 | Thaler; Stephen L. | Device for the autonomous generation of useful information |
US7925492B2 (en) | 2004-01-06 | 2011-04-12 | Neuric Technologies, L.L.C. | Method for determining relationships through use of an ordered list between processing nodes in an emulated human brain |
US7089218B1 (en) * | 2004-01-06 | 2006-08-08 | Neuric Technologies, Llc | Method for inclusion of psychological temperament in an electronic emulation of the human brain |
US6292809B1 (en) * | 1997-08-07 | 2001-09-18 | Adobe Systems Incorporated | Selecting cells in a table having individual cells and joined cells |
US6560582B1 (en) * | 2000-01-05 | 2003-05-06 | The United States Of America As Represented By The Secretary Of The Navy | Dynamic memory processor |
US6633855B1 (en) | 2000-01-06 | 2003-10-14 | International Business Machines Corporation | Method, system, and program for filtering content using neural networks |
US7809601B2 (en) * | 2000-10-18 | 2010-10-05 | Johnson & Johnson Consumer Companies | Intelligent performance-based product recommendation system |
US6768815B2 (en) | 2001-05-10 | 2004-07-27 | The United States Of America As Represented By The Secretary Of The Navy | Color sensor |
WO2003015027A2 (en) * | 2001-08-07 | 2003-02-20 | Board Of Trustees Operating Michigan State University | Self-programmable chip |
US7499594B2 (en) * | 2002-04-15 | 2009-03-03 | At&T Intellectual Property 1, L.P. | Multi-resolution predictive foveation for bandwidth reduction of moving images |
US7010169B2 (en) * | 2002-04-15 | 2006-03-07 | Sbc Technology Resources, Inc. | Multi-point predictive foveation for bandwidth reduction of moving images |
MY138544A (en) * | 2003-06-26 | 2009-06-30 | Neuramatix Sdn Bhd | Neural networks with learning and expression capability |
DE10345440A1 (en) * | 2003-09-30 | 2005-05-12 | Siemens Ag | Method, computer program with program code means and computer program product for analyzing influencing variables on a burning process in a combustion chamber using a trainable, statistical model |
US7333963B2 (en) * | 2004-10-07 | 2008-02-19 | Bernard Widrow | Cognitive memory and auto-associative neural network based search engine for computer and network located images and photographs |
US8473449B2 (en) * | 2005-01-06 | 2013-06-25 | Neuric Technologies, Llc | Process of dialogue and discussion |
JP2008533615A (en) * | 2005-03-14 | 2008-08-21 | エル ターラー、ステフエン | Neural network development and data analysis tools |
US20060229932A1 (en) * | 2005-04-06 | 2006-10-12 | Johnson & Johnson Services, Inc. | Intelligent sales and marketing recommendation system |
US7454388B2 (en) * | 2005-05-07 | 2008-11-18 | Thaler Stephen L | Device for the autonomous bootstrapping of useful information |
US20070011169A1 (en) * | 2005-07-05 | 2007-01-11 | Xerox Corporation | Method and system for collecting data from diverse sources and transforming the collected data into a user-friendly format |
JP2009503657A (en) * | 2005-07-22 | 2009-01-29 | エル ターラー、ステフエン | Rating system using neural network |
US20100312734A1 (en) * | 2005-10-07 | 2010-12-09 | Bernard Widrow | System and method for cognitive memory and auto-associative neural network based pattern recognition |
US20080033630A1 (en) * | 2006-07-26 | 2008-02-07 | Eun-Mi Lee | System and method of predicting traffic speed based on speed of neighboring link |
US20080201234A1 (en) * | 2007-02-16 | 2008-08-21 | Microsoft Corporation | Live entities internet store service |
US20080201338A1 (en) * | 2007-02-16 | 2008-08-21 | Microsoft Corporation | Rest for entities |
US7953680B2 (en) * | 2007-12-10 | 2011-05-31 | International Business Machines Corporation | Computer program product for excluding variations attributable to equipment used in semiconductor wafer manufacturing from split analysis procedures |
US20090210400A1 (en) * | 2008-02-15 | 2009-08-20 | Microsoft Corporation | Translating Identifier in Request into Data Structure |
US9122994B2 (en) | 2010-03-26 | 2015-09-01 | Brain Corporation | Apparatus and methods for temporally proximate object recognition |
US9405975B2 (en) | 2010-03-26 | 2016-08-02 | Brain Corporation | Apparatus and methods for pulse-code invariant object recognition |
US9311593B2 (en) | 2010-03-26 | 2016-04-12 | Brain Corporation | Apparatus and methods for polychronous encoding and multiplexing in neuronal prosthetic devices |
US8315305B2 (en) | 2010-03-26 | 2012-11-20 | Brain Corporation | Systems and methods for invariant pulse latency coding |
US8825745B2 (en) | 2010-07-11 | 2014-09-02 | Microsoft Corporation | URL-facilitated access to spreadsheet elements |
US9906838B2 (en) | 2010-07-12 | 2018-02-27 | Time Warner Cable Enterprises Llc | Apparatus and methods for content delivery and message exchange across multiple content delivery networks |
US9152915B1 (en) | 2010-08-26 | 2015-10-06 | Brain Corporation | Apparatus and methods for encoding vector into pulse-code output |
US9015093B1 (en) | 2010-10-26 | 2015-04-21 | Michael Lamport Commons | Intelligent control with hierarchical stacked neural networks |
US8775341B1 (en) | 2010-10-26 | 2014-07-08 | Michael Lamport Commons | Intelligent control with hierarchical stacked neural networks |
EA201391111A1 (en) | 2011-01-31 | 2013-12-30 | Лэндмарк Графикс Корпорейшн | SYSTEM AND METHOD FOR USE OF ARTIFICIAL NEURAL NETWORK FOR MODELING A HYDRAULIC OF PIPELINES IN THE SIMULATOR |
US9147156B2 (en) | 2011-09-21 | 2015-09-29 | Qualcomm Technologies Inc. | Apparatus and methods for synaptic update in a pulse-coded network |
US8990133B1 (en) | 2012-12-20 | 2015-03-24 | Brain Corporation | Apparatus and methods for state-dependent learning in spiking neuron networks |
US9047568B1 (en) | 2012-09-20 | 2015-06-02 | Brain Corporation | Apparatus and methods for encoding of sensory data using artificial spiking neurons |
US9070039B2 (en) | 2013-02-01 | 2015-06-30 | Brian Corporation | Temporal winner takes all spiking neuron network sensory processing apparatus and methods |
US9566710B2 (en) | 2011-06-02 | 2017-02-14 | Brain Corporation | Apparatus and methods for operating robotic devices using selective state space training |
US9104973B2 (en) | 2011-09-21 | 2015-08-11 | Qualcomm Technologies Inc. | Elementary network description for neuromorphic systems with plurality of doublets wherein doublet events rules are executed in parallel |
US8725658B2 (en) | 2011-09-21 | 2014-05-13 | Brain Corporation | Elementary network description for efficient memory management in neuromorphic systems |
US8719199B2 (en) | 2011-09-21 | 2014-05-06 | Brain Corporation | Systems and methods for providing a neural network having an elementary network description for efficient implementation of event-triggered plasticity rules |
US8725662B2 (en) | 2011-09-21 | 2014-05-13 | Brain Corporation | Apparatus and method for partial evaluation of synaptic updates based on system events |
US9460387B2 (en) * | 2011-09-21 | 2016-10-04 | Qualcomm Technologies Inc. | Apparatus and methods for implementing event-based updates in neuron networks |
US9412064B2 (en) | 2011-08-17 | 2016-08-09 | Qualcomm Technologies Inc. | Event-based communication in spiking neuron networks communicating a neural activity payload with an efficacy update |
US8712939B2 (en) | 2011-09-21 | 2014-04-29 | Brain Corporation | Tag-based apparatus and methods for neural networks |
US9213937B2 (en) | 2011-09-21 | 2015-12-15 | Brain Corporation | Apparatus and methods for gating analog and spiking signals in artificial neural networks |
US9146546B2 (en) | 2012-06-04 | 2015-09-29 | Brain Corporation | Systems and apparatus for implementing task-specific learning using spiking neurons |
US9098811B2 (en) | 2012-06-04 | 2015-08-04 | Brain Corporation | Spiking neuron network apparatus and methods |
US9104186B2 (en) | 2012-06-04 | 2015-08-11 | Brain Corporation | Stochastic apparatus and methods for implementing generalized learning rules |
US9015092B2 (en) | 2012-06-04 | 2015-04-21 | Brain Corporation | Dynamically reconfigurable stochastic learning apparatus and methods |
US9117176B2 (en) | 2011-09-21 | 2015-08-25 | Qualcomm Technologies Inc. | Round-trip engineering apparatus and methods for neural networks |
US10210452B2 (en) | 2011-09-21 | 2019-02-19 | Qualcomm Incorporated | High level neuromorphic network description apparatus and methods |
US9156165B2 (en) | 2011-09-21 | 2015-10-13 | Brain Corporation | Adaptive critic apparatus and methods |
CN106991475A (en) * | 2012-03-15 | 2017-07-28 | 美国高通技术公司 | The apparatus and method based on mark for neutral net |
US9129221B2 (en) | 2012-05-07 | 2015-09-08 | Brain Corporation | Spiking neural network feedback apparatus and methods |
US9224090B2 (en) | 2012-05-07 | 2015-12-29 | Brain Corporation | Sensory input processing apparatus in a spiking neural network |
US9412041B1 (en) | 2012-06-29 | 2016-08-09 | Brain Corporation | Retinal apparatus and methods |
US9256215B2 (en) | 2012-07-27 | 2016-02-09 | Brain Corporation | Apparatus and methods for generalized state-dependent learning in spiking neuron networks |
US9256823B2 (en) | 2012-07-27 | 2016-02-09 | Qualcomm Technologies Inc. | Apparatus and methods for efficient updates in spiking neuron network |
US9186793B1 (en) | 2012-08-31 | 2015-11-17 | Brain Corporation | Apparatus and methods for controlling attention of a robot |
US9440352B2 (en) | 2012-08-31 | 2016-09-13 | Qualcomm Technologies Inc. | Apparatus and methods for robotic learning |
US9189730B1 (en) | 2012-09-20 | 2015-11-17 | Brain Corporation | Modulated stochasticity spiking neuron network controller apparatus and methods |
US9367798B2 (en) | 2012-09-20 | 2016-06-14 | Brain Corporation | Spiking neuron network adaptive control apparatus and methods |
US8793205B1 (en) | 2012-09-20 | 2014-07-29 | Brain Corporation | Robotic learning and evolution apparatus |
US9311594B1 (en) | 2012-09-20 | 2016-04-12 | Brain Corporation | Spiking neuron network apparatus and methods for encoding of sensory data |
US9082079B1 (en) | 2012-10-22 | 2015-07-14 | Brain Corporation | Proportional-integral-derivative controller effecting expansion kernels comprising a plurality of spiking neurons associated with a plurality of receptive fields |
US9183493B2 (en) | 2012-10-25 | 2015-11-10 | Brain Corporation | Adaptive plasticity apparatus and methods for spiking neuron network |
US9218563B2 (en) | 2012-10-25 | 2015-12-22 | Brain Corporation | Spiking neuron sensory processing apparatus and methods for saliency detection |
US9111226B2 (en) | 2012-10-25 | 2015-08-18 | Brain Corporation | Modulated plasticity apparatus and methods for spiking neuron network |
US9275326B2 (en) | 2012-11-30 | 2016-03-01 | Brain Corporation | Rate stabilization through plasticity in spiking neuron network |
US9123127B2 (en) | 2012-12-10 | 2015-09-01 | Brain Corporation | Contrast enhancement spiking neuron network sensory processing apparatus and methods |
US9195934B1 (en) | 2013-01-31 | 2015-11-24 | Brain Corporation | Spiking neuron classifier apparatus and methods using conditionally independent subsets |
US9177245B2 (en) | 2013-02-08 | 2015-11-03 | Qualcomm Technologies Inc. | Spiking network apparatus and method with bimodal spike-timing dependent plasticity |
US9764468B2 (en) | 2013-03-15 | 2017-09-19 | Brain Corporation | Adaptive predictor apparatus and methods |
US8996177B2 (en) | 2013-03-15 | 2015-03-31 | Brain Corporation | Robotic training apparatus and methods |
US9008840B1 (en) | 2013-04-19 | 2015-04-14 | Brain Corporation | Apparatus and methods for reinforcement-guided supervised learning |
US9242372B2 (en) | 2013-05-31 | 2016-01-26 | Brain Corporation | Adaptive robotic interface apparatus and methods |
US9314924B1 (en) * | 2013-06-14 | 2016-04-19 | Brain Corporation | Predictive robotic controller apparatus and methods |
US9792546B2 (en) | 2013-06-14 | 2017-10-17 | Brain Corporation | Hierarchical robotic controller apparatus and methods |
US9384443B2 (en) | 2013-06-14 | 2016-07-05 | Brain Corporation | Robotic training apparatus and methods |
US9239985B2 (en) | 2013-06-19 | 2016-01-19 | Brain Corporation | Apparatus and methods for processing inputs in an artificial neuron network |
US9436909B2 (en) | 2013-06-19 | 2016-09-06 | Brain Corporation | Increased dynamic range artificial neuron network apparatus and methods |
US9552546B1 (en) | 2013-07-30 | 2017-01-24 | Brain Corporation | Apparatus and methods for efficacy balancing in a spiking neuron network |
US9296101B2 (en) | 2013-09-27 | 2016-03-29 | Brain Corporation | Robotic control arbitration apparatus and methods |
US9579789B2 (en) | 2013-09-27 | 2017-02-28 | Brain Corporation | Apparatus and methods for training of robotic control arbitration |
US10339041B2 (en) * | 2013-10-11 | 2019-07-02 | Qualcomm Incorporated | Shared memory architecture for a neural simulator |
US9489623B1 (en) | 2013-10-15 | 2016-11-08 | Brain Corporation | Apparatus and methods for backward propagation of errors in a spiking neuron network |
US9597797B2 (en) | 2013-11-01 | 2017-03-21 | Brain Corporation | Apparatus and methods for haptic training of robots |
US9463571B2 (en) | 2013-11-01 | 2016-10-11 | Brian Corporation | Apparatus and methods for online training of robots |
US9248569B2 (en) | 2013-11-22 | 2016-02-02 | Brain Corporation | Discrepancy detection apparatus and methods for machine learning |
US9358685B2 (en) | 2014-02-03 | 2016-06-07 | Brain Corporation | Apparatus and methods for control of robot actions based on corrective user inputs |
US9364950B2 (en) | 2014-03-13 | 2016-06-14 | Brain Corporation | Trainable modular robotic methods |
US9533413B2 (en) | 2014-03-13 | 2017-01-03 | Brain Corporation | Trainable modular robotic apparatus and methods |
US9987743B2 (en) | 2014-03-13 | 2018-06-05 | Brain Corporation | Trainable modular robotic apparatus and methods |
US9630317B2 (en) | 2014-04-03 | 2017-04-25 | Brain Corporation | Learning apparatus and methods for control of robotic devices via spoofing |
US9613308B2 (en) | 2014-04-03 | 2017-04-04 | Brain Corporation | Spoofing remote control apparatus and methods |
US9346167B2 (en) | 2014-04-29 | 2016-05-24 | Brain Corporation | Trainable convolutional network apparatus and methods for operating a robotic vehicle |
US10194163B2 (en) | 2014-05-22 | 2019-01-29 | Brain Corporation | Apparatus and methods for real time estimation of differential motion in live video |
US9713982B2 (en) | 2014-05-22 | 2017-07-25 | Brain Corporation | Apparatus and methods for robotic operation using video imagery |
US9939253B2 (en) | 2014-05-22 | 2018-04-10 | Brain Corporation | Apparatus and methods for distance estimation using multiple image sensors |
US9848112B2 (en) | 2014-07-01 | 2017-12-19 | Brain Corporation | Optical detection apparatus and methods |
US10057593B2 (en) | 2014-07-08 | 2018-08-21 | Brain Corporation | Apparatus and methods for distance estimation using stereo imagery |
US9849588B2 (en) | 2014-09-17 | 2017-12-26 | Brain Corporation | Apparatus and methods for remotely controlling robotic devices |
US9579790B2 (en) | 2014-09-17 | 2017-02-28 | Brain Corporation | Apparatus and methods for removal of learned behaviors in robots |
US9860077B2 (en) | 2014-09-17 | 2018-01-02 | Brain Corporation | Home animation apparatus and methods |
US9821470B2 (en) | 2014-09-17 | 2017-11-21 | Brain Corporation | Apparatus and methods for context determination using real time sensor data |
US9870617B2 (en) | 2014-09-19 | 2018-01-16 | Brain Corporation | Apparatus and methods for saliency detection based on color occurrence analysis |
US9630318B2 (en) | 2014-10-02 | 2017-04-25 | Brain Corporation | Feature detection apparatus and methods for training of robotic navigation |
US9881349B1 (en) | 2014-10-24 | 2018-01-30 | Gopro, Inc. | Apparatus and methods for computerized object identification |
US9426946B2 (en) | 2014-12-02 | 2016-08-30 | Brain Corporation | Computerized learning landscaping apparatus and methods |
CN104634706A (en) * | 2015-01-23 | 2015-05-20 | 国家电网公司 | Neural network-based soft measurement method for pulverized coal fineness |
US9717387B1 (en) | 2015-02-26 | 2017-08-01 | Brain Corporation | Apparatus and methods for programming and training of robotic household appliances |
US9840003B2 (en) | 2015-06-24 | 2017-12-12 | Brain Corporation | Apparatus and methods for safe navigation of robotic devices |
US10197664B2 (en) | 2015-07-20 | 2019-02-05 | Brain Corporation | Apparatus and methods for detection of objects using broadband signals |
US10295972B2 (en) | 2016-04-29 | 2019-05-21 | Brain Corporation | Systems and methods to operate controllable devices with gestures and/or noises |
US10592725B2 (en) * | 2017-04-21 | 2020-03-17 | General Electric Company | Neural network systems |
KR102549540B1 (en) * | 2017-09-22 | 2023-06-29 | 삼성전자주식회사 | Storage device and method of operating the same |
US10672389B1 (en) * | 2017-12-29 | 2020-06-02 | Apex Artificial Intelligence Industries, Inc. | Controller systems and methods of limiting the operation of neural networks to be within one or more conditions |
US10802489B1 (en) | 2017-12-29 | 2020-10-13 | Apex Artificial Intelligence Industries, Inc. | Apparatus and method for monitoring and controlling of a neural network using another neural network implemented on one or more solid-state chips |
US10620631B1 (en) * | 2017-12-29 | 2020-04-14 | Apex Artificial Intelligence Industries, Inc. | Self-correcting controller systems and methods of limiting the operation of neural networks to be within one or more conditions |
US10324467B1 (en) | 2017-12-29 | 2019-06-18 | Apex Artificial Intelligence Industries, Inc. | Controller systems and methods of limiting the operation of neural networks to be within one or more conditions |
US10802488B1 (en) | 2017-12-29 | 2020-10-13 | Apex Artificial Intelligence Industries, Inc. | Apparatus and method for monitoring and controlling of a neural network using another neural network implemented on one or more solid-state chips |
US10795364B1 (en) * | 2017-12-29 | 2020-10-06 | Apex Artificial Intelligence Industries, Inc. | Apparatus and method for monitoring and controlling of a neural network using another neural network implemented on one or more solid-state chips |
WO2019210275A2 (en) * | 2018-04-26 | 2019-10-31 | David Schie | Event driven mathematical engine and method |
CN110458288A (en) * | 2019-08-08 | 2019-11-15 | 单菊平 | Data forecasting system, method and device based on wavelet neural network |
US11366434B2 (en) | 2019-11-26 | 2022-06-21 | Apex Artificial Intelligence Industries, Inc. | Adaptive and interchangeable neural networks |
US11367290B2 (en) | 2019-11-26 | 2022-06-21 | Apex Artificial Intelligence Industries, Inc. | Group of neural networks ensuring integrity |
US10691133B1 (en) | 2019-11-26 | 2020-06-23 | Apex Artificial Intelligence Industries, Inc. | Adaptive and interchangeable neural networks |
US10956807B1 (en) | 2019-11-26 | 2021-03-23 | Apex Artificial Intelligence Industries, Inc. | Adaptive and interchangeable neural networks utilizing predicting information |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4591980A (en) * | 1984-02-16 | 1986-05-27 | Xerox Corporation | Adaptive self-repairing processor array |
US5058184A (en) * | 1985-11-02 | 1991-10-15 | Nippon Hoso Kyokai | Hierachical information processing system |
US5241620A (en) * | 1991-01-03 | 1993-08-31 | Promised Land Technologies, Inc. | Embedding neural networks into spreadsheet applications |
US5422961A (en) * | 1992-04-03 | 1995-06-06 | At&T Corp. | Apparatus and method for improving recognition of patterns by prototype transformation |
US5500905A (en) * | 1991-06-12 | 1996-03-19 | Microelectronics And Computer Technology Corporation | Pattern recognition neural network with saccade-like operation |
US5588091A (en) * | 1989-05-17 | 1996-12-24 | Environmental Research Institute Of Michigan | Dynamically stable associative learning neural network system |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0366804B1 (en) * | 1988-03-25 | 1997-12-10 | Hitachi, Ltd. | Method of recognizing image structures |
US4941122A (en) * | 1989-01-12 | 1990-07-10 | Recognition Equipment Incorp. | Neural network image processing system |
US5274714A (en) * | 1990-06-04 | 1993-12-28 | Neuristics, Inc. | Method and apparatus for determining and organizing feature vectors for neural network recognition |
US5197114A (en) * | 1990-08-03 | 1993-03-23 | E. I. Du Pont De Nemours & Co., Inc. | Computer neural network regulatory process control system and method |
JP2747104B2 (en) * | 1990-10-22 | 1998-05-06 | 株式会社東芝 | neural network |
JPH04160463A (en) * | 1990-10-24 | 1992-06-03 | Hitachi Ltd | Optimizing method by neural network |
US5214715A (en) * | 1991-01-31 | 1993-05-25 | Trustees Of Boston University | Predictive self-organizing neural network |
US5239594A (en) * | 1991-02-12 | 1993-08-24 | Mitsubishi Denki Kabushiki Kaisha | Self-organizing pattern classification neural network system |
US5239593A (en) * | 1991-04-03 | 1993-08-24 | Nynex Science & Technology, Inc. | Optical pattern recognition using detector and locator neural networks |
US5235673A (en) * | 1991-04-18 | 1993-08-10 | International Business Machines Corporation | Enhanced neural network shell for application programs |
US5303330A (en) * | 1991-06-03 | 1994-04-12 | Bell Communications Research, Inc. | Hybrid multi-layer neural networks |
US5452400A (en) * | 1991-08-30 | 1995-09-19 | Mitsubishi Denki Kabushiki Kaisha | Method of optimizing a combination using a neural network |
US5303329A (en) * | 1991-12-10 | 1994-04-12 | Synaptics, Incorporated | Continuous synaptic weight update mechanism |
US5299285A (en) * | 1992-01-31 | 1994-03-29 | The United States Of America As Represented By The Administrator, National Aeronautics And Space Administration | Neural network with dynamically adaptable neurons |
CA2149913A1 (en) * | 1992-11-24 | 1994-06-09 | James David Keeler | Method and apparatus for operating a neural network with missing and/or incomplete data |
US5424959A (en) * | 1993-07-19 | 1995-06-13 | Texaco Inc. | Interpretation of fluorescence fingerprints of crude oils and other hydrocarbon mixtures using neural networks |
US5692107A (en) * | 1994-03-15 | 1997-11-25 | Lockheed Missiles & Space Company, Inc. | Method for generating predictive models in a computer system |
US5701400A (en) * | 1995-03-08 | 1997-12-23 | Amado; Carlos Armando | Method and apparatus for applying if-then-else rules to data sets in a relational data base and generating from the results of application of said rules a database of diagnostics linked to said data sets to aid executive analysis of financial data |
US5729662A (en) * | 1995-06-07 | 1998-03-17 | Rozmus; J. Michael | Neural network for classification of patterns with improved method and apparatus for ordering vectors |
-
1996
- 1996-01-26 US US08/592,767 patent/US5845271A/en not_active Expired - Lifetime
-
1997
- 1997-01-17 EP EP97906423A patent/EP0892957A4/en not_active Withdrawn
- 1997-01-17 JP JP9526939A patent/JP2000504129A/en not_active Ceased
- 1997-01-17 AU AU21120/97A patent/AU716593B2/en not_active Ceased
- 1997-01-17 CA CA002243120A patent/CA2243120C/en not_active Expired - Fee Related
- 1997-01-17 WO PCT/US1997/000886 patent/WO1997027525A2/en active Application Filing
-
1998
- 1998-05-15 US US09/079,731 patent/US5852816A/en not_active Expired - Lifetime
- 1998-05-15 US US09/079,987 patent/US6014653A/en not_active Expired - Lifetime
- 1998-05-15 US US09/079,730 patent/US5852815A/en not_active Expired - Lifetime
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4591980A (en) * | 1984-02-16 | 1986-05-27 | Xerox Corporation | Adaptive self-repairing processor array |
US5058184A (en) * | 1985-11-02 | 1991-10-15 | Nippon Hoso Kyokai | Hierachical information processing system |
US5588091A (en) * | 1989-05-17 | 1996-12-24 | Environmental Research Institute Of Michigan | Dynamically stable associative learning neural network system |
US5241620A (en) * | 1991-01-03 | 1993-08-31 | Promised Land Technologies, Inc. | Embedding neural networks into spreadsheet applications |
US5500905A (en) * | 1991-06-12 | 1996-03-19 | Microelectronics And Computer Technology Corporation | Pattern recognition neural network with saccade-like operation |
US5422961A (en) * | 1992-04-03 | 1995-06-06 | At&T Corp. | Apparatus and method for improving recognition of patterns by prototype transformation |
Non-Patent Citations (4)
Title |
---|
ANTONINI R., "Spreadsheet Simulation of Artificial Neural Network", IJCNNS-91, July 1991. * |
PROCEEDINGS OF THE 1990 SYMPOSIUM ON APPLIED COMPUTING, April 1990, WALTER et al., "A Spreadsheet Method for Studying Neural Networks", pages 42-44. * |
PROCEEDINGS: THE FIRST INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE ON WALL STREET, October 1991, FREEDMAN et al., "Expert Systems in Spreadsheets: Modelling the Ewall Street Under Domain", pages 296-301. * |
See also references of EP0892957A2 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU734922B2 (en) * | 1997-07-18 | 2001-06-28 | International Business Machines Corporation | Optimizer with neural network estimator |
CN111914984A (en) * | 2019-05-07 | 2020-11-10 | 财团法人成大研究发展基金会 | Artificial neural network normalization system for identification device |
CN111914984B (en) * | 2019-05-07 | 2023-10-27 | 财团法人成大研究发展基金会 | Artificial neural network normalization system for identification device |
Also Published As
Publication number | Publication date |
---|---|
AU716593B2 (en) | 2000-03-02 |
AU2112097A (en) | 1997-08-20 |
EP0892957A2 (en) | 1999-01-27 |
WO1997027525A3 (en) | 1997-09-04 |
CA2243120A1 (en) | 1997-07-31 |
US5852816A (en) | 1998-12-22 |
JP2000504129A (en) | 2000-04-04 |
CA2243120C (en) | 2005-10-18 |
US5852815A (en) | 1998-12-22 |
US6014653A (en) | 2000-01-11 |
US5845271A (en) | 1998-12-01 |
EP0892957A4 (en) | 2007-01-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
AU716593B2 (en) | Non-algorithmically implemented artificial neural networks and components thereof | |
Bornholdt et al. | General asymmetric neural networks and structure design by genetic algorithms | |
Kaparthi et al. | An improved neural network leader algorithm for part-machine grouping in group technology | |
WO2006104896A2 (en) | Method, system and computer program for developing cortical algorithms | |
Kumara et al. | Function-to-structure transformation in conceptual design: an associative memory-based paradigm | |
Yin et al. | Applying neural net technology for multi-objective land use planning | |
AU739502B2 (en) | Neural network based database scanning system and method | |
AU739330B2 (en) | Neural network based data examining system and method | |
Schiffmann et al. | Performance evaluation of evolutionarily created neural network topologies | |
AU739197B2 (en) | Neural network based prototyping system and method | |
Cho et al. | Modular neural networks evolved by genetic programming | |
WO2001035336A1 (en) | A sequence generator | |
Moon et al. | Automatic generation of group technology families during the part classification process | |
Rust¹ et al. | Activity-based pruning in developmental artificial neural networks | |
Praczyk | Procedure application in assembler encoding | |
Cho | Evolutionary modular neural networks for intelligent systems | |
Willshaw | Non-symbolic approaches to artificial intelligence and the mind | |
Nicoletti | An analysis of neural networks as simulators and emulators | |
Cho et al. | Emergence of structure and function in evolutionary modular neural networks | |
Rust et al. | Towards computational neural systems through developmental evolution | |
McDonell et al. | Neural network construction using evolutionary search | |
Kim et al. | Modeling a class of decision problems using artificial neural networks | |
Fulcher | Experience with teaching a graduate neural networks course | |
Plagianakos et al. | Improved learning of neural nets through global search | |
Taylor | THE QUESTIONS |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AU CA DE GB IL JP RU |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
ENP | Entry into the national phase |
Ref document number: 2243120 Country of ref document: CA Ref country code: CA Ref document number: 2243120 Kind code of ref document: A Format of ref document f/p: F |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1997906423 Country of ref document: EP |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
WWP | Wipo information: published in national office |
Ref document number: 1997906423 Country of ref document: EP |