i Iiifi uiiiiii 111 iiiti mit mil Iiiil mil Iilk urn urn Iuul m um Eiii
United States Patent  [li] Patent Number: 5,283,855
Motomura et al.  Date of Patent: Feb. 1,1994
 NEURAL NETWORK AND METHOD FOR TRAINING THE NEURAL NETWORK
 Inventors: Shuji Motomura; Toshiyuki Furuta;
Hirotoshi Eguchi, all of Yokahama
 Assignee: Ricoh Company, Ltd., Tokyo, Japan
 Appl. No.: 795,952
 Filed: Not. 21,1991
 Foreign Application Priority Data
Nov. 22, 1990 [JP] Japan 2-320472
Apr. 23, 1991 [JP] Japan 3-091876
Jul. 19, 1991 [JP] Japan 3-203420
 Int. CI.' 395 23; 395 27;
 U.S. CI 395/27; 395/23
 Field of Search 395/23, 27, 24
 References Cited
U.S. PATENT DOCUMENTS
4,893,255 1/1990 Tomlinson, Jr., et al 395/24
5,101,361 3/1992 Eberhardt 395/24
5,129,039 7/1992 Hiraiwa 395/24
5,131,073 7/1992 Furuta et al 395/27
5,167,006 11/1992 Furuta et al 395/11
FOREIGN PATENT DOCUMENTS
1244567 9/1989 Japan .
Murry et al "Asynchronous VLSI Neural Networks Using Pulse-Stream Arithmetic", 1988 IEEE. McCluskey "Logic Design Principles," Prentice-Hall, 1986, Front Inside Cover.
1990 Spring National Convention Record, The Institute of Electronics, Information and Communication Engineers, vol. D-56, pp. 6-56, T. Furuta, et al. "Neuron Model Using Pulse Density And Its Self-Learning Circuit".
The Institute of Electronics, Information and Communication Engineers, Oct. 15, 1990, T. Furuta, et al., "Pulse Density Neuron Model Having Learning Function and How To Construct It By Hardwares".
A method and apparatus are disclosed that modify [ies] and generalize [s] the use in artificial neural networks of the error backpropagation algorithm. Each neuron unit first divides a plurality of weighted inputs into more than one group, then sums up weighted inputs in each group to provide each group's intermediate outputs, and finally processes the intermediate outputs to produce an output of the neuron unit. Since the method uses, when modifying each weight, a partial differential coefficient generated by partially-differentiating the output of the neuron unit by each weighted input, the weight can be properly modified even if the output of a neuron unit as a function of intermediate outputs has a plurality of variables corresponding to the number of groups. Since the conventional method uses only one differentia] coefficient, that is, the differential coefficient of the output of a neuron unit differentiated by the sum of all weighted inputs in a neuron unit, for all weights in a neuron unit, it may be said that the method according to the present invention generalizes the conventional method. The present invention is especially useful for pulse density neural networks which express data as an ON-bit density of a bit string.