WO1995024684A1

WO1995024684A1 - A neural network

Info

Publication number: WO1995024684A1
Application number: PCT/DK1995/000105
Authority: WO
Inventors: Preben ALSTRØM; Dimitris Elias Stassinopoulos
Original assignee: Alstroem Preben; Dimitris Elias Stassinopoulos
Priority date: 1994-03-08
Filing date: 1995-03-08
Publication date: 1995-09-14
Also published as: DE69504316T2; DE69504316D1; DK0749601T3; ATE170303T1; EP0749601A1; US5857177A; AU1889895A; EP0749601B1

Abstract

A neural network has a plurality of network units and a plurality of network connections which connect each network unit to one or more other network units. Each network unit has control parameters in the form of an associated threshold value and/or signal distribution when a signal is supplied to other network units, and supplies a signal to the output in response to a comparison between said threshold value and a signal received on the input. One or more network units serve as network inputs, which supply an output representation in dependence on the sensed parameters applied to the network input. The network has associated with it sensor units which register changes in the conditions under which the network works, and the control parameters of the network units are regulated in dependence on this.

Description

A neural network

The invention concerns a neural network of the type stated in the introductory portion of claim 1.

Neural networks are used for data processing purposes on the basis of a plurality of complexely related input parameters to give the best possible response thereto without necessarily knowing the relation between the indi- vidual input parameters. This is extremely advantageous when no such linear relation exists.

The starting point of neural networks is the ability of the human brain to identify the most important parameters in a decision-making process and to draw correct conclu¬ sions by experience. Neural networks are therefore con¬ structed according to the same basic principles as the human brain, comprising a multitude of decision-making cells or neurons as well as connections or synapses between these.

In order to make the best possible decisions artificial neural networks therefore go through a comprehensive learning procedure before they are used in practice, and the experience thus acquired is utilized for adjusting the control parameters for the neurons and the synapses.

As far as the neurons are concerned, these control para¬ meters comprise a threshold value which determines whether the neuron concerned fires or applies an electric pulse after having received corresponding pulses from other neu¬ rons. The fired pulses are transferred via one or more synapses to other neurons, and the strength or the ampli¬ tude of the individual pulses transferred is one of the adjustable control parameters in the network. A plurality of learning approaches is known, by means of which the parameters can be established according to given applications, which takes place prior to putting the network into service.

These include e.g. EP-A-492 641, which discloses a neural network and a learning procedure for it. The learning procedure comprises submitting to the network an input data signal and a learning signal containing both desired and undesired data. Hereby the network will subsequently respond more expediently. If, e.g., the network controls a process system, it would be fatal if the network would e.g. cause an increase in temperature owing to specific input data which would in turn result in an increase in pressure, if the pressure was then above the value which the system could stand.

US-5 107 454 discloses a neural network for use in pattern recognition, and this network is based on feedback, since the learning procedure is iterative, which means that the pattern concerned and the subsequent intermediate result patterns are run through the network.

EP-A-405 174 discloses a learning method which ensures that the network does not "drop" in a local maximum, which might happen if the correlation between input and output data in a region was of a certain size. The data proces¬ sing system could thus not get out of these conditions, and further adaptation would not be possible.

US-5 010 512 discloses implementation of a neural network as MOSFET transistor elements. The neurons may here be regarded as being threshold switches having several in¬ puts. The network can be operated in two modes, a learning mode in which the control parameters of the neurons are adjusted, and an associative mode in which the control parameters are constant.

It may be said about prior art neural networks that following completed learning procedure they respond in the same manner to the same data sets, notwithstanding the surrounding in which the network controls a process, traffic or the like, change.

Accordingly, the object of the invention is to provide a new generation of neural networks which possess greater adaptability, and where the network is currently capable of adapting to new conditions as well as new or changed surroundings.

The object of the invention is achieved in that the net¬ work has associated to it sensor unit which register changes in the condition under which the network works and adjust the control parameters of the network units in response thereto and in response to whether the network unit received and applied a signal prior to the evaluated response.

The network of the invention can therefore be used in tasks where the possible results are not known, or where they are difficult to define, and the network can current¬ ly adapt itself to variations in the task as well as changes in the desired goals. While prior art neural net¬ works are normally adapted to a given task, it may said that the network of the invention automatically adapts itself, the modes of the network units and the strengths of the compounds between these being currently updated during the execution of a task.

While prior art may be said to work with a learning mode and an associative memory mode, respectively, the network of the invention may be said to be in a performing mode, since the network continues its learning during the exe¬ cution of a task. Even though no external feedback to a network according to the invention takes place, it will still possess advantages over prior art neural networks because of a local reinforcement rule and a simple eva¬ luative feedback in the learning.

The network units are preferably constructed in layers, and all the network units in a layer receive signals from network units in the preceding layer and emit signals to network units in the subsequent layer. It is hereby ensured that a direct signal transfer path is established in the network without any risk of loops being formed internally in the network.

The signals which the network units emit to each other via the network connections are formed by short pulses, and the pulse height is one of the parameters that can be con¬ trolled when the surroundings of the network change dynamically. When the network of the invention is implemented electronically, the pulse height regulation will normally be incorporated in the network connections or the synapses. In other words, the network units may be considered as being pulse transmitters, and the pulses propagate through the network in a chain reaction. The time it takes for the pulses to propagate through the net¬ work, i.e. from when a parameter is received on the input of the network until it provides a response on the output of the network, may be considered as being the response time of the network.

One of the essential elements in the network of the inven¬ tion is an internal activity control, e.g. implemented through a self-monitoring threshold value or strength adjustment. The network, which may be called a performing network, differs from other known, artificial networks precisely by this activity control. In a preferred embodi¬ ment of the invention, it is attempted to keep down the total number of pulses in the network. This is done by using an internal control mechanism which keeps down the total number of pulses. The total signal from a region which can e.g. be identified with the output region, is detected. If this exceeds the unit signal, i.e. if more than one output unit fires, then the threshold value in¬ creases and if the total signal is below the unit signal (no signal), then the threshold value decreases.

If it is the entire firing frequency of the network that is monitored, it is desired that only few neurons in each layer fire. The threshold value can hereby be controlled so that the number of firing neurons approaches a number of the same order as the number of layers. This means that the number of firing neurons will be small with respect to the total number of neurons of the network, but will be sligthly larger than the number of neuron layers in the network. The network will hereby be very sensitive to changes in the dynamic conditions of the external system.

Since the network has a natural response time (transport of signal from input to output), this mechanism gives rise to a complex oscillating activity signal, e.g. represented by the threshold value. It is this property that gives the network its adaptability.

The sensitivity of the network is increased precisely because there is path of firing neurons through the net¬ work. If the input or feedback is changed, more new paths will be formed, and when new stationary modes of the external system occur, these new paths will be narrowed again so that there will again be a path through the net- work. Each output of the network will be associated with a predetermined action in a preferred embodiment. Corre- spondingly, the inputs of the network will be associated with sensing of corresponding action-specific parameters. This gives an unsurpassed adaptability to the network, since the networks adapts itself to the use concerned. Therefore, the network does not have to be tied to a specific use, it being possible to connect the inputs and the outputs randomly to the external system, since with time the feedback signal optimizes the parameters of the network to the given purpose.

The invention also concerns a method of optimizing a neu¬ ral network to changed conditions of operation, said method being characterized by the features defined in claim 8.

It should be noted that the threshold value in the pre¬ ferred embodiment is global, i.e. the same for all network neurons, while the strength of the connections is local.

It will appear from the detailed embodiments that the net¬ work of the invention lends itself for adaptation to a given application, without it having been adapted to the given task beforehand. However, the invention is particu¬ larly useful for tasks in which the surroundings change their nature. When the network has adapted itself to a given application, it can also adapt itself to changes occurring in that connection. The invention will thus be extremely useful in connection with networks in which learning takes place in a special learning phase, since the network of the invention will still be able to adapt itself to dynamic variations in the external system.

The invention will be explained more fully below in connection with the preferred embodiment and in connection with various examples of applications with reference to the drawing, in which: fig. 1 is a schematic view of a preferred embodiment of a performing network according to the invention;

fig. 2 shows a network unit in one layer and network units in other layers connected therewith;

fig. 3 is a schematic view of the coupling between the neurons of the network;

figs. 4a-d are schematic views of interesting details of the control principles in connection with the preferred embodiment of a performing network according to the invention;

fig. 5 shows the efficiency of a performing network according to the invention in connection with a first examples of application;

figs. 6a-c illustrate the activity of the network;

fig. 7 illustrates the power spectrum of the network shown in fig. 5;

figs. 8 and 9 show the efficiency of the performing net- work according to the invention in connection with a second and a third examples of application; and

fig. 10 shows the power spectrum of the network whose performance is shown in figs. 8 and 9.

Fig. 1 shows a preferred embodiment of a neural network according to the invention, and the network, which is of the performance type, will be explained below. It will be seen that the network is constructed as a matrix of net- work units 1 or neurons, each network unit 1 being connected to one or more other network units 1 through network connections 2 or synapses. In the embodiment shown in fig. 1, the neural network is layered, the network units 1 in one layer being connected to a plurality of network units 1 in the adjacent layers. Thus, a network unit 1 receives signals or firings from network units 1 in the previous layer and emits signals or firings to network units in the following layer. Neural networks of this type are called feed forward networks. It will be seen that the network has an input layer of network units, and these are designated 1. in the the drawing. These network units 1. are supplied in a manner known per se with an input signal in the form of the parameters to which the network is desirably to respond or react in response to the "expe¬ rience" which the network has. There is likewise an output layer having network units 1 which supply a response or a representation in dependence on the parameter complex applied to the input layer which is to be evaluated.

It is moreover shown schematically that the network is connected to an external system 3, and this external system may e.g. be a process which is controlled by the network. The network can control many different systems, examples being traffic control, control of foreign exchange transactions, control of meat quality, etc. The network units 1. on the input of the network receive data from the external system via a connection 5. These received data have been processed in advance in a manner known per se, so that data are represented in a correct data format. In dependence on this, the network provides the output with a response to the data applied, and this response is supplied via a connection 4 to the external system. This response will then contain instructions on the manner in which the external system is to be affected to achieve the desired processes. It will additionally be seen that there is a feedback connection 6 from the external system, and this feedback signal affects the mode of operation of the network itself, while ordinary input data transferred via the connection 5 solely affect the decision-making basis for the network. It should be noted here that the feedback signal from the external system contains an evaluation of whether the system control of the network itself is good or bad. To prevent useful information from getting lost among random firings from the network units, it is generally desirable that the firing level is kept as low as possible, typically just a few firings in each network layer. The outputs from the neurons may be coupled to a multiplier so that the signals are added. The output from this may be coupled to a comparator by means of which the amount of the firing may be coupled with a desired threshold value, and the thres- hold value of the network is adjusted in response to this.

Optimization of the number of firing neurons to a pre¬ determined desired number makes the network very sensitive to changes in the external system. It may be said that the network becomes sensitive to "criticism" from the external system.

A performing network according to the invention has four dynamic elements which vary in terms of time.

The first one of these is the network units comprising both network units contained in the actual neural network and units contained in the input and output layers. The temporal mode variation of the network units corresponds to prior art known per se, since the variation solely depends on the signal applied to the input of the network. The network units are pulse transmitters which fire or emit a pulse, if the received complete signal exceeds a given threshold value T. Thus, the network units may be described by a mode n(t), where n(t) at a given time t may be expressed by: n = 0 when no pulse is emitted, and n » 1 when a pulse is emitted. To use pulse transmitters as net¬ work units is essential according to the invention, but, naturally, it is no innovation per se.

One of the essential elements in a performing network is the activity control, e.g. in the form of the self-moni¬ toring threshold T(t). Here the performing network differs significantly from other artificial neural networks. The threshold value may e.g. be controlled in that the total signal from a plurality of network units in a region, e.g. the output region, is detected and compared with a desired signal, the threshold value being then adjusted in response to this comparison. This may mean that if the total signal exceeds the unit signal, i.e. if more than one output unit fires, then the threshold value increases, and if the total signal is below and there is thus no signal, then the threshold value decreases. This new idea is an internal control mechanism where the total number of pulses is kept down. The network may be composed of transistor logics, which will explained below, and it will therefore have a certain response time, i.e. the time it takes from the application of a signal to the input of the network to the production of a response to this signal. This mechanism will thus cause the threshold value to be complexely oscillating. It is i.a. this property that gives the network its adaptability. It should be mentioned here that the change in the threshold value must be relatively small over a period corresponding to the response time of the network.

The third dynamic element of a performing network is the connection between the individual network units. A con¬ nection between two units just permits pulse firing in one direction. A strength q(t) is associated with each of these connections, q being located in the range between 0 and 1. The strength q thus determines the size of a possible pulse which is transmitted through the connection concerned. Each network unit 1 is connected to other pulse transmitters, from which signals are received, through input connections. If the total, received signal exceeds the threshold value concerned, the network unit transmits a pulse which is spread through the output connections, the sum of the strengths of the output connections being precisely one. The strength concept is well-known in connection with artificial neural networks of a known type in which the strength values are updated by pattern learning. It is here desired to strengthen the connections that lead to correct patterns. This frequently takes place globally according to a complicated mathematical set of rules. The strength values are currently updated in per- forming networks too, and this is done according to a simple local rule. Connections providing a positive feed¬ back will be strengthened according to a performing net¬ work according to the invention, while connections pro¬ viding a negative feedback are correspondingly weakened. This is done according to the invention by currently and locally updating or adjusting the connections where pulses are transmitted sequentially between two units.

The performing network of the invention is connected to a peripherial user-selected system via an input and an out¬ put as well as a feedback which is essential to the in¬ vention and which may be described with a value r(t). This feedback will either be positive or negative in response to whether the network has performed its task well or bad- ly, a d it may therefore be described as being either r = r > 0 for a positive feedback or r = r_ < 0 for a nega¬ tive feedback. Greater differentiation may occur.

The feedback signal r(t) will normally not be stepped, since the individual feedbacks will be delta-shaped or in the form of voltage peaks. If the feedback signal occurs periodically, the signal r(t) may have the shape of a periodically jumping function with segments of exponential functions with a negative argument.

Referring back to fig. 1, it will be seen that each net¬ work unit in the central part of the network is connected to three other network units in the preceding layer and to three network units in the subsequent layer. This is illustrated in fig. 2, and it will be seen that a network unit 10 has three input connections from three network units 11-13 in the preceding layer with the states (n-., Tϊ_y , I.,. ) , and each of these input connections have the strength (q., q₂, q₃)« Correspondingly, the network unit 10 has three output connections with three network units 14-16 in the subsequent layer, and these network units 14- 16 have the states (n., n₂, n_), and the connections to these have the strengths (q.,, q,, q₃). Thus, the network unit receives a signal:

q₁(t)n₁(t) + q₂(t)n₂(t) + q₃(t)n₃(t), and if this expression is greater than the threshold value T(t), the state of the unit will be n(t + Δt) = 1, and a pulse is transmitted. Otherwise, n(t + Δt) = 0. If more than one network unit in the output layer fires, then the threshold value increases by a size δ, which means that the threshold value may be expressed by:

T(t + Δt) = T(t) + δ.

If no network units in the output layer fires, then the threshold value decreases by a corresponding size, and a corresponding expression may be set up. If precisely one network unit in the output layer fires, the threshold value will not be changed, and the following expression of the threshold value t may be set up: T( t + Δt ) = T( t ) .

The strengths q→_^ , q₂, and q₃ are updated according to the preferred embodiment of the invention in the following manner:

^ql ^{→ q}l ^{= q}l^{(t) + r(t)f(Q[}ι^(t))n(t)n₁ ⁽t + ^Δt⁾.

It is noted that r(t)f(q(t)) is just included in the adjustment of the strength of a connection when the two neurons, which the connection connects, fire successively so that n(t) and n,(t + Δt) both assume the value 1.

Similar expressions may be set up for q_ and q«, and the function f(q) is here selected as a positive function which assumes the value 0 at the extremes of the range, and in this case f(q) may assume the value q(l-q). Further, noise in the system must also be taken into consideration.

The strength values are fixed subsequently, so that the sum of strengths from a neuron assumes the value 1. Then the strength may be expressed by:

Similar expressions may be set up for the strengths q₂ and q₃. In the preferred embodiment the feedback value r(t) has one of two values at any time, and this value may either be expressed as r = r₊ > 0 or r = r_ < 0 for a positive feedback and a negative feedback, respectively. If no output unit fires, the value is r = r . However, this is optional. Fig. 3 shows a functional diagram of the coupling between the neurons in two network layers. A network unit 1 receives a signal on the input which is the sum of the input signals from three network units in the preceding layer. In a preferred embodiment, a network unit will be a flip-flop manufactured by IC technology.

Fig. 4a shows a D-flip-flop, where the output signal on the Q output follows the input signal on the D input in the followng manner: Q . = D . The threshold value of the network unit is incorporated by subtracting from the input signal V_IN a threshold value voltage corresponding to the threshold value VTh concerned and adding to said input signal a bias corresponding to the threshold value voltage ^Vflip-flop ^{θf he fli}P"^fl°P- ^τhis 9^ives V_D = V_IN - V_{th +}

^Vflip-flop- ^{If V}D ^is 9^{reater than V}flip-fiop' ^{the fl±}P^" flop recognizes the input signal as being a 1 (D = 1), and a corresponding 1 is applied on the output Q. If, on the other hand, V_Q is smaller than V_f, •_ _£•_] , the flip-flop recognizes the input signal as being a 0. Correspondingly, a 0 is applied on the output.

In a preferred embodiment, the output signal from the flip-flop is a delta voltage or a voltage pulse. The strength of a connection between two neurons or network units is adjusted as follows. The output signals 27, 28 of the two neurons (see fig. 4b) are passed to a logic AND gate 29, and the signal from this is integrated in an integrator 30, which may e.g. be an operation amplifier having an integrated feedback. The signal from the integrator 30 is used for adjusting the strength of the connection.

The actual strength adjustment takes place in the connection strength regulator 25, which is shown in fig.

3. The actual connection strength regulator 25 is shown in detail in fig. 4c, and it is the input VI that receives the strength regulating signal from the integrator 30.

The strength regulating element here consists of a dual gate MOSFET transistor. The gain changing, slowly varying input is passed to the gate electrode of the transistor through a resistor Rl. The firing signal is likewise passed to the gate electrode of the transistor in which DC components, if any, are removed by a decoupling capacitor CI. The resistor R2 is adjusted together with the resistor Rl, R3 and R4 to provide suitable working conditions for the transistor. The output 01 and 02 are inverted and non- inverted, respectively. The signal on the output 02 corre¬ sponds to the signal on the input V2, but with a changed strength or amplitude.

The signal on 02 is passed from the connection strength regulator 25 to a multiplier 26 shown in fig. 3. The firing signal is multiplied here with the feedback signal r(t), and it is worth noting that the AND gate 29 ensures that only connections between firing neurons are strengthened, as long as the network works successfully. The output from the multiplier 26 is coupled to a network unit 1 in the next layer of network units.

It is shown in fig. 4d how the feedback signal r(t) is generated. The actual neural network 34 forms part of a computing unit, which moreover has an input/output unit 35 which is connected via respective interfaces 37, 38 to the external system. The interface 37 generates input para¬ meters for the neural network, while the interface 38 controls the external system in response to the output of the network. A sensing unit 31 monitors the external system, and this sensing unit 31 may e.g. be an optical sensing unit that supplies a signal to the calculating unit. If the neural network tries to minimize the distance to the movement of a given object, the sensing unit 31 may supply a signal which represents the distance concerned. This signal is applied to a central unit 32 which compares this signal to a previously measured value of the dis- tance, and this value is obtained from a storage 33. If this distance has been made smaller, the central unit 32 supplies a positive feedback to the neural network 34, and the feedback value concerned is likewise obtained from the storage 33. If, on the other hand, the distance has in- creased, a negative feedback signal is supplied to the neural network 34.

In connection with a first example of application (fig. 5) the network is used for choosing a randomly selected acti- vity among many, said activity being considered desirable. The network is constructed as a matrix having 64 x 64 net¬ work elements, and there are thus 64 outputs. If the net¬ work selects the correct action, a positive feedback is applied to the network, and the positive feedback may be r = r = 0.01. If the network makes a wrong selection, the feedback will correspondingly be negative, and it may be r = r_ = -0.1. The performance P of the network is the part of the active actions of the network which is desired. Fig. 5 shows how the performance P of the network, which has been averaged over 1000 time intervals, as well as the threshold value T changes with time. It will be seen that the variation of the threshold value is confined between about 0.35 and 0.50. It is also noted that the performance P increases rapidly from 0 to about 0.8, and the value of P fluctuates until it reaches 1.

Fig. 6 shows how the average activity slowly narrows. The activity has been collected over 1000 time intervals and is concentrated within a range 20, which is slowly narrowed down to a single path 21. The neurons of the network will then form a pattern of firing neurons until either the input signal or the feedback signal changes.

Inset in fig. 5 is a section of the temporal fluctuations in the feedback signal r and the threshold value T. The signal with more than 25000 time intervals was divided into 50 segments having 1024 intervals each (i.e. with 50% overlap), and a standard Parsen window was used for obtaining the power spectrum from these segments. Quan¬ titatively, it will be seen that the power spectrum S(f) of the fluctuating reaction signal (feedback signal) exhibits a 1/f relation over a wide temporal range: S(f) = constant f~ , where α is approximately equal to 1.1.

Inset in fig. 7, the distribution D( →τ ) is determined on the basis of five independent runs of the time intervals T, in which r is constant (r = r or r = r_). In this case, this distribution may be expressed by

D(τ) = constant T -1.2

These graphs show that the adaptive performance in the process of learning leads to behavioural changes on a great variety of lapses of time.

In connection with the next example of application the network is caused to find and follow a movable target, which is first assumed to move along a straigth line. In this case the network is formed by network units arranged in a 16 x 16 matrix, in which each unit in the output layer is associated with a motion of the sighting point in a predetermined direction and with a given size. The adjacent unit may thus be related to various motions. If several units in the output layer fire simultaneously, the motion is taken as the weighted average value. If the sighting point moves away from the object, the feedback will be negative, otherwise the feedback will be positive. Fig. 8 shows the linear motion of the object in dashed line, while the sighting point is marked by a solid line, and this coincides with the motion of the object after a short sequence. The variations in the threshold value are likewise shown. The inset image shows how the sighting follows the object, and how the feedback signal reacts during this sequence.

Fig. 9 shows the same case, but here the objects move along a sine curve. Here, too, the sighting point rela¬ tively rapidly captures the object and follows it. Here, too, the feedback signal is shown in an inset image, and both the movement of the object and the movement of the sighting point may be followed. It is important to the invention that the feedback of the system does not lock, but constantly changes between negative and positive. It is primarily this that gives the performing network its adaptability and performance.

Fig. 10 shows the power spectrum of the applications shown in figs. 8 and 9, and it will be noted that peak values occur in the power spectrum S(f) and in the distribution D(r), indicating that some time intervals occur particu¬ larly frequently during the performing process. According to the example of application with the linear motion of the object there are two peak values in the distribution D(r), a narrow peak being present at r = 15 and a wide peak around r₂ = 80. A separation of the distribution D(τ) in two distributions D_(r) and D₊(τ) of a negative feed- back and a positive feedback, respectively, shows that r. is related to a negative feedback, and ₂ is related to a positive feedback. This indicates that peaks may occur in the distribution when the network is in the learning mode. Although just relatively simple examples of application have been discussed above, it will be clear to a skilled person that the network, although it has not been specially adapted to the given tasks, has been able to adapt itself on the basis of the feedback signal, which regulates the control parameters of the network units. The performance P of the network has increased to a satisfac- tory level. Although it is still worthwhile subjecting neural networks to a learning process, the parameters being adjusted at the satisfactory starting level, the performance network of the invention is unique in that it is capable of adapting itself to changes in the surround- ings. It will here be possible to compensate for ageing of the process system or to compensate for the possible circumstance that one or more input parameters for the network are vitiated by errors.

Claims

P a t e n t C l a i m s:

1. A neural network having a plurality of network units and a plurality of network connections which connect each network unit to one or more other network units;

said network units being associated with control para¬ meters in the form of threshold values and strength levels for signals applied on the output of the respective net¬ work unit;

said network units supplying signals on the output in dependence on signals received on the input and the asso- ciated threshold value;

said network having a network input where one or more net¬ work units receive a parameter representation applied to the network as well as a network output where one or more network units supply the response of the network in dependence on the supplied parameter representation;

means being provided for evaluating the response of the network and applying an evaluation signal in dependence thereon, said evaluation signal being used for regulating the control parameters for a group of network units;

c h a r a c t e r i z e d in that the network has an in¬ put for the evaluation signal which is passed via this in- put to the group of network units having adjustable con¬ trol parameters; and

the control parameters of the individual network units are regulated in dependence on the common evaluation signal and in dependence on whether the individual network unit transmitted and received a signal prior to the evaluated response .

2. A neural network according to claim 1, c h a r a c ¬ t e r i z e d in that the network is layered, and that all network units in a layer receive signals from network units in a preceding layer and apply signals to network units in a subsequent layer.

3. A neural network according to claim 1 or 2, c h a - r a c t e r i z e d in that the signals on the output of the network units are formed by short pulses or firings, and that the height of a transferred pulse depends on the strength level associated with the firing network units.

4. A neural network according to claim 3, c h a r a c ¬ t e r i z e d in that the threshold value of the indivi¬ dual network units is controlled in dependence on the firing frequency somewhere in the network.

5. A neural network according to claims 1-4, c h a ¬ r a c t e r i z e d in that the relative change in the threshold value over a period corresponding to the response time of the network is small.

6. A neural network according to claims 1-5, c h a ¬ r a c t e r i z e d in that the evaluation signal is supplied as a feedback signal.

7. A neural network according to claim 6, c h a r a c - t e r i z e d in that the strength level of a network connection is affected most by the evaluation signal when the network unit both transmits and receives firing sig¬ nals.

8. A method of optimizing a neural network having dyna¬ mically varying operating conditions, said neural network comprising a plurality of network units and a plurality of network connections which connect each network unit to one or more other network units;

said network units being associated with control para¬ meters in the form of threshold values and strength levels of signals applied to the output of the respective network unit;

said network units supplying signals on the output in dependence on the signal received on the input and in dependence on associated threshold values;

said network having a network input where one or more net- work units receive a parameter representation applied to the network, as well as a network output where one or more network units supply the response of the network in dependence on the parameter representation applied;

the response of the network being moreover evaluated, and an evaluation signal being applied in dependence on this for regulating the control parameters for a group of network units;

c h a r a c t e r i z e d by passing the evaluation sig¬ nal to the group of network units having adjustable con¬ trol parameters; and

regulating the control parameters of the individual net- work units in dependence on the common evaluation signal and in dependence on whether the individual network unit transmitted and received a signal prior to the evaluated response.