CA2421928C - Anomaly detection system and a method of teaching it - Google Patents
Anomaly detection system and a method of teaching it Download PDFInfo
- Publication number
- CA2421928C CA2421928C CA2421928A CA2421928A CA2421928C CA 2421928 C CA2421928 C CA 2421928C CA 2421928 A CA2421928 A CA 2421928A CA 2421928 A CA2421928 A CA 2421928A CA 2421928 C CA2421928 C CA 2421928C
- Authority
- CA
- Canada
- Prior art keywords
- input data
- time
- learning mechanism
- presentation
- behaviour
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Abstract
A method for teaching an anomaly detecting mechanism in a system comprising observable elements (302), at least one of which has a periodic time-dependent behaviour, the anomaly detecting mechanism comprising a computerized learning mechanism (314). The method comprises assembling indicators (304) indicating the behaviour of the elements (302) and arranging the assembled indicators such that each observable element's indicators are assigned to the same input data component. The learning mechanism (314) is taught so that the input data of the learning mechanism comprises the input data components which are based on the assembled indicators (304). Points which approximate the input data are placed in the input space. A presentation of time (420 - 424) is incorporated into at least one input data component wherein the presentation of time is periodic, continuous and unambiguous within the period of the at least one element with periodic time-dependent behaviour.
Description
Anomaly detection system and a method of teaching it Background of the invention The invention relates to anomaly detection in a computer and tele-communication networks in which the concept of normal behaviour varies with time. More particularly, the invention relates especially to teaching an anomaly detection mechanism. An example of such an anomaly detection mechanism is based on self-organizing maps (SOM).
Mechanisms for detecting abnormal situations belong to one of two major categories, namely rule-based detection mechanisms and anomaly de-tection mechanisms. Rule-based detection mechanisms attempt to recognize certain behaviour patterns which are known to be improper. Thus, rule-based detection mechanisms have two severe limitations: they can only detect prob-lems which have occurred before and which have been explicitly taught to the detection system or programmed into it. Anomaly detection systems (ADS), as used in this application, reverse the detection problem: they are taught what normal behaviour is, and anything deviating significantly (by a predetermined margin) from the norm is considered anomalous. ADS mechanisms are capa-ble of detecting potentially problematic situations without explicit training of such situations. An example of an ADS is disclosed in reference 1. Thus an ADS is defined as a mechanism which is trained with normal behaviour of the target system. Accordingly, an ADS flags every significant deviation from nor-mal as a potential anomaly. In contrast, a rule-based detection system is trained with known modes of abnormal behaviour and it can only detect the problems that have been taught to it.
Reference 1 discloses an ADS for a Unix-based computer system.
The system consists of a data-gathering component, a user-behaviour visuali-zation component, an automatic anomaly detection component and a user in-terface. The system reduces the amount of data necessary for anomaly detec-tion by selecting a set of features which characterizes user behaviour in the system. The automatic anomaly detection component approximates users' daily profiles with self-organizing maps (SOM), originally created by Teuvo Ko-honen. A crucial parameter of an SOM is a Best Mapping Unit (BMU) distance.
The BMUs of the SOMs are used to detect deviations from the daily profiles. A
measure of such deviations is expressed as an anomaly P-value. According to reference 1, the ADS has been tested and found capable of detecting a wide range of anomalous behaviour.
2 .
A problem with known SOM-based ADS mechanisms is that they are restricted to detecting problems in systems having a well-defined normal behaviour. In most telecommunication networks the concept of "normal behav-iour" is, at best, vague. A network element's behaviour at peak time is very dif-ferent from its behaviour at the quiet hours just before dawn. More precisely, most often it is the users who cause the variation in what is called normal.
In other words, known ADS mechanisms do not readily lend themselves to de-tecting problems in systems or elements whose normal behaviour varies with time.
Disclosure of the invention Accordingly, it is an object of the invention to provide a mechanism for teaching ADS mechanisms which rely on the concept of normal behaviour in a system in which the normal behaviour varies significantly with time. In this context, "significantly" means that a behaviour which is normal at certain times is to be considered anomalous at other times.
This object is achieved with a method and equipment which are characterized by what is disclosed in the attached independent claims. Pre-ferred embodiments of the invention are disclosed in the attached dependent claims.
The invention is partially based on the idea that time is used as a component of the input data to the ADS. But it is not sufficient to include time in the input data, if time is represented as a quantity which increases linearly from a fixed start point. This is because such a presentation of time is not re-peating, and the ADS would not know when a certain behaviour was normal and when anomalous. It is also not sufficient to introduce time as a periodic quantity (such as a 24-hour clock) because the daily jumps from 23:59 to 00:00 would introduce severe discontinuities to the input data.
Accordingly, the invention is also based on formulating a presenta-tion of time which is suitable for solving the problem caused by the time-varying normal behaviour of systems such as telecommunication networks.
According to the invention, the presentation of time which is used as a compo-nent of the input data is 1) periodic, 2) continuous and 3) unambiguous (within the period of the input data). A preferred example of such a presentation of time (t) is a projection to x and y components such that x=sin(27ct/L) and y=cos(2Tct/L) where L is the length of the period of variation, typically 24 hours or a week. At first sight, such a two-dimensional presentation of time would seem to use both dimensions of a two-dimensional SOM map, but such SOM
maps are for visualization purposes only, and inside a computer memory, an SOM map can have an arbitrary number of dimensions.
The continuity requirement for the presentation of time should be in-terpreted with the constraints of reality in, mind, however. All digital systems have a finite resolution, which means that no presentation of time can be per-fectly continuous. In addition, some memory can be saved when storing the observations by omitting some of the least significant bits of the observations, ie by quantization. For the purposes of the invention, a presentation of time is sufficiently continuous (="large-scale continuous") if it does not contain discon-tinuities which are large enough to affect a decision between normal and anomalous behaviour. For example, in a telecommunication network with a usage period of 24 hours, discontinuities (quantizations) of up to about 10 or minutes may be considered acceptable if.there are no times at which user 15 behaviour changes so fast that a certain type of behaviour is considered nor-mal at a certain point of time but anomalous 10 or 15 minutes later. In con-trast, the presentation of time for a system which opens and closes (or radi-cally changes its behaviour in other ways) at well-defined times must have considerably smaller discontinuities.
Some memory can be saved if it is known beforehand that changes in the behaviour of the observable elements are small and/or gradual during certain parts of the period (such as nights) and more pronounced during other parts (such as days). In such a case, the presentation of time can be such that the resolution is variable within the period. This means that one bit may repre-sent, say, 30 minutes during the quiet parts of the period and 5 - 15 during the more active parts of the period.
In some cases a single period (typically 24 hours) is sufficient, but sometimes two or three nested periods may be required. For example, the presentation of time may comprise one component with a 24-hour period and 3o another with a one-week period. For locations or situations strongly affected by seasonal changes, a third component with a one-year period may be required.
The invention is not limited to self-organizing maps but can be used with other clustering techniques such as k-means and Learning Vector Quanti-zation.
Mechanisms for detecting abnormal situations belong to one of two major categories, namely rule-based detection mechanisms and anomaly de-tection mechanisms. Rule-based detection mechanisms attempt to recognize certain behaviour patterns which are known to be improper. Thus, rule-based detection mechanisms have two severe limitations: they can only detect prob-lems which have occurred before and which have been explicitly taught to the detection system or programmed into it. Anomaly detection systems (ADS), as used in this application, reverse the detection problem: they are taught what normal behaviour is, and anything deviating significantly (by a predetermined margin) from the norm is considered anomalous. ADS mechanisms are capa-ble of detecting potentially problematic situations without explicit training of such situations. An example of an ADS is disclosed in reference 1. Thus an ADS is defined as a mechanism which is trained with normal behaviour of the target system. Accordingly, an ADS flags every significant deviation from nor-mal as a potential anomaly. In contrast, a rule-based detection system is trained with known modes of abnormal behaviour and it can only detect the problems that have been taught to it.
Reference 1 discloses an ADS for a Unix-based computer system.
The system consists of a data-gathering component, a user-behaviour visuali-zation component, an automatic anomaly detection component and a user in-terface. The system reduces the amount of data necessary for anomaly detec-tion by selecting a set of features which characterizes user behaviour in the system. The automatic anomaly detection component approximates users' daily profiles with self-organizing maps (SOM), originally created by Teuvo Ko-honen. A crucial parameter of an SOM is a Best Mapping Unit (BMU) distance.
The BMUs of the SOMs are used to detect deviations from the daily profiles. A
measure of such deviations is expressed as an anomaly P-value. According to reference 1, the ADS has been tested and found capable of detecting a wide range of anomalous behaviour.
2 .
A problem with known SOM-based ADS mechanisms is that they are restricted to detecting problems in systems having a well-defined normal behaviour. In most telecommunication networks the concept of "normal behav-iour" is, at best, vague. A network element's behaviour at peak time is very dif-ferent from its behaviour at the quiet hours just before dawn. More precisely, most often it is the users who cause the variation in what is called normal.
In other words, known ADS mechanisms do not readily lend themselves to de-tecting problems in systems or elements whose normal behaviour varies with time.
Disclosure of the invention Accordingly, it is an object of the invention to provide a mechanism for teaching ADS mechanisms which rely on the concept of normal behaviour in a system in which the normal behaviour varies significantly with time. In this context, "significantly" means that a behaviour which is normal at certain times is to be considered anomalous at other times.
This object is achieved with a method and equipment which are characterized by what is disclosed in the attached independent claims. Pre-ferred embodiments of the invention are disclosed in the attached dependent claims.
The invention is partially based on the idea that time is used as a component of the input data to the ADS. But it is not sufficient to include time in the input data, if time is represented as a quantity which increases linearly from a fixed start point. This is because such a presentation of time is not re-peating, and the ADS would not know when a certain behaviour was normal and when anomalous. It is also not sufficient to introduce time as a periodic quantity (such as a 24-hour clock) because the daily jumps from 23:59 to 00:00 would introduce severe discontinuities to the input data.
Accordingly, the invention is also based on formulating a presenta-tion of time which is suitable for solving the problem caused by the time-varying normal behaviour of systems such as telecommunication networks.
According to the invention, the presentation of time which is used as a compo-nent of the input data is 1) periodic, 2) continuous and 3) unambiguous (within the period of the input data). A preferred example of such a presentation of time (t) is a projection to x and y components such that x=sin(27ct/L) and y=cos(2Tct/L) where L is the length of the period of variation, typically 24 hours or a week. At first sight, such a two-dimensional presentation of time would seem to use both dimensions of a two-dimensional SOM map, but such SOM
maps are for visualization purposes only, and inside a computer memory, an SOM map can have an arbitrary number of dimensions.
The continuity requirement for the presentation of time should be in-terpreted with the constraints of reality in, mind, however. All digital systems have a finite resolution, which means that no presentation of time can be per-fectly continuous. In addition, some memory can be saved when storing the observations by omitting some of the least significant bits of the observations, ie by quantization. For the purposes of the invention, a presentation of time is sufficiently continuous (="large-scale continuous") if it does not contain discon-tinuities which are large enough to affect a decision between normal and anomalous behaviour. For example, in a telecommunication network with a usage period of 24 hours, discontinuities (quantizations) of up to about 10 or minutes may be considered acceptable if.there are no times at which user 15 behaviour changes so fast that a certain type of behaviour is considered nor-mal at a certain point of time but anomalous 10 or 15 minutes later. In con-trast, the presentation of time for a system which opens and closes (or radi-cally changes its behaviour in other ways) at well-defined times must have considerably smaller discontinuities.
Some memory can be saved if it is known beforehand that changes in the behaviour of the observable elements are small and/or gradual during certain parts of the period (such as nights) and more pronounced during other parts (such as days). In such a case, the presentation of time can be such that the resolution is variable within the period. This means that one bit may repre-sent, say, 30 minutes during the quiet parts of the period and 5 - 15 during the more active parts of the period.
In some cases a single period (typically 24 hours) is sufficient, but sometimes two or three nested periods may be required. For example, the presentation of time may comprise one component with a 24-hour period and 3o another with a one-week period. For locations or situations strongly affected by seasonal changes, a third component with a one-year period may be required.
The invention is not limited to self-organizing maps but can be used with other clustering techniques such as k-means and Learning Vector Quanti-zation.
According to a preferred embodiment of the invention, all variables (components of the input data), including the presentation of time, are scaled such that the variance of each variable is the same, preferably one.
The invention can be implemented as software routines in a com-puter system having access to the elements to be observed. Ideally, the inven-tive mechanism is comprised in a single network element, such as in an opera-tions and maintenance centre.
Brief description of the drawings The invention will be described in more detail by means of preferred embodiments with reference to the appended drawing wherein:
Figure 1 shows a self-organizing map;
Figure 2 is a variation of Figure 1, with circles centred around the neurons of the SOM;
Figure 3 is a process chart illustrating a preferred embodiment of the invention; and Figure 4A to 4C illustrate different presentations of time.
Detailed description of the invention Preferred embodiments of the invention will be described in connec-tion with self-organizing map (SOM) technology. Figure 1 shows a self-organizing map. The objective with a SOM,test for anomaly is to test if the cur-rent behaviour of an object is anomalous or not. The hypothesis to be tested is:
HO: The most recent observation is not anomalous.
Hl: The most recent observation is anomalous.
The behaviour of an object can be very consistent, which means that it is concentrated to one or a couple of regions in the feature space. On the other hand, the behaviour can also be more scattered in the feature space, which would signify a more irregular behaviour. The idea of the SOM test for anomaly is to approximate the normal behaviour of an object with a small ob-ject-specific SOM. The previous behaviour, is assumed to represent the normal behaviour of the object. Anomalous observations can be omitted from the pre-vious behaviour when training the SOM.
The SOM shown in Figure 1 is a one-dimensional (8*1) SOM with 200 points of artificial data, commonly depicted by reference number 13. Fig-ure 2 shows the same SOM with circles or ellipses 21 plotted using the neu-rons 14 of the SOM as centres. For clarity, Figures 1 and 2 are shown with only two features 11 and 12, but in reality, the number of observable features can be much larger than two.
200 points of artificial data for two features have been plotted in the 5 plane together with the neurons of a map of size 8*1 trained with the data.
The one-dimensional SOM approximates two clusters (having four ellipses 21 each) of data quite well. Note that the data in Figure 1 is two-dimensional to allow visualization to humans. In a computer system, the number of dimen-sions can be much larger than two.
The Best Matching Unit (BMU) for a data point fk in an SOM is the neuron w; having the smallest distance to the data point. This is expressed in equation (1), where dist stands for the distance.
BMU = argmin{dist(fk,w;)} (1) Here, we assume that a Euclidean distance to the BMU is used to measure how much an observation deviates from the normal object-specific behaviour, but other types of distance measurements can be used. The anomaly P-value is a measure of the degree of anomaly for an observation.
On the basis of this value, the hypothesis Ho is accepted or rejected. Calcula-tion of the anomaly P-value will be described in connection with the use phase of the SOM-based ADS.
An ADS mechanism involves three major phases, design, teaching and use. The design phase typically involves human decisions and comprises the following steps:
1. Selecting a set of features describing the target object. The fea-ture vector describing the object is denoted by f. (The target object is the ob-ject to be observed, such as a network element.) This step is described in de-tail in reference 1. For the purposes of the present invention, it suffices to say that the features are parameters which can be used to make a distinction be-tween normal and anomalous behaviour.
2. Formulating a hypothesis for detecting anomalous behaviour.
The objective is to test the most recent observation fõ+1 for anomaly. The hy-pothesis to be tested is Ho: The most recent observation fõ+l is not anomalous.
The alternative hypothesis is Hi: The most recent observation fõ+l is anoma-lous. (The suffix n will be described in connection with the use phase.) The teaching phase typically comprises the following steps:
The invention can be implemented as software routines in a com-puter system having access to the elements to be observed. Ideally, the inven-tive mechanism is comprised in a single network element, such as in an opera-tions and maintenance centre.
Brief description of the drawings The invention will be described in more detail by means of preferred embodiments with reference to the appended drawing wherein:
Figure 1 shows a self-organizing map;
Figure 2 is a variation of Figure 1, with circles centred around the neurons of the SOM;
Figure 3 is a process chart illustrating a preferred embodiment of the invention; and Figure 4A to 4C illustrate different presentations of time.
Detailed description of the invention Preferred embodiments of the invention will be described in connec-tion with self-organizing map (SOM) technology. Figure 1 shows a self-organizing map. The objective with a SOM,test for anomaly is to test if the cur-rent behaviour of an object is anomalous or not. The hypothesis to be tested is:
HO: The most recent observation is not anomalous.
Hl: The most recent observation is anomalous.
The behaviour of an object can be very consistent, which means that it is concentrated to one or a couple of regions in the feature space. On the other hand, the behaviour can also be more scattered in the feature space, which would signify a more irregular behaviour. The idea of the SOM test for anomaly is to approximate the normal behaviour of an object with a small ob-ject-specific SOM. The previous behaviour, is assumed to represent the normal behaviour of the object. Anomalous observations can be omitted from the pre-vious behaviour when training the SOM.
The SOM shown in Figure 1 is a one-dimensional (8*1) SOM with 200 points of artificial data, commonly depicted by reference number 13. Fig-ure 2 shows the same SOM with circles or ellipses 21 plotted using the neu-rons 14 of the SOM as centres. For clarity, Figures 1 and 2 are shown with only two features 11 and 12, but in reality, the number of observable features can be much larger than two.
200 points of artificial data for two features have been plotted in the 5 plane together with the neurons of a map of size 8*1 trained with the data.
The one-dimensional SOM approximates two clusters (having four ellipses 21 each) of data quite well. Note that the data in Figure 1 is two-dimensional to allow visualization to humans. In a computer system, the number of dimen-sions can be much larger than two.
The Best Matching Unit (BMU) for a data point fk in an SOM is the neuron w; having the smallest distance to the data point. This is expressed in equation (1), where dist stands for the distance.
BMU = argmin{dist(fk,w;)} (1) Here, we assume that a Euclidean distance to the BMU is used to measure how much an observation deviates from the normal object-specific behaviour, but other types of distance measurements can be used. The anomaly P-value is a measure of the degree of anomaly for an observation.
On the basis of this value, the hypothesis Ho is accepted or rejected. Calcula-tion of the anomaly P-value will be described in connection with the use phase of the SOM-based ADS.
An ADS mechanism involves three major phases, design, teaching and use. The design phase typically involves human decisions and comprises the following steps:
1. Selecting a set of features describing the target object. The fea-ture vector describing the object is denoted by f. (The target object is the ob-ject to be observed, such as a network element.) This step is described in de-tail in reference 1. For the purposes of the present invention, it suffices to say that the features are parameters which can be used to make a distinction be-tween normal and anomalous behaviour.
2. Formulating a hypothesis for detecting anomalous behaviour.
The objective is to test the most recent observation fõ+1 for anomaly. The hy-pothesis to be tested is Ho: The most recent observation fõ+l is not anomalous.
The alternative hypothesis is Hi: The most recent observation fõ+l is anoma-lous. (The suffix n will be described in connection with the use phase.) The teaching phase typically comprises the following steps:
1. Observing normal behaviour of the target object. For example, n measurements (f1, f2, . . . , fõ) of the feature vector are collected.
2. Training an SOM with m neurons using the measurements (fl, f2, .... fõ) as training data. The number of neurons in the map, m, is selected to be much smaller than n, for example n/10.
The use phase typically comprises the following steps:
1. Omitting neurons in the SOM that are not Best Mapping Units (BMU) for any of the data points (f1, f2, . . . , fn).
2. Calculating the BMU distances for (fi, f2, . . . , fõ) from the trained SOM. These distances are denoted by (Di, D2, ... , Dn).
3. Calculating the BMU distance for the observation fõ+1. This dis-tance is denoted by Dõ+1.
4. Calculating the anomaly P-value. Let B be the number of the Best Mapping Unit distances (DI, D2, ... Dõ) higher than Dõ+1. The anomaly P-value for a certain object is then calculated from:
Põ+1 = n (2) 5. Accepting or rejecting the null hypothesis on the basis of the anomaly P-value. If the anomaly P-value is higher than the anomaly P-value threshold, the null hypothesis Ho is accepted (the most recent observation is considered normal). If, on the other hand, the anomaly P-value is smaller than the anomaly P-value threshold, the null hypothesis Ho is rejected and the most recent data point is assumed anomalous.
If the test indicates that the object behaviour is anomalous (Ho is re-jected), the k most significantly deviating features can be determined. The k features (components of the feature vector) with the biggest absolute contribu-tion to the BMU distance are the k most significantly deviating features. Equa-tion (3) shows how the most deviating feature can be calculated. This compo-nent of the feature vector is given the sub-index and in equation (3). In equa-tion (3) BMU stands for the Best Mapping Unit of the feature vector fõ+1, and j takes values from zero to the number of features. The other k-1 most deviating features are calculated in a corresponding manner.
f imid = arg max{abs(fr+1 j - BMU) } (3) The situation shown in Figure 1 can be used as an example. Figure 1 shows two anomalies, commonly depicted with reference numeral 15. The anomaly P-value for anomaly 1 is 0/200 = 0. Since none of the BMU distances for the data points have a BMU distance greater than that of anomaly 1, the value of the numerator is zero. Correspondingly, the anomaly P-value for anomaly 2 is 7/200 = 0.035.
If the Anomaly P-value is smaller than the Anomaly P-value thresh-old, the null hypothesis Ho is rejected and an alarm is triggered. The Anomaly P-value threshold can be interpreted as the fraction of observations that will be rejected if the behaviour of the monitored object does not deviate from the the same object's earlier behaviour which was used during the teaching phase.
That is, if the null hypothesis is true:
number of alarms = P-value threshold * observations (4) On the other hand, if the null hypothesis is not true (the new data is anomalous), the number of rejections (alarms) is higher.
Figure 2 shows how a selected P-value threshold can be illustrated for object i using d-dimensional spheres (d-spheres) centred at the neurons of the object-specific map. With two-dimensional input data, the d-spheres are circles. Here d stands for the number of dimensions in the input data (f1, f2, .. .
, fn). In other words, each input data element f, through fn is itself a vector with d dimensions. The number of observations for object i falling outside the spheres corresponds to the numerator B in equation (2). The two-dimensional example in Figure 2 shows such a situation. Here B is 13, which corresponds to quite high a P-value threshold of about 6.50.
Figure 3 is a process chart illustrating a preferred embodiment of the invention. Reference number 302 points to an element of a physical sys-tem such as a telecommunication network (as distinguished from a neural network). A physical element may comprise several observable elements. For example, if the physical system element 302 is a telecommunication ex-change, its observable elements may comprise throughput, waiting time, num-ber (or percentage) of failed calls and the like. For each unit of time, an indica-tor collector 306 collects an indicator tuple 304. The tuples are stored in an in-dicator database 310. Reference 312 points to a data set used for training the neural network (or another learning mechanism) 314. The data set 312 should indicate normal behaviour of the physical element 302. A storage 318 contains trained neural networks. When a physical element 302 is to be observed, the corresponding trained neural network 320 is retrieved from the storage 318 and applied as one input to the anomaly detection mechanism 322. The anomaly detection mechanism's other input is the indicator set 324 to be tested for anomalous behaviour. If the anomaly detection mechanism 322 de-cides that the behaviour described by the indicator set 324 is anomalous, the anomaly P-value and the most deviating indicators 326 are stored in an anomaly history database 328. At the same time, an alarm 330 is given to a monitoring device 332, such as a computer screen.
Figures 4A to 4C illustrate different presentations of time, some of which are acceptable and some unacceptable. In Figure 4A, the horizontal axis is the time in units of L where L is the period of input data, which is as-sumed to be 24 hours. Line 400 shows a straight presentation of time. Refer-ences 401 to 403 point to three instances of a repeating event which occurs at 24-hour intervals. A problem with this presentation of time is that the presenta-tions of the times are different, and the ADS cannot recognize events 401 to 403 as a recurring event.
The saw-tooth line 405 is a 24-hour presentation of time, or in other words, a modulo function of time. In this presentation, events occurring at the same time each day have identical representations, but the day changes in-troduce discontinuities into the input data.
In Figure 4B, the sine wave 410 is periodic and continuous, but it is not ambiguous. Events 411 and 412 occur at different times but have identical presentations of time. Assuming that event 411 was normal in the morning, the ADS would not recognize a similar event as an anomaly if it occurred in the evening.
Figure 4C shows three acceptable presentations of time. They are all based on the idea that time is represented as a coordinate pair x,y. The cir-cle 420 represents time as {x=sin(2irt/L); y=cos(2ict/L)} where L is the length of the variation period, and 2tt/L is an angle from the x axis. The ellipse 422 is also acceptable as long as it is not too flat to introduce an ambiguity as to whether a point is on the top half or the bottom half of the ellipse. Even a rec-tangle 424 can be used. Although several points have identical x or y coordi-nates, no two points of the rectangle have identical x/y coordinate pairs.
The sine/cosine combination of the circle 420 is considered a pre-ferred presentation of time because events which are equidistant in time are also equidistant in the presentation of time. However, the sinetcosine combination may be computationally intensive, and some approximations, such as a pair of triangular wave functions with a 90-degree phase shift, can be used. As stated earlier, in some situations the presentation of time may require more than one component. For example, there may be up to three sine/cosine pairs with periods of 24 hours, one week and one year.
Although preferred embodiments of the invention have been de-scribed in connection with neural networks and self-organizing maps, the in-vention is not limited to these examples. As an alternative, the invention can Io be generalized to other clustering techniques such as k-means and Learning Vector Quantization, in which case the neurons are replaced by codebook vec-tors.
Reference:
1. Hoglund, Albert: An Anomaly Detection System for Computer Networks, Master of Science thesis, Helsinki University of Technology 1997
2. Training an SOM with m neurons using the measurements (fl, f2, .... fõ) as training data. The number of neurons in the map, m, is selected to be much smaller than n, for example n/10.
The use phase typically comprises the following steps:
1. Omitting neurons in the SOM that are not Best Mapping Units (BMU) for any of the data points (f1, f2, . . . , fn).
2. Calculating the BMU distances for (fi, f2, . . . , fõ) from the trained SOM. These distances are denoted by (Di, D2, ... , Dn).
3. Calculating the BMU distance for the observation fõ+1. This dis-tance is denoted by Dõ+1.
4. Calculating the anomaly P-value. Let B be the number of the Best Mapping Unit distances (DI, D2, ... Dõ) higher than Dõ+1. The anomaly P-value for a certain object is then calculated from:
Põ+1 = n (2) 5. Accepting or rejecting the null hypothesis on the basis of the anomaly P-value. If the anomaly P-value is higher than the anomaly P-value threshold, the null hypothesis Ho is accepted (the most recent observation is considered normal). If, on the other hand, the anomaly P-value is smaller than the anomaly P-value threshold, the null hypothesis Ho is rejected and the most recent data point is assumed anomalous.
If the test indicates that the object behaviour is anomalous (Ho is re-jected), the k most significantly deviating features can be determined. The k features (components of the feature vector) with the biggest absolute contribu-tion to the BMU distance are the k most significantly deviating features. Equa-tion (3) shows how the most deviating feature can be calculated. This compo-nent of the feature vector is given the sub-index and in equation (3). In equa-tion (3) BMU stands for the Best Mapping Unit of the feature vector fõ+1, and j takes values from zero to the number of features. The other k-1 most deviating features are calculated in a corresponding manner.
f imid = arg max{abs(fr+1 j - BMU) } (3) The situation shown in Figure 1 can be used as an example. Figure 1 shows two anomalies, commonly depicted with reference numeral 15. The anomaly P-value for anomaly 1 is 0/200 = 0. Since none of the BMU distances for the data points have a BMU distance greater than that of anomaly 1, the value of the numerator is zero. Correspondingly, the anomaly P-value for anomaly 2 is 7/200 = 0.035.
If the Anomaly P-value is smaller than the Anomaly P-value thresh-old, the null hypothesis Ho is rejected and an alarm is triggered. The Anomaly P-value threshold can be interpreted as the fraction of observations that will be rejected if the behaviour of the monitored object does not deviate from the the same object's earlier behaviour which was used during the teaching phase.
That is, if the null hypothesis is true:
number of alarms = P-value threshold * observations (4) On the other hand, if the null hypothesis is not true (the new data is anomalous), the number of rejections (alarms) is higher.
Figure 2 shows how a selected P-value threshold can be illustrated for object i using d-dimensional spheres (d-spheres) centred at the neurons of the object-specific map. With two-dimensional input data, the d-spheres are circles. Here d stands for the number of dimensions in the input data (f1, f2, .. .
, fn). In other words, each input data element f, through fn is itself a vector with d dimensions. The number of observations for object i falling outside the spheres corresponds to the numerator B in equation (2). The two-dimensional example in Figure 2 shows such a situation. Here B is 13, which corresponds to quite high a P-value threshold of about 6.50.
Figure 3 is a process chart illustrating a preferred embodiment of the invention. Reference number 302 points to an element of a physical sys-tem such as a telecommunication network (as distinguished from a neural network). A physical element may comprise several observable elements. For example, if the physical system element 302 is a telecommunication ex-change, its observable elements may comprise throughput, waiting time, num-ber (or percentage) of failed calls and the like. For each unit of time, an indica-tor collector 306 collects an indicator tuple 304. The tuples are stored in an in-dicator database 310. Reference 312 points to a data set used for training the neural network (or another learning mechanism) 314. The data set 312 should indicate normal behaviour of the physical element 302. A storage 318 contains trained neural networks. When a physical element 302 is to be observed, the corresponding trained neural network 320 is retrieved from the storage 318 and applied as one input to the anomaly detection mechanism 322. The anomaly detection mechanism's other input is the indicator set 324 to be tested for anomalous behaviour. If the anomaly detection mechanism 322 de-cides that the behaviour described by the indicator set 324 is anomalous, the anomaly P-value and the most deviating indicators 326 are stored in an anomaly history database 328. At the same time, an alarm 330 is given to a monitoring device 332, such as a computer screen.
Figures 4A to 4C illustrate different presentations of time, some of which are acceptable and some unacceptable. In Figure 4A, the horizontal axis is the time in units of L where L is the period of input data, which is as-sumed to be 24 hours. Line 400 shows a straight presentation of time. Refer-ences 401 to 403 point to three instances of a repeating event which occurs at 24-hour intervals. A problem with this presentation of time is that the presenta-tions of the times are different, and the ADS cannot recognize events 401 to 403 as a recurring event.
The saw-tooth line 405 is a 24-hour presentation of time, or in other words, a modulo function of time. In this presentation, events occurring at the same time each day have identical representations, but the day changes in-troduce discontinuities into the input data.
In Figure 4B, the sine wave 410 is periodic and continuous, but it is not ambiguous. Events 411 and 412 occur at different times but have identical presentations of time. Assuming that event 411 was normal in the morning, the ADS would not recognize a similar event as an anomaly if it occurred in the evening.
Figure 4C shows three acceptable presentations of time. They are all based on the idea that time is represented as a coordinate pair x,y. The cir-cle 420 represents time as {x=sin(2irt/L); y=cos(2ict/L)} where L is the length of the variation period, and 2tt/L is an angle from the x axis. The ellipse 422 is also acceptable as long as it is not too flat to introduce an ambiguity as to whether a point is on the top half or the bottom half of the ellipse. Even a rec-tangle 424 can be used. Although several points have identical x or y coordi-nates, no two points of the rectangle have identical x/y coordinate pairs.
The sine/cosine combination of the circle 420 is considered a pre-ferred presentation of time because events which are equidistant in time are also equidistant in the presentation of time. However, the sinetcosine combination may be computationally intensive, and some approximations, such as a pair of triangular wave functions with a 90-degree phase shift, can be used. As stated earlier, in some situations the presentation of time may require more than one component. For example, there may be up to three sine/cosine pairs with periods of 24 hours, one week and one year.
Although preferred embodiments of the invention have been de-scribed in connection with neural networks and self-organizing maps, the in-vention is not limited to these examples. As an alternative, the invention can Io be generalized to other clustering techniques such as k-means and Learning Vector Quantization, in which case the neurons are replaced by codebook vec-tors.
Reference:
1. Hoglund, Albert: An Anomaly Detection System for Computer Networks, Master of Science thesis, Helsinki University of Technology 1997
Claims (15)
1. A method, comprising:
assembling indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator corresponding to each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time-dependent behaviour;
programming a computerized learning mechanism of an anomaly detecting mechanism in the system such that input data of the learning mechanism comprises input data components which are based on the assembled indicators, and wherein the learning mechanism comprises an input space to define the input data comprising the input data components;
placing points which approximate the input data in the input space; and incorporating a presentation of time into at least one input data component, wherein the presentation of time is periodic, continuous and unambiguous within a period of the at least one element with periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
assembling indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator corresponding to each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time-dependent behaviour;
programming a computerized learning mechanism of an anomaly detecting mechanism in the system such that input data of the learning mechanism comprises input data components which are based on the assembled indicators, and wherein the learning mechanism comprises an input space to define the input data comprising the input data components;
placing points which approximate the input data in the input space; and incorporating a presentation of time into at least one input data component, wherein the presentation of time is periodic, continuous and unambiguous within a period of the at least one element with periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
2. A method according to claim 1, wherein the learning mechanism is or comprises a self-organizing map.
3. A method according to claim 1, wherein the presentation of time has a first period and at least one second period which is a multiple of the first period.
4. A method according to claim 1, further comprising:
scaling the input data components such that each component has the same variance, preferably one.
scaling the input data components such that each component has the same variance, preferably one.
5. A method according to claim 1, wherein the presentation of time has a variable resolution such that one bit corresponds to different units of time depending on changes in the time-dependent behaviour.
6. A system, comprising:
a computerized learning mechanism comprising an input space for defining input data comprising input data components;
means for assembling indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator corresponding to each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time-dependent behaviour;
means for programming the learning mechanism such that the input data of the learning mechanism comprises the input data components which are based on the assembled indicators;
means for placing points which approximate the input data in the input space;
and at least one input data component comprising a presentation of time to detect anomalies in the system, wherein the presentation of time is periodic, continuous and unambiguous within the period of the at least one element with periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
a computerized learning mechanism comprising an input space for defining input data comprising input data components;
means for assembling indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator corresponding to each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time-dependent behaviour;
means for programming the learning mechanism such that the input data of the learning mechanism comprises the input data components which are based on the assembled indicators;
means for placing points which approximate the input data in the input space;
and at least one input data component comprising a presentation of time to detect anomalies in the system, wherein the presentation of time is periodic, continuous and unambiguous within the period of the at least one element with periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
7. A system according to claim 6, wherein the learning mechanism is or comprises a self-organizing map.
8. A system according to claim 6, wherein the presentation of time has a first period and at least one second period which is a multiple of the first period.
9. A system according to claim 6, wherein the system is comprised in a single network element.
10. A computer readable medium with a computer program embodied thereon, the computer program being configured to control a processor to perform:
setting up an input space for defining input data comprising input data components;
receiving indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator of each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time dependent behaviour;
programming a computerized learning mechanism of an anomaly detecting mechanism in the system such that input data of the learning mechanism comprises input data components which are based on the assembled indicators;
placing points which approximate the input data in the input space; and incorporating a presentation of time into at least one input data component, wherein the presentation of time is periodic, continuous, and unambiguous within a period of the at least one element with periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
setting up an input space for defining input data comprising input data components;
receiving indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator of each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time dependent behaviour;
programming a computerized learning mechanism of an anomaly detecting mechanism in the system such that input data of the learning mechanism comprises input data components which are based on the assembled indicators;
placing points which approximate the input data in the input space; and incorporating a presentation of time into at least one input data component, wherein the presentation of time is periodic, continuous, and unambiguous within a period of the at least one element with periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
11. A system, comprising:
a computerized learning mechanism configured to comprise an input space to define input data comprising input data components;
an assembling unit configured to assemble indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator corresponding to each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time-dependent behaviour;
a programming unit configured to program the learning mechanism such that the input data of the learning mechanism comprises the input data components which are based on the assembled indicators;
a placing unit configured to place points which approximate the input data in the input space; and at least one input data component comprising a presentation of time to detect anomalies in the system, wherein the presentation of time is periodic, continuous and unambiguous within the period of the at least one element with the periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
a computerized learning mechanism configured to comprise an input space to define input data comprising input data components;
an assembling unit configured to assemble indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator corresponding to each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time-dependent behaviour;
a programming unit configured to program the learning mechanism such that the input data of the learning mechanism comprises the input data components which are based on the assembled indicators;
a placing unit configured to place points which approximate the input data in the input space; and at least one input data component comprising a presentation of time to detect anomalies in the system, wherein the presentation of time is periodic, continuous and unambiguous within the period of the at least one element with the periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
12. A system according to claim 11, wherein the learning mechanism is or comprises a self-organizing map.
13. A system according to claim 11, wherein the presentation of time has a first period and at least one second period which is a multiple of the first period.
14. A system according to claim 11, wherein the system is comprised in a single network element.
15. A computer readable medium with a computer program product comprising a program code embodied thereon, said program code being configured to control a processor to perform:
setting up an input space for defining input data comprising input data components;
receiving indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator of each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time dependent behaviour;
programming a computerized learning mechanism of an anomaly detecting mechanism in the system such that input data of the learning mechanism comprises input data components which are based on the assembled indicators;
placing points which approximate the input data in the input space; and incorporating a presentation of time into at least one input data component, wherein the presentation of time is periodic, continuous, and unambiguous within a period of the at least one element with periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
setting up an input space for defining input data comprising input data components;
receiving indicators indicating behaviour of observable elements in a system and arranging the assembled indicators such that each indicator of each observable element is assigned to the same input data component, wherein at least one of the observable elements has a periodic time dependent behaviour;
programming a computerized learning mechanism of an anomaly detecting mechanism in the system such that input data of the learning mechanism comprises input data components which are based on the assembled indicators;
placing points which approximate the input data in the input space; and incorporating a presentation of time into at least one input data component, wherein the presentation of time is periodic, continuous, and unambiguous within a period of the at least one element with periodic time-dependent behaviour, and wherein the programmed computerized learning mechanism is configured to detect an anomaly in the system.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| FI20001997A FI114749B (en) | 2000-09-11 | 2000-09-11 | Anomaly detection system and method for teaching it |
| FI20001997 | 2000-09-11 | ||
| PCT/FI2001/000783 WO2002021242A1 (en) | 2000-09-11 | 2001-09-10 | Anomaly detection system and a method of teaching it |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2421928A1 CA2421928A1 (en) | 2002-03-14 |
| CA2421928C true CA2421928C (en) | 2011-11-15 |
Family
ID=8559059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2421928A Expired - Fee Related CA2421928C (en) | 2000-09-11 | 2001-09-10 | Anomaly detection system and a method of teaching it |
Country Status (8)
| Country | Link |
|---|---|
| US (2) | US7519860B2 (en) |
| EP (2) | EP1325588A1 (en) |
| JP (1) | JP4436042B2 (en) |
| CN (1) | CN1196984C (en) |
| AU (2) | AU2001269056A1 (en) |
| CA (1) | CA2421928C (en) |
| FI (1) | FI114749B (en) |
| WO (2) | WO2002021774A1 (en) |
Families Citing this family (106)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FI114749B (en) | 2000-09-11 | 2004-12-15 | Nokia Corp | Anomaly detection system and method for teaching it |
| US6862698B1 (en) | 2002-01-22 | 2005-03-01 | Cisco Technology, Inc. | Method of labeling alarms to facilitate correlating alarms in a telecommunications network |
| US7694190B2 (en) * | 2003-01-16 | 2010-04-06 | Nxp B.V. | Preventing distribution of modified or corrupted files |
| US7016806B2 (en) * | 2003-03-31 | 2006-03-21 | Lucent Technologies Inc. | Method and apparatus for event monitoring in an information processing system |
| US9143393B1 (en) | 2004-05-25 | 2015-09-22 | Red Lambda, Inc. | System, method and apparatus for classifying digital data |
| US7631222B2 (en) * | 2004-08-23 | 2009-12-08 | Cisco Technology, Inc. | Method and apparatus for correlating events in a network |
| US20070028220A1 (en) * | 2004-10-15 | 2007-02-01 | Xerox Corporation | Fault detection and root cause identification in complex systems |
| WO2006056223A1 (en) * | 2004-11-26 | 2006-06-01 | Telecom Italia S.P.A. | Instrusion detection method and system, related network and computer program product therefor |
| CN100440796C (en) * | 2004-12-04 | 2008-12-03 | 华为技术有限公司 | Method for obtaining network key property indication and key property indication module |
| US7996814B1 (en) | 2004-12-21 | 2011-08-09 | Zenprise, Inc. | Application model for automated management of software application deployments |
| US7937197B2 (en) * | 2005-01-07 | 2011-05-03 | GM Global Technology Operations LLC | Apparatus and methods for evaluating a dynamic system |
| US9129226B2 (en) * | 2011-12-04 | 2015-09-08 | Beyondcore, Inc. | Analyzing data sets with the help of inexpert humans to find patterns |
| US10127130B2 (en) | 2005-03-18 | 2018-11-13 | Salesforce.Com | Identifying contributors that explain differences between a data set and a subset of the data set |
| GB0513294D0 (en) * | 2005-06-29 | 2005-08-03 | Nokia Corp | Quality assessment for telecommunications network |
| JP4626852B2 (en) * | 2005-07-11 | 2011-02-09 | 日本電気株式会社 | Communication network failure detection system, communication network failure detection method, and failure detection program |
| JP2008134705A (en) * | 2006-11-27 | 2008-06-12 | Hitachi Ltd | Data processing method and data analysis device |
| US20080229153A1 (en) * | 2007-03-13 | 2008-09-18 | At&T Knowledge Ventures, Lp | System and method of network error analysis |
| EP2003604B1 (en) * | 2007-05-23 | 2018-10-24 | Deutsche Telekom AG | Self-organizing map with virtual map units |
| US20090030752A1 (en) * | 2007-07-27 | 2009-01-29 | General Electric Company | Fleet anomaly detection method |
| US8327395B2 (en) | 2007-10-02 | 2012-12-04 | The Nielsen Company (Us), Llc | System providing actionable insights based on physiological responses from viewers of media |
| US7941382B2 (en) * | 2007-10-12 | 2011-05-10 | Microsoft Corporation | Method of classifying and active learning that ranks entries based on multiple scores, presents entries to human analysts, and detects and/or prevents malicious behavior |
| SG152081A1 (en) * | 2007-10-18 | 2009-05-29 | Yokogawa Electric Corp | Metric based performance monitoring method and system |
| JP2011505175A (en) * | 2007-10-31 | 2011-02-24 | エムセンス コーポレイション | System and method for providing distributed collection and centralized processing of physiological responses from viewers |
| US8793363B2 (en) * | 2008-01-15 | 2014-07-29 | At&T Mobility Ii Llc | Systems and methods for real-time service assurance |
| JP4984162B2 (en) * | 2008-01-17 | 2012-07-25 | 日本電気株式会社 | Monitoring control method and monitoring control apparatus |
| US8676964B2 (en) * | 2008-07-31 | 2014-03-18 | Riverbed Technology, Inc. | Detecting outliers in network traffic time series |
| US8213308B2 (en) | 2008-09-11 | 2012-07-03 | Juniper Networks, Inc. | Methods and apparatus for defining a flow control signal related to a transmit queue |
| US8325749B2 (en) * | 2008-12-24 | 2012-12-04 | Juniper Networks, Inc. | Methods and apparatus for transmission of groups of cells via a switch fabric |
| US8154996B2 (en) | 2008-09-11 | 2012-04-10 | Juniper Networks, Inc. | Methods and apparatus for flow control associated with multi-staged queues |
| US8254255B2 (en) | 2008-12-29 | 2012-08-28 | Juniper Networks, Inc. | Flow-control in a switch fabric |
| US20100169169A1 (en) * | 2008-12-31 | 2010-07-01 | International Business Machines Corporation | System and method for using transaction statistics to facilitate checkout variance investigation |
| US8023513B2 (en) * | 2009-02-24 | 2011-09-20 | Fujitsu Limited | System and method for reducing overhead in a wireless network |
| US7962797B2 (en) * | 2009-03-20 | 2011-06-14 | Microsoft Corporation | Automated health model generation and refinement |
| US8522085B2 (en) * | 2010-01-27 | 2013-08-27 | Tt Government Solutions, Inc. | Learning program behavior for anomaly detection |
| WO2011008198A1 (en) * | 2009-07-14 | 2011-01-20 | Hewlett-Packard Development Company, L.P. | Determining a seasonal effect in temporal data |
| US8516471B2 (en) * | 2009-10-06 | 2013-08-20 | International Business Machines Corporation | Detecting impact of operating system upgrades |
| US20110090820A1 (en) | 2009-10-16 | 2011-04-21 | Osama Hussein | Self-optimizing wireless network |
| US9264321B2 (en) | 2009-12-23 | 2016-02-16 | Juniper Networks, Inc. | Methods and apparatus for tracking data flow based on flow state values |
| EP2438511B1 (en) | 2010-03-22 | 2019-07-03 | LRDC Systems, LLC | A method of identifying and protecting the integrity of a set of source data |
| US8805839B2 (en) | 2010-04-07 | 2014-08-12 | Microsoft Corporation | Analysis of computer network activity by successively removing accepted types of access events |
| US9602439B2 (en) | 2010-04-30 | 2017-03-21 | Juniper Networks, Inc. | Methods and apparatus for flow control associated with a switch fabric |
| US9065773B2 (en) | 2010-06-22 | 2015-06-23 | Juniper Networks, Inc. | Methods and apparatus for virtual channel flow control associated with a switch fabric |
| US8584241B1 (en) * | 2010-08-11 | 2013-11-12 | Lockheed Martin Corporation | Computer forensic system |
| US8553710B1 (en) | 2010-08-18 | 2013-10-08 | Juniper Networks, Inc. | Fibre channel credit-based link flow control overlay onto fibre channel over ethernet |
| US8719930B2 (en) * | 2010-10-12 | 2014-05-06 | Sonus Networks, Inc. | Real-time network attack detection and mitigation infrastructure |
| US8595556B2 (en) | 2010-10-14 | 2013-11-26 | International Business Machines Corporation | Soft failure detection |
| US8683591B2 (en) * | 2010-11-18 | 2014-03-25 | Nant Holdings Ip, Llc | Vector-based anomaly detection |
| US9660940B2 (en) | 2010-12-01 | 2017-05-23 | Juniper Networks, Inc. | Methods and apparatus for flow control associated with a switch fabric |
| US9032089B2 (en) | 2011-03-09 | 2015-05-12 | Juniper Networks, Inc. | Methods and apparatus for path selection within a network based on flow duration |
| US8806645B2 (en) * | 2011-04-01 | 2014-08-12 | Mcafee, Inc. | Identifying relationships between security metrics |
| US8509762B2 (en) | 2011-05-20 | 2013-08-13 | ReVerb Networks, Inc. | Methods and apparatus for underperforming cell detection and recovery in a wireless network |
| US9007919B2 (en) | 2011-07-22 | 2015-04-14 | Empirix Inc. | Systems and methods for network monitoring and testing using dimension value based KPIs |
| EP2754271B1 (en) | 2011-09-09 | 2019-11-13 | Reverb Networks Inc. | Methods and apparatus for implementing a self optimizing-organizing network manager |
| US8811183B1 (en) | 2011-10-04 | 2014-08-19 | Juniper Networks, Inc. | Methods and apparatus for multi-path flow control within a multi-stage switch fabric |
| US9258719B2 (en) | 2011-11-08 | 2016-02-09 | Viavi Solutions Inc. | Methods and apparatus for partitioning wireless network cells into time-based clusters |
| US10796232B2 (en) | 2011-12-04 | 2020-10-06 | Salesforce.Com, Inc. | Explaining differences between predicted outcomes and actual outcomes of a process |
| US10802687B2 (en) | 2011-12-04 | 2020-10-13 | Salesforce.Com, Inc. | Displaying differences between different data sets of a process |
| EP2815541B1 (en) | 2012-02-17 | 2018-06-27 | Osama Tarraf | Methods and apparatus for coordination in multi-mode networks |
| US10268974B2 (en) * | 2012-09-28 | 2019-04-23 | Rex Wiig | System and method of a requirement, compliance and resource management |
| US9953281B2 (en) * | 2012-09-28 | 2018-04-24 | Rex Wiig | System and method of a requirement, compliance and resource management |
| US9246747B2 (en) * | 2012-11-15 | 2016-01-26 | Hong Kong Applied Science and Technology Research Co., Ltd. | Adaptive unified performance management (AUPM) with root cause and/or severity analysis for broadband wireless access networks |
| WO2014183784A1 (en) * | 2013-05-14 | 2014-11-20 | Telefonaktiebolaget L M Ericsson (Publ) | Resource budget determination for communications network |
| US9218570B2 (en) | 2013-05-29 | 2015-12-22 | International Business Machines Corporation | Determining an anomalous state of a system at a future point in time |
| WO2014205421A1 (en) * | 2013-06-21 | 2014-12-24 | Arizona Board Of Regents For The University Of Arizona | Automated detection of insider threats |
| US9414244B2 (en) | 2013-07-22 | 2016-08-09 | Motorola Solutions, Inc. | Apparatus and method for determining context-aware and adaptive thresholds in a communications system |
| GB2517147A (en) | 2013-08-12 | 2015-02-18 | Ibm | Performance metrics of a computer system |
| US9727821B2 (en) | 2013-08-16 | 2017-08-08 | International Business Machines Corporation | Sequential anomaly detection |
| WO2015025379A1 (en) * | 2013-08-21 | 2015-02-26 | 株式会社日立製作所 | Monitoring device, monitoring method, and recording medium |
| TWI510109B (en) * | 2013-09-25 | 2015-11-21 | Chunghwa Telecom Co Ltd | The recursive method of network traffic anomaly detection |
| WO2015077917A1 (en) * | 2013-11-26 | 2015-06-04 | Telefonaktiebolaget L M Ericsson (Publ) | Method and apparatus for anomaly detection in a network |
| US10122747B2 (en) * | 2013-12-06 | 2018-11-06 | Lookout, Inc. | Response generation after distributed monitoring and evaluation of multiple devices |
| FR3015757B1 (en) * | 2013-12-23 | 2019-05-31 | Electricite De France | METHOD FOR QUANTITATIVE ESTIMATING OF THE PLATE COATING OF A STEAM GENERATOR |
| CN103869053B (en) * | 2014-03-24 | 2015-07-15 | 焦振志 | Regional geochemical survey sample analysis and abnormal point sampling inspection method |
| EP3221793B1 (en) * | 2014-11-18 | 2018-10-17 | Vectra Networks, Inc. | Method and system for detecting threats using passive cluster mapping |
| WO2016093836A1 (en) | 2014-12-11 | 2016-06-16 | Hewlett Packard Enterprise Development Lp | Interactive detection of system anomalies |
| US10320824B2 (en) * | 2015-01-22 | 2019-06-11 | Cisco Technology, Inc. | Anomaly detection using network traffic data |
| US9113353B1 (en) | 2015-02-27 | 2015-08-18 | ReVerb Networks, Inc. | Methods and apparatus for improving coverage and capacity in a wireless network |
| EP3304820B1 (en) | 2015-05-27 | 2019-12-18 | Telefonaktiebolaget LM Ericsson (publ) | Method and apparatus for analysing performance of a network by managing network data relating to operation of the network |
| US20170034720A1 (en) * | 2015-07-28 | 2017-02-02 | Futurewei Technologies, Inc. | Predicting Network Performance |
| US10803074B2 (en) | 2015-08-10 | 2020-10-13 | Hewlett Packard Entperprise Development LP | Evaluating system behaviour |
| US10861031B2 (en) | 2015-11-25 | 2020-12-08 | The Nielsen Company (Us), Llc | Methods and apparatus to facilitate dynamic classification for market research |
| US20170167890A1 (en) * | 2015-12-09 | 2017-06-15 | Watersmart Software, Inc. | System and method for providing a platform for detecting pattern based irrigation |
| WO2017108106A1 (en) | 2015-12-22 | 2017-06-29 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and network node for identifiying specific area of wireless communication system |
| US10609587B2 (en) | 2016-05-01 | 2020-03-31 | Teoco Corporation | System, method, and computer program product for location-based detection of indicator anomalies |
| US9942085B2 (en) * | 2016-07-13 | 2018-04-10 | Incelligent P.C. | Early warning and recommendation system for the proactive management of wireless broadband networks |
| US10694487B2 (en) * | 2016-09-15 | 2020-06-23 | Cisco Technology, Inc. | Distributed network black box using crowd-based cooperation and attestation |
| WO2018130284A1 (en) * | 2017-01-12 | 2018-07-19 | Telefonaktiebolaget Lm Ericsson (Publ) | Anomaly detection of media event sequences |
| US10419269B2 (en) | 2017-02-21 | 2019-09-17 | Entit Software Llc | Anomaly detection |
| US11055631B2 (en) * | 2017-03-27 | 2021-07-06 | Nec Corporation | Automated meta parameter search for invariant based anomaly detectors in log analytics |
| US11271960B2 (en) * | 2017-12-06 | 2022-03-08 | Ribbon Communications Operating Company, Inc. | Communications methods and apparatus for dynamic detection and/or mitigation of anomalies |
| US10931696B2 (en) | 2018-07-13 | 2021-02-23 | Ribbon Communications Operating Company, Inc. | Communications methods and apparatus for dynamic detection and/or mitigation of threats and/or anomalies |
| US10548032B2 (en) * | 2018-01-26 | 2020-01-28 | Verizon Patent And Licensing Inc. | Network anomaly detection and network performance status determination |
| US10685652B1 (en) * | 2018-03-22 | 2020-06-16 | Amazon Technologies, Inc. | Determining device groups |
| US11188865B2 (en) * | 2018-07-13 | 2021-11-30 | Dimensional Insight Incorporated | Assisted analytics |
| US11012421B2 (en) | 2018-08-28 | 2021-05-18 | Box, Inc. | Predicting user-file interactions |
| US11921612B2 (en) * | 2018-08-29 | 2024-03-05 | Oracle International Corporation | Identification of computer performance anomalies based on computer key performance indicators |
| US11228506B2 (en) | 2018-09-06 | 2022-01-18 | Hewlett Packard Enterprise Development Lp | Systems and methods for detecting anomalies in performance indicators of network devices |
| US10834106B2 (en) | 2018-10-03 | 2020-11-10 | At&T Intellectual Property I, L.P. | Network security event detection via normalized distance based clustering |
| KR102424694B1 (en) * | 2018-12-26 | 2022-07-25 | 삼성전자주식회사 | Apparatus and method for monitoring performance of network device in wireless communication system |
| CN111949496B (en) * | 2019-05-15 | 2022-06-07 | 华为技术有限公司 | Data detection method and device |
| US11799890B2 (en) * | 2019-10-01 | 2023-10-24 | Box, Inc. | Detecting anomalous downloads |
| US11216666B2 (en) | 2019-12-11 | 2022-01-04 | Fujifilm Business Innovation Corp. | Understanding normality of an environment using semantic information from images |
| WO2022019728A1 (en) * | 2020-07-24 | 2022-01-27 | Samsung Electronics Co., Ltd. | Method and system for dynamic threshold detection for key performance indicators in communication networks |
| US11499892B2 (en) * | 2020-11-30 | 2022-11-15 | Kcf Technologies, Inc. | Optimization for anomaly detection |
| US20220214948A1 (en) * | 2021-01-06 | 2022-07-07 | Kyndryl, Inc. | Unsupervised log data anomaly detection |
| US11558238B1 (en) | 2022-01-08 | 2023-01-17 | Bank Of America Corporation | Electronic system for dynamic latency reduction through edge computation based on a multi-layered mechanism |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FI910512A (en) * | 1991-02-01 | 1992-08-02 | Esko Antero Hirvonen | Kontrollsystem. |
| US5317725A (en) | 1991-03-12 | 1994-05-31 | Hewlett-Packard Company | Landmark data abstraction paradigm to diagnose data communication networks |
| US5819226A (en) * | 1992-09-08 | 1998-10-06 | Hnc Software Inc. | Fraud detection using predictive modeling |
| US5365514A (en) * | 1993-03-01 | 1994-11-15 | International Business Machines Corporation | Event driven interface for a system for monitoring and controlling a data communications network |
| US5446874A (en) * | 1993-12-23 | 1995-08-29 | International Business Machines Corp. | Automated benchmarking with self customization |
| US6006016A (en) * | 1994-11-10 | 1999-12-21 | Bay Networks, Inc. | Network fault correlation |
| ATE191112T1 (en) * | 1995-02-02 | 2000-04-15 | Cabletron Systems Inc | METHOD AND ARRANGEMENT FOR LEARNING BEHAVIOR TRENDS OF NETWORKS AND PREDICTING THE FUTURE BEHAVIOR OF DATA TRANSMISSION NETWORKS |
| GB2303275B (en) * | 1995-07-13 | 1997-06-25 | Northern Telecom Ltd | Detecting mobile telephone misuse |
| EP0849910A3 (en) * | 1996-12-18 | 1999-02-10 | Nortel Networks Corporation | Communications network monitoring |
| GB2321362A (en) * | 1997-01-21 | 1998-07-22 | Northern Telecom Ltd | Generic processing capability |
| US6609217B1 (en) * | 1998-03-30 | 2003-08-19 | General Electric Company | System and method for diagnosing and validating a machine over a network using waveform data |
| US6105149A (en) * | 1998-03-30 | 2000-08-15 | General Electric Company | System and method for diagnosing and validating a machine using waveform data |
| US6442542B1 (en) * | 1999-10-08 | 2002-08-27 | General Electric Company | Diagnostic system with learning capabilities |
| US6609036B1 (en) * | 2000-06-09 | 2003-08-19 | Randall L. Bickford | Surveillance system and method having parameter estimation and operating mode partitioning |
| FI114749B (en) | 2000-09-11 | 2004-12-15 | Nokia Corp | Anomaly detection system and method for teaching it |
-
2000
- 2000-09-11 FI FI20001997A patent/FI114749B/en active
-
2001
- 2001-06-06 EP EP01947345A patent/EP1325588A1/en not_active Withdrawn
- 2001-06-06 WO PCT/EP2001/006405 patent/WO2002021774A1/en not_active Application Discontinuation
- 2001-06-06 AU AU2001269056A patent/AU2001269056A1/en not_active Abandoned
- 2001-06-06 US US10/363,745 patent/US7519860B2/en not_active Expired - Fee Related
- 2001-09-10 WO PCT/FI2001/000783 patent/WO2002021242A1/en active Application Filing
- 2001-09-10 CA CA2421928A patent/CA2421928C/en not_active Expired - Fee Related
- 2001-09-10 CN CNB018154476A patent/CN1196984C/en not_active Expired - Fee Related
- 2001-09-10 JP JP2002524792A patent/JP4436042B2/en not_active Expired - Fee Related
- 2001-09-10 EP EP01967371A patent/EP1334417A1/en not_active Withdrawn
- 2001-09-10 AU AU2001287759A patent/AU2001287759A1/en not_active Abandoned
-
2003
- 2003-03-07 US US10/383,224 patent/US7613668B2/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| WO2002021774A1 (en) | 2002-03-14 |
| US20040039968A1 (en) | 2004-02-26 |
| CA2421928A1 (en) | 2002-03-14 |
| AU2001269056A1 (en) | 2002-03-22 |
| US7613668B2 (en) | 2009-11-03 |
| FI20001997A (en) | 2002-03-12 |
| CN1455890A (en) | 2003-11-12 |
| CN1196984C (en) | 2005-04-13 |
| JP2004508631A (en) | 2004-03-18 |
| WO2002021242A1 (en) | 2002-03-14 |
| EP1334417A1 (en) | 2003-08-13 |
| US7519860B2 (en) | 2009-04-14 |
| JP4436042B2 (en) | 2010-03-24 |
| AU2001287759A1 (en) | 2002-03-22 |
| FI114749B (en) | 2004-12-15 |
| FI20001997A0 (en) | 2000-09-11 |
| EP1325588A1 (en) | 2003-07-09 |
| US20030225520A1 (en) | 2003-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2421928C (en) | Anomaly detection system and a method of teaching it | |
| JP7105932B2 (en) | Anomaly detection using deep learning on time series data related to application information | |
| US20180260723A1 (en) | Anomaly detection for context-dependent data | |
| CN107566163A (en) | A kind of alarm method and device of user behavior analysis association | |
| US20210192586A1 (en) | Systems and Methods for Detecting and Responding to Anomalous Traffic Conditions | |
| CN105320727A (en) | Method for detecting anomalies in real time series | |
| EP1360557A2 (en) | Adaptive modeling of changed states in predictive condition monitoring | |
| US10268836B2 (en) | System and method for detecting sensitivity content in time-series data | |
| Fu et al. | Online temporal-spatial analysis for detection of critical events in cyber-physical systems | |
| US20080168375A1 (en) | Systems and methods for simultaneous summarization of data cube streams | |
| CN114780644B (en) | Ship navigation data processing method, device, equipment and storage medium | |
| CN115561408A (en) | Air pollution early warning method and device, electronic equipment and storage medium | |
| KR20180024331A (en) | Learning method on recent data considering external effect in early alarm system, and system using thereof | |
| CN110099089A (en) | The self-tuing on line of multiple data flows in sensor network | |
| US20230107337A1 (en) | Managing machine operations using encoded multi-scale time series data | |
| US7756682B1 (en) | System and method for statistically monitoring and analyzing sensed conditions | |
| Goenawan et al. | Dynamics signature based anomaly detection | |
| WO2020183781A1 (en) | Abnormality diagnosis device | |
| CN110245844A (en) | Abnormal index detection method and device | |
| US20240112071A1 (en) | Anomaly detection using hash signature generation for model-based scoring | |
| CN112600944B (en) | Differential cloud storage method and system suitable for time sequence data of Internet of things | |
| CN117114500B (en) | Behavior baseline establishment method, system and medium based on data decomposition | |
| Alberg et al. | Predicting aircraft maintenance timetofailure using the interval gradient regression tree algorithm | |
| CN116311795A (en) | Geological disaster early warning method, device, equipment and storage medium | |
| Peter et al. | Hybrid--approach for outlier detection using minimum spanning tree |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |
Effective date: 20160912 |