US 20090185496 A1 Abstract Network performability characteristics with improved accuracy are derived by taking into account, in the various analyzed network failure states, attributes of elements at the logical level other than just the capacities of edges, as well as by taking into account one or more “abstract components,” such as scheduled maintenance, and by using multiple traffic matrices.
Claims(18) 1. A method for computing a performability characteristic for a communications network that comprises a plurality of network components including a plurality of interconnected links, the method comprising
computing a performance measure associated with each of a plurality of failure states of the network, each of at least ones of the failure states including at least one failed network component, and computing the performability characteristic based on the performance measures computed for the plurality of failed network states, the performance measure being computed based on at least one attribute of one or more of the network components that is other than the traffic-handling capacity of one of the links. 2. The method of 3. The method of 4. The method of 5. The method of 6. The method of 7. A method for computing a performability characteristic for a communications network that comprises interconnected links, the method comprising
computing a performance measure associated with each of a plurality of failure states of the network, each of at least ones of the failure states including at least one failed network component, the performance measure being computed based on attributes of edges of a logic level graph of the network, each of the edges of the logic level graph representing a route through the network over one or more corresponding ones of the links, the attributes of the edges being determined at least by attributes of the one or more corresponding links in said each of the failed network states, and computing the performability characteristic based on the performance measures computed for the plurality of failed network states, the at least one attribute being other than the traffic-handling capacity of said one or more corresponding links. 8. The method of 9. The method of 10. The method of 11. The method of 12. The method of 13. The method of wherein the performance measure is computed further based on attributes of nodes of the logic level graph, each of the nodes of the logic level graph representing a associated network element connected to at least one of the links, and wherein the at least one attribute is an attribute of the network element represented by one of the nodes of the logic level graph. 14. The method of 15. The method of 16. The method of 17. The method of 18. A computer-readable medium on which are stored instructions that are executable by a processor to carry out the method defined by Description This application claims the benefit of U.S. Provisional application 61/011,814 filed Jan. 22, 2008. The present disclosed method relates to techniques for evaluating the performance and/or reliability of communications networks, or of any system that can be modeled as a communications network. As communications networks have grown in size and complexity, the evaluation of their performance and reliability has become more critical. Network service providers usually guarantee a certain level of service, in terms of down time, restoration delay, packet delay, etc., to their enterprise customers. Violating these agreements results in a penalty to the service providers. In turn, the enterprise customers incur business losses if their networks do not perform as expected. There are even some legal requirements on U.S. businesses to be aware of their network's “risk profile” (e.g., the US Sarbanes-Oxley act of 2002). Some aspects of the service level have to do purely with reliability, such as downtime, while others are pure performance measures, such as packet delay. Each can be evaluated or predicted in isolation by an appropriate model and technique, but as networks are becoming larger and more complex, it is becoming less realistic to consider only pure reliability measures or to evaluate performance measures as if the network were always in its perfect (no failure) state. Even though the complexity of real networks often requires that performance and reliability evaluations be treated separately from each other, a combined, simultaneous evaluation, known as performability analysis, is very informative. Performability analysis consists in characterizing failures probabilistically and evaluating a performance measure (or measures) over a large set of network states. There is one “perfect” network state in which all of the network components are assumed to be operating properly. Every other network state is characterized by the failure of one or more of the network components. Among the various network states that would be evaluated are a) states in which a particular component is assumed to have failed (in combination with the failure of zero or more other components) and b) other states in which that particular component is assumed to have not failed. The probability of various kinds of components failing is known, based on experience, information from manufacturers, and statistical sampling techniques, and that probability is a factor in the probability of the occurrence of those states in which the component in question is assumed to have failed and ultimately, then, in computing the overall performability characteristic in question. An individual network component may actually have more than one failure mode. For example, a link that normally has bandwidth X may fail in such a way that the available bandwidth is X/2, X/3, etc. Another failure mode is one in which the link has failed completely and thus no bandwidth is available. Any one or these possibilities can occur in some network states. A typical performance measure is packet delay, as mentioned above. Another is the percentage of offered traffic that cannot be delivered due to the particular failures represented by the network state, referred to herein as “percent traffic lost.” Such performance measures are computed based on an assumed traffic matrix, i.e., an assumed amount of traffic demand between each origin/destination node pair in the network and, in a typical application, are computed after a network restoration algorithm has been applied to the network in the assumed state in order to reroute as much of the traffic as possible. The performance measure value computed for each network state is then multiplied by the probability of occurrence of that state (which is, in turn, a function, of the probability of failure of the components assumed to have failed for that state) and the resultant products are used to derive a performability characteristic. A simple performability characteristic is what is called the “expectation” of a particular performance measure, given by the sum of the products just described. A more sophisticated performability characteristic that can be derived from this analysis is a performability guarantee of the form: “with X% probability, at most Y% of the network traffic will have no path.” A whole set of performability characteristics can be developed in this way by considering multiple performance measures for each network state. For each network state, there is a characterization of the network at the component level. In order to carry out the analysis just described, a logical level characterization is generated for each network state. The logical level characterization characterizes the network in terms of nodes and edges that interconnect them. The nodes at the logical level correspond to packet routers or other similar components, whereas the edges represent routes for traffic through the network which may traverse one or more physical links. A failure of one or more components at the component level—including optical fiber spans, routers, repeaters or other physical components—may affect the capacity of one or more edges at the logical level, capacity being a measure of the amount of traffic that can traverse an edge as reflected by, for example, the bandwidth of the edge or the bit rate or packet rate that it can support. Given an assumed traffic matrix as mentioned above, one can then use known network analysis techniques to compute the desired performance measure—such as, again, packet delay or percent traffic lost—after any restoration algorithm used in the network has been run on to the network state under investigation. One can then compute desired performability characteristic(s) such as those mentioned above, i.e., the performance measure's expectation or some performance guarantee. In determining the effects on traffic, at the logical level, of the failures that characterize a particular network state, prior art approaches focus on the capacities of edges, as described above. In accordance with one aspect of our method, we have recognized that more accurate performability characteristics can be derived by taking into account, in the various analyzed network states, attributes of elements at the logical level other than just the capacities of edges. In particular, we have recognized that more accurate performability characteristics can be derived by taking into account in the various analyzed network states a) attributes of edges for a given network state other than just their capacities, and b) attributes of logical level elements other then edges. Examples of a) include an edge's latency and an edge's so-called network routing “cost” or “weight.” An example of b) is whether a particular node is capable of routing additional traffic, such as traffic that might be desired to be rerouted through that node due to a failure somewhere else in the network. In the drawing, The physical network comprises four packet routers Each of the depicted endpoints Each of the various network components shown in A failure in a physical component of the network at the component level has implications for the routing of traffic over the edges of the logical network. For example, The physical network of Multiple concurrent failures of components of the network are also possible, and the resulting logical networks would represent network states in which the paths take all such failures into account. Examples of that are given below. The process begins (as indicated at A logical level “master” graph is then generated ( A traffic matrix must also be assumed ( The process of identifying the network states then begins ( The identified state is now considered ( The real-life network being modeled may employ a restoration algorithm that attempts to reroute traffic after network failures. In that case, the particular restoration algorithm that is used in that network is applied ( We can now compute the performance measure(s) of interest, e.g., packet delay or percent traffic lost using known network analysis techniques ( The performance measure thus computed is only for a particular network state. What we care about is an overall performability characteristic of the network, taking into account a) the various ways in which components can fail, both singly and in various combinations, and b) the failure probabilities. Accordingly, if there are more states to be considered ( It is possible to compute the desired performance measure, and thus the overall performability characteristic, over the entire set of possible network states. However, as noted above, there are typically too many states to consider in order for the computation to be carried out in a reasonably short time. What one can do is to consider the network states at Associated with network states The assumed performability characteristic of interest is the expectation {tilde over (F)} of the performance measure F. As shown in In determining the effects on traffic, at the logical level, of the failures that characterize a particular network state, prior art approaches focus on the capacities of edges, as described above. In accordance with one aspect of our method, we have recognized that more accurate performability characteristics can be derived by taking into account attributes of elements at the logical level other than just the capacities of edges. In particular, we have recognized that more accurate performability characteristics can be derived by taking into account in each analyzed network state a) attributes of edges for a given network state other than just their capacities and b) attributes of logical level elements other then edges. These will now be discussed in turn. Latency is the time it takes for a packet to travel from its point of origin to its point of destination. In the example of The length of a route can also affect various measures of network performance. Thus the network states that are evaluated in computing various performability characteristics pursuant to this aspect of our disclosed method may include states in which the length of the edges of the logical network differ from “normal” as the result of failures. Another example related to latency arises in the context of routes between nodes in an underlying network layer that is not explicitly included in the model, such as a SONET ring. Normally, traffic between two points A and B is routed in a particular direction around the ring. If a component in that direction should fail, the traffic between A and B could be re-routed in the other direction, which could be the long way around the ring, thus increasing the latency of the edge between nodes A and B of the ring. Links in a network are often assigned a so-called administrative cost that is a function of, for example, the physical length of the link. Algorithms that determine routes through the network can take link costs into account in doing so, with the goal being to develop routes that minimize the use of network resources or to maximize throughput or to achieve some other goal. The routing algorithms are designed such that they tend to avoid routing traffic over links having relatively high costs. In addition to assigning baseline, or fixed, administrative costs to links, network management software can artificially increase a link's cost if for any reason it is desired to “discourage” a routing algorithm from routing additional traffic over that link. For example, if one or more of a router's line cards fail, or if maintenance is being performed on the router, its packet-handling capabilities are reduced. In such a situation the network operator may simply take the router out of service for the period of repair or maintenance. A better operations policy is to discourage the routing of network traffic via the impaired router by increasing the administrative cost associated with all links to which that router is connected; this makes the routing algorithms steer traffic away from the router in question. An example is shown in Increasing the link costs is a better network operations policy because it effectively removes the router in question from the network, but still allows it to be used in the event that a failure elsewhere in the network is such that a demand affected by it has no path to its destination other than (being rerouted) through this impaired router. As an example, consider router The above example is one in which an attribute of a link other than just its capacity is affected not by a failure on the link itself, but of some other component of the network. For another example, consider the loss of packets in a network employing radio, microwave, or laser links. Each link has a “packet loss probability” attribute, whose value is affected by components representing atmospheric conditions or conditions in space. The overall loss in the network is a function of these link loss attributes. The loss probability attribute may also depend on the available buffer (queue) space of routers to which the link is connected. Thus, if one of N buffers in a router fails, there is an increased probability of loss in the associated links since the probability that buffers will overflow, and thus the rate at which packets are dropped, is increased. In accordance with this aspect of our disclosed method, however, step The above examples are ones in which, pursuant to one aspect of our disclosed method, one or more attributes of edges other than just their respective capacities are taken into account in defining the network states. We saw how those edge attributes might be affected by a failure within the associated physical link and/or within some other network component. In accordance with another aspect of our disclosed method, however, the network states may include states in which a failure affects the attributes of a node. An example of such a state is depicted in In accordance with an aspect of our method, we have recognized that more accurate performability characteristics can be derived by including in the component-level characterization of the network one or more of what we refer to as “abstract components.” These are aspects of the network and/or its operation that can have the same kind of effect at the logical level—and thus on traffic and the performance measure—as an actual physical component failure. An example, illustrated in The scheduled maintenance of routers may be handled by network managers by artificially increasing to a very high number the “cost” assigned to all links connected to the router, so that the only way any traffic will go through this router is if a failure somewhere else makes paths using this router unavoidable. In the example where router As to the “probability” associated with the scheduled maintenance subcomponent of, say, router In accordance with another aspect of our method, we have recognized that more accurate performability characteristics can be derived by computing a performance measure using not a single traffic matrix but, rather, two or more traffic matrices. As an example, The amount of traffic from node The traffic amounts indicated in matrices II and III follow a similar labeling scheme. A first way of taking multiple traffic matrices into account is to compute the value of the performance measure F for each network state using each traffic matrix in turn. One can then amalgamate the resulting performance measure values F into a composite performance measure value for the network state under consideration, weighting them in accordance with the length of time over which the traffic associated with the various traffic matrices is assumed to subsist. This first approach is illustrated in Thus if a particular performance measure F A second approach for taking multiple traffic matrices into account is to define an abstract component—per the discussion of abstract components above—and to make the traffic matrix part of the network state. In this case the abstract component would be a traffic component. In any given network state the traffic component will be in (in this example) one of the three modes (corresponding to the three traffic matrices). The probability associated with each traffic mode used for computing the probability of the overall network state is the associated percentage of time that network traffic is characterized by that particular matrix. The second approach may yield the same computed performability characteristic as the first if all of the possible network states are considered in the computation. If significantly less than all of the possible network states are considered in the computation, a different, but probably statistically insignificant, value of the performability characteristic might be expected. There are, however, certain advantages to using the second approach. One of them is that if the implementing software allows for abstract components, as discussed above, the inclusion of different traffic matrices in the computation is readily accomplished without having to generate program code specifically directed to multiple-traffic traffic matrix considerations. Moreover, in certain situations the second approach will yield a more accurate computation. As an example of that, assume that the software in the network's routers gets upgraded only when the network traffic is low (say between 2 AM and 4 AM). In that case, we want the abstract component “router software upgrade” to be correlated with low traffic. Indeed, the implementing software will typically already have been configured to allow for correlated component modes. Accordingly, those states in which the router is in software upgrade mode will always have the abstract traffic component in its mode associated with the time frame that encompasses 2 AM to 4 AM. That is, we will not consider any network state in the overall computation in which the router is in software upgrade mode but the traffic matrix is for a time frame that does not encompass 2 AM to 4 AM. This is correct to do because no such a state ever occurs in the postulated scenario. If, by contrast, the first approach, as shown in The balance of this Detailed Description presents some theoretical underpinnings of our disclosed method as well as further examples. We describe the problem of network performability evaluation in an abstract manner and, for clarity of exposition, restricted to networks with only binary components. More complex multi-mode models are discussed hereinbelow Let C={c A state of the network is an assignment of a state to each component and can be thought of as an n-bit vector. The set of all network states S (C) has size 2 The most basic characteristic of F is its expected value over the space of network states:
Our main purpose is to compute more sophisticated characteristics of F than just its expectation. These are performability guarantees which, in the simplest case have the form, e.g., “with 99.9% probability, at most 5% of the total traffic is down,” and in a more complex case the form “with 99.9% probability at most 5% of the total traffic is down, and with 99.99% probability at most 10% is down.” Formally, these guarantees are statements of the type that apply to the entire space S. These guarantees, referred to as “multi-part performability specifications” are much stronger and more useful than just the expectation (2.1). For instance, they can be used to set, or to check, service-level agreements (SLAs) for the network. An alternative assessment uses multi-attribute risk, examined in §4.4. What is the meaning of Since the size of S (C) is normally very large, the straightforward computation of (2.1) or (2.2) is generally prohibitive. The complexity of the Performability Evaluation problem Given a set C and a polynomial-time computable function F (•) defined on S (C), compute the expectation
By viewing states as sets of failed components, F is monotone increasing if F (s)≦F (s′) whenever s A generalization of most probable states is the notion of “most important states”, i.e. those which contribute the most to the value of State generation algorithms There are two fundamentally different types of algorithms for generating states in order of probability. One type of algorithm views the space S as a partially-ordered set (lattice), while the other treats it as a binary tree. Both these algorithms work only on systems of binary components, and they are best suited to different problem domains. For a monotone F, the second algorithm produces better bounds on Bounds on the CDF So far we have discussed bounds only on the expectation of a measure F. The same techniques allow us to derive bounds on the performability guarantee Pr(F≦x), (CDF of F) for a given x. Let the notation [C] yield 1 if the condition C is true and 0 otherwise. Then
is a sum of the form (2.1), so bounds of the type (2.3) can be found for it as well (recall that P is the probability of the k highest-probability states):
Our experience indicates that many practical telecommunications networks can be adequately modeled by binary components together with a relatively small number of components with multiple failure modes. There is no satisfactory state generation algorithm for such a system. To our knowledge, the so-called GC algorithm is the most efficient for systems of purely binary components, but cannot handle any non-binary components. Other algorithms work with multi-mode components, including binary, but they are not nearly as efficient as the GC algorithm in the purely binary case. A method is known for combining an arbitrary binary algorithm (for binary components only) and an arbitrary multi-mode algorithm (for any components) into a hybrid algorithm which works for combinations of binary and non-binary components, but is particularly efficient for mostly binary systems of components. By “mostly binary” we mean such that the number of binary failure modes is at least as large as the total number of multi-mode failure modes. All state generation algorithms that we are aware of use a priority queue which stores network states according to probability, and a set of rules for generating next states from a current state. The hybrid algorithm follows the same pattern, and is described in Efficiency We now present a more complete discussion of the efficiency of the hybrid algorithm. Given the components C={c Notice that for a given set C and ε>0 any algorithm, hybrid or not, for generating states in order of probability, will produce exactly the same sequence of states (ignoring the ordering of states with equal probabilities). So the algorithms M and (B, M′) will generate the same s Claim Claim
In a “mostly binary” network as defined above we have b>M, and for relatively small k the ratio in (2.5) is comparable to 1, approaching 1 as b becomes larger with respect to M. E.g., for k≦3, if M=b/10, the ratio is ≧0.75 for b≧100. Now if M is comparable to b, the fraction of 1-states in S Suppose we sort all b+M failure modes in the decreasing order of probability. A network state corresponds to a subset of these modes and its probability is proportional to the product of the corresponding mode probabilities. Let us consider the states in S In other words, more than half of the states of S To gain some perspective on what has been said so far and to understand some of the compromises involved, it is useful to consider performability evaluation in a general setting. Ideally, one would like to model the network under consideration as a system of (perhaps interacting) components c There are two difficulties in applying such methods to our network performability evaluation problem. First, the number of components is large: n≈700 is not unusual, leading to a large state space. Second, our performance measure is complex: it involves a multi-level network model (cf. §3), and the execution of a complicated network restoration algorithm (cf. §5). Of these, the second is the most difficult: the simpler the performance measure defined on the system, the easier it is to deal with the size of the system (e.g. by some approximation or reduction technique) and its evolution in time, and the more detailed one can make the models for the individual components. So the formulation of §2 reflects a compromise, where we have basically traded off accurate representation of the network and its dynamics (2-state models for the components, assumption of steady state, assumption that multiple failures occur at a single instant in time and restorations are single events) in order to be able to handle the complexity of the mechanisms involved in the performance measure. The remainder of this discussion will clarify these assumptions. The performability model view has a demand level, a graph level (multigraph, in general), a component level, and a reliability level (not to be confused with network layers). The demand level specifies the amount (and possibly type) of traffic flowing from one node of the graph to another via the graph's edges. The graph is the level at which the network routing and restoration algorithms operate, so graph edges have associated capacities and (routing) costs. At the component level, each component corresponds to an independent failure mechanism, as explained in §2; failure of a component may affect a whole set of graph-level elements. The components may, in fact, have an arbitrary number of failure modes; see §3.2 and §3.3. In general, the effect of a component entering one of its failure modes on a graph node or edge is to change some of its attributes. For example, the capacity of an edge may decrease, or a node may become unable to perform routing. Depending on the final values of the attributes, e.g. edge capacity 0, the graph element may be considered “failed”. Finally, at the reliability level, failure modes of components are described simply by their mean times between failures and mean times to repair. The model view in Today's typical commercial packet network consists of IP routers whose links (between the routers) are transported by an underlying optical network. For such a network we illustrate how we model the layers Traffic layer Based on an estimate of the peak or average traffic pattern, we create a matrix giving the demand (typically called “flows”) between each pair of routers. A demand has a rate, and possibly a type associated with it. Transport layer nodes A network node represents an IP router which contains a data plane, a control plane, and ports. The data plane, or switching fabric, is responsible for routing packets, while the control plane computes routing tables, etc. We model the data and control planes as separate components, because newer routers have the ability to continue forwarding packets even when the control plane fails. The nodes of the graph level model represent routers. At the component level this node explodes into one data plane, one control plane, and one hardware and software upgrade component. When a data plane component fails, all the graph edges incident to its node fail. When a control plane component fails, the existing flows are unaffected, but the node cannot participate in restoration. We also model components for each port of the network node. Failure of a port component brings down the corresponding edge(s). Finally, at the reliability level we specify an MTBF and an MTTR for each class of component. Transport layer links Two network nodes may be connected by multiple parallel links. These are modeled at the graph level by parallel edges. For the purposes of IP routing, these parallel edges may form an “aggregate” link whose capacity is the sum of the capacities of the edges, and the nodes see only the aggregate links. When a constituent edge fails, the capacity of a link is reduced; the link fails if and only if all its constituent edges fail. As stated earlier, edges fail if the corresponding port components fail. Edges can also fail because of network span failures, discussed next. Physical layer spans We use the term “span” to refer to the network equipment at the physical layer that carries the transport-layer links. We define a component for each network span. Failure of this component affects all graph-level edges which are routed over this span. When the optical layer uses Dense Wavelength Division Multiplexing (DWDM), a span consists of a concatenation of point-to-point Optical Transport Systems (OTS) (more complex WDM systems have various optically transparent “add/drop” capabilities which, for simplicity, we do not discuss here). In turn, an OTS is composed of optical multiplexers and demultiplexers, optical amplifiers, and optical transponders. The length of a span and the maximum permitted distance between optical amplifiers determine how many OTSs need to be concatenated. To avoid proliferation of components, we do not define a component for each piece of equipment appearing in a span, but represent the entire span by a single component; this component may affect multiple edges, as many as the number of wavelengths. A basic assumption about spans in typical transport networks is that they are considered to be working only if both directions of the span are working. With this assumption, it is not difficult to compute the failure probability of a span based on the failure probabilities of its individual components. However, at the reliability level we want a more detailed characterization of a span, in terms of an equivalent failure rate Λ (MTBF) and an equivalent repair rate M (MTTR). Since both directions of a span must be working for the span to be operational, we can represent the whole span by laying out the components of both directions into a series connection of independent blocks. The MTBF and MTTR of this connection are then calculated as explained in §3.3. Other components We may also have other components, in addition to the above types. For example, a bundle of fibers that is likely to fail together because they are in a single conduit can be represented by a component that brings down all graph edges carried by this fiber bundle. Other types of catastrophic failures of entire sets of graph nodes and edges may be similarly represented. Simplifying the graph-level access topology Commercial transport networks are usually partitioned into “access”, and a “backbone” or “core” segment. The topology of the access to the backbone or core is the most complex part of the network, with many details that are very difficult to model. Moreover, it undergoes frequent changes, as older equipment is replaced by newer equipment and customer interfaces change. To analyze the availability and performance of the backbone segment of our IP-over-Optical example network, at the graph level we create a simplified model (and hence the accompanying network “roll-up” approximation) of the access topology based on the point-to-point demands and assume only a single type of router port at the SONET OC-48 (2.5 Gb/s) rate. The model is created as follows. Given a backbone office, we go through all the demands originating from it and try to fit them into as few ports as possible. These become the “customer-facing” ports of the location. We then distribute these ports into groups of a certain size, which is a parameter. Each group is assigned to a virtual edge node, which has the same structure as an ordinary node: it contains ports, both customer- and network-facing, a data plane, and a control plane. (The network-facing ports of an edge node and the links to backbone nodes are determined by the network design, as described in §6.) A distinguishing feature of our model is that it allows graph nodes or edges to have any number of (possibly vector-valued) attributes, whose values can be set by components. Having such attributes at the graph level greatly increases the representational power of the model. As an example, consider modeling what is known as a “composite network link”. Such a link represents a set of parallel edges which, for routing purposes, is treated as a single edge, with capacity equal to the sum of the constituent capacities, and which fails if some of its constituent links fail and its composite capacity becomes less than a certain threshold. Suppose we want to model a composite link that consists of two edges e We can calculate F in a binary model by examining the states of p The use of components with multiple failure modes is not new in performability analysis, so here we just mention an example in the IP networking context. Upgrading the software of a router results in a short outage for the router. To minimize these outages, the commercial network maintenance policy is that only a single router is upgraded at a time. We model these mutually exclusive “failures” by using a component that, for an n-router network, has n+1 modes, with mode i corresponding to upgrade of the ith router and the last mode representing no router in upgrade state. Our model also allows the combination of multi-mode components and (vector) graph element attributes. Each component of the network model is represented by a simple continuous-time Markov process with states G and B (“good” and “bad”), and parameters λ and μ (failure and repair rates), as shown in If we have a series connection of k independent binary blocks, the corresponding Markov model has a single “up” state and 2
and λ Given a network specified by the hierarchical model of §3, we define the following performance measures on every network state s: t t t t Here “traffic” means the sum of the corresponding demands. A demand is affected when an edge on its route fails. A demand fails when a link (multiedge) on its route fails. A failed demand is lost, and remains so if no path for it has been found after the restoration process completes. If the network routing allows congestion, the total traffic lost because of congestion can be computed by finding the maximum possible total flow F that can be carried without congestion and subtracting it from the total traffic in the network. Let D be the set of all subdemands (a demand may be split routed over multiple paths; we refer to each of these as a “subdemand”), and D
where c However, for IP networks, in general this provides a crude estimate of packet loss, because packet loss depends on a model for the traffic, a queuing model for the nodes, and details of the protocol. E.g., TCP will “throttle down” traffic as soon as it detects congestion, so that packet loss is mitigated; in that case it is more accurate to call the t Whether a traffic measure is monotone or not (as defined in §2) depends both on the measure itself and on the routing/restoration method used by the network. It is clear that the measure t When computing the measures in (4.1) we also want to take into account the duration τ of a failure as well as the time τ _{b}(s)=t _{bb}(s)+(t _{ba}(s)−t _{bb}(s))e ^{−τ} ^{ r } ^{/{tilde over (w)}} ^{ s }. (4.3)For example, when “lost because of congestion” is substituted for “bad” and the appropriate “before” and “after” sets of demands are computed, the result of (4.3) is to be taken as the value of t The analysis just given clearly involves some idealizations. For example, the set of demands D Given a set of measures such as the ones in (4.1), one way to assess the performability of a network is to check if their CDFs satisfy certain requirements, the performability guarantees of (2.2). An alternative to looking at performability guarantees is to examine the space of explored failures from the viewpoint of risk. Given a performance measure, let the risk of a failure with respect to this measure be the product of the probability of the failure and the value of the measure. With an m-dimensional measure, the risk associated with a failure has m aspects, or attributes. An immediate question is “what are the riskiest failures for the network?” Because of the multiplicity of attributes, the set of network states cannot be totally ordered with respect to risk, and there will usually not exist a failure that is worse than all the others. But we can say that a failure state s dominates (is worse than) another state s′ if the risk attributes of s are ≧the corresponding attributes of s′, with strict inequality for at least one attribute. Given a set of network states, we can eliminate from consideration all states whose risk is dominated by that of some other state. What remains is known as the Pareto boundary or Pareto frontier of the risk set. The frontier can be extracted from the set by what is known as a “Pareto filter”. The filtering problem is: if V is a set of d-dimensional vectors in R _{i}≧v_{i }for all i with strict inequality for at least one i. Suppose that determining whether xy takes d units of time. Then given n points, a straightforward algorithm for eliminating all dominated points takes time O(dn^{2}). A much better algorithm for d=2, requiring only O(n log n) time, is given as follows. Denote a point by (x_{i}, y_{i}) and sort the points in order of increasing x, with increasing y for equal values of x_{i}. Without loss of generality, let the result be (x_{1}, y_{1}), (x_{2}, y_{2}), . . . , (x_{n}, y_{n}). Then it is clear that any point can be dominated only by a point to its right. Starting from the end, (x_{n−1}, y_{n−1}) is dominated by a point on its right if and only if y_{n−1}<y_{n}. (x_{n−2}, y_{n−2}) is dominated by a point on its right if and only if y_{n−2}<max(y_{n−1}, y_{n}), etc. Finally, (x_{1}, y_{1}) is dominated by some point to its right if and only if y_{1}<max(y_{2}, . . . , y_{n}). Denoting the maxima by m_{i}, the algorithm finds them in linear time by making a backward pass through the list to determine m_{n−1}=y_{n}, m_{n−2}=max(y_{n}, y_{n−1}), m_{n−3}=max(y_{n}, y_{n−1}, y_{n−2}), . . . , m_{1}=max(y_{n}, . . . , y_{2}). (The filtering problem may also be solved in O(n log n) time with a more general bi-criterion shortest path algorithm, but the procedure we described here is much simpler.)
We have implemented many of the concepts described in previous sections in software to evaluate commercial networks. This section describes the routing methods we implemented to model the IP layer for commercial IP-over-optical layered networks. However, we have also applied these methods to other layered networks. In each network state we have a new graph (a set of nodes fail, a set of edges fail or their capacities are reduced, etc.), and a set of demands affected by a failure that needs to be rerouted. The performance measures of §4 depend partly on how we select new routes for the failed demands. We use two schemes for selecting paths during network restoration: 1. Shortest-path routing, which models OSPF (Open Shortest Path First) routing where packets are sent over the shortest (minimum cost) paths. The path computation does not depend on the capacities or utilizations of the links. Note that the other main IP-layer routing protocol, IS-IS, would behave similarly in our context. 2. Optimal routing, which selects a set of minimum-cost paths that maximizes the amount of restored traffic while respecting the link capacities. This models a scheme where a centralized server has complete knowledge of the entire network and computes optimal routing paths. OSPF-TE (MPLS) may be considered to be an approximation to this. The shortest-path routing, because it does not take link capacities into account, is likely to result in more congested links and thus larger values of t Clearly, the above are idealized representations of real network routing and restoration protocols, in which many details are not taken into account. Notably, we do not model in detail the full set of complicated message exchanges (known as “signaling”) that take place among network nodes to implement the routing algorithm. We first construct a graph G=(V, E) where V is the set of network nodes and E represents the OSPF links. The link costs are equal to the OSPF administrative weights. The capacity of a link (multiedge) is equal to the sum of the edge capacities between this pair of nodes (recall §3.1). Capacities are not used for the minimum-cost (min-cost or shortest-path) computation, but are used for determining the set of congested links for calculation of t If there is more than one min-cost path between a pair of nodes, we balance the traffic as follows: for each source and destination pair (s, d) we compute NextHops(s, d), the set of neighbors of s that are on at least one min-cost path from s to d. Then we distribute all packets from s to d equally among all nodes in NextHops(s, d), and repeat recursively. To compute the all-pairs min-cost matrix we use the Floyd-Warshall algorithm). For a graph with n nodes and m edges, the complexity of this algorithm is O(n The network is specified as a directed graph with edges that have capacities and costs and a set of indivisible demands. The idealized problem that we want to solve is optimal restoration, i.e. “route as much total demand as possible subject to the capacity constraints, and so that the total cost is minimized”. This problem involves optimizing both the quantity of traffic routed and the cost (good-ness) of the routing. In network flow terminology, we have an integer min-cost max-multicommodity flow problem, i.e. among the maximal flows find one with least cost (the cost of a flow, equal to Σ Even very simple versions of this problem are hard. For example, deciding whether just two demands can be routed in a graph with edges all of the same capacity and no costs is NP-complete. But if the integrality (indivisibility) requirement on the edge capacities and flows is removed, min-cost multicommodity flow is solvable in polynomial time via linear programming. We find an approximate solution to the optimal restoration problem by using a greedy heuristic which reduces the problem to a set of 1-commodity flow problems, and at the same time takes care of the integrality requirement. This is depicted in Finally, the heuristic lends itself well to speedup by parallelization. Given a set of M processors, R/M permutations are given to each, and, when finished, each processor reports its best result to the coordinating processor. Our implementation includes the option of parallel evaluation, using the Parallel Virtual Machine [PVM] infrastructure. It is not unusual for a network performability analysis to generate hundreds of thousands of states, and paths for the network demands need to be computed in every one of these states. We describe some improvements in the overall efficiency of the path computation that exploit the fact that when states are generated in order of probability, the state sequence exhibits a certain locality: typically, successive network states differ in only one component. In the case of min-cost routing, this implies that the all-pairs min-cost (classical shortest-distance) matrix typically changes little between adjacent states. One way to take advantage of this is via a dynamic graph algorithm. There are several such algorithms with strong worst-case asymptotic guarantees, but they are not easy to implement. In contrast, the improvements we describe below are very easy to implement and result in near-optimal running time for our specific application. Improvement 1: affected node pairs Let D Proposition. For any graph G(V, E), construct a graph G′ (V, E′) as follows. Initialize E′ to E. Then pick an arbitrary pair of nodes u, v and add an edge (u, v) to E′ with cost equal to the min-cost path between u and v in G. Repeat this step for an arbitrary number of node pairs. Then the min-cost matrices for G and G′ are identical. This can be proved by induction on the number of extra edges added to E′. For each extra edge, a path of the same cost existed in the graph and thus none of the costs change because of the addition of this edge. Because the effect of initialization is the same as adding an extra edge of cost equal to the minimum cost path, this also completes our proof of the dynamic all-pairs min-cost algorithm. Given the matrix D Improvement 2: matrix caching Both the number of failed edges and the size of AffectedNodePairs depend on the size of the failure. Our next trick is to “reduce” the case of multiple component failures to the case of single component failures. This reduction is possible quite often because of the locality in the sequence of states produced by the algorithms of §2.2. When the reduction is possible, its effect is that “number of failed edges+size of AffectedNodePairs” corresponds to a single component failure, significantly improving the speed of the algorithm in Based on these observations, we modify the algorithm to maintain a cache of (set, matrix) pairs. In a state corresponding to a set of failed components C, we first check if the cache contains an entry (C In our simulations of this scheme with a cache size of 25 and least-recently used re-placement policy, we found a cache hit rate of nearly 99%. Moreover, as we searched the cache from top to bottom, about 80% of the time we had a match among the first 2 entries, and about 90% of the time a match among the first 5 entries. Thus the average complexity (per state) of the all-pairs min-cost paths algorithm becomes a small constant times n Here we report the results of running our tool on three network designs, variations on the IP-over-optical backbone network of a large Internet Service Provider. Each design consists of a number of backbone locations. Each location houses one or two backbone routers (BRs), and has a set of associated edge routers (ERs). The ERs are connected to the BRs within the location. Each BR is connected to the other BR in the location, and to a set of BRs in other locations; see The “fat” design. Each location has two BRs, and each ER is connected to both the BRs in its associated location. The network has enough capacity to protect against any single span, port, or BR failure. The “thin” design. This is similar to the fat design except that we do not allocate restoration capacity for BR failures. The “skinny” design. Here we have only one BR per location, and each ER is connected to the single BR in the location. The network has enough capacity to protect against any single span and port (but no BR) failure. All of these networks use OSPF-style restoration, and the designs use the “80/95” rule, namely that link packet utilization should be below 80% under no-failure conditions, and not exceed 95% after a failure. Specifically, the edge capacities are assigned so that each network meets the 80/95 rule for a certain set of single failures. This approach is common among most large ISPs, although the utilization levels and network design specifics differ. After a failure that is not in the “covered” set, some links may be utilized over 95% and it is even possible that some flows may not be restored. Our goal is to quantitatively evaluate effect of using reduced network capacity and fewer routers on the performability of the network. We also evaluate the effects of some improvements to routers, namely “hitless” software/hardware upgrades and N:1 port protection. For each network design we compute the CCDF (complementary CDF) of two measures: traffic lost because of no route, and traffic lost because of congestion, the t In all these CCDF graphs, the plots are smoothed versions of staircase-like functions. The abrupt changes in slope of the CCDF curves are due to the discrete nature of the probability space and the complex mapping from this space to the value of the measure. In particular, the relatively flat segments are not caused by any particular set of failures, but by the absence of failures that, via the discrete mappings from the physical to the traffic layer, cause traffic loss of a certain magnitude. One may note in Here we look at two possible improvements to the reliability of routers and assess the effect of these improvements on the entire network from the viewpoint of lost traffic. Here we investigate the effect of adding N:1 protection on router ports. We define groups of N ports ahead of time, each with an (N+1)th protection port. When a port fails and the protection port in its group is working, then the link is switched to the protection port, which takes over the role of the failed port. If multiple ports within the same protection group fail, at most one of the failed ports can be restored; the rest remain failed. Without port protection, the probability that a port does not fail is 1−q. With protection, a port stays up in any state in which it has not failed; the set of all these states has probability 1−q. The port also stays up in any state in which it has failed but every other port, including the protection port in its group, is up; this set of states has total probability q(1−q) This shows that for typical (small) values of q, the effect of protection is, surprisingly, almost independent of the protection group size N. The conclusion for the network in this case study is that software upgrades are a major contributor to network downtime. If the upgrades were made hitless, port protection would further improve the network's performability, and there is even a promise of being able to reduce restoration capacity and still maintain the same performability levels. In §6.1 and §6.2 we evaluated three network designs by looking at performability guarantees, recall (2.2), for the lost traffic measure. Another, complementary, assessment consists in examining the space of explored failures from the viewpoint of risk and the Pareto frontier. As explained in §4.4, risk is defined with respect to a particular traffic measure: at each network state, the value of the measure is multiplied by the probability of the state. With the 4-dimensional traffic measure of (4.1), the risk associated with a failure has 4 attributes. The foregoing is merely illustrative. For example, although the invention has been illustrated in the context of packet networks, the principles of the invention are equally applicable to other kinds of networks. For example, the invention could be used to determine a performability characteristic for circuit-switched networks. In such an application of the invention, a typical performance measure would be “Is there a path available between source A and destination B?” Another might be, “Is there a path available between source A and every possible destination B, C, D, etc” Another might be “Is there a path available between each possible source/destination pair.” In each of these cases, the performance measure would have the value of “1” or “0” for each different network state depending on whether or not the path(s) in question are available in that network state. The performability characteristic, then, would be in an indication of the extent to which it is likely, across all network states considered, that the path(s) in question would, in fact, be available. It will thus be appreciated that those skilled in the art will be able to design other arrangements and methods which, although not explicitly shown herein, embody our inventive principles and are thus within their spirit and scope. Referenced by
Classifications
Legal Events
Rotate |