US 8060614 B2 Abstract A streaming operator assignment system and method for determining a streaming operator assignment that minimizes overload in a data processing system. Embodiments of the streaming operator assignment system include an optimization goals definition module, which defines optimization goals in terms of fundamental quantities that system administrators and application writers want to control, such as minimizing the worst case latency over all periods of time, or minimizing how much the system is backlogged with work. Embodiments of the streaming operator assignment system also include an optimization goals solution module that optimizes and solves a selected optimization goal. A specialized optimization technique is used to find the best operator (or load) assignment using the optimization goals to measure of the value of the assignment. This technique minimizes an optimization goal by iterating over all possible operators assignments over all possible nodes to find the operator assignment that minimizes the desired optimization goal.
Claims(20) 1. A computer-implemented method for solving a load assignment optimization goal of a data processing system on a computer network having a plurality of nodes, comprising:
determining a most overloaded node in the data processing system as measured by the optimization goal;
determining a high-quality placement, with respect to the optimization goal, by iterating over all possible streaming operator placements for streaming operators on the most overloaded node as defined by the optimization goal;
assigning streaming operators to nodes based on the streaming operator placement to solve the optimization goal; and
processing data in real time as the data arrives at each of the plurality of nodes using the assigned streaming operators.
2. The computer-implemented method of
removing a streaming operator on the most overloaded node; and
placing the streaming operator on each of the other nodes in the system one at a time.
3. The computer-implemented method of
4. The computer-implemented method of
5. The computer-implemented method of
determining that at least one of the streaming operator placements improves performance of the data processing system as measured by the optimization goal; and
implementing a streaming operator placement that gives a greatest performance increase of the data processing system as measured by the optimization goal for operators being considered for placement.
6. The computer-implemented method of
determining that none of the streaming operator placements improve performance off the data processing system as measured by the optimization goal; and
selecting an operator on the most overloaded node in the data processing system having a next highest load as measured by the optimization goal.
7. The computer-implemented method of
8. The computer-implemented method of
9. The computer-implemented method of
10. The computer-implemented method of
11. The computer-implemented method of
12. A method for assigning streaming operators to nodes in a Computer network, comprising:
defining a plurality of optimization goals for distributed operator placement;
selecting one of the plurality of optimization goals to use as a measure for streaming operator placement optimization to obtain a selected optimization goal;
determining a most overloaded node in a data processing system as measured by the selected optimization goal; and
optimizing the selected optimization goal using the most overloaded node to obtain a optimized streaming operator placement; and
assigning the streaming operators to the nodes in conformance with the optimized streaming operator placement.
13. The method of
14. The method of
15. The method of
16. The method of
where N
_{i }represents a node, N represents a set of all nodes in the system, [t_{0}, t_{1}] is a time interval AccLoad(N_{i}, t) is an accumulated overload of the node N_{i }that occurs at a time t, and CPU_{i }is a CPU capacity of the node N_{i}.17. The method of
18. The method of
where N
_{i }represents a node, N represents a set of all nodes in the system, t_{0 }is a time that is less than t_{1}, t_{1}−t_{0}+1 is a length of a given sequence of load combinations, AccLoad(N_{i}, t) is an accumulated overload of the node N_{i }that occurs at a time t, and CPU_{i }is a CPU capacity of the node N_{i}.19. A method for using hill-climbing with random restart technique to minimize an optimization goal to solve a load assignment optimization problem in a data processing system having a plurality of nodes connected by a computer network, comprising:
defining a plurality of optimization goals for distributed operator placement;
selecting one of the plurality of optimization goals to optimize to obtain a selected optimization goal;
determining a most overloaded node in the data processing system as measured by the selected optimization goal;
removing a streaming operator on the most overloaded node and placing the removed streaming operator on each remaining node in the computer network one node at a time;
determining whether any of the streaming operator assignments improve performance of the data processing system based on the selected optimization goal;
if any streaming operator assignments does improve the performance, then implementing an optimal streaming operator assignment that gives the greatest performance improvement of the data processing system;
if none of the streaming operator assignments improves the performance, then selecting an operator on the most overloaded node in the data processing system having a next highest load as measured by the selected optimization goal; and
repeating the above process until one of the following occurs; (a) a streaming operator assignment is found that achieves better performance than a current streaming operator assignment; (b) if no streaming operator assignment can be found that achieves better performance than a current streaming operator assignment, then select a streaming operator assignment that gives a best performance.
20. The method of
defining a maximum instantaneous overload of the data processing system based on a history of the data processing system;
defining an average instantaneous overload of the data processing system based on the history of the data processing system;
defining a maximum accumulated overload of the data processing system using a given load sequence over a time interval; and
defining an average accumulated overload of the data processing system using the given load sequence over the time interval.
Description Processing of a data stream can be quite a resource intensive procedure. A data stream is a sequence of an order list of values (called a “tuple”). Many established and emerging applications can be naturally modeled as data stream applications. In order to monitor a data stream, a user registers continuous queries with the Data Stream Management System (DSMS). These queries continuously update their state and produce new output for newly arriving stream tuples. In a typical data stream application users expect at least quasi-real time results from their continuous queries, even if the stream has a high rate of arrival. Due to these requirements, data stream processing can be very resource intensive. Examples of a data stream are monitoring of networks and computing systems, consumer credit card purchases, telephone calls dialed by callers, monitoring of sensor networks, and supply chain management and inventory tracking based on RFID tags. Another example of a data stream is measurement data, such as IP traffic at router interfaces, sensor networks, and road traffic measuring. Even publish-subscribe and filtering and dissemination of RSS feeds (such as for monitoring the “blogosphere”) can be viewed as data stream applications. One way in which this resource intensive problem has been addressed is to distribute the processing load over multiple nodes in a network. A fundamental challenge, however, of such a distributed stream processing system is to select the correct criterion for distributing load in the system. Load balancing in traditional distributed and parallel systems is a well-studied problem. These techniques do not carry over to data stream processing, because load balancing decisions on a per-tuple basis are too costly. In load balancing techniques, incoming jobs (queries) have to be assigned to processing nodes such that throughput is maximized or latency (response time) is minimized. This is usually achieved by some type of load balancing, which takes into account the availability of input data at the processing nodes and communication costs for moving data between nodes. In a data stream processing system, the roles of queries and data are reversed from traditional distributed systems. Namely, queries are continuously active while new data tuples are streaming in at a high rate. This creates new challenges for a data stream processing system compared to traditional distributed systems. In a data stream processing system the individual input tuples are small. It is therefore too costly to decide for each tuple individually to which processing node it should be routed. Furthermore, for operators with state (such as sliding window joins), re-routing tuples would also require migrating operator state to the new processing nodes. In order to amortize the optimization cost, tuple routing decisions should be made such that they benefit many stream tuples. This is achieved by assigning operators to processing nodes. These are operators that take real-time data from things like network monitoring sensors, stock market, etc., and perform some form of processing on the data (such as rank the data, correlating the data, or filter the data). The data is not stored somewhere and then processed, but instead the data is processed in real time as it arrives to the operator. In other words, each operator inputs a stream of events, processes the stream of events, and outputs a processed stream of events. Given a set of these streaming operators that are running on a data processing system, and given that there is a collection of computing devices or processors connected together, the goal is to determine how to best assign these streaming operators to those processors. This is called operator placement. The operator placement, and hence the routing pattern for the tuples, are used for a large number of input tuples and are only changed when system statistics change significantly. This can be called a Distributed Operator Placement (DOP) problem. Several techniques have been proposed for placing operators in a distributed streaming system for the purpose of balancing load and improving query latency. These techniques are based on some type of operator placement strategy. One obvious solution to the DOP problem is to assign operators to nodes such that system load is balanced for a “typical” case. Optimizing for the “typical” load is not sufficient. This is because data streams in practice tend to be “bursty”, meaning that data is received in large waves at one time and a trickle of data the next moment. This bursty nature makes it virtually impossible to react to short-duration load bursts with any kind of load re-balancing. While the system is busy adapting to a burst, the load situation might already have changed significantly again to require another adaptation. To address this problem, some techniques use resilient operator placements, where the system can handle a wide variety of load situations without any node being over-loaded. A related idea for distributed stream processing is to prevent load spikes by placing operators with uncorrelated load behavior onto the same node and to maximize load correlations between different nodes. Other techniques for distributing load for data stream processing include distributing the load of a single operator. Queuing theory has provided valuable insights into scheduling decisions in multi-operator and multi-resource queuing systems, but results are usually limited by high computational cost and strong assumptions about underlying data and processing cost distributions. The problem with all of these operator placement strategies is that they are heuristics that have been found to achieve good overall results. In other words, these approaches take some heuristic that researchers believe result in some good placement of the streaming operators without any type of mathematical structure, and then apply that placement that provides good results. The point is that these existing techniques do not provide a solid mathematical foundation or a solid optimization foundation to the DOP problem. While an assignment of streaming operators to processors is made, left unanswered are questions about the quality of the assignment and even how to measure the quality of the assignment in a principled, precise way. Thus, these placement strategies are based on trial and error and these heuristic solutions are not designed to directly optimize a specific, application oriented optimization goal. This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Embodiments of the streaming operator assignment system and method find the most effective assignment for streaming operators in a data processing system. This system is on a network having many nodes, and the streaming operators input data, process the data, and then output the data. By finding the most effective (or best) assignment of streaming operators to network nodes, the efficiency of the processing of data in the system can be greatly increased. Embodiments of the streaming operator assignment system define the optimization goals in terms of fundamental quantities that system administrators and application writers want to control, such as minimizing the worst case latency over all periods of time, or minimizing how much the system is backlogged with work. Embodiments of the streaming operator assignment system then define a series of optimization problems in terms of the optimization goals and how the data processing system should behave during periods of overload. Embodiments of the streaming operator assignment system include an optimization goals definition module and an optimization goals solution module. The optimization goals are goal that system administrators and application writers feel are important to measure the efficiency of the data processing system. Embodiments of the optimization goals definition module formally models and defines performance goals for throughput and latency of the data processing system. Embodiments of the optimization goals definition module define four optimization goals. The first two optimization goals formalize throughput goals (namely, maximum instantaneous overload and average instantaneous overload), while the other two optimization goals formalize latency goals (namely, maximum accumulated overload and average accumulated overload). For all optimization goals, performance of the data processing system is determined by the node having the worst performance (which is known as the bottleneck node). This bottleneck-focused approach provides the strongest guarantees for system performance. The maximum instantaneous overload optimization goal seeks to minimize the worst instantaneous overload on the system. Moment by moment, by looking at the worst state that has occurred in the system across all time based on a system history, this optimization goal seeks to know how overloaded would the system have been during that worst case overload. This optimization goal is to reduce that overload at that instant as much as possible over all time and over all nodes. The average instantaneous overload optimization goal seeks to ensure that at any instant in time the system does not have any node that is continuously unstressed. This is based on a history of the system and load sequences over time intervals. The history of the system can be any desired time interval of the system. This history is used to define the optimization goal and to determine how well the system is performing according the optimization goal. For example, if it is desired to minimize the worst overload over all time, then a history of what happened, say, in the last week or year, is used. A particular operator placement is applied to that period having the worst overload of the system. It then can be determined if that particular streaming operator placement was used at that worst time within the given history, how well the system would have performed with this particular placement. This is known as the “goodness” of the placement. The maximum accumulated overload optimization goal does not look at only a moment in time, but rather looks at an accumulation of the system being overloaded or underloaded for a particular period of time. For example, if the system is overloaded, overloaded, overloaded, and overloaded, that is worse than being overloaded, underloaded, underloaded, and overloaded. In other words, maximum accumulated overload optimization goal is a measure of how far behind is the system at its worst moment. The average accumulated overload optimization goal is an average of the worst accumulated overload over a time interval. The idea is to minimize the average accumulative overload over all possible operator assignments over the nodes. Embodiments of the streaming operator assignment system include an optimization goals solution module. Embodiments of the optimization goals solution module optimize and solve the previously-defined optimization goals for the data processing system. Once the optimization goals are defined, a specialized optimization technique is used to find the best operator (or load) assignment using the optimization goals as a measure of the value of the assignment. This is called a streaming operator hill-climbing technique with random restarts. The general idea for this technique is to minimize an optimization goal by iterating over all possible operators assignments over all possible nodes to find the operator assignment that minimizes the desired optimization goal. Embodiments of the streaming operator assignment system formally achieve dramatic improvements, both in result quality and especially in run-time performance. The result is a formal framework for expressing, analyzing, and implementing various distributed optimization goals. This framework can be extended as needs evolve. It should be noted that alternative embodiments are possible, and that steps and elements discussed herein may be changed, added, or eliminated, depending on the particular embodiment. These alternative embodiments include alternative steps and alternative elements that may be used, and structural changes that may be made, without departing from the scope of the invention. Referring now to the drawings in which like reference numbers represent corresponding parts throughout: In the following description of embodiments of the streaming operator assignment system and method reference is made to the accompanying drawings, which form a part thereof, and in which is shown by way of illustration a specific example whereby embodiments of the streaming operator assignment system and method may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the claimed subject matter. I. System Overview The streaming operator assignment system The streaming operators shown in Referring to II. Operational Overview In general, embodiments of the streaming operator assignment system Once an optimization goal is selected, a streaming operator placement is determined that minimizes the selected optimization goal (box III. Operational Details The operational details of embodiments of the streaming operator assignment system III.A. Optimization Goals Definition Module As noted above, embodiments of the streaming operator assignment system The module The optimization goals definition module III.A.1 Maximum Instantaneous Overload One of the optimization goals that formalize throughput goals is the maximum instantaneous overload. Ideally, operators can be assigned to nodes such that there is never an overloaded node in the data processing system. This guarantees that stream tuples will always be processed immediately on arrival and will not spend time waiting in queues of operators that “fell behind”. In other words, system throughput is high enough to keep up with any occurring load combination. This notion is captured by the concept of “instantaneous overload”. Instantaneous overload measures by how much the load produced at each moment in time exceeds the available central processing unit (CPU) capacity of a node. Overload is considered relative to a node's CPU capacity to also handle heterogeneous clusters whose nodes have widely varying CPU capacities. The formal definition of the maximum instantaneous overload is as follows. Maximum instantaneous overload for a time interval [t By way of example, assume there is a node N III.A.2 Average Instantaneous Overload A second optimization goal that formalizes throughput goals is the average instantaneous overload. It will not always be possible to guarantee that no node is ever overloaded. For many applications (such as a free service for filtering and dissemination of news), such performance guarantees are not even necessary. Instead, it is desirable to guarantee that the data processing system can keep up with the input streams “in the long run”. Some processing nodes might temporarily fall behind during a load spike, but eventually they will catch up and process all their input tuples. In other words, the data processing system throughput is high enough to keep up with the expected load. This concept can be captured by the following notion of average instantaneous overload. Mathematically, for a time interval [t
By way of example, for a node N
The maximum over all nodes is taken to ensure that every node is able to keep up with its input streams. Thus, the average instantaneous overload is non-positive if and only if none of the nodes in the system are overloaded on expectation. A negative value indicates that every node in the system could handle that much more average load. The optimization goal is to minimize the average instantaneous overload. III.A.3 Maximum Accumulated Overload and Average Accumulated Overload The two optimization goals that formalize latency goals are the maximum accumulated overload and the average accumulated overload. If no DOP solution with a non-positive maximum instantaneous overload can be found, then average instantaneous overload is not the only alternative goal. The average instantaneous overload has the disadvantage of “averaging out” periods of poor system performance. For example, a node on average might only need 50% of its CPU resources to keep up with its input streams, but it can still have extended periods of overload. During these times tuples might accumulate in operator queues, causing significant delays. To minimize such delays, the optimization goals definition module Latency is difficult to model, because it depends on operator scheduling decisions, queue length, and per-tuple processing times of the individual operators. Therefore, the optimization goals definition module The optimization goals definition module The optimization goals definition module
Moreover, the average accumulated overload is defined as: The defined optimization goals are directly dependent on actual performance requirements of the data processing system. Based on previously observed load combinations, stream rates, and operator properties, the optimization goals definition module As long as future load combinations are as predicted, no further re-optimization of embodiments of the streaming operator assignment system The optimization goals definition module The optimization goals definition module
This optimization goal is similar to average instantaneous overload, but it averages the average instantaneous overload across all nodes rather than taking the average instantaneous overload of the bottleneck node. For this metric a configuration like (−5,3), meaning that one node have an average instantaneous overload of −5 and another node has an average instantaneous overload of 3, is equivalent to a configuration (−1,−1), meaning that both nodes have an average instantaneous overload of −1. In practice the latter configuration is preferable, because none of the nodes is overloaded. Similar arguments apply to the other optimization goals that average across different nodes. Intuitively, optimizing based on bottleneck performance forces the most balanced load distribution. III.B. Optimization Goals Solution Module Embodiments of the streaming operator assignment system III.B.1 Solving the Optimization Goals The optimization goals solution module The individual steps taken are different for each optimization problem to which the hill climbing with random restarts technique is applied. For the optimization goals defined above, the optimization goals solution module The streaming operator assignment hill-climbing technique with random restarts ensures that, at any given moment, progress is being made towards the best answer. One reason the technique is called hill climbing is because you start with an operator (or load) assignment, and then according to the optimization goal it is decided how “good” is a particular operator assignment. The measure of how “good” is the particular operator assignment is based on and measured by the particular optimization goal itself. The streaming operator assignment hill-climbing technique with random restarts computes the actual overload of the system at the worst moment to determine how well that particular operator assignment fared. Then, the operator assignment is changed by some small increment and in such a way to guarantee that the new operator assignment will always be “better” than the old operator assignment. This streaming operator assignment hill-climbing technique with random restarts guarantees that progress is being made and that the solution is getter closer to the best possible answer for that optimization goal. Random restarts are used to ensure that a best solution is found. For example, sometimes when using a hill climbing technique a point is reached in the problem where the solution cannot be improved further. This may or may not be the best solution. It may only be a local optimum, which means that there are no current modifications that can be made to improve the solution. The way around this is to start in a totally different place in the problem. This will likely result in a different local optimum being reached. After this random restart, the hill climbing technique is used until another peak or local optimum is reached. Once this occurs, another random restart is performed along with the subsequent hill climbing to reach yet another peak or local optimum. This continues until it is believed that a good solution has been reached. For example, if ten peaks have been found and their solutions have been good then it can be assumed a solution has been found. This is not generally true for all hill climbing-based approaches, but it seems to be true for this particular problem. Abstractly, the streaming operator assignment hill-climbing technique greedily transforms one streaming operator placement to another one, such that the value of the optimization goal improves. This streaming operator assignment hill-climbing technique keeps iterating until no further improvement is possible, or the optimization budget is exhausted. The streaming operator assignment hill-climbing technique is based on the fact that for proposed optimization problems system performance is determined by the bottleneck node (or the most overloaded node). The module This operator is assigned to the target node that among all nodes in the system would have the lowest overload score after this operator is added there. However, the operator move is only permitted if the overload score on the target node after adding the operator there is below the overload score of the bottleneck node before the move. In other words, a determination is made as to whether any of the streaming operator assignments improve performance based on the selected optimization goal (box The streaming operator assignment hill-climbing technique for the various optimization goals are similar, except that a different optimization goal is used to determine the bottleneck node (or most overloaded node), and the node performance after removing or adding an operator. This greatly simplifies implementation of specialized algorithms for the different optimization problems. In other words, the same streaming operator assignment hill-climbing technique can be used except that a different optimization goal may be selected as the performance metric. III.B.2 Runtime Complexity of Solving the Optimization Goals The streaming operator assignment hill-climbing technique of the optimization goals solution module The overall complexity of the streaming operator assignment hill-climbing technique depends on the number of successful operator migration steps. First, the worst case cost of such a step is analyzed for the optimization goal maximum accumulated overload. It costs O(m·s) to find the bottleneck node, where s=t
Computing maximum accumulated overload for node N
Let N Hence, the worst case cost of a streaming operator assignment hill-climbing step for the maximum accumulated overload is O(m
The cost analysis for the average accumulated overload performance metric is identical. For the maximum instantaneous overload metric, the cost of computing the per-node performance can be reduced by maintaining Load(N The following should be noted about s, which is the number of load combinations in the considered time interval [t III.B.3 Alternate Embodiments of the Optimization Goals Solution Module In some embodiments of the streaming operator assignment system Network resources are included in the streaming operator assignment hill-climbing technique by modeling the effect on the network in addition to modeling the effect on the processors. Note, however, that the streaming operator assignment hill-climbing technique will become more complex, because moving operators from one node to another not only affects the CPU load, but also some network links. Specifically, when a node is removed from the processor the transport of the data that that operator receives over the network link is also removed. The loads on both the incoming and the outgoing network links associated with the operator being moved must be adjusted. These embodiments of the streaming operator assignment hill-climbing technique assume that there is some sort of graph that describes how the nodes are connected. Using this graph, and the process above, when an operator is removed from one node and the operator is placed on another node, it can be determined which network link loads need to be adjusted. Similarly, when considering overload, if a network link, rather than a node, is the most overloaded, then the streaming operator assignment system IV. Exemplary Operating Environment Embodiments of the streaming operator assignment system The full-text lattice indexing and searching system The streaming operator assignment system Components of the computer The computer Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computer The system memory The computer Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive The drives and their associated computer storage media discussed above and illustrated in Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, radio receiver, or a television or broadcast video receiver, or the like. These and other input devices are often connected to the processing unit The computer When used in a LAN networking environment, the computer The foregoing Detailed Description has been presented for the purposes of illustration and description. Many modifications and variations are possible in light of the above teaching. It is not intended to be exhaustive or to limit the subject matter described herein to the precise form disclosed. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims appended hereto. Patent Citations
Non-Patent Citations
Referenced by
Classifications
Legal Events
Rotate |