|Publication number||US20060080388 A1|
|Application number||US 11/242,303|
|Publication date||Apr 13, 2006|
|Filing date||Oct 3, 2005|
|Priority date||Jun 20, 2001|
|Also published as||US7117242, US20030028636|
|Publication number||11242303, 242303, US 2006/0080388 A1, US 2006/080388 A1, US 20060080388 A1, US 20060080388A1, US 2006080388 A1, US 2006080388A1, US-A1-20060080388, US-A1-2006080388, US2006/0080388A1, US2006/080388A1, US20060080388 A1, US20060080388A1, US2006080388 A1, US2006080388A1|
|Inventors||Ludmila Cherkasova, Magnus Karlsson|
|Original Assignee||Ludmila Cherkasova, Magnus Karlsson|
|Export Citation||BiBTeX, EndNote, RefMan|
|Referenced by (15), Classifications (15)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present invention relates to the field of web servers. Specifically, the present invention relates to a method for workload-aware request distribution in cluster-based network servers.
Web server clusters are a popular hardware platform in a web hosting infrastructure. Servers based on clusters of workstations are used to meet the growing traffic demands imposed by the World Wide Web. A cluster of servers, arranged to act as a single unit, provides an incremental scalability as it has the ability to grow gradually with demand. However, for clusters to be able to achieve the scalable performance with the cluster size increase, mechanisms and policies are employed for “balanced” request distribution.
Traditional load balancing solutions are represented by two major groups: 1) Domain Name System (DNS) based approaches; and 2) Internet Protocol (IP)/Transmission Control Protocol (TCP)/Hypertext Transfer Protocol (HTTP) redirection based approaches.
In a DNS based approach, the DNS server returns the IP address list (e.g., a list of nodes in a cluster which can serve this content, placing a different address first in the list for each successive request) to distribute the requests among the nodes in the cluster. Thus, different clients are mapped to different server nodes in the cluster. DNS based approaches are widely used, as they require minimal setup time and provide reasonable load balancing. Further, it uses the existing DNS infrastructure (e.g., there is no additional cost). However, DNS based approaches do not recognize either the load of the nodes in a cluster or the content of the request.
The second group, IP/TCP/HTTP redirection based approaches, employ a specialized front-end node, the load-balancer, which acts as a single point of contact for the clients and distributes the requests among back-end server nodes in the cluster. These solutions can be classified in the following groups:
These terms refer to the techniques by which the systems in the cluster are configured together. In a L4/2 and L4/3 cluster, the load-balancer determines the least loaded server (this decision is the job of the proprietary algorithms implemented in different products) to which server in a cluster the packet has to be sent.
Traditional load balancing solutions for a web server cluster (L4/2 and L4/3) try to distribute the requests among all the back-end machines based on some load information.
The load-balancer can be either a switch or a load-balancing server (e.g., hardware solution) or a software load balancer (e.g., software solution). In both solutions, the load-balancer determines the least loaded server in a cluster to which the packet should be sent.
Load-balancing servers operate by intelligently distributing the incoming requests across multiple web servers. They determine where to send an incoming request, taking into account the processing capacity of attached servers, monitoring the responses in real time and shifting the load onto servers that can best handle the traffic. Load-balancing servers are typically positioned between a router (connected to the Internet) and a local area network (LAN) switch which fans traffic to the Web servers.
Traditional load balancing solutions for a web server try to distribute the requests evenly among all the back-end machines based on some load information. This adversely affects efficient memory usage because the content is redundantly replicated across the caches of all the web servers, thus resulting in a significant decrease in overall system performance.
Content-aware request distribution (e.g., L7 switching) takes into account the content (can be a Uniform Resource Locator (URL) name, URL type, or cookies) when making a decision to which back-end server the request has to be routed. Content-aware request distribution mechanisms enable intelligent routing inside the cluster to support additional quality of service requirements for different types of content and to improve overall cluster performance. Policies distributing the requests based on cache affinity lead to significant performance improvements compared to the strategies taking into account only load information.
There are three main components comprising a cluster configuration with content aware request distribution strategy: the dispatcher which implements the request distribution strategy, it decides which web server will be processing a given request; the distributor which interfaces the client and implements the mechanism that distributes the client requests to a specific web server; and the web server which processes HTTP requests.
In the content-aware request distribution approach, the cluster nodes are partitioned in two sets: front end and back ends. The front end acts as a smart router or a switch, its functionality is similar to the aforementioned load-balancing software servers. The front end node implements the policy which routes the incoming requests to an appropriate node (e.g., web server) in the cluster. Content-aware request distribution can take into account both document locality and current load. In this configuration, the typical bottleneck is due to front-end node that combines the functions of distributor and dispatcher.
To be able to distribute the requests on a base of requested content, the distributor component should implement either a form of TCP handoff or the splicing mechanism. Splicing is an optimization of the front-end relaying approach, with the traffic flow represented in
Thus, another recent solution is shown in
In this architecture the distributor is decoupled from the request distribution strategy defined by the centralized dispatcher module. The switch in front of the cluster can be a simple LAN switch or L4 level load-balancer. For simplicity, we assume that the clients directly contact distributor, for instance via RR-DNS. In this case, the typical client request is processed in the following way. 1) Client web browser uses TCP/IP protocol to connect to the chosen distributor; 2) the distributor component accepts the connection and parses the request; 3) the distributor contacts the dispatcher for the assignment of the request to a server; 4) the distributor hands off the connection using TCP handoff protocol to the server chosen by the dispatcher (since in this design the centralized dispatcher is the most likely bottleneck, the dispatcher module resides on a separate node in a typical configuration, as shown in
This design shows good scalability properties when distributing requests with the earlier proposed LARD policy. The main idea behind LARD is to logically partition the documents among the cluster nodes, aiming to optimize the usage of the overall cluster RAM. Thus, the requests to the same document will be served by the same cluster node that will most likely have the file in RAM. Clearly, the proposed distributed architecture eliminates the front-end distributor bottleneck, and improves cluster scalability and performance.
However, under the described policy in a sixteen-node cluster, each node statistically will serve only 1/16 of the incoming requests locally and will forward 15/16 of the requests to the other nodes using the TCP handoff mechanism. TCP handoff is an expensive operation. Besides, the cost of the TCP handoff mechanism can vary depending on the implementation and specifics of the underlying hardware. It could lead to significant forwarding overhead, decreasing the potential performance benefits of the proposed solution.
Web server performance greatly depends on efficient RAM usage. A web server operates much faster when it accesses files from a cache in the RAM. Additionally, the web servers throughput is much higher too.
Accordingly, a need exists for a request distribution strategy that maximizes the number of requests served from the total cluster memory by partitioning files to be served by different servers. A need also exists for a request distribution strategy that minimizes the forwarding and the disk access overhead. Furthermore, a need also exists for a request distribution strategy that accomplishes the above needs and that improves web server cluster throughput.
The present invention provides a content-aware request distribution strategy that maximizes the number of requests served from the total cluster memory by logically partitioning files to be served by different servers. The present invention also provides a request distribution strategy that minimizes the forwarding and the disk access overhead by assigning a small set of most frequent files (referred to as the core files) to be served by any node in the cluster.
A method and system for workload-aware request distribution in cluster-based network servers are described. The present invention provides a web server cluster having a plurality of nodes wherein each node comprises a distributor component, a dispatcher component and a server component. The distributor component operates to distribute a request to a specific node. The dispatcher component has stored upon it routing information for the plurality of nodes which is replicated across the plurality of nodes. The routing information indicates which node is assigned for processing a request. The server component operates to process the request. In one embodiment, the plurality of nodes are coupled to a network.
In another embodiment, the present invention provides a method for managing request distribution of a set of files stored on a web server cluster. A request for a file is received at a first node of a plurality of nodes, each node comprising a distributor component, a dispatcher component and a server component. If the request is for a core file, the request is processed at the first node. If the request is for a partitioned file, it is determined whether the request is assigned to be processed by the first node (e.g., processed locally). If the request for a partitioned files is assigned to be processed by the first node, the request is processed at the first node. If the request for a partitioned file is assigned to be processed by another node, the request is forwarded to the correct node for processing (e.g., processed remotely). If the request is not for a core file or a partitioned file, the request is processed at the first node.
In one embodiment, the web server cluster also comprises a set of base files, wherein the base files are a set of frequently accessed files fitting into a cluster memory (RAM) of the web server cluster.
In one embodiment, the present invention provides a method for identifying a set of frequently accessed files on a server cluster comprising a number of nodes. A set of base files is defined wherein the base files are a set of frequently accessed files fitting into the cluster memory of the server cluster. The base files are ordered by decreasing frequency of access. The base files are logically partitioned into a subset of core files having a core size, a subset of partitioned files having a partitioned size, and a subset of on disk files which are evicted from the cluster memory (RAM) to a disk. Each subset of files is ordered by decreasing frequency of access, respectively. The core files and partitioned files are identified wherein the total of the partitioned size added to the product of the number of nodes multiplied by the core size is less than or equal to the cluster memory (RAM). The total overhead due to the base files is minimized wherein the total overhead equals the overhead of the core files plus the overhead of the partitioned files plus the overhead of the on disk files.
These and other objects and advantages of the present invention will become obvious to those of ordinary skill in the art after having read the following detailed description of the preferred embodiments which are illustrated in the various drawing figures.
The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention:
In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are not described in detail in order to avoid obscuring aspects of the present invention.
Some portions of the detailed descriptions which follow are presented in terms of procedures, steps, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. A procedure, computer executed step, logic block, process, etc., is here and generally conceived to be a self-consistent sequence of steps of instructions leading to a desired result. The steps are those requiring physical manipulations of data representing physical quantities to achieve tangible and useful results. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers or the like.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present invention, discussions utilizing terms such as “accessing”, “determining”, “storing”, “receiving”, “requesting” or the like, refer to the actions and processes of a computer system, or similar electronic computing device. The computer system or similar electronic device manipulates and transforms data represented as electronic quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission, or display devices.
Portions of the present invention are comprised of computer-readable and computer executable instructions which reside, for example, in computer-usable media of a computer system. It is appreciated that the present invention can operate within a number of different computer systems including general purpose computer systems, embedded computer systems, and stand alone computer systems specially adapted for controlling automatic test equipment.
In one embodiment, the present scalable web cluster design implements a workload-aware request distribution (WARD) load balancing strategy in which a content-aware distribution is performed by each of the nodes in a web cluster. The current architecture is fully distributed. Each node in a cluster performs three different functions:
In the present embodiment, the dispatcher component, which is replicated across all the cluster nodes, has the same routing information in all the nodes. The routing information indicates which node of the cluster is for processing which requested file. This routing information is defined by the off-line workload analysis process and a workload-aware distribution strategy (WARD). The distributor component of each node distributes a request to a specific node in the cluster. The server component of each node processes the request.
In the present embodiment, each node, after receiving a request, reviews the local dispatcher component routing table. The node then either accepts the request for local processing by the server component or forwards the request to the server component of another node for remote processing.
The present invention takes into account workload access patterns and cluster parameters such as number of nodes in a cluster, node RAM size, TCP handoff overhead, and disk access overhead. The present invention utilizes more efficiently the overall cluster RAM leading to improved web server cluster performance. The distribution (routing) strategy WARD is defined by off-line analysis of the joint set of all web server cluster logs during a certain time interval (e.g., daily analysis). The off-line analysis logically splits all the files into the following three groups:
In one embodiment, base files 440 are the set of files that fit into the cluster RAM (e.g., memory unit 410). In the present embodiment, base files 440 comprise web files for use in permitting a remote client 420 to access the files over the Internet. Disk 470 is a remote location distinct from the cluster RAM, wherein files evicted from the cluster RAM are stored on disk 470.
Under the strategy presented in the present invention, the base files 440 are represented by the three groups of files: Filescore and Filespart in the ClusterRAM (e.g., memory unit 410), and Fileson disk consisting of files evicted from RAM to disk (e.g., disk 470) due to the expansion of the Filescore. Each node comprises a core section 450 and a partitioned section 460 for storing Filescore and Filesparts, respectively.
Web server performance greatly depends on efficient memory usage. The throughput of a web server is higher when it reads pages from a cache in memory than from disk. If all files of the web site fit in memory the web server demonstrates excellent performance because only the first request for a file will require a disk access, and all the following file accesses will be served from memory. The present invention provides a method and system for achieving the goals of maximizing the number of requests served from the total cluster memory by partitioning files to be served by different servers and minimizing the forwarding overhead by identifying the subset of core files to be processed on any node, (e.g., allowing the replication of these files in the memories across the nodes).
It is appreciated that processing the requests to the core files locally by each cluster node helps to minimize the forwarding overhead. However, it may result in additional, initial disk accesses to core files on all those nodes and extra disk accesses because more files will reside on disk due to the expansion of the core files. This is why the ultimate goal here is to identify such a subset of core files for which the forwarding overhead savings are higher than the additional cost of the disk accesses caused by the core files.
At step 510 of process 500, a request for a file is received at a node of the web server cluster. In one embodiment, the request is an HTTP request sent by a remote client. Each node comprises a dispatcher component, a distributor component, and a server component (see scalable web cluster configuration 300 of
At step 520, it is determined whether the requested file is a core file (e.g., a frequently accessed file assigned to be served by any node). In one embodiment, the dispatcher component reviews the routing information to determine whether the requested file is a core file.
If it is determined that the requested file is a core file, as shown at step 530, the server component of the receiving node processes the requested file.
If it is determined that the requested file is not a core file, as shown at step 540, it is then determined whether the requested file is a partitioned file (e.g., a file assigned to be served by a particular node in a cluster). In one embodiment, the dispatcher component reviews the routing information to determine whether the requested file is a partitioned file.
If the requested file is not a partitioned file, as shown at step 550, the requested file is served locally from the receiving node.
If the requested file is a partitioned file, as shown at step 560, it is determined whether the requested file is assigned to be processed by the receiving node. If it is determined that the requested file assigned to be processed by the receiving node, as shown at step 530, the requested file is served locally from the receiving node. In one embodiment, the server component processes the requested file.
If it is determined that the requested file is not assigned to be processed by the receiving node, as shown at step 570, the distributor component forwards the request to the remote node designated by the dispatcher component. In one embodiment, the request is processed at the remote node by the server component of the remote node.
At step 580, process 500 ends. Process 500 is repeated for every request received by the cluster.
At step 610 of process 600, a set of base files is defined. The base files are a set of frequently accessed files fitting into the cluster memory (RAM) of a web server cluster. In one embodiment, the cluster memory is RAM. In one embodiment, the base files are ordered by decreasing frequency of access.
At step 620, the base files are logically partitioned into a set of core files having a core size, a set of partitioned files having a partitioned size, and a set of on disk files. In one embodiment, the base files comprising each set of files are ordered by decreasing frequency of access.
At step 630, the files comprising the core files and the partitioned files are identified, wherein the total of the partitioned size added to the product of the number of nodes multiplied by the core size is less than or equal to the cluster memory.
In one embodiment, the frequencies of access (the number of times a file was accessed) and sizes of individual files is used to determine the core set of files. These are denoted by FileFreq and FileSize, respectively. These are gathered by analyzing web-server access logs from the cluster. Freq-Size is the table of all accessed files with their frequency and the files sizes. This table is sorted in decreasing frequency order. The determination of the contents of the core files assumes that the cache replacement policy of the file cache in the web-server has the property that the most frequent files will most likely be in the ClusterRAM, wherein ClusterRAM is defined as the total size of all the file caches in the cluster.
If all the files were partitioned across the cluster nodes, the most probable files to be in the cluster RAM would be the most frequent files that fit into the cluster RAM. The set of files that fit into the cluster RAM is called BaseFiles (e.g., base files 440 of
of the request coming to each node of the total N nodes have to be handed off. Under the present invention, BaseFiles are represented by three groups of files as shown in Equation 1: Filescore and Filespart in the ClusterRAM, and Fileson disk consisting of BaseFiles that do not fit into ClusterRAM due to the expansion of Filescore. They satisfy Equations 1 and 2:
BaseFiles=Filespart+Filescore+Fileson disk Equation 1
Filescore are the files belonging to the core, the requests to these files are served locally by any node, and having a size Sizecore, the combined size (in bytes) of the files in Filescore;
The ideal case for web server request processing is when a request is processed locally (e.g., it does not incur an additional forwarding overhead (ForwardOH)) and it is processed from the node RAM (e.g., it does not incur an additional disk access overhead (DiskOH)). The goal is to identify a set of Filescore and a set of Filespart that minimizes the total overhead due to BaseFiles:
OH BaseFiles =OH core +OH part +OH on disk. Equation 3
Still with reference to
First, analyze what the additional overhead incurred by processing the requests to Filespart is, denoted as OHpart. Assuming all these files are partitioned to be served by different nodes, statistically a file in the partition incurs forwarding overhead on the average
times, where N is the number of nodes in the cluster. The file from partition will also incur one disk access on the node it is assigned to the first time it is read from disk. This reasoning gives the following overhead for the partition files:
PenaltyDiskAccess=FileSize∞DiskOH Equation 5
where ForwardOH is the processing time in μsec the TCP handoff operation consumes, and DiskOH is the extra time in μsec it generally takes to read one byte from disk compared to from RAM.
Determine the additional overhead incurred by processing the requests to Filescore. If a file belongs to the core then the request to such file can be processed locally, (e.g., with no additional forwarding overhead for these files). The drawback is that the files have to be read from disk into memory once on all the nodes in the cluster and that the number of files in Fileson disk increases due to the expansion of Filescore, creating additional disk access overhead. However, this is under the assumption that the files are accessed frequently enough that at least one request for each file will end up on all nodes. For files that are accessed less frequently this number is expected to be lower, thus it is necessary to calculate the expected value of the number of nodes that get at least one access to a file given a certain frequency f and a number of nodes N.
Here P(f,i) is the probability that exactly i nodes will have the file after f references to it. It can be calculated using the following recursion and starting conditions.
The overhead due to extra disk accesses to core files, denoted as OHcore, can then be calculated as follows.
Finally, the requests to Fileson disk will incur additional disk overhead every time these files are accessed, which gives the following equation.
Using the reasoning and the equations above, a set Filescore that minimizes the total overhead due to BaseFiles can be computed.
At step 710 of process 700, for a combined set of web server access logs in a cluster, a fileset profile is built for a combined set of web server access logs. In one embodiment, the table of all accessed files with their file frequency (number of times a file was accessed during the observed period) and their file size is built. This table is sorted in decreasing file frequency order.
At step 720 a WARD mapping is built. Using process 600 of
At step 730, once the WARD mapping is built, the dispatcher component in each cluster node will enforce the following WARD routing strategy.
If in core: serve locally If in partition and local: serve locally If in partition and remote: send to designated remote node Everything else: serve locally
At step 740, the distributor component in the each cluster node will send the request to be processed either locally or forward it to a corresponding node in the cluster, accordingly to directions of its corresponding dispatcher component.
By monitoring the traffic to a web cluster and analyzing it (for example, on a daily basis), WARD proposes a new balancing schema where the files (and requests to them) are classified into three groups: Filescore, Filespart and Fileson disk.
The preferred embodiment of the present invention, a method and system for workload-aware request in cluster-based network servers, is thus described. While the present invention has been described in particular embodiments, it should be appreciated that the present invention should not be construed as limited by such embodiments, but rather construed according to the below claims.
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7209967 *||Jun 1, 2004||Apr 24, 2007||Hitachi, Ltd.||Dynamic load balancing of a storage system|
|US7555484 *||Jan 19, 2005||Jun 30, 2009||Microsoft Corporation||Load balancing based on cache content|
|US8185654||Jun 4, 2008||May 22, 2012||International Business Machines Corporation||Systems and methods for content-aware load balancing|
|US8209700 *||Dec 22, 2005||Jun 26, 2012||International Business Machines Corporation||System, method, and program product for providing local load balancing for high-availability servers|
|US8239535 *||Dec 20, 2005||Aug 7, 2012||Adobe Systems Incorporated||Network architecture with load balancing, fault tolerance and distributed querying|
|US8346824 *||May 21, 2009||Jan 1, 2013||Translattice, Inc.||Data distribution system|
|US8417679||May 21, 2009||Apr 9, 2013||Translattice, Inc.||Fast storage writes|
|US8713186 *||Mar 12, 2008||Apr 29, 2014||Oracle International Corporation||Server-side connection resource pooling|
|US8775373||May 21, 2009||Jul 8, 2014||Translattice, Inc.||Deleting content in a distributed computing environment|
|US8862644 *||Nov 28, 2012||Oct 14, 2014||Translattice, Inc.||Data distribution system|
|US20050267950 *||Jun 1, 2004||Dec 1, 2005||Hitachi, Ltd.||Dynamic load balancing of a storage system|
|US20060274761 *||Dec 20, 2005||Dec 7, 2006||Error Christopher R||Network architecture with load balancing, fault tolerance and distributed querying|
|US20080228923 *||Mar 12, 2008||Sep 18, 2008||Oracle International Corporation||Server-Side Connection Resource Pooling|
|US20120102226 *||Jun 17, 2011||Apr 26, 2012||Microsoft Corporation||Application specific web request routing|
|US20130159366 *||Nov 28, 2012||Jun 20, 2013||Translattice, Inc.||Data distribution system|
|International Classification||H04L29/06, H04L29/08, G06F15/16|
|Cooperative Classification||H04L69/329, H04L67/1002, H04L67/1017, H04L67/327, H04L67/1019, H04L2029/06054, H04L29/06|
|European Classification||H04L29/08N9A1G, H04L29/08N9A1F, H04L29/08N31Y, H04L29/06|