US 20050138162 A1
A system for managing computer resource usage quotas includes a distributed gateway service, a plurality of clients, an internal network, a distributed storage service a system management service, a configuration database, and a distributed metadata service. A method for managing computer resource usage quotas applied to a plurality of users of data includes the steps of defining one or more resource classes, defining at least one usage policy for the resource classes, tracking quota usage for the usage quotas in accordance with the usage policy, and enforcing quota usage for the usage quotas, and storing the data in accord with the resource class definitions. Optionally, the quota usage may be aggregated.
14. A method for managing the cost of utilization of computer resources, said method comprising the steps of:
a) defining classes of computer resources based on quality of service attributes the classes having assigned costs;
b) defining computer resource usage policies based on said class definitions;
c) managing the cost of utilization of the computer resources by using the computer resources according to said usage policies; and
d) controlling usage of the computer resources by particular users based on the assigned costs and usage policies.
15. The method as defined in
16. The method as defined in
17. The method as defined in
18. The system as defined in
19. The method as defined in
20. A method for managing computer resources, comprising the steps of:
a) defining classes of persistent computer resources, said classes having assigned costs;
b) defining computer resource usage policies;
c) applying said usage policies to said classes;
d) generating reports on the usage of said classes; and
e) controlling usage of said persistent computer resources by particular individuals according to said defined classes in accordance with said computer resource usage policies and assigned costs.
22. The method as defined in
i) defining quality of service attributes.
23. The method as defined in
24. The method as defined in
25. The method as defined in
i) assigning usage quotas to said defined classes.
26. The method as defined in
i) tracking, aggregating, and enforcing said usage quotas.
27. The method as defined in
28. The method as defined in
29. The method as defined in
30. The method as defined in
31. The method as defined in
32. The method as defined in
The present application is related to commonly owned, co-pending U.S. application Ser. No. 09/709,187 entitled “Scalable Storage System” by David Raccah et al., filed Nov. 10, 2000; U.S. application Ser. No. 09/731,418, entitled “Symmetric Shared Storage System,” by Olaf Manczak et al., filed Dec. 5, 2000; and U.S. application Ser. No. 09/664,677, entitled “File Storage System Having Separation of Components,” by George Feinberg et al., filed Sep. 19, 2000; each commonly owned by the present assignee, the contents of each being incorporated herein by reference in their entirety.
The present invention relates generally to computing systems, and more particularly to a method and system that define, track, and enforce resource usage quotas based on classes.
Computer network technology has improved the exchange of and access to information. As a result, computer use has become pervasive for both business and personal purposes. With increased use, however, has come the requirement for enterprise and co-location hosting facilities to store vast amounts of data. The increased demand for storage presents several challenges to providing a tracking and enforcement system that manages resource usage.
In any computing system, including distributed scalable storage systems, resources are not infinite and must be managed. Among the types of resources that must be managed, are for example and without limitation, storage resources available to system users, bandwidth, and even the number of files. Managing resources requires defining, tracking, and enforcing user quotas associated with each resource. For example, conventional quota tracking and enforcement methods for storage resources often define user quotas in physical terms, such as how much space a user is allocated in a given file system. Although these conventional storage quota management methods are useful in some respects, they fail in others. For example, they do not provide for the definition of other characteristics about a storage resource that may be useful to manage, and they do not track or enforce quotas through the use of logical groupings, also called classes, which allow for more flexibility in quota definition, tracking and enforcement.
Using conventional methods to manage storage quotas, the quotas are viewed in terms of physical bytes. However the cost of resources in a distributed computing system and more particularly a distributed scalable storage system is also influenced by a multitude of other factors. Therefore, traditional storage quota management methods fail to adequately define, track, and enforce usage quotas and appropriately allocate their respective costs.
Thus, there exists in the industry a need to provide an improved quota tracking and enforcement method and system, which apply logical groupings, that is, classes, to user quotas. More particularly, the industry needs a method, which operates in a distributed computing system, which uses classes to define usage quotas, track resource usage, and enforce the defined quotas. The method needs to be applicable throughout the distributed system wherever quotas need to be allocated for a system resource, for example and without limitation, storage quotas. In the case of storage quotas, a user may want to use classes based on quality of service attributes of the system to determine how file space will be allocated. Classes add a level of abstraction to the concept of assigning resources to users. Moreover, a specific cost for storing a file can be associated with each storage class. This means a storage class quota provides information regarding how much storage space a user is allowed to use, while a storage class identifies the cost associated with the type of space that the user can use. Additionally, the method and system should be flexible enough to allow tracking and enforcement of quotas for system attributes for example and without limitation bandwidth and number of files, and not simply those attributes that are defined on a per-directory or per-file system basis.
In light of the above, it would be desirable to arrive at an approach to a quota tracking and enforcement system and method that may be used to monitor and control usage quotas without some or all of the drawbacks to conventional approaches described above.
Accordingly, it is an advantage of the present invention to provide a quota management system that allows the use of classes to provide highly customizable and easy to use resource usage management.
Another advantage of the present invention is to provide a quota management system in variety of process models such as but not limited to, a single process, multiple processes, or as distributed processes on multiple machines.
A further advantage of the present invention is to provide a highly scalable quota management system that can easily be scaled to provide for the expanding needs of customers.
Another advantage of the invention is to provide a quota management system for allowing the definition of classes that are stored in a configuration database and can be used for resource management.
Another advantage of the present invention is to provide a quota management system for allowing a tracking mechanism that monitors usage quotas.
Another advantage of the invention is to provide quota usage tracking for gathering quota usage information for a partition of a file system.
Another advantage of the present invention is to provide a quota management system for allowing an enforcement mechanism for usage quotas based on classes.
A further advantage of the present invention is to provide a quota management system for allowing the aggregation of the quota information for all partitions of a file system.
In view of the foregoing advantages and others, the present invention provides a system and method for defining, tracking, and enforcing quotas based on classes. According to the disclosed embodiments, a quota usage system may include a quota usage server function, quota usage updater function, and a quota usage database that implements tracking and enforcement services. In addition, the disclosed system and method are scalable. This is in contrast to conventional approaches that include quota tracking and enforcement, but cannot provide class-based quota management or highly scalable and customizable quota management.
These and other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures, wherein:
The present invention will now be described in detail with reference to the drawings, which are provided as illustrative examples of the invention so as to enable those skilled in the art to practice the invention. Notably, the implementation of certain elements of the present invention may be accomplished using software, hardware or any combination thereof, as would be apparent to those of ordinary skill in the art, and the figures and examples below are not meant to limit the scope of the present invention. Moreover, where certain elements of the present invention can be partially or fully implemented using known components, only those portions of such known components that are necessary for an understanding of the present invention will be described, and detailed descriptions of other portions of such known components will be omitted so as not to obscure the invention. Further, the present invention encompasses present and future known equivalents to the known components referred to herein by way of illustration. The preferred embodiment of the present invention will be described herein with reference to an exemplary implementation in a distributed, highly scalable storage system. However, the present invention is not limited to this exemplary implementation, but can be practiced in any computing environment including a distributed computing environment.
Looking now to
The distributed, scalable MDS 15, which is a component of the storage system 300, can be made up of one or more metadata partitions (84 a-84 n). A metadata partition cannot belong to more than one file system and each file system has at least one partition. It should be noted that a file system is a logical hierarchical representation (“tree”) of, for example and without limitation, files, directories, and devices on a single physical server with a specified “root”. Usage quotas apply on a file system basis and not on an individual metadata partition basis.
Each metadata partition (84 a-84 n) is made up of an MDS Server (101 a-101 n) and an MDS database (100 a-100 n). Each MDS Server (101 a-101 n) includes a Quota Usage Server (20 a-20 n) function and Quota Usage Updater function (40 a-40 n). Each MDS Database (100 a-100 n) includes a Quota Usage Database portion (30 a-30 n). The terms Quota Usage Server function and Quota Usage Server are equivalent and can be used interchangeably. The terms Quota Usage Updater function and Quota Usage Updater are equivalent and can be used interchangeably. The terms Quota Usage Database portion and Quota Usage Database are equivalent and can also be used interchangeably. Together, the Quota Usage Server, Quota Usage Updater, and Quota Usage Database make up the Quota Usage System 10 a-10 n. It should be noted that the Quota Usage Server, Quota Usage Updater, and Quota Usage Database can be implemented as a function within one or more processes or as individual processes while operating within the spirit and scope of the invention.
The distributed, scalable Storage Service 16, which is a component of the storage system 300, can be made up of storage servers 85 a-85 n and storage devices 87 a-87 n. Each storage server 85 a-85 n can include one or more storage classes 88 a-88 n. Storage classes provide a level of abstraction to quotas by assigning a collection of Quality of Service (QoS) attributes to a particular storage class. Thus, instead of quotas being viewed in terms of physical bytes, quotas can now be viewed in terms of cost. This concept of cost can also be applied to other storage characteristics, such as replicating data locally or remotely, storing data redundantly (using Redundant Arrays of Inexpensive Disks (RAID) or some other device technology), and so forth. QoS attributes can define storage device characteristics, for example and without limitation, performance (speed), reliability (redundancy), and accessibility (availability). Therefore, associated with each storage class is the cost of storing the data. The designation of storage classes is provided by way of example only and those of ordinary skill in the art will understand that numerous other resources such as network bandwidth, CPU usage, and memory usage may be designated and resource classes can be used to provide a level of abstraction to quotas by assigning a collection of resource-specific QoS to a particular resource class. Similarly, instead of quotas being viewed in terms traditional to the resource, quotas for resources can now be viewed in terms of cost. A distributed, scalable Quota Enforcement Service 19, which is an optional component of the storage system 300, can be made up of one or more Quota Aggregation and Enforcement Service nodes 50 a-50 n. Each service node aggregates quota usage and quota usage limit information, and initiates quota enforcement for a single file system, that is, for all the partitions that make up a file system.
Each MDS 15 manages one or more MDS partitions 84 a-84 n, but each MDS partition 84 a-84 n is associated with only one MDS 15. An MDS database 100 a-100 n for a single file system can contain data for one or more (not shown) metadata partitions 84 a-84 n. A Quota Usage System 10 a-10 n handles quota usage information 160 a-160 n, which applies to a file system as a whole rather than to metadata partitions 84 a-84 n separately. However, the quota usage information 160 a-160 n can be maintained on an individual metadata partition 84 a-84 n basis.
The storage system 300 is illustrated as comprising a distributed Gateway Service 13, made up of Gateway Service nodes 70 a-70 n, a distributed Storage Service 16, made up of storage servers 85 a-85 n and storage devices 87 a-87 n, and a distributed Metadata Service 15, made up of MDS servers 101 a-101 n and MDS databases 100 a-100 n, but those skilled in the relevant art(s) will understand, based on the teachings contained herein, that fewer or additional, Gateway Service nodes 70 a-70 n, storage servers 85 a-85 n, storage devices 87 a-87 n, MDS servers 101 a-101 n and MDS databases 100 a-100 n, as well as other services may be removed from, or added to the illustrated storage system and still be within the spirit and scope of the invention.
It should be noted that the Storage System 300 as shown in
Looking now to
When at least one class is defined, the management of resource usage quotas method can proceed to the usage-policies-definition phase. During this phase, usage quota policies are defined for a plurality of users (step 240). A usage quota policy helps a customer to automate the quota assignment process, that is, a customer can define a usage policy that can be applied to any number of users. A policy can be defined as a set quotas of one or more storage classes. Any user defined with that policy would have the storage classes associated with the policy assigned to that user. Usage quota policies automate the quota assignment process. Instead of individually assigning quotas of various storage classes to each user individually, policies can be used to easily assign quotas to a class of users. For example, Usage Policy A can assign to each user assigned this policy 10 MB of Very Reliable Storage, 20 MB of Reliable Storage, 10 MB of Adequately Reliable Storage and 5 MB of Very Highly Reliable Storage. Usage Policy B can assign to each user assigned this policy 20 MB of Reliable Storage and 40 MB of Adequately Reliable Storage. The customer can assign Usage Policy A to Users 1-6, and Usage Policy B to Users 7-12. A customer typically configures several users and/or several user groups, that is, a class of users, and for each selected user or user group assigns selected usage quotas for each class defined in steps 210 and 220, for example and without limitation, storage class, as needed, or sets a usage policy. The SMS 17 then creates, in the CDB 18, the quota limits for each user or file system based on the configuration information, that is, specific usage quota assignment or policy, provided by a customer. Defining and reporting quotas in the present invention will be described below in more detail in connection with
When at least one usage policy is defined, the management of resource usage quotas method can proceed to the application-of-quotas phase. This phase is done without operator intervention. During this phase, the Quota Usage system (10 a-10 n on
When quotas are defined for a user or file system in the CDB (18 in
Quota usage information is maintained on an individual metadata partition basis. Aggregating quota usage information and enforcement allows quota usage information maintained by the Quota Usage System on an individual metadata partition basis to be collected together for an entire file system, that is, for all partitions that make up a file system (optional step 260). A Quota Aggregation and Enforcement Service (19 in
In addition, because quotas can be defined, tracked, and enforced by file owner or an entire file system (that is, one or more metadata partitions), many different reports relating to quota usage information can be generated (step 280). The customer can customize reports and thereby has flexibility in determining the types of reports to be generated.
Upon successful resource class definition, usage policy definition, and usage quota application (that is, tracking, aggregating, and enforcing), data can be stored accordingly in a Storage Service 16 (made up of interconnected Storage Servers 85 a-85 n and Storage devices 87 a-87 n in
Looking now to
Looking now to
Looking now to
Referring now to
To gather quota limit information, a Quota Usage Aggregation and Enforcement service polls the CDB 18 for quota limits (step 710). More particularly, each Quota Aggregation and Enforcement Service node 50 a-50 n polls the CDB 18 for quota limits pertinent to a file system for which it is gathering information. This includes quota limit information about the file system and the users that can own files in that file system. Upon receipt of this information, a Quota Aggregation and Enforcement Service node 50 a-50 n creates a quota table, which includes quota limit information for a file system, and all users and storage classes associated with that file system for which a quota limit has been set (step 715). Storage classes are defined per user and per file system. When a Quota Aggregation and Enforcement Service node 50 a-50 n creates a quota table, it can periodically poll the Quota Usage Database in each partition for quota usage information (step 720). The Quota Usage Database sends quota information to a Quota Usage Aggregation and Enforcement Service node (step 725). It can then add the information it receives from the Quota Usage Database to the quota table (step 728). The quota table can then contain the quota limit information as well as the current quota usage information, thereby providing a snapshot of the use of resources by class for each file system and user. The Quota Usage Aggregation and Enforcement Service node 50 a-50 n checks whether a quota limit for a file system, storage class, or user is exceeded or nearly exceeded (step 730). The quota limit can be a hard quota or a soft quota. A hard quota would be a limit that cannot be exceeded under any circumstances. A soft quota, for example and without limitation, can be exceeded for a predefined period of time with perhaps warning messages sent to the user. A soft quota is more like a policy that determines behavior when nearing the allowable use of resources. For example and without limitation, a soft limit can be made up of three numbers, a soft quota storage size, a predefined time that specifies how long a user can be over the soft quota, and a hard quota. The behavior of an enforcement system when a user is nearing the usage limit can be simple to elaborate depending on customer requirements. For example, once a quota limit is exceeded or nearly exceeded (step 735), the Quota Aggregation and Enforcement Service node 50 a-50 n can send a notification to the Gateway Service nodes 70 a-70 n, which can update the relevant file system. The notification can indicate that the quota limit for the file system, storage class, or user has been exceeded or is about to be exceeded (step 740). It should be further apparent to those skilled in the relevant art(s) that policies for handling soft or hard quotas can vary widely and still be within the spirit and scope of the invention.
Whether the Quota Aggregation and Enforcement Service 19 sends a notification when a limit has been exceeded or some measure before can be configurable. It should be noted that the Gateway Service node 70 a-70 n performs the actual enforcement by rejecting or accepting a request to create or modify a file from a client.
Although the present invention has been particularly described with reference to the preferred embodiments thereof, it should be readily apparent to those of ordinary skill in the art that changes and modifications in the form and details may be made without departing from the spirit and scope of the invention. It is intended that the appended claims include such changes and modifications. It should be further apparent to those skilled in the art that the various embodiments are not necessarily exclusive, but that features of some embodiments may be combined with features of other embodiments while remaining with the spirit and scope of the invention.