|Publication number||US20070043860 A1|
|Application number||US 11/503,090|
|Publication date||Feb 22, 2007|
|Filing date||Aug 10, 2006|
|Priority date||Aug 15, 2005|
|Also published as||US8799431, US20140344462, WO2007021836A2, WO2007021836A3|
|Publication number||11503090, 503090, US 2007/0043860 A1, US 2007/043860 A1, US 20070043860 A1, US 20070043860A1, US 2007043860 A1, US 2007043860A1, US-A1-20070043860, US-A1-2007043860, US2007/0043860A1, US2007/043860A1, US20070043860 A1, US20070043860A1, US2007043860 A1, US2007043860A1|
|Original Assignee||Vipul Pabari|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (7), Referenced by (206), Classifications (35), Legal Events (1)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application claims priority to co-pending U.S. Provisional Patent Application No. 60/708,473, filed on Aug. 15, 2005, the contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates generally to computer network systems and, more particularly, to management of computer network virtualization environments.
2. Description of the Related Art
Information Technology (IT) management tasks can be characterized into two general areas, managing present day operations and forecasting capacity for future operations. Ensuring the well being of the current IT environment while using current trends to predict needs and trends for the future business needs is a fine balance and a highly refined skill. Management needs all the help and tools it can find to assist it in these tasks. Today, all major IT management platforms support the International Telecommunications Union (ITU) standard for Element Management Systems (EMS), wherein general functionality can be split into five key areas: Fault, Configuration, Accounting, Performance, and Security (FCAPS).
This conventional methodology has created an element-driven management system, with a focus on ensuring that each of the individual elements are running to their full potential. As the number of elements grew, the need for aggregated and correlated information increased. As the number of data-center locations grew, the need for global visibility and control increased. Conventionally, IT capacity planning for day-to-day operations is typically carried out with a bottom-up data aggregation and with the use of forecasting methods such as trending, simulation, and custom analytics.
Capacity planning for resources is also typically completed when new business applications are rolled out or during an application upgrade cycle. In this capacity planning scenario, the planning is typically carried out at the individual device level, which is then multiplied by the number of consumers and/or producers and further multiplied by the number of locations that need to be supported, giving a large number in the aggregate: (number of individual devices)×(number of consumers/producers)×(number of locations requiring support).
In order to estimate what resource capacity an enterprise will need to support the core business applications it provides, an enterprise will typically evaluate the worst case usage scenario, and bolster its capacity to ensure that a worst case scenario will be adequately supported. What is often overlooked is that this type of worst-case capacity planning typically is driven by the vendors who have built ROI calculators that are to their benefit.
By planning and bulking up resources to combat this evasive worst case scenario, enterprises typically end up with under-utilized IT resources. Applying the Pareto Principle, an estimate of how much under-utilized capacity can exist in a single enterprise would be as follows: only 20% of the available capacity ends up being used during 80% of the time during a given timeframe.
Many improvements have been made in the enterprise scenario for management of IT resources. At the macro-level, IT resources can be classified into 4 categories: (1) client resources (client resource examples include desktop machines, wireless, handheld devices, client grids (such as SETI@home and the like); (2) server resources (server resource examples include mainframe, File Server, Web server, peer-to-peer servers, blade servers, grid servers etc.); (3) network resources (network resource example include routers, switches, bridges, infiniband, wireless, radio, optical, fiber channel, link aggregation technologies (such as BitTorrent and the like); (4) storage resources (storage resource example include databases, network attached storage, storage area networks, data grids etc.). While one can describe a capacity planning scenario for each of the categories above, they all follow a very similar capacity planning process.
In the following example, we will describe a typical server resource capacity planning scenario. Servers in the enterprise have evolved with new application architecture. Application topologies have evolved from Mainframe-Green Screen interaction, to Client/Server, to Client/Web Server/Application Server/Database, Peer-to-Peer, and so forth. Server resource capacity planning is typically achieved by stress-testing the application with a certain predetermined workload and a set, acceptable application response time. A hardware specification is defined to support a user-defined worst-case scenario. The application is rolled out on the new hardware into a production environment.
Server resource and application utilization is monitored by a FCAPS-compliant management platform to provide complete visibility over operations. In order to provide an aggregated summary view, such management platforms typically roll-up element-level metrics into higher level metrics through data correlation techniques.
Conventionally, with the emergence of resource virtualization and the increase in use of Web services, combined with service-oriented architectures (SOAs), the number of moving pieces that need to be managed for the enterprise continues to rise. For example, imagine an enterprise running composite applications. That enterprise would include using a mixture of legacy, local, and external Web services, running on virtual infrastructure spread globally across the enterprise, and frustrated end users can't complete their mission critical business tasks. It is difficult to achieve sufficient visibility and control to manage such an environment, and knowing where to begin to manage such an environment can be difficult.
Traditional methods of resource planning at the individual physical resource level begin to show their age. For example, correlation and aggregation of element level data also becomes compute-intensive with the increase in the number of managed elements. It has been said that “The information technology industry is in a strange situation. We have enormously sophisticated engines that we're running—in the form of CPUs and communications equipment and so forth, but the way we keep them running is through an outdated vision.” Doug Busch, Intel Vice President and CIO-Technology quoted in Intel Magazine, September/October 2004 (available at the URL of: www.intel.com/update/contents/it09041.htm as of June 2005). Management of virtual assets can be achieved conventionally with virtualization software tools, but such techniques are typically labor intensive and require manual selection and implementation of configurations and utilize relatively cumbersome configuration change management.
In network systems, with virtualization, it is possible to deploy physical resources in the form of virtual assets. The assets can thereby provide the functional equivalent of desktops (user interfaces), operating systems, applications, servers, data bases, and the like. Adding additional applications can be implemented by remotely executed software operations in virtual environments on one or more computers, rather than physical installations involving personnel with an installation CD media at each physical location (computer) of a network where the additional applications are desired.
The management of such virtual assets, however, is becoming increasingly complex and unwieldy. Many tools to assist in the management of virtualization environments are proprietary and work only with virtual environments from particular vendors. Similarly, some virtualization tools might only work with specific central processor units (CPUs) of machines that host the virtual environment, or might only work with specific operating systems or virtualization platforms of either the host machine or in the virtual environment. This characteristic can make it necessary to have multiple tools on hand for the various platforms and vendors that might be deployed throughout a network, as well as making it necessary to acquire and maintain the skill sets necessary to use such tools. The mere fact of requiring such diverse tools is, itself, inefficient. Thus, although virtualization trends show much promise for more efficient utilization of physical resources by optimal deployment of virtual assets, the virtualization environment management task is daunting.
It would be advantageous if more efficient means for managing virtual environments across computer networks were available. The present invention satisfies this need.
The present invention provides methods and apparatus for management of one or more virtual environments regardless of any underlying central processing unit (CPU) specification and regardless of any underlying operating system (OS) or virtualization environment. In one embodiment of the present invention the virtualization environment is managed through a Control Center application that provides an interface to virtualization environments in communication with the Control Center computer. The system, through the Control Center, provides active management of the virtualization environment by initiating automatic responses to operational situations that influence dynamic demands on the physical resources and virtual assets. In this way, multiple virtual environments can be managed through a single user interface of a Control Center application even where the underlying CPU of the system physical resources are different from that of the Control Center, even where the operating systems of the Control Center, physical resources, and virtual assets are different, and even where the virtualization environments being managed are different from each other.
In one embodiment, the Control Center can comprise a collection of functional elements characterized as an Asset Manager, a Provisioning Manager, a Dynamic Application Router, an Optimizer, a Performance Manager, and a Capacity Planning Manager. With these functional elements, the process of managing the virtual environment comprises a sequence of building an inventory of available physical resources and virtual assets, provisioning the assets for a desired network virtual configuration, optimizing the mix of physical resources and virtual assets, reporting on the system performance, and planning for future trends and forecasting for needed capacity.
In another embodiment, management of computer network virtualization environments is provided by performing one or more functions from among the set of functions including (1) identification and management of network resources and virtual assets, (2) provisioning of virtual assets in response to network workflow demands, (3) dynamic deployment of virtual assets across the computer network, (4) performance measurement and reporting of resources and virtual assets, and (5) planning and forecasting of resource demands and asset utilization of the virtualization environment, such that the functions are carried out without regard to processors, operating systems, virtualization platforms, and application software of the virtualization environment. In this way, an inventory of resources and assets available at a network virtualization environment is determined, prioritization is assigned to an inventory of available resources and assets, and the inventory is utilized by allocating the virtual assets in the virtualization environment. The allocated virtualization environment is automatically managed by determining real time performance metrics for the environment, and producing a reallocation of the inventory based on the real time performance metrics. In this way, automatic and efficient virtualization management of a computer network is provided.
Other features and advantages of the present invention should be apparent from the following description of the preferred embodiment, which illustrates, by way of example, the principles of the invention.
Various embodiments of the present invention taught herein are illustrated by way of example, and not by way of limitation, in the FIG. s of the accompanying drawings, in which:
In the drawings, like reference numerals refer to like structures. It will be recognized that some or all of the Figures are schematic representations for purposes of illustration and do not necessarily depict the actual relative sizes or locations of the elements shown. The Figures are provided for the purpose of illustrating one or more embodiments of the invention with the explicit understanding that they will not be used to limit the scope or the meaning of the claims.
In the following paragraphs, the present invention will be described in detail by way of example with reference to the attached drawings. While this invention is capable of embodiment in many different forms, there is shown in the drawings and will herein be described in detail specific embodiments, with the understanding that the present disclosure is to be considered as an example of the principles of the invention and not intended to limit the invention to the specific embodiments shown and described. That is, throughout this description, the embodiments and examples shown should be considered as exemplars, rather than as limitations on the present invention. Descriptions of well known components, methods and/or processing techniques are omitted so as to not unnecessarily obscure the invention. As used herein, the “present invention” refers to any one of the embodiments of the invention described herein, and any equivalents. Furthermore, reference to various feature(s) of the “present invention” throughout this document does not mean that all claimed embodiments or methods must include the referenced feature(s).
In this description, physical devices such as computers, printers, routers, and other “boxes” will be referred to as resources, whereas virtual devices that exist only as software instantiations of equipment objects in a virtual environment will be referred to as virtual assets. The resource-asset dichotomy will be maintained throughout this discussion.
The present invention provides automated management of network-based virtual environments through Control Center software that supports on-demand operation of one or more functions including (1) identification and management of enterprise resources and virtual assets, (2) provisioning of virtual assets in response to network workflow demands, (3) dynamic deployment (routing) of virtual assets across the network, (4) performance measurement and reporting of virtual assets and resources, and (5) planning and forecasting of resource demands and asset utilization. Such operations are carried out without regard to the mix of otherwise proprietary processors, operating systems, virtualization platforms, application software, and protocols. These features and functions are provided in a modular fashion so that desired functions can be included in the system, and functions not desired can be excluded from the system. In any implementation in accordance with the present invention, the virtualization management system is transparent to the virtual environment being managed, in that the system can support multiple virtual environments with different protocols and operating specifications. Thus, the disclosed system is platform-independent.
Each Control Center computer 110, 112 that is equipped with the virtualization management application described herein will have management of one or more physical assets within a domain or other subnet arrangement associated with the respective Control Center.
The first set of resources 120 are illustrated as comprising two blade servers, indicated in
As described further below, the illustrated embodiment of the Control Center 202 is configured in a modular fashion, such that the installation of the Control Center of the computers 110, 112 in
The Asset Manager 204 provides visibility and management of all global virtual assets available to the Control Center. This component provides a global view of all the resources used in a virtualized infrastructure along with a physical and logical topology view. Thus, a user can view a global topology view and an inventory of virtual infrastructure from a central console. In addition, the component will support discovery and configuration of virtual resources, allow topology views for physical and logical infrastructure, provide inventory reports at local, remote, and global context, and simplify management of applications, hardware resources, virtual assets, and operating systems.
The Provisioning Manager 206 provides an on-demand solution for provisioning of virtual assets for an automated workflow management. This component provides an automated solution allowing users to request and schedule their individual virtual asset needs and IT management to prioritize and provision the required assets on demand. The component can be used to provide a portal for users' virtual asset requests showing available virtual assets, as well as a supervisor portal to prioritize needs and approve asset needs by time and priority, and also provides an IT manager to provision the needed resources and keeping track of used and available assets, an automated workflow system for users and IT to track needs and resources, supports the ability to keep mission critical application and assets to put them on line on demand for emergency and disaster recovery needs, and provides a repository of virtual machines supporting VMWare, Microsoft and Xen virtual assets and their needed hardware components. The Provisioning Manager provides central management of global virtual machine images, provides an enterprise workflow system for adds, moves and changes in virtual infrastructure, and can be used to standardize and optimize virtual infrastructure use from lab to production environment, such as needed in different development testing scenarios.
Dynamic Application Router
The Dynamic Application Router 208 provides real time and dynamic routing of applications running on virtual infrastructure. With this component, users of the Control Center can move applications running on virtual assets by comparing application usage with business policy and drivers, and scheduling appropriate routing actions. This component can be used to move applications to different virtual assets as global business need changes, provide dynamic allocation of global virtual assets for workload optimization, enable zero down time in upgrading virtual environments, provide an optimizer for balancing resource and asset inventory against needs, a scheduler to auto schedule actions and provide reports, and provide alarms and triggers to notify users of out of balance inventory items. The component also provides an aggregated action library for all virtual infrastructure, performs zero downtime virtual asset maintenance and upgrades, ensures a high availability plan for mission critical applications running on virtual infrastructure, can compare business policy against real time asset usage and then make load balance changes, and can provide top 10 recommendations for solutions.
The Optimizer 210 component operates on the virtual assets of a physical resource to provide efficient configuration of the assets in accordance with a set of business rules. For example, the Optimizer component may operate on a user desktop or on a server that manages a virtual environment so as to configure an efficient combination of applications and assets, such as virtual devices of the host machine. In this way, the Optimizer provides a user with control over how virtual assets use the underlying physical resources. Users can set business policies based on application criticality and can let the Optimizer allocate appropriate physical resources in accordance with the business policies. The Optimizer can allocate resources such as CPU, network, memory, hard disk, and the like in accordance with the business policies that have been set by the user. Configuration of the business policies lets the user have a high degree of control over the allocation of resources. For example, users can set a maximum limit on how much each virtual asset can use the underlying physical resource.
The Performance Manager component 212 provides availability and performance management for all virtual assets in the system. This component shows performance of virtual asset usage, provides key performance matrices and metrics, and identifies future potential bottlenecks. The component can be used to provide real time monitoring and viewing of all virtual assets usage, measure and trend performance and show it against plan, provide triggers, alerts, and alarms for key performance matrices and metrics, identify current and predicted bottlenecks, and provides a cross-platform solution.
Capacity Planning Manager
The Capacity Planning Manager component 214 provides capacity planning, trending and forecasting based on usage and needs. This component provides capacity planning, trending and forecasting of virtual assets based on historical trends and future projected business needs. The component can be used to show used versus available capacity of virtual assets, trends capacity usage and compares it with thresholds, provides historical trend reports, provides forecasting of future needs, and provides alarms and triggers based on capacity usage and thresholds.
The computer device 300 includes an application that provides a virtualization layer 322 that manages virtual assets 326. Thus, the computer comprises a host machine for the virtualization environment. In
In accordance with the invention, a Control Center (such as illustrated in
For example, if the Asset Manager component 204 can communicate directly with the Virtualization Layer using native interfaces of the virtualization layer, then the Control Agent 324 is not needed. This would likely be the case if, for example, the virtualization layer is provided by VMWare, which includes controls needed for external communications. Other virtualization software, such as provided by Xen and Microsoft, does not typically include such native controls and therefore a Control Agent 324 would be necessary. Those skilled in the art will appreciate that such controls typically include actions such as “Move VM”, “Migrate VM”, “Clone VM”, and the like, which support movement, migration, and cloning of virtual machines and which usually can be invoked programmatically by all virtualization platforms. It is these controls that may be implemented by the Control Agent 324, in the absence of a native control in the virtualization platform. If the host machine does include a Control Agent 324, then the Control Agent 324 will have components to facilitate communication between the Control Center and the Virtualization Layer 322. In an alternative embodiment, the Control Agent 324 can be integrated into the Control Center itself, such that the Control Center can communicate directly with the Virtualization Layer 322 of the host machine, regardless of the virtualization middleware that is actually installed at the host machine. Those skilled in the art will understand how to integrate the functionality of the Control Agent 324 described herein into the Control Center, without further explanation, in view of the description provided.
In the Control Agent 324, an Action Event Receiver 404 receives notifications about incoming events for which a response or action is required. Such notifications will typically involve, for example, changes in status of a virtual machine or requests for action or service. The Control Agent 324 also includes a Server Monitor 404 that checks for status of the virtual applications of the host machine, and also checks the status of the external Control Center with which it is communicating. As noted above, each host machine is associated, or managed, by a designated Control Center. The Control Agent 324 also includes an Event Dispatcher 408, which initiates actions by the host machine in response to the incoming events received by the Action Event Receiver 404. A Control Center Interface Layer 414 provides a monitoring interface function, a management interface function, and a statistics interface function for data exchange between the Control Agent 324 and the associated Control Center. A Virtual Platform Abstraction Layer 416 permits communications between the host machine 302 resources and a set of multiple middleware adapters 418, as described further below.
In the server virtualization of the illustrated embodiment, the Control Agent 324 acts as a proxy between the Control Center 110, 112 and server instances. The Control Agent manages the messaging between the Control Center and the actual VMs. The Control Agent 324 will be created per host and per VM, as required. The Control Agent communicates with the Control Center through a communications protocol, such as the WS Management Catalog Protocol specification, wherein the Control Agent 324 is implemented as a service and will have the set of virtual assets that it manages. Other communications schemes can be used, as will be known to those skilled in the art. The VM's and Hosts are the available resources, as specified by the protocol. The Control Agent 324 is assigned a unique URI and will provide a selector to select a VM instance running within the server.
The Control Agent 324 can create a VM using a Resource configuration feature. The Control Agent 324 is responsible for a variety of tasks, including:
The Control Agent 324 also supports communication between the Abstraction Layer 416 and different Virtualization servers (virtualization platforms). The Abstraction Layer supports interfaces for monitoring, managing, and collecting statistics from the VMs and hosts.
The Control Center 110, 112 can start the Control Agent 324. The Control Agent 324 has its lifecycle independent of the Control Center and can handle reconnection with the Control Center, when it starts.
The Control Agent 324 also controls and monitors the health of the host machine. The Control Agent 324 is responsible for mining the system performance statistics such as the network performance, memory usage, and the disk usage of the host and passing these statistics on to the Control Center.
If a particular VM fails, the Control Agent 324 tries to restart the VM, or if that fails, sends out an alarm message to the virtual server.
Data constructs, such as Communication Objects, implement the communication protocols between the Control Agent 324 and the Control Center 110, 112. The communication protocol can be based on the WS Catalog protocol, in use by VMWare. Other suitable protocols will occur to those skilled in the art, in view of this description.
The Control Center has virtualization management responsibilities that include:
The Control Center includes the following functional components to perform the above-mentioned responsibilities:
The operation of the components described above can be better understood with reference to
At box 508, the Control Agent 324 message is received at an Event Receiver, and Business Rules are executed to determine the available hosts for supporting the type of virtual machine that has failed. At box 510, the Load Balancing procedure identifies a host machine from the list of available hosts. Next, at box 512, the Event Dispatcher sends the failure event message to the Control Agent 324 of the selected host machine. Lastly, the Action Event Receiver of the selected host machine receives the event message and performs the action (startup VM) to restore the failed virtual machine.
Returning to the description of
The Administration aspect of the Control Agent 324 provides all the methods that are required for managing a virtual server, such as the GSX/ESX server platform from VMWare, and the virtual machines under its supervision. All methods related to starting, stopping, cloning, and moving a virtual machine are managed through the Administration interface.
The Monitoring aspect of the Control Agent 324 provides methods to check virtual machine status and review heartbeat information for a virtual machine. The monitoring interface includes methods to get the statistical information and compare it with specified thresholds and generate Action Events. The Control Agent 324 operation includes a “getAllEvents” method that validates each threshold value and generates the necessary Action Events.
The Statistical aspect of the Control Agent 324 collects the statistical information. The methods used by the Control Agent in these duties are responsible for obtaining performance statistics for CPU performance, disk performance, memory performance, and network performance. Those skilled in the art will understand the various performance metrics by which such performance is typically judged. The Control Agent 324 operation therefore includes methods such as getCPUperfStats, getDiskPerfStats, getMemoryPerfStats and getNetworkPerfStats, which are responsible for returning corresponding specific statistical objects such as CPUStats, DiskStats, MemoryStats, and NetworkStats. These methods are supported for a variety of servers, such as ESX servers, and some servers will require the Control Agent 324 to make system calls to obtain the information, such as for GSX servers.
As noted above, the Virtual Platform Abstraction Layer 416 of the Control Agent 324 includes a set of multiple middleware adapters 418. The middleware adapters communicate with the virtualization environments of the host machine 302. These virtualization environments are shown in
To provide an abstraction layer over a variety of virtualization servers such as from VMWare, Xen, and Microsoft, the Abstraction Layer 416 of the Control Agent 324 provides a common API access for the virtualization servers. To do so, the Control Agent 324 includes components in accordance with the virtualization platform and communication management protocol, components such as:
For example, the Control Agent 324 may include a “VMWareAgentImpl” class for implementation of a Control Agent 324 interface to VMWare systems. This implementation class makes calls to classes called “VMServerCtl” and “VMCtl”, wherein the VMServerCtl class includes methods related to the GSX and ESX servers. The VMServerCtl class is implemented as a singleton class. There would be only one instance of the Server object. This discussion assumes that there will be only one server per host (GSX server or ESX server). A host will not have the virtual machines of multiple virtualization servers (such as VmWare, Xen, and Microsoft). The server object will maintain a map of the VMName-to-VMCtl objects. The lookup will be on VMNames. The user can give the same name to multiple virtual machines on a server. Currently, VMWare does support duplicate names. The VMCtl and VMServerCTL classes invoke the com/perl interfaces of VMWare using JNI wrapper and JPL, respectively. The JNIWrapper is a DLL file which exposes methods from the VMCOM object.
The operations of the Control Center and Control Agent 324 to manage the virtual environment will support various use cases, or operating scenarios, including creation and startup of virtual machines (VMs). Such operations are described as follows for a VMWare environment (corresponding operations for other virtualization environments will be known to those skilled in the art, in view of this description):
Other actions can be supported by suitable methods, as desired:
The setting pane on the right side, below the OS listing pane, shows the user interface feature for setting operating parameters of the Control Center. The priority of the virtual machine under management (VM Priority) can be set with a display slider from low priority to high priority. A high priority setting means that the VM will have a high probability of being instantiated on the host machine under management. A low priority setting means that the VM is a likely candidate for deletion from the virtualization of the host machine if resource utilization is great and if other VMs have a higher priority setting. The virtualization parameters that can be set for the virtualization environment through the user interface of
The installed Control Center application provides a network-based, intelligent orchestration engine for automatic management of virtual assets across multiple computer virtualization platforms around a network. The exact management functions that can be performed by the Control Center will depend on the number of components selected for installation on the Control Center computer. The full complement of components are illustrated in
The Control Center application provides a centralized repository for virtual infrastructure configuration change management with an audit trail for IT compliance requirements. In accordance with the application, macro-level policies are defined based on business needs. These are implemented as Application Criticality Rules (ACRs), described further below. The Control Center operates as a virtual controller that can mediate between physical server, network, applications; and storage resources. Such mediation can occur based on a combination of portable, application-provided as well as user-defined, Knowledge Blocks (KBs). The Control Center can also provide Adaptive Application Routing (AAR) for virtual assets based on the ACRs. With the Knowledge Blocks, the Control Center can take intelligent actions based on the KBs when any of the ACRs are violated.
The ACRs allow a user to specify business rules for controlling priority settings for applications. For example, in a system with a set of five application App1, App2, App3, App4, and App5, one of the rules could specify that, each morning from 6:00 am to 10:00 am, the five applications will have priorities set as follows: App1 has high priority, App2 has low priority, App3 has medium priority, App4 has low priority, and App5 has high priority. If preferred, the Control Center can provide a priority setting that is numerical, such as a range of integral numbers between 0 (zero) and 10 (ten). Other priority range indicators can be used as desired, such as colors or other indexes. If traffic to an application is detected as being above a threshold level, another business rule could specify a dynamic response, that the priority for the corresponding application can be adjusted higher. Conversely, action could be specified to reduce priority if traffic is detected as abnormally low. Another business rule could be set for prospective action, for example, such that a particular application is given an increased priority according to time of day.
Control Center management of virtual assets and mediation between physical resources in conjunction with the ACRs provides a robust load balancing functionality with the ability to distribute server load among potential host machines, and determine when and how to increase or decrease the number of VM's to improve overall system throughput.
For purposes of the load balancing function, load can be calculated on a server by following several alternative criteria, or parameters. One parameter can be CPU run queue length, which returns the load as the average number of processes in the operating system (OS) run queue over a specific time period in seconds, normalized over the number of processors. Another parameter is CPU utilization, which returns the load as the CPU usage percentage. Other suitable parameters can include Network Performance (traffic throughput), Disk Performance, which can be measured by the number of bytes read and/or written in a specific time period in seconds, and Memory Performance, mainly by the ratio of the active memory used as compared with total managed memory. If desired, a weighted formula can be used for computing the load on a particular host based on these parameters.
With such parameters available, the load balancing strategy can be implemented with two different strategies: Static load balancing and dynamic load balancing. Static load balancing is similar to having a pre-defined set of minimum and maximum number of VM's that can run on a given host. The Dynamic load balancing approach suggests that, based on the current and the future load on a host, a decision is made as to which VM should run on which host. Once a set of hosts are identified, then the load balancer processing can pick a particular host, by a round-robin or a weighted round-robin method. The load balancer processing would first try to assign the high priority VM's to hosts with minimum load.
The Control Center also provides an interface to the installed configuration management application, which operates in conjunction with observed real-time events on the network. This is achieved through a four-step process that includes (1) monitor, discover and alert tasks, (2) applying user-guided fix automation tasks, (3) automatically applying basic KBs, and automatically applying more complex KBs and user-defined KBs. In addition, the Control Center provides a mechanism to centrally schedule resource re-allocation in response to defined business events.
The Knowledge Block feature can be used to specify analysis rules that can detect application performance degradation. In response to such detection, an alert can be sent over the network, such as an alert message being sent to a network administrator. After an alert message has been sent, the configuration management application can wait for an administrative action, or the application can be set up so as to execute an automatic action. For example, in response to a “crash” of a virtual machine, the application can find the last known “good” VM image from a repository and can deploy that image to the affected machine. In response to overtaxed applications, the application can deploy a new application instance, and can add new application instance information into a network load balancer to indicate the new instance is available and should be considered as part of the “available application pool”. In addition, the application can remove a degrading application from the available application pool, or can move a degrading application VM from a group of production servers to a debug server pool. Other functions that can be performed by the application in accordance with Knowledge Blocks include the capture of log files from the degrading application VM, the capture of archive logs and sending of email alert messages to an administrator with the archive log location, and the halting of a degrading application VM and removal of it from the debug server pool.
The server virtualization implementation described thus far can be used to control and manage a complete virtualization environment, including assets such as storage assets, virtual routers, and virtual desktops. Such configurations are depicted in
After the Control Center determines the collection of available virtualization assets, the next operation is for the application to apply the application rules 1604. These include the Application Critical Rules such as illustrated above in
If configuration changes are called for, an affirmative outcome at the decision box 1804, then the Control Center issues commands to the virtualization layer software to implement the desired configuration changes 1806. Those skilled in the art will understand how to implement such configuration changes without further explanation, given the description herein. If no configuration changes are called for, a negative outcome at the decision box 1804, then the Control Center next checks to determine if any of the ACRs or Knowledge Block rules have been violated by any system configuration settings or performance metrics, as indicated by the decision box 1808. If there has been a rules violation, an affirmative outcome at the decision box, then the application reports the violation 1810. The report can take the form of an alert email message generated by the Control Center application that is sent to a network administrator or other predetermined email mailing address. After the alert email message has been sent 1810, the Control Center issues commands to the virtualization layer software to implement the desired configuration changes 1812. Those skilled in the art will understand how to implement such configuration changes without further explanation, given the description herein. Operation then returns to determining real-time system performance metrics 1802 and the loop repeats for as long as the Control Center application is executing. If there was no rules violation, a negative outcome at the decision box 1808, then no configuration change is carried out and, instead, operation returns to determining the performance metrics and the loop repeats itself.
In this way, the Control Center provides automatic virtualization management for computer network systems that include network virtual assets. A wide range of installation enhancements can be implemented, including business (application critical) rules, import and export of rules, and administrator alert messages.
The present invention has been described above in terms of presently preferred embodiments so that an understanding of the present invention can be conveyed. One skilled in the art will appreciate that the present invention can be practiced by other than the above-described embodiments, which are presented in this description for purposes of illustration and not of limitation. The specification and drawings are not intended to limit the exclusionary scope of this patent document. It is noted that various equivalents for the particular embodiments discussed in this description may practice the invention as well. That is, while the present invention has been described in conjunction with specific embodiments, it is evident that many alternatives, modifications, permutations and variations will become apparent to those of ordinary skill in the art in light of the foregoing description. Accordingly, it is intended that the present invention embrace all such alternatives, modifications and variations as fall within the scope of the appended claims. The fact that a product, process or method exhibits differences from one or more of the above-described exemplary embodiments does not mean that the product or process is outside the scope (literal scope and/or other legally-recognized scope) of the following claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US20020166117 *||Sep 10, 2001||Nov 7, 2002||Abrams Peter C.||Method system and apparatus for providing pay-per-use distributed computing resources|
|US20030236882 *||Jun 25, 2002||Dec 25, 2003||Yong Yan||Automatic management of e-services|
|US20030236956 *||Jun 20, 2002||Dec 25, 2003||International Business Machines Corpoaration||File system backup in a logical volume management data storage environment|
|US20050044301 *||Apr 26, 2004||Feb 24, 2005||Vasilevsky Alexander David||Method and apparatus for providing virtual computing services|
|US20060069761 *||Sep 14, 2004||Mar 30, 2006||Dell Products L.P.||System and method for load balancing virtual machines in a computer network|
|US20060149842 *||Jan 6, 2005||Jul 6, 2006||Dawson Christopher J||Automatically building a locally managed virtual node grouping to handle a grid job requiring a degree of resource parallelism within a grid environment|
|US20060184937 *||Feb 11, 2005||Aug 17, 2006||Timothy Abels||System and method for centralized software management in virtual machines|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7441135||Jan 14, 2008||Oct 21, 2008||International Business Machines Corporation||Adaptive dynamic buffering system for power management in server clusters|
|US7805511 *||Apr 30, 2008||Sep 28, 2010||Netapp, Inc.||Automated monitoring and reporting of health issues for a virtual server|
|US7870360 *||Sep 14, 2007||Jan 11, 2011||International Business Machines Corporation||Storage area network (SAN) forecasting in a heterogeneous environment|
|US7890615||Sep 8, 2008||Feb 15, 2011||Kace Networks, Inc.||Architecture and protocol for extensible and scalable communication|
|US7987432||Apr 24, 2007||Jul 26, 2011||Parallels Holdings, Ltd.||Seamless integration and installation of non-native application into native operating system|
|US8065676 *||Apr 24, 2007||Nov 22, 2011||Hewlett-Packard Development Company, L.P.||Automated provisioning of virtual machines for a virtual machine buffer pool and production pool|
|US8074218 *||Mar 29, 2007||Dec 6, 2011||International Business Machines Corporation||Method and system for constructing virtual resources|
|US8103751||Jan 27, 2011||Jan 24, 2012||Kace Networks, Inc.||Architecture and protocol for extensible and scalable communication|
|US8117554||Mar 15, 2009||Feb 14, 2012||Parallels Holdings, Ltd.||Seamless integration of non-native widgets and windows with dynamically scalable resolution into native operating system|
|US8127290||Oct 4, 2007||Feb 28, 2012||Red Hat, Inc.||Method and system for direct insertion of a virtual machine driver|
|US8135824 *||Oct 1, 2007||Mar 13, 2012||Ebay Inc.||Method and system to detect a network deficiency|
|US8141090||Apr 24, 2007||Mar 20, 2012||Hewlett-Packard Development Company, L.P.||Automated model-based provisioning of resources|
|US8146080||Mar 30, 2007||Mar 27, 2012||Novell, Inc.||Tessellated virtual machines conditionally linked for common computing goals|
|US8146098||Sep 7, 2007||Mar 27, 2012||Manageiq, Inc.||Method and apparatus for interfacing with a computer user via virtual thumbnails|
|US8166143 *||May 7, 2007||Apr 24, 2012||Netiq Corporation||Methods, systems and computer program products for invariant representation of computer network information technology (IT) managed resources|
|US8171485 *||Sep 21, 2007||May 1, 2012||Credit Suisse Securities (Europe) Limited||Method and system for managing virtual and real machines|
|US8175863 *||May 8, 2012||Quest Software, Inc.||Systems and methods for analyzing performance of virtual environments|
|US8185893||Oct 27, 2006||May 22, 2012||Hewlett-Packard Development Company, L.P.||Starting up at least one virtual machine in a physical machine by a load balancer|
|US8185894||Sep 24, 2008||May 22, 2012||Hewlett-Packard Development Company, L.P.||Training a virtual machine placement controller|
|US8191141||Jun 22, 2007||May 29, 2012||Red Hat, Inc.||Method and system for cloaked observation and remediation of software attacks|
|US8201016 *||Jun 28, 2007||Jun 12, 2012||Alcatel Lucent||Heartbeat distribution that facilitates recovery in the event of a server failure during a user dialog|
|US8219063 *||Jun 26, 2009||Jul 10, 2012||Vmware, Inc.||Controlling usage in mobile devices via a virtualization software layer|
|US8219358||May 9, 2008||Jul 10, 2012||Credit Suisse Securities (Usa) Llc||Platform matching systems and methods|
|US8234641||Nov 27, 2007||Jul 31, 2012||Managelq, Inc.||Compliance-based adaptations in managed virtual systems|
|US8239526 *||Nov 13, 2009||Aug 7, 2012||Oracle International Corporation||System and method for performance data collection in a virtual environment|
|US8239557 *||Jun 25, 2008||Aug 7, 2012||Red Hat, Inc.||Virtualization management using a centralized server|
|US8250666 *||Jul 3, 2008||Aug 21, 2012||Sap Ag||Method and apparatus for improving security in an application level virtual machine environment|
|US8259616||Jan 21, 2009||Sep 4, 2012||Aerohive Networks, Inc.||Decomposition of networking device configuration into versioned pieces each conditionally applied depending on external circumstances|
|US8266280||Mar 17, 2010||Sep 11, 2012||International Business Machines Corporation||System and method for a storage area network virtualization optimization|
|US8291411 *||Mar 6, 2008||Oct 16, 2012||International Business Machines Corporation||Dynamic placement of virtual machines for managing violations of service level agreements (SLAs)|
|US8296760||Oct 27, 2006||Oct 23, 2012||Hewlett-Packard Development Company, L.P.||Migrating a virtual machine from a first physical machine in response to receiving a command to lower a power mode of the first physical machine|
|US8301737||Jan 23, 2012||Oct 30, 2012||Dell Products L.P.||Architecture and protocol for extensible and scalable communication|
|US8332847||Sep 23, 2008||Dec 11, 2012||Hewlett-Packard Development Company, L. P.||Validating manual virtual machine migration|
|US8336108||Oct 4, 2007||Dec 18, 2012||Red Hat, Inc.||Method and system for collaboration involving enterprise nodes|
|US8341626||Sep 29, 2008||Dec 25, 2012||Hewlett-Packard Development Company, L. P.||Migration of a virtual machine in response to regional environment effects|
|US8347355||Jan 21, 2009||Jan 1, 2013||Aerohive Networks, Inc.||Networking as a service: delivering network services using remote appliances controlled via a hosted, multi-tenant management system|
|US8352608||Apr 9, 2009||Jan 8, 2013||Gogrid, LLC||System and method for automated configuration of hosting resources|
|US8359594 *||May 27, 2010||Jan 22, 2013||Sychron Advanced Technologies, Inc.||Automated rapid virtual machine provisioning system|
|US8364639 *||Oct 10, 2008||Jan 29, 2013||Parallels IP Holdings GmbH||Method and system for creation, analysis and navigation of virtual snapshots|
|US8364802||Apr 9, 2009||Jan 29, 2013||Gogrid, LLC||System and method for monitoring a grid of hosting resources in order to facilitate management of the hosting resources|
|US8365169||Sep 23, 2008||Jan 29, 2013||Hewlett-Packard Development Company, L.P.||Migrating a virtual machine across processing cells connected to an interconnect that provides data communication without cache coherency support|
|US8374929||Aug 7, 2007||Feb 12, 2013||Gogrid, LLC||System and method for billing for hosted services|
|US8407688||Nov 27, 2007||Mar 26, 2013||Managelq, Inc.||Methods and apparatus for storing and transmitting historical configuration data associated with information technology assets|
|US8412809 *||Oct 24, 2007||Apr 2, 2013||International Business Machines Corporation||Method, apparatus and computer program product implementing multi-tenancy for network monitoring tools using virtualization technology|
|US8417910||Nov 2, 2010||Apr 9, 2013||International Business Machines Corporation||Storage area network (SAN) forecasting in a heterogeneous environment|
|US8418173||Nov 27, 2007||Apr 9, 2013||Manageiq, Inc.||Locating an unauthorized virtual machine and bypassing locator code by adjusting a boot pointer of a managed virtual machine in authorized environment|
|US8418176||Apr 9, 2009||Apr 9, 2013||Gogrid, LLC||System and method for adapting virtual machine configurations for hosting across different hosting systems|
|US8423998||Jun 4, 2010||Apr 16, 2013||International Business Machines Corporation||System and method for virtual machine multiplexing for resource provisioning in compute clouds|
|US8429748||Nov 27, 2009||Apr 23, 2013||Red Hat, Inc.||Network traffic analysis using a dynamically updating ontological network description|
|US8443077||Jul 21, 2010||May 14, 2013||Gogrid, LLC||System and method for managing disk volumes in a hosting system|
|US8453144||Apr 9, 2009||May 28, 2013||Gogrid, LLC||System and method for adapting a system configuration using an adaptive library|
|US8458695 *||Nov 27, 2007||Jun 4, 2013||Manageiq, Inc.||Automatic optimization for virtual systems|
|US8458717||Apr 9, 2009||Jun 4, 2013||Gogrid, LLC||System and method for automated criteria based deployment of virtual machines across a grid of hosting resources|
|US8468535 *||Apr 9, 2009||Jun 18, 2013||Gogrid, LLC||Automated system and method to provision and allocate hosting resources|
|US8473587||Jul 21, 2010||Jun 25, 2013||Gogrid, LLC||System and method for caching server images in a hosting system|
|US8495512||Jul 21, 2010||Jul 23, 2013||Gogrid, LLC||System and method for storing a configuration of virtual servers in a hosting system|
|US8510439||Aug 6, 2012||Aug 13, 2013||Oracle International Corporation||System and method for performance data collection in a virtual environment|
|US8516284||Nov 4, 2010||Aug 20, 2013||International Business Machines Corporation||Saving power by placing inactive computing devices in optimized configuration corresponding to a specific constraint|
|US8527793||Sep 4, 2012||Sep 3, 2013||International Business Machines Corporation||Method for saving power in a system by placing inactive computing devices in optimized configuration corresponding to a specific constraint|
|US8533305||May 25, 2012||Sep 10, 2013||Gogrid, LLC||System and method for adapting a system configuration of a first computer system for hosting on a second computer system|
|US8539071||May 9, 2012||Sep 17, 2013||International Business Machines Corporation||System and method for a storage area network virtualization optimization|
|US8539570 *||Apr 28, 2008||Sep 17, 2013||Red Hat, Inc.||Method for managing a virtual machine|
|US8566823||Feb 5, 2010||Oct 22, 2013||Tripwire, Inc.||Systems and methods for triggering scripts based upon an alert within a virtual infrastructure|
|US8566835||Oct 31, 2008||Oct 22, 2013||Hewlett-Packard Development Company, L.P.||Dynamically resizing a virtual machine container|
|US8566941||Feb 29, 2012||Oct 22, 2013||Red Hat, Inc.||Method and system for cloaked observation and remediation of software attacks|
|US8601226||Jul 21, 2010||Dec 3, 2013||Gogrid, LLC||System and method for storing server images in a hosting system|
|US8612596 *||Mar 31, 2010||Dec 17, 2013||Amazon Technologies, Inc.||Resource planning for computing|
|US8612971||Oct 17, 2006||Dec 17, 2013||Manageiq, Inc.||Automatic optimization for virtual systems|
|US8631226||Dec 28, 2006||Jan 14, 2014||Verizon Patent And Licensing Inc.||Method and system for video monitoring|
|US8644194||Oct 14, 2011||Feb 4, 2014||International Business Machines Corporation||Virtual switching ports on high-bandwidth links|
|US8656018||Apr 9, 2009||Feb 18, 2014||Gogrid, LLC||System and method for automated allocation of hosting resources controlled by different hypervisors|
|US8676944||Jul 20, 2001||Mar 18, 2014||Trendium, Inc.||Network models, methods, and computer program products for managing a service independent of the underlying network technology|
|US8677353 *||Jan 10, 2008||Mar 18, 2014||Nec Corporation||Provisioning a standby virtual machine based on the prediction of a provisioning request being generated|
|US8694679 *||Jul 13, 2011||Apr 8, 2014||Fujitsu Limited||Control device, method and program for deploying virtual machine|
|US8694989||Jul 17, 2008||Apr 8, 2014||Apple Inc.||Virtual installation environment|
|US8718070||Jul 6, 2011||May 6, 2014||Nicira, Inc.||Distributed network virtualization apparatus and method|
|US8725689 *||Jan 15, 2013||May 13, 2014||Parallels IP Holdings GmbH||Method and system for creation, analysis and navigation of virtual snapshots|
|US8732310 *||Apr 22, 2010||May 20, 2014||International Business Machines Corporation||Policy-driven capacity management in resource provisioning environments|
|US8732607||Feb 14, 2012||May 20, 2014||Parallels IP Holdings GmbH||Seamless integration of non-native windows with dynamically scalable resolution into host operating system|
|US8732699||Oct 27, 2006||May 20, 2014||Hewlett-Packard Development Company, L.P.||Migrating virtual machines between physical machines in a define group|
|US8738972||Feb 3, 2012||May 27, 2014||Dell Software Inc.||Systems and methods for real-time monitoring of virtualized environments|
|US8743888||Jul 6, 2011||Jun 3, 2014||Nicira, Inc.||Network control apparatus and method|
|US8745601 *||Jul 17, 2008||Jun 3, 2014||Apple Inc.||Methods and systems for using data structures for operating systems|
|US8750119||Jul 6, 2011||Jun 10, 2014||Nicira, Inc.||Network control apparatus and method with table mapping engine|
|US8750165 *||Aug 2, 2012||Jun 10, 2014||Hitachi, Ltd.||Configuration management method of logical topology in virtual network and management server|
|US8751654 *||Nov 30, 2008||Jun 10, 2014||Red Hat Israel, Ltd.||Determining the graphic load of a virtual desktop|
|US8751656||Oct 20, 2010||Jun 10, 2014||Microsoft Corporation||Machine manager for deploying and managing machines|
|US8752045||Nov 27, 2007||Jun 10, 2014||Manageiq, Inc.||Methods and apparatus for using tags to control and manage assets|
|US8752066 *||Nov 23, 2009||Jun 10, 2014||Raytheon Company||Implementing a middleware component using factory patterns|
|US8756599 *||Jan 17, 2011||Jun 17, 2014||International Business Machines Corporation||Task prioritization management in a virtualized environment|
|US8761036||Jul 6, 2011||Jun 24, 2014||Nicira, Inc.||Network control apparatus and method with quality of service controls|
|US8762525 *||Mar 8, 2012||Jun 24, 2014||International Business Machines Corporation||Managing risk in resource over-committed systems|
|US8763084||Sep 4, 2012||Jun 24, 2014||Aerohive Networks, Inc.||Networking as a service|
|US8775607 *||Dec 10, 2010||Jul 8, 2014||International Business Machines Corporation||Identifying stray assets in a computing enviroment and responsively taking resolution actions|
|US8782266 *||Dec 18, 2009||Jul 15, 2014||David A. Daniel||Auto-detection and selection of an optimal storage virtualization protocol|
|US8798982 *||Aug 20, 2012||Aug 5, 2014||Nec Corporation||Information processing device, information processing method, and program|
|US8799432 *||Oct 31, 2006||Aug 5, 2014||Hewlett-Packard Development Company, L.P.||Managed computer network caching requested and related data from remote computers|
|US8799453||Oct 20, 2010||Aug 5, 2014||Microsoft Corporation||Managing networks and machines for an online service|
|US8817621||Jul 6, 2011||Aug 26, 2014||Nicira, Inc.||Network virtualization apparatus|
|US8826289 *||Mar 26, 2012||Sep 2, 2014||Vmware, Inc.||Method and system for managing virtual and real machines|
|US8832691||Jun 7, 2012||Sep 9, 2014||Manageiq, Inc.||Compliance-based adaptations in managed virtual systems|
|US8839221 *||Sep 10, 2008||Sep 16, 2014||Moka5, Inc.||Automatic acquisition and installation of software upgrades for collections of virtual machines|
|US8850433||Jun 7, 2012||Sep 30, 2014||Manageiq, Inc.||Compliance-based adaptations in managed virtual systems|
|US8850550||Nov 23, 2010||Sep 30, 2014||Microsoft Corporation||Using cached security tokens in an online service|
|US8856174 *||Sep 7, 2012||Oct 7, 2014||Fujitsu Limited||Asset managing apparatus and asset managing method|
|US8868987 *||Feb 5, 2010||Oct 21, 2014||Tripwire, Inc.||Systems and methods for visual correlation of log events, configuration changes and conditions producing alerts in a virtual infrastructure|
|US8875129||Feb 5, 2010||Oct 28, 2014||Tripwire, Inc.||Systems and methods for monitoring and alerting events that virtual machine software produces in a virtual infrastructure|
|US8880657||Jun 28, 2011||Nov 4, 2014||Gogrid, LLC||System and method for configuring and managing virtual grids|
|US8886816 *||Dec 18, 2009||Nov 11, 2014||David A. Daniel||Auto-detection and selection of an optimal I/O system resource virtualization protocol|
|US8887158 *||Mar 7, 2008||Nov 11, 2014||Sap Se||Dynamic cluster expansion through virtualization-based live cloning|
|US8903983 *||Feb 27, 2009||Dec 2, 2014||Dell Software Inc.||Method, system and apparatus for managing, modeling, predicting, allocating and utilizing resources and bottlenecks in a computer network|
|US8904213||Aug 19, 2013||Dec 2, 2014||International Business Machines Corporation||Saving power by managing the state of inactive computing devices according to specific constraints|
|US8910152||Sep 23, 2008||Dec 9, 2014||Hewlett-Packard Development Company, L.P.||Migrating a virtual machine by using a hot-plug event|
|US8910153 *||Jul 13, 2009||Dec 9, 2014||Hewlett-Packard Development Company, L. P.||Managing virtualized accelerators using admission control, load balancing and scheduling|
|US8910163||Feb 26, 2013||Dec 9, 2014||Parallels IP Holdings GmbH||Seamless migration of non-native application into a virtual machine|
|US8914598 *||Sep 24, 2009||Dec 16, 2014||Vmware, Inc.||Distributed storage resource scheduler and load balancer|
|US8924967 *||Apr 28, 2011||Dec 30, 2014||Vmware, Inc.||Maintaining high availability of a group of virtual machines using heartbeat messages|
|US8929253||Jan 31, 2014||Jan 6, 2015||International Business Machines Corporation||Virtual switching ports on high-bandwidth links|
|US8930945||Nov 15, 2007||Jan 6, 2015||Novell, Inc.||Environment managers via virtual machines|
|US8935500 *||Nov 10, 2011||Jan 13, 2015||Vmware, Inc.||Distributed storage resource scheduler and load balancer|
|US8935701||Mar 9, 2009||Jan 13, 2015||Dell Software Inc.||Unified management platform in a computer network|
|US8949825||Oct 17, 2006||Feb 3, 2015||Manageiq, Inc.||Enforcement of compliance policies in managed virtual systems|
|US8949826||Nov 27, 2007||Feb 3, 2015||Managelq, Inc.||Control and management of virtual systems|
|US8949827||Jan 11, 2008||Feb 3, 2015||Red Hat, Inc.||Tracking a virtual machine|
|US8954487 *||Apr 8, 2010||Feb 10, 2015||Samsung Electronics Co., Ltd.||Management server and method for providing cloud computing service|
|US8955108 *||Jun 17, 2009||Feb 10, 2015||Microsoft Corporation||Security virtual machine for advanced auditing|
|US8959055 *||Apr 22, 2014||Feb 17, 2015||Parallels IP Holdings GmbH||Method and system for creation, analysis and navigation of virtual snapshots|
|US8966035||Apr 1, 2010||Feb 24, 2015||Nicira, Inc.||Method and apparatus for implementing and managing distributed virtual switches in several hosts and physical forwarding elements|
|US8972223||Jun 19, 2012||Mar 3, 2015||Credit Suisse Securities (Usa) Llc||Platform matching systems and methods|
|US8984504||Jan 11, 2008||Mar 17, 2015||Red Hat, Inc.||Method and system for determining a host machine by a virtual machine|
|US9009851 *||Mar 29, 2011||Apr 14, 2015||Brainlab Ag||Virtual machine for processing medical data|
|US9015177||Feb 15, 2013||Apr 21, 2015||Microsoft Technology Licensing, Llc||Dynamically splitting multi-tenant databases|
|US9015703||Nov 27, 2007||Apr 21, 2015||Manageiq, Inc.||Enforcement of compliance policies in managed virtual systems|
|US9038062||Nov 27, 2007||May 19, 2015||Manageiq, Inc.||Registering and accessing virtual systems for use in a managed system|
|US9043370||Apr 8, 2013||May 26, 2015||Microsoft Technology Licensing, Llc||Online database availability during upgrade|
|US9043452||Nov 3, 2011||May 26, 2015||Nicira, Inc.||Network control apparatus and method for port isolation|
|US9047351||Apr 11, 2011||Jun 2, 2015||Sandisk Enterprise Ip Llc||Cluster of processing nodes with distributed global flash memory using commodity server technology|
|US9058239 *||Jun 20, 2013||Jun 16, 2015||International Business Machines Corporation||Hypervisor subpartition as concurrent upgrade|
|US9069774 *||Nov 3, 2009||Jun 30, 2015||Infoblox Inc.||Graphical visualization and management of networks|
|US9075661 *||Oct 20, 2010||Jul 7, 2015||Microsoft Technology Licensing, Llc||Placing objects on hosts using hard and soft constraints|
|US9076311 *||Dec 28, 2006||Jul 7, 2015||Verizon Patent And Licensing Inc.||Method and apparatus for providing remote workflow management|
|US9083609||Sep 26, 2008||Jul 14, 2015||Nicira, Inc.||Network operating system for managing and securing networks|
|US9086917||Oct 17, 2006||Jul 21, 2015||Manageiq, Inc.||Registering and accessing virtual systems for use in a managed system|
|US9092248||Aug 21, 2013||Jul 28, 2015||Symantec Corporation||Systems and methods for restoring distributed applications within virtual data centers|
|US9092250||Oct 27, 2006||Jul 28, 2015||Hewlett-Packard Development Company, L.P.||Selecting one of plural layouts of virtual machines on physical machines|
|US9094292 *||Dec 18, 2009||Jul 28, 2015||Accenture Global Services Limited||Method and system for providing access to computing resources|
|US9106587||Aug 25, 2011||Aug 11, 2015||Nicira, Inc.||Distributed network control system with one master controller per managed switching element|
|US9110729||Feb 17, 2012||Aug 18, 2015||International Business Machines Corporation||Host system admission control|
|US20080295096 *||Mar 6, 2008||Nov 27, 2008||International Business Machines Corporation||DYNAMIC PLACEMENT OF VIRTUAL MACHINES FOR MANAGING VIOLATIONS OF SERVICE LEVEL AGREEMENTS (SLAs)|
|US20080320583 *||Apr 28, 2008||Dec 25, 2008||Vipul Sharma||Method for Managing a Virtual Machine|
|US20090006885 *||Jun 28, 2007||Jan 1, 2009||Pattabhiraman Ramesh V||Heartbeat distribution that facilitates recovery in the event of a server failure during a user dialog|
|US20090070752 *||Sep 6, 2007||Mar 12, 2009||International Business Machines Corporation||Method and system for optimization of an application|
|US20090100420 *||Sep 10, 2008||Apr 16, 2009||Moka5, Inc.||Automatic Acquisition and Installation of Software Upgrades for Collections of Virtual Machines|
|US20090185500 *||Jan 21, 2009||Jul 23, 2009||Carl Steven Mower||Virtualization of networking services|
|US20090199175 *||Jan 31, 2008||Aug 6, 2009||Microsoft Corporation||Dynamic Allocation of Virtual Application Server|
|US20090254927 *||Apr 6, 2009||Oct 8, 2009||Installfree, Inc.||Techniques For Deploying Virtual Software Applications On Desktop Computers|
|US20090300423 *||May 28, 2008||Dec 3, 2009||James Michael Ferris||Systems and methods for software test management in cloud-based network|
|US20090313160 *||Dec 17, 2009||Credit Suisse Securities (Usa) Llc||Hardware accelerated exchange order routing appliance|
|US20100058342 *||Jan 10, 2008||Mar 4, 2010||Fumio Machida||Provisioning system, method, and program|
|US20100125665 *||Nov 13, 2009||May 20, 2010||Oracle International Corporation||System and method for performance data collection in a virtual environment|
|US20100161768 *||Dec 18, 2009||Jun 24, 2010||Daniel David A||Auto-detection and selection of an optimal storage virtualization protocol|
|US20100161814 *||Dec 18, 2009||Jun 24, 2010||Daniel David A||Auto-detection and selection of an optimal I/O system resource virtualization protocol|
|US20100161838 *||Dec 23, 2009||Jun 24, 2010||Daniel David A||Host bus adapter with network protocol auto-detection and selection capability|
|US20100251255 *||Mar 26, 2010||Sep 30, 2010||Fujitsu Limited||Server device, computer system, recording medium and virtual computer moving method|
|US20100312809 *||Jun 5, 2009||Dec 9, 2010||Microsoft Corporation||Geographic co-location service for cloud computing|
|US20100325191 *||Apr 8, 2010||Dec 23, 2010||Samsung Electronics Co., Ltd.||Management server and method for providing cloud computing service|
|US20100325727 *||Jun 17, 2009||Dec 23, 2010||Microsoft Corporation||Security virtual machine for advanced auditing|
|US20110010721 *||Jul 13, 2009||Jan 13, 2011||Vishakha Gupta||Managing Virtualized Accelerators Using Admission Control, Load Balancing and Scheduling|
|US20110055712 *||Mar 3, 2011||Accenture Global Services Gmbh||Generic, one-click interface aspects of cloud console|
|US20110072208 *||Sep 24, 2009||Mar 24, 2011||Vmware, Inc.||Distributed Storage Resource Scheduler and Load Balancer|
|US20110126212 *||Nov 23, 2009||May 26, 2011||Raytheon Company||Implementing a middleware component using factory patterns|
|US20110264805 *||Oct 27, 2011||International Business Machines Corporation||Policy-driven capacity management in resource provisioning environments|
|US20120030349 *||Feb 2, 2012||Fujitsu Limited||Control device, method and program for deploying virtual machine|
|US20120102199 *||Oct 20, 2010||Apr 26, 2012||Microsoft Corporation||Placing objects on hosts using hard and soft constraints|
|US20120130738 *||May 24, 2012||Compressus, Inc.||System Management Dashboard|
|US20120131480 *||May 24, 2012||International Business Machines Corporation||Management of virtual machine snapshots|
|US20120151036 *||Jun 14, 2012||International Business Machines Corporation||Identifying stray assets in a computing enviroment and responsively taking resolution actions|
|US20120185848 *||Jul 19, 2012||International Business Machines Corporation||Task prioritization management in a virtualized environment|
|US20120203536 *||Aug 31, 2010||Aug 9, 2012||International Business Machines Corporation||Method and system for software behaviour management|
|US20120203824 *||Mar 29, 2011||Aug 9, 2012||Nokia Corporation||Method and apparatus for on-demand client-initiated provisioning|
|US20120233236 *||Mar 7, 2011||Sep 13, 2012||Min-Shu Chen||Cloud-based system for serving service request of embedded device by cloud computing and related cloud-based processing method thereof|
|US20120240114 *||Sep 20, 2012||Credit Suisse Securities (Europe) Limited||Method and System for Managing Virtual and Real Machines|
|US20120254437 *||Oct 4, 2012||Robert Ari Hirschfeld||Information Handling System Application Decentralized Workload Management|
|US20120278801 *||Nov 1, 2012||Vmware, Inc.||Maintaining high availability of a group of virtual machines using heartbeat messages|
|US20120297216 *||May 19, 2011||Nov 22, 2012||International Business Machines Corporation||Dynamically selecting active polling or timed waits|
|US20120303322 *||Nov 29, 2012||Rego Charles W||Incorporating memory and io cycle information into compute usage determinations|
|US20120304170 *||Nov 29, 2012||Morgan Christopher Edwin||Systems and methods for introspective application reporting to facilitate virtual machine movement between cloud hosts|
|US20120331004 *||Dec 27, 2012||Fujitsu Limited||Asset managing apparatus and asset managing method|
|US20130019011 *||Jan 17, 2013||International Business Machines||Policy-driven capacity management in resource provisioning environments|
|US20130055246 *||Aug 26, 2011||Feb 28, 2013||Rapid7, LLC.||Systems and methods for identifying virtual machines in a network|
|US20130083693 *||Apr 4, 2013||Hitachi, Ltd.||Configuration management method of logical topology in virtual network and management server|
|US20130097601 *||Apr 18, 2013||International Business Machines Corporation||Optimizing virtual machines placement in cloud computing environments|
|US20130239114 *||Mar 7, 2012||Sep 12, 2013||Darpan Dinker||Fine Grained Adaptive Throttling of Background Processes|
|US20130297283 *||Aug 20, 2012||Nov 7, 2013||Nec Corporation||Information processing device, information processing method, and program|
|US20130326639 *||Mar 29, 2011||Dec 5, 2013||Brainlab Ag||Virtual machine for processing medical data|
|US20140040447 *||Jul 31, 2012||Feb 6, 2014||Hitachi, Ltd.||Management system and program product|
|US20140189127 *||Dec 27, 2012||Jul 3, 2014||Anjaneya Reddy Chagam||Reservation and execution image writing of native computing devices|
|US20140258506 *||Mar 11, 2013||Sep 11, 2014||Amazon Technologies, Inc.||Tracking application usage in a computing environment|
|US20140380297 *||Jun 20, 2013||Dec 25, 2014||International Business Machines Corporation||Hypervisor subpartition as concurrent upgrade|
|WO2008118464A1 *||Mar 26, 2008||Oct 2, 2008||Credit Suisse Securities Usa L||Method and system for managing virtual and real machines|
|WO2009014493A1 *||Jul 20, 2007||Jan 29, 2009||Eg Innovations Pte Ltd||Monitoring system for virtual application environments|
|WO2009033172A1 *||Sep 8, 2008||Mar 12, 2009||Michael Gray||Architecture and protocol for extensible and scalable communication|
|WO2009070661A1 *||Nov 26, 2008||Jun 4, 2009||Manageiq Inc||Automatic optimization for virtual systems|
|WO2009146165A1 *||Apr 14, 2009||Dec 3, 2009||Blade Network Technologies, Inc.||Network virtualization for a virtualized server data center environment|
|WO2010068458A2 *||Nov 24, 2009||Jun 17, 2010||Citrix Systems, Inc.||Systems and methods for gslb remote service monitoring|
|WO2010151859A1 *||Jun 28, 2010||Dec 29, 2010||Vmware, Inc.||Controlling usage in virtualized mobile devices|
|Cooperative Classification||G06F9/455, H04L41/147, H04L47/78, G06F9/50, H04L67/1017, H04L67/1029, H04L67/1034, H04L67/1008, H04L67/1002, H04L41/0896, H04L41/0816, H04L41/0886, H04L41/0806, H04L41/0823, H04L41/22, H04L41/046, H04L41/12, H04L41/5025, H04L43/08, G06F9/5072, H04L12/24, H04L41/18, H04L41/00|
|European Classification||H04L29/08N9A1F, H04L29/08N9A7, H04L41/08D3, H04L41/00, H04L41/08A3, G06F9/50C4, H04L41/08A1, H04L29/08N9A1B, H04L41/14C, H04L12/24|
|Oct 27, 2006||AS||Assignment|
Owner name: TOUTVIRTUAL, INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PABARI, VIPUL;REEL/FRAME:018446/0334
Effective date: 20061027