|Publication number||US8209707 B2|
|Application number||US 12/013,349|
|Publication date||Jun 26, 2012|
|Priority date||Jan 11, 2008|
|Also published as||US20090183174, WO2009087165A2, WO2009087165A3|
|Publication number||013349, 12013349, US 8209707 B2, US 8209707B2, US-B2-8209707, US8209707 B2, US8209707B2|
|Inventors||Dinesh Kumar Subhraveti|
|Original Assignee||Google Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (15), Non-Patent Citations (13), Classifications (7), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of the Invention
The present invention relates to a method, system, and article of manufacture for gathering state information for an application and kernel components called by the application.
2. Description of the Related Art
An operating system may designate processes that are assigned to execute applications and components as running in a user space or kernel space. Typically, user applications are assigned to execute in the user address space and essential operations are assigned to execute in the kernel address space, such as resource allocation, low-level hardware interfaces, security, etc. A process may comprise one or more threads allocated to processor and computer resources to execute computer instructions. Multiple processes or multiple threads in one process may concurrently execute instructions for one application, or multiple processes may concurrently execute instructions for multiple applications. Multiple processes may execute on multiple processors. The term “process” as used herein refers to a process, thread or any other unit of execution allocated to execute instructions.
Certain applications may gather state information on the execution of an application to store in checkpoint information. The state of an application maintained in checkpoint information may be used for debugging, development and record and replay purposes. A record and replay program would allow the recreation of an application state to recreate the application so that it may run from the state as indicated in the checkpoint information.
To fully record the state information of an application, the state of kernel components the application calls would also have to be recorded. For instance, the application may call a graphics engine or network stack for network communication that are implemented in the kernel space. Proprietary operating systems, such as Windows®, a registered trademark of Microsoft Corporation, may not provide the tools to enable easy access to the state and system information of kernel components that are called by an application.
There is a need in the art for improved techniques to gather state and system information related to application execution to allow improved recording of the checkpoint information for the application.
Provided are a method, system, and article of manufacture for gathering state information for an application and kernel components called by the application. An implementation of a component to execute in a kernel space is instantiated in a user space. An application invokes a call to the component in the kernel space. The call to the component in the kernel space is invoked and an implementation of the component in the user space is invoked to execute the call in the user space. State information related to the execution of the call to the component in the user space to include in checkpoint information for the application is gathered.
In a further embodiment, the checkpoint information includes system information related to operations of the application and the execution of the calls to the implementation of the component in the user space, and a state of memory regions used by the application.
In a further embodiment, the intercepted call is translated to a format compatible with implementation of component in the user space, wherein the translated call is invoked to the implementation of the component.
In a further embodiment, the implementation of the component in the user space comprises a subsystem that executes in the user space, and wherein the state information of the subsystem related to the call from the application is gathered for the checkpoint information.
In a further embodiment, the application is loaded and initialized and an agent is loaded in response to loading the application. The agent intercepts the application calls to the component in the kernel space and invokes the call to the implementation of the component in the user space.
In a further embodiment, the agent instantiates the implementation of the component executing in a kernel space in the user space.
In a further embodiment, wherein intercepting the call comprises modifying, by the agent, the call made by the application to invoke the implementation of the component in the user space to execute the call in the user space.
In a further embodiment, wherein the component comprises a network communication data structure residing in the kernel space that is used to process network communication packets. The implementation of the component in the user space comprises the network communication data structure in the user space used to process the network communication packets.
In a further embodiment, the component comprises a graphics engine residing in the kernel space that is used to calls to a graphics engine to render graphics, and wherein the implementation of the component in the user space comprises a graphics subsystem in the user space used to process the calls to the graphics engine and render graphics.
The operating system 10 may allocate processes operating in the memory 6 into a kernel space 14 and a user space 16. Higher priority applications and operating system processes may execute in the kernel space 14. The application 12 executing in the user space 16 may invoke component application programming interfaces (APIs) 18 to call kernel components 20 executing in the kernel space 14, such as a network stack storing packets for network transmission or a graphics engine to render graphics on a display monitor. The kernel components 20 may comprise other high priority applications and subsystems that typically execute in the kernel space 14.
An agent 22 is loaded and initialized when the application 12 is loaded and initialized. The agent 22 initializes a user space implementation 24 of the kernel components 14 called by the application 12 in the user space 16, or user space component implementation 24. The agent 22 may gather information related to the application 12's execution of the user space component implementation 24. The agent 22 may store the gathered information as checkpoint information 26 to provide state information related to the application 12 execution, including system information on the execution of the components called by the application 12. This checkpoint information 26 may be used to replicate the application 12 state on the same or different systems 2 by restoring the application 12 to the state indicated in the checkpoint information 26. The applications 12 and other data may be stored and loaded from a storage 28, such as a non-volatile storage device.
In one embodiment, the kernel component 20 and user space component implementation 24 may exist simultaneously. In alternative embodiments, only the user space component 24 may remain in memory 8.
Once the state and system information is stored with the checkpoint information 26, a checkpoint manager or other program may resume the operation of the application 12 from the state represented in the checkpoint information 26. The address space of the recreated application 12 may be populated with the memory state information stored in the checkpoint information 26. To recreate an application 12, a new process for the application is created. The newly created process contains the executable image, ntdll.dll, PEB and other system regions such as ANSI code page, shared memory data, mapped at the top of the process address space. The data segment portions of the executable image and ntdll.dll are overwritten from the respective contents saved in the checkpoint information 26. The rest of the address space of the process may be populated with the memory regions described by the checkpoint information 26. The application 12 default heap, thread stacks, regions containing the loader data and process environment variables, etc., are restored by mapping memory regions with appropriate size and attributes and overwriting them with the contents saved in the checkpoint information 26 without regard to their internal structure. In particular, the memory region containing the agent 22 is also mapped, so that the restarted instance of the process already has the agent 22 for subsequent checkpoints.
Described embodiments provide techniques to implement kernel space components 20 as user space component implementations 24 accessed by an application in the user space 16 to allow the gathering of state information related to the component execution from the user space 16. In certain proprietary operating systems, it may be difficult to access and gather state information on the kernel components an application calls from the kernel space 14. The described embodiments circumvent this limitation with proprietary and other operating systems by executing kernel components called by applications in the user space where system and state information on the component execution can be more easily gathered.
The described operations may be implemented as a method, apparatus or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The described operations may be implemented as code maintained in a “computer readable storage medium”, where a processor may read and execute the code from the computer storage readable medium. A computer readable storage medium comprises non-transitory storage media such as magnetic storage medium (e.g., hard disk drives, floppy disks, tape, etc.), optical storage (CD-ROMs, DVDs, optical disks, etc.), volatile and non-volatile memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, DRAMs, SRAMs, Flash Memory, firmware, programmable logic, etc.), etc. The code implementing the described operations may further be implemented in hardware logic implemented in a hardware device (e.g., an integrated circuit chip, Programmable Gate Array (PGA), Application Specific Integrated Circuit (ASIC), etc.). Still further, the code implementing the described operations may be implemented in “transmission signals”, where transmission signals may propagate through space or through a transmission media, such as an optical fiber, copper wire, etc. The transmission signals in which the code or logic is encoded may further comprise a wireless signal, satellite transmission, radio waves, infrared signals, Bluetooth, etc. The “article of manufacture” may comprise a transmitting station and/or a receiving station for transmitting and receiving transmission signals in which the code or logic is encoded, where the code or logic encoded in the transmission signal may be decoded and stored in hardware or a computer readable storage medium at the receiving and transmitting stations or devices. An “article of manufacture” comprises a computer readable storage medium, hardware device, and/or transmission transmitters or receivers in which code or logic may be implemented. Those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope of the present invention, and that the article of manufacture may comprise suitable information bearing medium known in the art.
In the described embodiments, the access parameters are changed for process accesses to memory addresses in a shared memory. In an alternative embodiment, access patterns may be monitored for computer resources other than memory addresses, such as addresses in a storage device and other computer resources.
The terms “an embodiment”, “embodiment”, “embodiments”, “the embodiment”, “the embodiments”, “one or more embodiments”, “some embodiments”, and “one embodiment” mean “one or more (but not all) embodiments of the present invention(s)” unless expressly specified otherwise.
The terms “including”, “comprising”, “having” and variations thereof mean “including but not limited to”, unless expressly specified otherwise.
The enumerated listing of items does not imply that any or all of the items are mutually exclusive, unless expressly specified otherwise.
The terms “a”, “an” and “the” mean “one or more”, unless expressly specified otherwise.
Devices that are in communication with each other need not be in continuous communication with each other, unless expressly specified otherwise. In addition, devices that are in communication with each other may communicate directly or indirectly through one or more intermediaries.
A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention.
Further, although process steps, method steps, algorithms or the like may be described in a sequential order, such processes, methods and algorithms may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order practical. Further, some steps may be performed simultaneously.
When a single device or article is described herein, it will be readily apparent that more than one device/article (whether or not they cooperate) may be used in place of a single device/article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device/article may be used in place of the more than one device or article or a different number of devices/articles may be used instead of the shown number of devices or programs. The functionality and/or the features of a device may be alternatively embodied by one or more other devices which are not explicitly described as having such functionality/features. Thus, other embodiments of the present invention need not include the device itself.
The illustrated operations of
The foregoing description of various embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto. The above specification, examples and data provide a complete description of the manufacture and use of the composition of the invention. Since many embodiments of the invention can be made without departing from the spirit and scope of the invention, the invention resides in the claims hereinafter appended.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US6105148||Jun 16, 1995||Aug 15, 2000||Lucent Technologies Inc.||Persistent state checkpoint and restoration systems|
|US6161219||Jul 3, 1997||Dec 12, 2000||The University Of Iowa Research Foundation||System and method for providing checkpointing with precompile directives and supporting software to produce checkpoints, independent of environment constraints|
|US6766471 *||Dec 28, 2000||Jul 20, 2004||International Business Machines Corporation||User-level checkpoint and restart for groups of processes|
|US7161926 *||Jul 3, 2002||Jan 9, 2007||Sensoria Corporation||Low-latency multi-hop ad hoc wireless network|
|US7536591 *||Nov 17, 2004||May 19, 2009||Virginia Tech Intellectual Properties, Inc.||Transparent checkpointing and process migration in a distributed system|
|US7644413 *||Jan 5, 2010||Microsoft Corporation||User mode device driver interface|
|US7657787 *||Feb 2, 2010||Hewlett-Packard Development Company, L.P.||Method of restoring communication state of process|
|US7773096 *||Dec 12, 2005||Aug 10, 2010||Microsoft Corporation||Alternative graphics pipe|
|US7793153||Sep 7, 2010||International Business Machines Corporation||Checkpointing and restoring user space data structures used by an application|
|US20030088807 *||Nov 7, 2001||May 8, 2003||Mathiske Bernd J.W.||Method and apparatus for facilitating checkpointing of an application through an interceptor library|
|US20060085679||Aug 26, 2005||Apr 20, 2006||Neary Michael O||Method and system for providing transparent incremental and multiprocess checkpointing to computer applications|
|US20060150010||Dec 13, 2005||Jul 6, 2006||Stiffler Jack J||Memory-controller-embedded apparatus and procedure for achieving system-directed checkpointing without operating-system kernel support|
|US20080002578 *||Jun 30, 2006||Jan 3, 2008||Jerrie Coffman||Network with a constrained usage model supporting remote direct memory access|
|US20080229323 *||Mar 12, 2007||Sep 18, 2008||Timothy Mackey||Systems and Methods for Error Detection|
|US20100057932 *||Jul 9, 2007||Mar 4, 2010||Solarflare Communications Incorporated||Onload network protocol stacks|
|1||"Inside Microsoft Windows 2000, Third Edition", Microsoft, [online] [retrieved Nov. 30, 2007] http://www.microsoft.com/mspress/books/sampchap/4354a.aspx.|
|2||"Microsoft Windows Internals, Fourth Edition: Microsoft Windows Server 2003, Windows XP, and Windows 2000", Microsoft, [online] [retrieved Nov. 25, 2007] http://book.itzero.com/read/microsoft/0507/Microsoft.Press.Microsoft.|
|3||Dieter, et al., "User-Level Checkpointing for Linux Threads Programs", Proceedings of the FREENIX Track, Usenix Annual Technical Conference, Boston, MA, Jun. 2001, pp. 81-92.|
|4||Final Office Action dated Sep. 26, 2011, pp. 1-24, for U.S. Appl. No. 12/013,344, filed Jan. 11, 2008, by inventors Dinesh K. Subhraveti et al.|
|5||First Office Action for U.S. Appl. No. 12/013,344, dated Apr. 14, 2011, 15 pgs.|
|6||Litzkow, et al., "Condor-A Hunter of Idle Workstations", IEEE, 8th International Conference on Distributed Computing Systems, San Jose, CA, 1988, pp. 104-111.|
|7||Litzkow, et al., "Supporting Checkpointing and Process Migration Outside the Unix Kernel", Usenix Winter Conference, San Francisco, CA 1992.|
|8||Litzkow, et al., "Condor—A Hunter of Idle Workstations", IEEE, 8th International Conference on Distributed Computing Systems, San Jose, CA, 1988, pp. 104-111.|
|9||Mogul, et al., "Unveiling the Transport", ACM SIGCOMM Computer Communications Review, vol. 34, No. 1, Jan. 2004, pp. 99-106.|
|10||Osman, et al., "The Design and Implementation of Zap: A System for Migrating Computing Environments", Proceedings of the 5th Operating Systems Design and Implementation (OSDI 2002), Boston, MA, Dec. 2002.|
|11||Response to First Office Action for U.S. Appl. No. 12/013,344, dated Jul. 14, 2011, 12 pgs.|
|12||US Patent Application entitled "Checkpointing and Restoring User Space Data Structures Used by an Application", Serial No. unknown, filed Jan. 11, 2008, by inventor D.K. Subhraveti.|
|13||US Patent Application entitled "Gathering Pages Allocated to an Application to Include in Checkpoint Information", Serial No. unknown, filed Jan. 11, 2008, by inventor D.K. Subhraveti.|
|U.S. Classification||719/317, 719/331, 707/649|
|International Classification||G06F9/44, G06F7/00|
|Mar 27, 2008||AS||Assignment|
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SUBHRAVETI, DINESH KUMAR;REEL/FRAME:020712/0137
Effective date: 20071210
|Dec 30, 2011||AS||Assignment|
Owner name: GOOGLE INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:027463/0594
Effective date: 20111228
|Dec 28, 2015||FPAY||Fee payment|
Year of fee payment: 4