CA2285096C - Logical mapping of data objects using data spaces - Google Patents
Logical mapping of data objects using data spaces Download PDFInfo
- Publication number
- CA2285096C CA2285096C CA002285096A CA2285096A CA2285096C CA 2285096 C CA2285096 C CA 2285096C CA 002285096 A CA002285096 A CA 002285096A CA 2285096 A CA2285096 A CA 2285096A CA 2285096 C CA2285096 C CA 2285096C
- Authority
- CA
- Canada
- Prior art keywords
- page
- data
- data space
- sub
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000013507 mapping Methods 0.000 title claims description 92
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 description 13
- 239000000872 buffer Substances 0.000 description 10
- 230000007246 mechanism Effects 0.000 description 10
- 238000013519 translation Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000005192 partition Methods 0.000 description 3
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000012536 storage buffer Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/0292—User address space allocation, e.g. contiguous or non contiguous base addressing using tables or multilevel address translation means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0877—Cache access modes
- G06F12/0882—Page mode
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/10—Address translation
- G06F12/109—Address translation for multiple virtual address spaces, e.g. segmentation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99944—Object-oriented database structure
- Y10S707/99945—Object-oriented database structure processing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99952—Coherency, e.g. same view to multiple users
- Y10S707/99953—Recoverability
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99956—File allocation
Abstract
Method and means are provided for simulating a contiguous data space within a computer memory, and for placing and accessing data objects of various sizes within the simulated contiguous data space. Multiple, sub-data spaces are concatenated in such a way that each page and each sub-data space in the contiguous data space are uniquely identified.
Data objects are placed in the contiguous data space and at the first reference to a page of the data object, only the segment containing the referenced page in the contiguous data space is mapped to the database storage disk. Once a data space page is mapped, the operating system can read the page into memory without requesting a disk operation from the database manager. On modifying a page, if the database disk page location is changed, the contiguous data space page is re-mapped without changing the page address in the data space. Also, modified data pages are re-written to the database storage disk in an ongoing manner set by the user, instead of at intervals set by the operating system.
Data objects are placed in the contiguous data space and at the first reference to a page of the data object, only the segment containing the referenced page in the contiguous data space is mapped to the database storage disk. Once a data space page is mapped, the operating system can read the page into memory without requesting a disk operation from the database manager. On modifying a page, if the database disk page location is changed, the contiguous data space page is re-mapped without changing the page address in the data space. Also, modified data pages are re-written to the database storage disk in an ongoing manner set by the user, instead of at intervals set by the operating system.
Description
LOGICAL MAPPING OF DATA OBJECTS USING DATA SPACES
Field of the Invention The invention relates in general to method and means for mapping data objects from a simulated contiguous data space to the memory of a computer system.
The Background Computerized data processing systems rely on various types of storage spaces to process and store data. For example, "main storage" is program-addressable storage from which data can be loaded directly into the registers of the central processing unit (CPU) for processing. "Auxiliary storage" is addressable storage other than main storage that can be accessed by means of input/output (I/O) channels, and includes direct access storage devices such as magnetic storage disks. "Expanded storage" is a high-speed, high-volume electronic extension of main storage that is accessed synchronously in multiple-byte increments e.g. 4096 (4K) bytes, sometimes referred to as a "page".
Data on storage disks exists in sets of fixed length records accessible in pages. The size of a page varies depending on the system in use; on some systems a page is 4096 (4K) bytes. Also, on some computers e.g. virtual machines, the operating system keeps track of disk space in blocks of
Field of the Invention The invention relates in general to method and means for mapping data objects from a simulated contiguous data space to the memory of a computer system.
The Background Computerized data processing systems rely on various types of storage spaces to process and store data. For example, "main storage" is program-addressable storage from which data can be loaded directly into the registers of the central processing unit (CPU) for processing. "Auxiliary storage" is addressable storage other than main storage that can be accessed by means of input/output (I/O) channels, and includes direct access storage devices such as magnetic storage disks. "Expanded storage" is a high-speed, high-volume electronic extension of main storage that is accessed synchronously in multiple-byte increments e.g. 4096 (4K) bytes, sometimes referred to as a "page".
Data on storage disks exists in sets of fixed length records accessible in pages. The size of a page varies depending on the system in use; on some systems a page is 4096 (4K) bytes. Also, on some computers e.g. virtual machines, the operating system keeps track of disk space in blocks of
2 0 256 pages called segments. This is necessary for them because the hardware requires that a control block be maintained for each segment for virtual address translation.
When data is to be transferred from auxiliary storage to the CPU, pages of data are transferred to a storage buffer in segments.
In order for data to execute in the CPU, it normally should be in main storage. But main 2 5 storage is limited and is therefore not used to store large amounts of data permanently. On the other hand, vast amounts of data may be stored on data disks. However, accessing data from disks is slow compared to the rate at which it can be processed in main storage. To compensate for the difference in access rates, a data buffer is used. A data buffer is a portion of storage used to hold input and output data temporarily. The buffer can reside in main storage or expanded
When data is to be transferred from auxiliary storage to the CPU, pages of data are transferred to a storage buffer in segments.
In order for data to execute in the CPU, it normally should be in main storage. But main 2 5 storage is limited and is therefore not used to store large amounts of data permanently. On the other hand, vast amounts of data may be stored on data disks. However, accessing data from disks is slow compared to the rate at which it can be processed in main storage. To compensate for the difference in access rates, a data buffer is used. A data buffer is a portion of storage used to hold input and output data temporarily. The buffer can reside in main storage or expanded
3 0 storage.
On mufti-user computing systems, concurrent users time-share the resources on the computer systems through "virtual machines". In a virtual machine, which is a functional simulation of the real machine, each user is allowed to address the computer main storage as though it were real. Addressable main storage in a virtual machine is called "virtual storage". The size of virtual storage is limited by the addressing scheme of the computer system and by the amount of real disk storage available. Data in virtual storage is mapped to real addresses when they are to be operated on by the CPU. Mapping is the establishment of correspondences between the physical storage and virtual storage locations. An address translation table is maintained by the operating system for this purpose. On virtual machines, each user's reference to a memory address is referred to as a virtual address, and each range of addressable space available to a user is called an address space. When a user references a virtual storage location, the page containing that location may be on disk or expanded storage (a "flag" in the address translation table would indicate this). When a page has to be copied to main storage, the operating system would read the page into an available real storage page location. When completed, the page translation table is updated to reflect the new page location. If no real storage space is available, the operating system frees up main storage space by "paging out" least recently used pages.
Drawings Fig. 1 shows key information maintained in a database directory relevant to the present invention.
2 0 Fig. 2. shows an overall representation of the key elements in this invention.
Fig. 3 shows how multiple independent sub-data spaces are made to appear contiguous by using a table of pointers.
Fig. 4 shows data objects placed in specific data spaces using logical mapping.
Fig. 5 shows how a page address can be converted to a data space address and the relevant 2 5 portion of the address used by this invention.
A typical database storage system is shown in Fig. 1. It comprises a directory disk (1), one or more data disks (2), and one or two log disks similar to the data disks (not as illustrated).
The directory disk contains information on the mapping of the database pages from virtual 3 0 storage to their real physical location and other information describing the physical configuration of the data base. The data disks stores data, while the log disks record transactions against the database. In a database system, users are assigned logical pages on disk to store data objects. A
data object is a logical set of pages in the database. For example, in a relational database system, a data object may be viewed as set pages containing records of rows and columns of a table where each row is a separate record, and each column is a different data field. As shown in Fig 1, when a data object is created, entries are inserted in the database directory to indicate which data object pages contain data (3), and their physical location on disk (4).
Initially, only directory space is taken, but as a user inserts data into a data object, pages are allocated on disk and the directory is updated to point to them.
In order to maintain data integrity, the database system has a mechanism to take "checkpoints"
of the database at certain intervals to ensure that a consistent version of the database is saved. For example when a database page is modified, a copy of the page as of the previous checkpoint is kept unchanged, the modified version of the page is copied to disk, and the page directory is updated to point to the new location. Hence at checkpoint time the modified version of the database becomes the current copy of the database.
On virtual machines, data space pages can be mapped to disk by various techniques. For example contiguous virtual storage pages can be mapped to contiguous disk pages; this is referred to as physical mapping. Alternatively, contiguous virtual pages can be mapped to non-contiguous disk pages; this is referred to as logical mapping.
On some types of virtual machines, users can access multiple address spaces.
Some of these address spaces, however, can contain only data (not computer instructions) and are referred 2 0 to as data spaces. Furthermore data space pages can be mapped to disk in such a manner as to eliminate the need for the data base program manager to execute page I/O
operations in order to move data between data disk and main storage. On these systems the location on disk containing a data object page is known to the operating system. When the page is referenced by a user, it is read from its disk location by the operating system without requiring a specific disk operation 2 5 from the database program manager. When a page is directly accessible by the operating system, the database system operates more efficiently with less demands on CPU
processing cycles.
On database systems, there is frequently a continuing need to improve the overall efficiency of the system to handle increasingly large amounts of data. In particular, there is a need for faster response to queries and data changes; efficient use of real storage; and improved 3 0 efficiency in handling data objects. This in turn implies systems that optimize use of data spaces;
systems that handle only those pages that are referenced rather than the whole data object; and systems that minimize database downtime at checkpoint. In the prior art, various schemes are available to use data spaces, but no method or means has been found that uses data spaces in virtual memory that handle data objects of various sizes multi-user systems.
Example of prior art involving virtual storage but not addressing this deficiency include:
U.S. Patent 4,742,447, ("Method To Control I/O Accesses In Multi-tasking Virtual Memory Virtual Machine Type Data Processing System") which discloses a method for accessing information in a page segmented virtual memory data processing system in which virtual machines running UNIX type operating systems are concurrently established, and in which a memory manager controls the transfer of information between primary storage and secondary storage devices in response to the occurrence of page faults.
U.S. Patent 4,843,541 ("Logical Resource Partitioning of a Data Processing System") discloses a method and means for partitioning the resources in a data processing system into a plurality of logical partitions. The main storage, expanded storage, the channel and sub-channel resources of the system are assigned to the different logical partitions in the system to enable a plurality of preferred guest programming systems to run simultaneously in the different partitions.
U.S. Patent 4,843,542 ("Virtual Memory Cache for Use in Multi Processing Systems) discloses a system for maintaining data consistency among distributed processors, each having its associated cache memory.
U.S Patent 4,922,415 ("Data Processing System for Converting Virtual to Real Addresses 2 0 Without Requiring Instructions from the Central Processing) discloses a method in which a controller performs the translation functions for the inter-operability of the processor and memory and does so without requiring instructions from the processor.
U.S. Patent 4,961,134 ("Method For Minimizing Locking and Reading in a Segmented Storage Space") discloses a page accessing method in a segmented table-space which eliminates 2 5 unnecessary reading and locking. The table-space comprises data pages grouped into identical sizes segments, each segment storing data for a single table. A status indicator for each data page of segment is kept in a separate segment control block stored in a space map page.
Consequently there is an unfulfilled need for means to create and use data spaces to accommodate large data objects of different sizes, including data objects which may be much 3 0 larger than the size of a single real data space.
Also there is a unfulfilled need for means to make more efficient use of real storage by mapping to data disk, only those data pages from a data space that are referenced by the CPU, rather than the entire data object.
Yet another unfulfilled need is a means to efficiently save modified pages back to data disk so as to reduce the database down-time at checkpoint.
A full understanding of how the present invention addresses the above unfulfilled needs may be had by referring to the following description and claims taken in conjunction with the accompanying drawings:
General Description of Invention The present invention provides a method for simulating a contiguous addressable data space in a computer memory, which can be a virtual memory. Multiple addressable sub-data spaces are concatenated to simulate the contiguous addressable data space. The location of each concatenated sub-dataspace is kept track of within the contiguous addressable data space. Each sub-data space is identified by an assigned unique sub-data space identifier.
Each sub-data space has a known number of pages and each page has a known number of addressable storage locations. Each page is assigned a unique page identifier within the contiguous addressable data space, and each addressable location within the contiguous addressable data space can be referenced based on its unique page identifier and sub-data space identifier containing the page.
Another aspect of the invention includes means or apparatus for simulating a contiguous addressable data space in a computer memory. One embodiment can use computer programs 2 0 stored or carried on suitable storage media such as magnetic or optical media. The program includes means for concatenating multiple addressable sub-data spaces to simulate contiguous addressable data space and means for keeping track of the location of each concatenated sub-dataspace within the contiguous addressable data space An identification portion of the program provides means for identifying each of the sub-data space by an assigned unique sub-data space 2 5 identifier. Each page has a unique page identifier so that each addressable location within the contiguous addressable data space can be referenced, based on its unique page identifier and sub-data space identifier containing the page.
Another aspect of the invention provides a computer system with a simulated contiguous addressable data space in computer memory. Multiple addressable sub-data spaces simulating the 3 0 contiguous addressable data space are concatenated using an index containing the location of the sub-data spaces. Each sub-data space is identified by an unique sub-data space identifier, and each sub-data space has a pre-defined number of pages; and each page has a pre-defined number of addressable storage locations. The addressable storage locations are mappable to corresponding data in another storage. Each page is identified by an unique page identifier within the contiguous addressable data space and each addressable location within the contiguous addressable data space is capable of being referenced by means of a unique page identifier and the sub-data space identifier.
Detailed Description An embodiment of the present invention as illustrated in Fig. 2 is incorporated in a database system that supports data objects and data space services. However it is to be understood that the invention is not restricted to the specific system described, but can be applied on a computer system that supports data spaces, data objects and data disks in the manner described herein.
The invention can be summarized as comprising of the following steps which are described in more detail in subsequent sections:
Step 1. As shown in Fig 4., on database initialization, the present invention provides for the creation, by the use of pointers (5) and a data space table (6), of a contiguous data space (7) comprising of linked, but separately identifiable, sub-data spaces (8). Each data page within the contiguous area is identifiable from an assigned page number, and each sub-area within the contiguous data space area is identifiable from an assigned sub-data space identifier.
2 0 Step 2. On first reference to a data object in the database by a user, the invention provides for enough pages in the simulated contiguous data space to be reserved to place the entire data object starting from the first available "empty" page location. In the simulated contiguous data space, the pages to be reserved start from the first available free page, and the page number is incremented sequentially to match the size of a data object. The invention provides means to 2 5 identify any page of a data object within the simulated contiguous data space once the starting page number of the data object is established and its size expressed in pages is known.
Step 3. On first reference by a user to a page in the data object within the contiguous area, the invention provides for a mapping to disk only the segment of the data object containing the desired page. This partial mapping of the data object, rather than mapping the whole data object, 3 0 minimizes the use of main storage because for each segment mapped, the operating system has to maintain a control block which uses up real storage. Delaying the mapping to a segment on the first reference means that a control block is only allocated if the segment is actually referenced.
Therefore data space segments that are never referenced, and thus not mapped, will not require the control block.
Step 4. After pages have been modified, the invention provides for saving the modified pages at predetermined time intervals set by the user, rather than waiting for the database system to make a global save at checkpoint. This provision of the invention allows pages to be saved in an on-going manner that minimizes the database down time associated with system initiated checkpoint.
Details on the above steps are provide in the following sections.
Since a data object can be larger than the maximum size of the real data space available to a user, the first step as provided by this invention and shown in Fig 3. is to create a large apparent contiguous data space (7) by linking separately identifiable fixed-sized sub-data spaces (8). This may done advantageously by the following steps:
i) Logically link multiple fixed-sized sub-data spaces using an array of pointers, and maintain a data space table for this purpose. See Fig 3.
ii) For each page in the simulated contiguous data space, assign a unique page number as provided for by this invention.
iii) For each page in the simulated contiguous data space, establish the page number as provided for by the present invention.
2 0 In order to keep track of the identity of each data object and sub-data space within the simulated contiguous data space, a table of pointers is set up to identify the start and end of each data space and sub-data space.
Within the simulated contiguous data space, any page number can be located as provided by the present invention. For example, if the size of each constituent data space is 524,288 pages, 2 5 and the page being referenced is page number 1,217,436, then we can calculate that the referenced page is 168,860 in the third sub-data space. After the page has been identified, it can be mapped to the correct real storage page.
In pseudo code format the procedure to determine the page address as a function of its location on a data space can expressed as follows:
3 0 If the Starting_page number of the data object(n), is greater or equal to 0 (i.e. the data object was placed in the simulated contiguous data space, then do;
Data space number = [(Starting-page number of data object,(n)) + m ] / [the size of 1 data space expressed in pages]
Similarly the page number within the contiguous area can be determined as follows:
Page address = [(Starting-page number of data object, (n)) + m] // [(The size of 1 data space expressed in pages) * (the size of 1 page expressed in bytes)]
end;
In the above pseudo code the Starting-page number is the location of the first page of the data object in the simulated contiguous data space; the symbol (/) represent an integer division;
the symbol (/~ represents the remainder of an integer division; (*) is the multiplication operator;
and (n) is the identity of the data object; and (m) is the page number within the data object containing the required data.
If the data object Starting-page number is not known, it can be determined as follows:
i) On database initialization, an array named "Starting-page number" is defined to contain the starting page location of each data object to be placed in the simulated contiguous data space. The dimension of the array is the same as the number of data objects in the database.
The array is initialized to negative values to indicate that the data objects have not yet been 2 0 placed in the simulated contiguous data space.
ii) A variable "Next available-page" is defined and initialized to 0 in a counter to indicate that this is the beginning page of the empty simulated contiguous data space.
iii) At the first user reference to a data object, the next available set of pages in the simulated contiguous data space is reserved for the referenced data object, by setting the 2 5 Starting_page number of the said data object to the Next available data space page number taken from the counter.
iv) The Next available-page counter is then increased by the size of the data object (in pages) rounded up to the next segment so as to start all data objects on a segment boundary. In this invention it is convenient to start the data object on a new page segment boundary, but this is 3 0 not an essential requirement if the systems does not handle pages in segments.
With the above procedure, any address within the simulated contiguous data space can be determined once the location of the object is known. Segments of the data object can then be mapped to its disk using information in the database directory.
The same result can be obtained by shifting bits. For example, on a 32-bit computer architecture, bits 0 to 12 can be used to represent the data space number, and bits 13 to 31 shifted left 12 positions (equivalent to multiplication by 4096) as the address in the data space. This alternative procedure is illustrated in Fig. 5.
Once the simulated contiguous data space is created, a data object can be placed in the data space as shown in Fig 3. For example, data object number 1 (9) consisting of 102,528 pages is placed in pages 0 to 102,527 in sub-data space 1 (10). This may be followed by another data object number 5 (11), say, consisting of 1,024,000 pages placed from pages 102,656 (which marks the beginning of a new segment in data space 1 ) to pages spanning sub-data spaces 2 (8) and 3 (8). In this example, data object 1 would therefore be placed in the first 400 segments of the simulated contiguous data space, while data object 5 would be placed from the first page of segment number 401 starting from page number 102,565.
Similarly, since the Starting-page number for each data object is known, any request for a data page can be converted to a page number within the simulated contiguous data space. For example if data object(n) starts at page 1,126,656 and we request data page 90,780 of data object (n), we can calculate the page number to be 1,217,436 or page 168,860 located in the third sub-data space.
An important benefit in allowing a data object to start on a segment boundary instead of 2 0 at the beginning of data space is the ease in supporting both very large data objects (spanning more than one data space) and very small data objects (spanning much less than the size of one data space). This invention eliminates the need to have one contiguous data space for each data object, and makes it possible to have many small and large data objects placed in the same simulated contiguous data space.
2 5 Once the data object has been placed in the simulated contiguous data space, it can be mapped to the correct disk pages based on information contained in the database directory.
However mapping an entire data object at its first reference is expensive and would require considerable real storage. For this reason this invention provides a mechanism to delay the mapping until first reference of a data object, and then to map only a segment consisting of the 3 0 desired pages of the data object by a procedure called "mapping on demand."
Mapping on demand can be explained as follows: For each sub-data space a bit map is created to indicate if each segment in the sub-data space is currently "referenced". A segment is referenced if the pages containing data in that segment have been mapped to disk. Initially, the bit map is set to '0' and the entire sub-data space is considered not referenced. The bit is referred to as the "segment valid bit" and one bit map is used per data space. For example if a sub-data space is made up of 524 288 pages and each segment is 256 pages, then there are 2048 segments and the bit map is therefore 2048/8 = 256 bytes per sub-data space which is allocated when the sub-data space is created.
Although the entire data object is placed in the simulated contiguous data space when it is first referenced, the sub-data spaces and the bit maps may not all exist. The mechanism provided by this invention to calculate the page address of any page in the contiguous area detects if this is the first reference to a sub-data space and/or a segment of the data object.
If this is the first reference, a sub-data space is created and a segment is mapped. The steps comprising mapping on demand can be summarized as follows:
Step 1. At first reference to a data object, place it in the simulated contiguous data space by allocating the starting and ending data space address as previously described.
Step 2. Calculate the address of the referenced page as previously described.
Step 3. If the sub-data space was not created, then create the sub-data space as previously described, set the data space identifier value used to address that sub-data space, and allocate and initialize the segment validity bit map.
Step 4. If the segment containing the page is not mapped, then map it and turn on the bit 2 0 to indicate that the segment was mapped.
This procedure is summarized in the following pseudo code:
If Starting-page number for data object(n) is not assigned then do;
2 5 Starting_page number(n) = Next available_ page;
Reserved space = ((Data object size(n)/(z) + 1) * (z);
Next available_page = Next available_page + Reserved_ end;
3 0 do;
space;
Data space number = (Starting_page_number(n) + m)/x;
Page-address = ((Starting_page_number(n) + m)//x) * y;
end;
If Data space-pointer(Data space number) was not created then do;
Create the data space Set Data space-pointer(Data space number) to the value used to address that data space.
Allocate the segment valid bit map for that data space and initialize it to off. end;
If Segment valid(Page_address / y*z) is off then do;
Map the segment containing Page-m Set segment valid(Page address / y*z) on;
end;
In the above pseudo code; Starting-page number is an array of as many elements as the number of data objects in the database; (~ represent the division function;
(/n represent the remainder of a division; (*) represent the multiplication operator; (z) is the number of pages in a segment; (y) is the size of a page in bytes; (m) is the referenced page from the start of the data object; (n) is the identity of the data object.
2 0 Checking the "segment valid" bit at each page request is inexpensive as the bit map is small. If the segment is not mapped, then the database manager program has to read the directory blocks containing information on the pages required for that segment and issue a request to map the data space pages that contain data to their locations on disk. Once mapped, the "segment valid bit" is turned on and any subsequent reference to read a page in that segment will be translated 2 5 directly to a virtual storage address without reference to the database directory. The net result is an improved query response time.
If a request is made for an update (the page was modified), the page may have to be shadowed (moved to a new disk location); alternatively it may have to be inserted if it had not existed before (first insert). This will require a change to the mapping in which case the database 3 0 manager will determine from the database directory if the page needs to be (re)mapped. If so, a new data page is allocated, the directory is updated and a new mapping is done. This will result in a mapping change, not a change in the virtual address of a page.
An example of mapping on demand as provided by one aspect of the present invention can be illustrated using the IBM~ SQL/DSTM database management system operating on an IBM
System/390TM computer with the IBM VM/ESATM operating system as follows.
Before updating a page, the IBM SQL/DS management system will copy the requested page to a buffer. It will then re-issue the MAPDISK DEFINE request with the PAGEVIEW = ZERO parameter before copying the data back from the local buffer to the data space. Note, PAGEVIEW
= ZERO
informs VM/ESA operating system that it should provide an empty real storage page for that data space page rather than reading the contents of the disk when the page is first referenced.
If updating without the use of a local buffer is desired, the page can be remapped with the PAGEVIEW = RETAIN parameter. This first reads the copy of the page from its current disk (old) location before re-setting the mapping to the new location. Consequently the user is presented with the old copy of the page which will be written to the new location.
When the local buffer is used, the re-mapping can be deferred to when the page is moved back from the local buffer to the data space. Performance optimization is implemented by delaying the remapping in an attempt to group multiple mapping request into one MAPMDISK
DEFINE call.
For this purpose, several MAPMDISK DEFINE request blocks can be initialized and used as required. Each request can be batched for several consecutive data space pages.
When a page is moved to the local buffer and a MAPMDISK DEFINE is required, the 2 0 following is done for a new request:
i) If a MAPMDISK DEFINE request exists for the previous or the next page of the same data space, the two requests are combined into one.
ii) If the new request cannot be attached to any existing request so that it would be consecutive, and there exists an unused request block, use it to store the new request.
2 5 iii) If all the pre-allocated request areas are in use, issue the DEFINE
for the requests already in one of the request control blocks and reset the block for the current page.
Since, by using an embodiment of the present invention, the operating system is now responsible for performing the database I/O operations through its paging mechanism, an interface is used to ensure that all modified data is written to the database at regular intervals, 3 0 maintaining integrity of the database. Without such mechanism, integrity of the database.
In the prior art, mechanisms exist to request that changes to a database be written to disk while the database manager is waiting. However such mechanisms may cause significant delays at checkpoint time as many pages may be needed to be written to disk when data spaces are used.
To resolve this problem, some operating system have implemented a mechanism whereby the database manager can specify a subset of the database to be made current on disk (i.e. to be written to disk if the pages in memory were modified). Further the mechanism is said to be asynchronous in that a save request can be issued, and processing continued, while the I/O
operations are performed. An indicator is presented to the database manager when all I/O needed for that save have completed.
One embodiment of the present invention takes advantage of the above mechanism to implement a partial save. Like mapping on demand which allows the database manager to delay the mapping to the first reference of a page, the invention provides for a partial save which allows the database manager to request page saves at short intervals and for only the modified pages of a data object.
Partial save can be illustrated using the IBM SQL/DS database management system as follows:
Like the Segment Valid Bit map, a bit map is allocated and set to '0' when a data space is referenced for the first time. In this case the bit map is made of 1 bit for every 32 pages (or 4 096 bytes per data space) which we refer to as 'MODIFIED BIT MAP'. The bit representing a set of 32 pages can be calculated by dividing the page address by 131,072 (32 pages of 4 096 bytes each). Before moving a page from the local buffer back to the data space, the corresponding bit in 2 0 the 'MODIFIED BIT MAP' is checked. If it is OFF (0), it is then turned ON
(1) and a counter is incremented to keep track of the number of bits currently ON.
When the counter reaches a predefined limit (set by the user) the data base manager would issue a MAPMDISK SAVE using as the list of addresses to save, the list corresponding to the pages represented by the ON in the 'MODIFIED BIT MAP'. The database manager would 2 5 then reset the bit map to OFF (0). Since the request is asynchronous, the database manager would continue processing while the I/O operations take place. The value of the counter determines the frequency with which the partial saves are issued (a lower counter causes more frequent partial saves). The counter also determines the maximum number of outstanding page sets for which the SAVE has been issued to save all the database pages on disk.
3 0 The steps comprising partial save can be summarized as follows:
Step 1. When a data space is created, allocate a bit map made of 1 bit per 'n' pages of data spaces and initialize the bit map to '0'. Initialize a counter (Modified_group count) for that data space to 0.
Step 2. When modifying the page in the data space, check the corresponding bit in the bit map. If'0', increment the Modified_group count for that data space and change the bit to ' 1'.
Step 3. If a Modified_group count reaches the save interval threshold build a save request made of all the pages corresponding to the ' 1' bits in the bit map for that data space. Reset the bit map to '0' and issue the SAVE request as an asynchronous request.
Step 4. At checkpoint time, build a save request for each data space of the pages corresponding to the ' I' bits in the bit map of the data space. Reset the bit map to '0' and issue the save request. Wait for all the request to complete before marking the checkpoint as completed.
The data space creation procedure of the present invention can be represented in pseudo code as shown as follows:
If Data space_pointer(Data space number) = 0 (this data space does not already exist) then do;
Create the data space Set Data space-pointer(Data space number) to the ALET
value used to address that data space.
Allocate the segment valid bit map for that data 2 0 space and initialize it to off.
Allocate the section is modified bit map for that data data space of size(m/n) and initialize it to off.
Set Modified_group count(Data space number) to 0.
2 5 end;
Similarly the page modification procedure can be represented in pseudo code as shown as follows:
Modify the page.
3 0 If Modified_group count(Data space number) >
Save interval and Section is modified(page-address/(n*y) is off then do;
Initialize a save request Do i = 1 to (m/n) If section is modified(i) is on then Add pages from (i*n) to ((i*n)+n) to the save request end;
Reset Section is modified to off Issue the save request without wait for completion end;
If Section is modifled(page-address/(n*y) is off then do; /* none of the pages in this section were modified */ Increment modified-group count(Data space number) by 1. Set Section is modifled(page-address/(n*y) on.
End;
Similarly the checkpoint procedure can be represented in pseudo code as shown as follows:
2 0 Do j = 1 to max(Data space number) Initialize a save request Do i = 1 to (m/n) If section is modified(i) for data space(j) is on Add pages from (i*n) to ((i*n)+n) to the save 2 5 request End;
Reset Section is modified for data space(j) to off Issue the save request without wait for completion End;
3 0 End;
Wait for all outstanding save requests to complete.
In the above pseudo code, (*) is the multiplication operator,(x) is the number of pages in one data space; (y) is the number of bytes per page; (m) is the size in pages of the data object; (n) is the number of pages represented by 1 bit; (i) is a counter.
For each SAVE request issued, the data base manager would keep the control block built to issue the request until the request completes. On completion, if the operating system indicates that some of the I/O could not be done due to a hardware problem, the data base manager can take corrective action.
No restrictions are placed on pages that are referenced in a MAPMDISK SAVE
request so that they can be modified after the request is issued even before the SAVE
completion is received. No assumptions is made on what version of the page was written out, but the system will save the latest copy at a later time.
With this invention, the database manager I/O function is bypassed.
Furthermore, page I/O is implemented so that the operating system informs the database manager that a paging I/O
is needed (and will inform it when it is completed) allowing the database manager to perform other tasks in parallel with the I/O. This technique has been proven to result in very significant improvement in data base response time without compromising data integrity.
On mufti-user computing systems, concurrent users time-share the resources on the computer systems through "virtual machines". In a virtual machine, which is a functional simulation of the real machine, each user is allowed to address the computer main storage as though it were real. Addressable main storage in a virtual machine is called "virtual storage". The size of virtual storage is limited by the addressing scheme of the computer system and by the amount of real disk storage available. Data in virtual storage is mapped to real addresses when they are to be operated on by the CPU. Mapping is the establishment of correspondences between the physical storage and virtual storage locations. An address translation table is maintained by the operating system for this purpose. On virtual machines, each user's reference to a memory address is referred to as a virtual address, and each range of addressable space available to a user is called an address space. When a user references a virtual storage location, the page containing that location may be on disk or expanded storage (a "flag" in the address translation table would indicate this). When a page has to be copied to main storage, the operating system would read the page into an available real storage page location. When completed, the page translation table is updated to reflect the new page location. If no real storage space is available, the operating system frees up main storage space by "paging out" least recently used pages.
Drawings Fig. 1 shows key information maintained in a database directory relevant to the present invention.
2 0 Fig. 2. shows an overall representation of the key elements in this invention.
Fig. 3 shows how multiple independent sub-data spaces are made to appear contiguous by using a table of pointers.
Fig. 4 shows data objects placed in specific data spaces using logical mapping.
Fig. 5 shows how a page address can be converted to a data space address and the relevant 2 5 portion of the address used by this invention.
A typical database storage system is shown in Fig. 1. It comprises a directory disk (1), one or more data disks (2), and one or two log disks similar to the data disks (not as illustrated).
The directory disk contains information on the mapping of the database pages from virtual 3 0 storage to their real physical location and other information describing the physical configuration of the data base. The data disks stores data, while the log disks record transactions against the database. In a database system, users are assigned logical pages on disk to store data objects. A
data object is a logical set of pages in the database. For example, in a relational database system, a data object may be viewed as set pages containing records of rows and columns of a table where each row is a separate record, and each column is a different data field. As shown in Fig 1, when a data object is created, entries are inserted in the database directory to indicate which data object pages contain data (3), and their physical location on disk (4).
Initially, only directory space is taken, but as a user inserts data into a data object, pages are allocated on disk and the directory is updated to point to them.
In order to maintain data integrity, the database system has a mechanism to take "checkpoints"
of the database at certain intervals to ensure that a consistent version of the database is saved. For example when a database page is modified, a copy of the page as of the previous checkpoint is kept unchanged, the modified version of the page is copied to disk, and the page directory is updated to point to the new location. Hence at checkpoint time the modified version of the database becomes the current copy of the database.
On virtual machines, data space pages can be mapped to disk by various techniques. For example contiguous virtual storage pages can be mapped to contiguous disk pages; this is referred to as physical mapping. Alternatively, contiguous virtual pages can be mapped to non-contiguous disk pages; this is referred to as logical mapping.
On some types of virtual machines, users can access multiple address spaces.
Some of these address spaces, however, can contain only data (not computer instructions) and are referred 2 0 to as data spaces. Furthermore data space pages can be mapped to disk in such a manner as to eliminate the need for the data base program manager to execute page I/O
operations in order to move data between data disk and main storage. On these systems the location on disk containing a data object page is known to the operating system. When the page is referenced by a user, it is read from its disk location by the operating system without requiring a specific disk operation 2 5 from the database program manager. When a page is directly accessible by the operating system, the database system operates more efficiently with less demands on CPU
processing cycles.
On database systems, there is frequently a continuing need to improve the overall efficiency of the system to handle increasingly large amounts of data. In particular, there is a need for faster response to queries and data changes; efficient use of real storage; and improved 3 0 efficiency in handling data objects. This in turn implies systems that optimize use of data spaces;
systems that handle only those pages that are referenced rather than the whole data object; and systems that minimize database downtime at checkpoint. In the prior art, various schemes are available to use data spaces, but no method or means has been found that uses data spaces in virtual memory that handle data objects of various sizes multi-user systems.
Example of prior art involving virtual storage but not addressing this deficiency include:
U.S. Patent 4,742,447, ("Method To Control I/O Accesses In Multi-tasking Virtual Memory Virtual Machine Type Data Processing System") which discloses a method for accessing information in a page segmented virtual memory data processing system in which virtual machines running UNIX type operating systems are concurrently established, and in which a memory manager controls the transfer of information between primary storage and secondary storage devices in response to the occurrence of page faults.
U.S. Patent 4,843,541 ("Logical Resource Partitioning of a Data Processing System") discloses a method and means for partitioning the resources in a data processing system into a plurality of logical partitions. The main storage, expanded storage, the channel and sub-channel resources of the system are assigned to the different logical partitions in the system to enable a plurality of preferred guest programming systems to run simultaneously in the different partitions.
U.S. Patent 4,843,542 ("Virtual Memory Cache for Use in Multi Processing Systems) discloses a system for maintaining data consistency among distributed processors, each having its associated cache memory.
U.S Patent 4,922,415 ("Data Processing System for Converting Virtual to Real Addresses 2 0 Without Requiring Instructions from the Central Processing) discloses a method in which a controller performs the translation functions for the inter-operability of the processor and memory and does so without requiring instructions from the processor.
U.S. Patent 4,961,134 ("Method For Minimizing Locking and Reading in a Segmented Storage Space") discloses a page accessing method in a segmented table-space which eliminates 2 5 unnecessary reading and locking. The table-space comprises data pages grouped into identical sizes segments, each segment storing data for a single table. A status indicator for each data page of segment is kept in a separate segment control block stored in a space map page.
Consequently there is an unfulfilled need for means to create and use data spaces to accommodate large data objects of different sizes, including data objects which may be much 3 0 larger than the size of a single real data space.
Also there is a unfulfilled need for means to make more efficient use of real storage by mapping to data disk, only those data pages from a data space that are referenced by the CPU, rather than the entire data object.
Yet another unfulfilled need is a means to efficiently save modified pages back to data disk so as to reduce the database down-time at checkpoint.
A full understanding of how the present invention addresses the above unfulfilled needs may be had by referring to the following description and claims taken in conjunction with the accompanying drawings:
General Description of Invention The present invention provides a method for simulating a contiguous addressable data space in a computer memory, which can be a virtual memory. Multiple addressable sub-data spaces are concatenated to simulate the contiguous addressable data space. The location of each concatenated sub-dataspace is kept track of within the contiguous addressable data space. Each sub-data space is identified by an assigned unique sub-data space identifier.
Each sub-data space has a known number of pages and each page has a known number of addressable storage locations. Each page is assigned a unique page identifier within the contiguous addressable data space, and each addressable location within the contiguous addressable data space can be referenced based on its unique page identifier and sub-data space identifier containing the page.
Another aspect of the invention includes means or apparatus for simulating a contiguous addressable data space in a computer memory. One embodiment can use computer programs 2 0 stored or carried on suitable storage media such as magnetic or optical media. The program includes means for concatenating multiple addressable sub-data spaces to simulate contiguous addressable data space and means for keeping track of the location of each concatenated sub-dataspace within the contiguous addressable data space An identification portion of the program provides means for identifying each of the sub-data space by an assigned unique sub-data space 2 5 identifier. Each page has a unique page identifier so that each addressable location within the contiguous addressable data space can be referenced, based on its unique page identifier and sub-data space identifier containing the page.
Another aspect of the invention provides a computer system with a simulated contiguous addressable data space in computer memory. Multiple addressable sub-data spaces simulating the 3 0 contiguous addressable data space are concatenated using an index containing the location of the sub-data spaces. Each sub-data space is identified by an unique sub-data space identifier, and each sub-data space has a pre-defined number of pages; and each page has a pre-defined number of addressable storage locations. The addressable storage locations are mappable to corresponding data in another storage. Each page is identified by an unique page identifier within the contiguous addressable data space and each addressable location within the contiguous addressable data space is capable of being referenced by means of a unique page identifier and the sub-data space identifier.
Detailed Description An embodiment of the present invention as illustrated in Fig. 2 is incorporated in a database system that supports data objects and data space services. However it is to be understood that the invention is not restricted to the specific system described, but can be applied on a computer system that supports data spaces, data objects and data disks in the manner described herein.
The invention can be summarized as comprising of the following steps which are described in more detail in subsequent sections:
Step 1. As shown in Fig 4., on database initialization, the present invention provides for the creation, by the use of pointers (5) and a data space table (6), of a contiguous data space (7) comprising of linked, but separately identifiable, sub-data spaces (8). Each data page within the contiguous area is identifiable from an assigned page number, and each sub-area within the contiguous data space area is identifiable from an assigned sub-data space identifier.
2 0 Step 2. On first reference to a data object in the database by a user, the invention provides for enough pages in the simulated contiguous data space to be reserved to place the entire data object starting from the first available "empty" page location. In the simulated contiguous data space, the pages to be reserved start from the first available free page, and the page number is incremented sequentially to match the size of a data object. The invention provides means to 2 5 identify any page of a data object within the simulated contiguous data space once the starting page number of the data object is established and its size expressed in pages is known.
Step 3. On first reference by a user to a page in the data object within the contiguous area, the invention provides for a mapping to disk only the segment of the data object containing the desired page. This partial mapping of the data object, rather than mapping the whole data object, 3 0 minimizes the use of main storage because for each segment mapped, the operating system has to maintain a control block which uses up real storage. Delaying the mapping to a segment on the first reference means that a control block is only allocated if the segment is actually referenced.
Therefore data space segments that are never referenced, and thus not mapped, will not require the control block.
Step 4. After pages have been modified, the invention provides for saving the modified pages at predetermined time intervals set by the user, rather than waiting for the database system to make a global save at checkpoint. This provision of the invention allows pages to be saved in an on-going manner that minimizes the database down time associated with system initiated checkpoint.
Details on the above steps are provide in the following sections.
Since a data object can be larger than the maximum size of the real data space available to a user, the first step as provided by this invention and shown in Fig 3. is to create a large apparent contiguous data space (7) by linking separately identifiable fixed-sized sub-data spaces (8). This may done advantageously by the following steps:
i) Logically link multiple fixed-sized sub-data spaces using an array of pointers, and maintain a data space table for this purpose. See Fig 3.
ii) For each page in the simulated contiguous data space, assign a unique page number as provided for by this invention.
iii) For each page in the simulated contiguous data space, establish the page number as provided for by the present invention.
2 0 In order to keep track of the identity of each data object and sub-data space within the simulated contiguous data space, a table of pointers is set up to identify the start and end of each data space and sub-data space.
Within the simulated contiguous data space, any page number can be located as provided by the present invention. For example, if the size of each constituent data space is 524,288 pages, 2 5 and the page being referenced is page number 1,217,436, then we can calculate that the referenced page is 168,860 in the third sub-data space. After the page has been identified, it can be mapped to the correct real storage page.
In pseudo code format the procedure to determine the page address as a function of its location on a data space can expressed as follows:
3 0 If the Starting_page number of the data object(n), is greater or equal to 0 (i.e. the data object was placed in the simulated contiguous data space, then do;
Data space number = [(Starting-page number of data object,(n)) + m ] / [the size of 1 data space expressed in pages]
Similarly the page number within the contiguous area can be determined as follows:
Page address = [(Starting-page number of data object, (n)) + m] // [(The size of 1 data space expressed in pages) * (the size of 1 page expressed in bytes)]
end;
In the above pseudo code the Starting-page number is the location of the first page of the data object in the simulated contiguous data space; the symbol (/) represent an integer division;
the symbol (/~ represents the remainder of an integer division; (*) is the multiplication operator;
and (n) is the identity of the data object; and (m) is the page number within the data object containing the required data.
If the data object Starting-page number is not known, it can be determined as follows:
i) On database initialization, an array named "Starting-page number" is defined to contain the starting page location of each data object to be placed in the simulated contiguous data space. The dimension of the array is the same as the number of data objects in the database.
The array is initialized to negative values to indicate that the data objects have not yet been 2 0 placed in the simulated contiguous data space.
ii) A variable "Next available-page" is defined and initialized to 0 in a counter to indicate that this is the beginning page of the empty simulated contiguous data space.
iii) At the first user reference to a data object, the next available set of pages in the simulated contiguous data space is reserved for the referenced data object, by setting the 2 5 Starting_page number of the said data object to the Next available data space page number taken from the counter.
iv) The Next available-page counter is then increased by the size of the data object (in pages) rounded up to the next segment so as to start all data objects on a segment boundary. In this invention it is convenient to start the data object on a new page segment boundary, but this is 3 0 not an essential requirement if the systems does not handle pages in segments.
With the above procedure, any address within the simulated contiguous data space can be determined once the location of the object is known. Segments of the data object can then be mapped to its disk using information in the database directory.
The same result can be obtained by shifting bits. For example, on a 32-bit computer architecture, bits 0 to 12 can be used to represent the data space number, and bits 13 to 31 shifted left 12 positions (equivalent to multiplication by 4096) as the address in the data space. This alternative procedure is illustrated in Fig. 5.
Once the simulated contiguous data space is created, a data object can be placed in the data space as shown in Fig 3. For example, data object number 1 (9) consisting of 102,528 pages is placed in pages 0 to 102,527 in sub-data space 1 (10). This may be followed by another data object number 5 (11), say, consisting of 1,024,000 pages placed from pages 102,656 (which marks the beginning of a new segment in data space 1 ) to pages spanning sub-data spaces 2 (8) and 3 (8). In this example, data object 1 would therefore be placed in the first 400 segments of the simulated contiguous data space, while data object 5 would be placed from the first page of segment number 401 starting from page number 102,565.
Similarly, since the Starting-page number for each data object is known, any request for a data page can be converted to a page number within the simulated contiguous data space. For example if data object(n) starts at page 1,126,656 and we request data page 90,780 of data object (n), we can calculate the page number to be 1,217,436 or page 168,860 located in the third sub-data space.
An important benefit in allowing a data object to start on a segment boundary instead of 2 0 at the beginning of data space is the ease in supporting both very large data objects (spanning more than one data space) and very small data objects (spanning much less than the size of one data space). This invention eliminates the need to have one contiguous data space for each data object, and makes it possible to have many small and large data objects placed in the same simulated contiguous data space.
2 5 Once the data object has been placed in the simulated contiguous data space, it can be mapped to the correct disk pages based on information contained in the database directory.
However mapping an entire data object at its first reference is expensive and would require considerable real storage. For this reason this invention provides a mechanism to delay the mapping until first reference of a data object, and then to map only a segment consisting of the 3 0 desired pages of the data object by a procedure called "mapping on demand."
Mapping on demand can be explained as follows: For each sub-data space a bit map is created to indicate if each segment in the sub-data space is currently "referenced". A segment is referenced if the pages containing data in that segment have been mapped to disk. Initially, the bit map is set to '0' and the entire sub-data space is considered not referenced. The bit is referred to as the "segment valid bit" and one bit map is used per data space. For example if a sub-data space is made up of 524 288 pages and each segment is 256 pages, then there are 2048 segments and the bit map is therefore 2048/8 = 256 bytes per sub-data space which is allocated when the sub-data space is created.
Although the entire data object is placed in the simulated contiguous data space when it is first referenced, the sub-data spaces and the bit maps may not all exist. The mechanism provided by this invention to calculate the page address of any page in the contiguous area detects if this is the first reference to a sub-data space and/or a segment of the data object.
If this is the first reference, a sub-data space is created and a segment is mapped. The steps comprising mapping on demand can be summarized as follows:
Step 1. At first reference to a data object, place it in the simulated contiguous data space by allocating the starting and ending data space address as previously described.
Step 2. Calculate the address of the referenced page as previously described.
Step 3. If the sub-data space was not created, then create the sub-data space as previously described, set the data space identifier value used to address that sub-data space, and allocate and initialize the segment validity bit map.
Step 4. If the segment containing the page is not mapped, then map it and turn on the bit 2 0 to indicate that the segment was mapped.
This procedure is summarized in the following pseudo code:
If Starting-page number for data object(n) is not assigned then do;
2 5 Starting_page number(n) = Next available_ page;
Reserved space = ((Data object size(n)/(z) + 1) * (z);
Next available_page = Next available_page + Reserved_ end;
3 0 do;
space;
Data space number = (Starting_page_number(n) + m)/x;
Page-address = ((Starting_page_number(n) + m)//x) * y;
end;
If Data space-pointer(Data space number) was not created then do;
Create the data space Set Data space-pointer(Data space number) to the value used to address that data space.
Allocate the segment valid bit map for that data space and initialize it to off. end;
If Segment valid(Page_address / y*z) is off then do;
Map the segment containing Page-m Set segment valid(Page address / y*z) on;
end;
In the above pseudo code; Starting-page number is an array of as many elements as the number of data objects in the database; (~ represent the division function;
(/n represent the remainder of a division; (*) represent the multiplication operator; (z) is the number of pages in a segment; (y) is the size of a page in bytes; (m) is the referenced page from the start of the data object; (n) is the identity of the data object.
2 0 Checking the "segment valid" bit at each page request is inexpensive as the bit map is small. If the segment is not mapped, then the database manager program has to read the directory blocks containing information on the pages required for that segment and issue a request to map the data space pages that contain data to their locations on disk. Once mapped, the "segment valid bit" is turned on and any subsequent reference to read a page in that segment will be translated 2 5 directly to a virtual storage address without reference to the database directory. The net result is an improved query response time.
If a request is made for an update (the page was modified), the page may have to be shadowed (moved to a new disk location); alternatively it may have to be inserted if it had not existed before (first insert). This will require a change to the mapping in which case the database 3 0 manager will determine from the database directory if the page needs to be (re)mapped. If so, a new data page is allocated, the directory is updated and a new mapping is done. This will result in a mapping change, not a change in the virtual address of a page.
An example of mapping on demand as provided by one aspect of the present invention can be illustrated using the IBM~ SQL/DSTM database management system operating on an IBM
System/390TM computer with the IBM VM/ESATM operating system as follows.
Before updating a page, the IBM SQL/DS management system will copy the requested page to a buffer. It will then re-issue the MAPDISK DEFINE request with the PAGEVIEW = ZERO parameter before copying the data back from the local buffer to the data space. Note, PAGEVIEW
= ZERO
informs VM/ESA operating system that it should provide an empty real storage page for that data space page rather than reading the contents of the disk when the page is first referenced.
If updating without the use of a local buffer is desired, the page can be remapped with the PAGEVIEW = RETAIN parameter. This first reads the copy of the page from its current disk (old) location before re-setting the mapping to the new location. Consequently the user is presented with the old copy of the page which will be written to the new location.
When the local buffer is used, the re-mapping can be deferred to when the page is moved back from the local buffer to the data space. Performance optimization is implemented by delaying the remapping in an attempt to group multiple mapping request into one MAPMDISK
DEFINE call.
For this purpose, several MAPMDISK DEFINE request blocks can be initialized and used as required. Each request can be batched for several consecutive data space pages.
When a page is moved to the local buffer and a MAPMDISK DEFINE is required, the 2 0 following is done for a new request:
i) If a MAPMDISK DEFINE request exists for the previous or the next page of the same data space, the two requests are combined into one.
ii) If the new request cannot be attached to any existing request so that it would be consecutive, and there exists an unused request block, use it to store the new request.
2 5 iii) If all the pre-allocated request areas are in use, issue the DEFINE
for the requests already in one of the request control blocks and reset the block for the current page.
Since, by using an embodiment of the present invention, the operating system is now responsible for performing the database I/O operations through its paging mechanism, an interface is used to ensure that all modified data is written to the database at regular intervals, 3 0 maintaining integrity of the database. Without such mechanism, integrity of the database.
In the prior art, mechanisms exist to request that changes to a database be written to disk while the database manager is waiting. However such mechanisms may cause significant delays at checkpoint time as many pages may be needed to be written to disk when data spaces are used.
To resolve this problem, some operating system have implemented a mechanism whereby the database manager can specify a subset of the database to be made current on disk (i.e. to be written to disk if the pages in memory were modified). Further the mechanism is said to be asynchronous in that a save request can be issued, and processing continued, while the I/O
operations are performed. An indicator is presented to the database manager when all I/O needed for that save have completed.
One embodiment of the present invention takes advantage of the above mechanism to implement a partial save. Like mapping on demand which allows the database manager to delay the mapping to the first reference of a page, the invention provides for a partial save which allows the database manager to request page saves at short intervals and for only the modified pages of a data object.
Partial save can be illustrated using the IBM SQL/DS database management system as follows:
Like the Segment Valid Bit map, a bit map is allocated and set to '0' when a data space is referenced for the first time. In this case the bit map is made of 1 bit for every 32 pages (or 4 096 bytes per data space) which we refer to as 'MODIFIED BIT MAP'. The bit representing a set of 32 pages can be calculated by dividing the page address by 131,072 (32 pages of 4 096 bytes each). Before moving a page from the local buffer back to the data space, the corresponding bit in 2 0 the 'MODIFIED BIT MAP' is checked. If it is OFF (0), it is then turned ON
(1) and a counter is incremented to keep track of the number of bits currently ON.
When the counter reaches a predefined limit (set by the user) the data base manager would issue a MAPMDISK SAVE using as the list of addresses to save, the list corresponding to the pages represented by the ON in the 'MODIFIED BIT MAP'. The database manager would 2 5 then reset the bit map to OFF (0). Since the request is asynchronous, the database manager would continue processing while the I/O operations take place. The value of the counter determines the frequency with which the partial saves are issued (a lower counter causes more frequent partial saves). The counter also determines the maximum number of outstanding page sets for which the SAVE has been issued to save all the database pages on disk.
3 0 The steps comprising partial save can be summarized as follows:
Step 1. When a data space is created, allocate a bit map made of 1 bit per 'n' pages of data spaces and initialize the bit map to '0'. Initialize a counter (Modified_group count) for that data space to 0.
Step 2. When modifying the page in the data space, check the corresponding bit in the bit map. If'0', increment the Modified_group count for that data space and change the bit to ' 1'.
Step 3. If a Modified_group count reaches the save interval threshold build a save request made of all the pages corresponding to the ' 1' bits in the bit map for that data space. Reset the bit map to '0' and issue the SAVE request as an asynchronous request.
Step 4. At checkpoint time, build a save request for each data space of the pages corresponding to the ' I' bits in the bit map of the data space. Reset the bit map to '0' and issue the save request. Wait for all the request to complete before marking the checkpoint as completed.
The data space creation procedure of the present invention can be represented in pseudo code as shown as follows:
If Data space_pointer(Data space number) = 0 (this data space does not already exist) then do;
Create the data space Set Data space-pointer(Data space number) to the ALET
value used to address that data space.
Allocate the segment valid bit map for that data 2 0 space and initialize it to off.
Allocate the section is modified bit map for that data data space of size(m/n) and initialize it to off.
Set Modified_group count(Data space number) to 0.
2 5 end;
Similarly the page modification procedure can be represented in pseudo code as shown as follows:
Modify the page.
3 0 If Modified_group count(Data space number) >
Save interval and Section is modified(page-address/(n*y) is off then do;
Initialize a save request Do i = 1 to (m/n) If section is modified(i) is on then Add pages from (i*n) to ((i*n)+n) to the save request end;
Reset Section is modified to off Issue the save request without wait for completion end;
If Section is modifled(page-address/(n*y) is off then do; /* none of the pages in this section were modified */ Increment modified-group count(Data space number) by 1. Set Section is modifled(page-address/(n*y) on.
End;
Similarly the checkpoint procedure can be represented in pseudo code as shown as follows:
2 0 Do j = 1 to max(Data space number) Initialize a save request Do i = 1 to (m/n) If section is modified(i) for data space(j) is on Add pages from (i*n) to ((i*n)+n) to the save 2 5 request End;
Reset Section is modified for data space(j) to off Issue the save request without wait for completion End;
3 0 End;
Wait for all outstanding save requests to complete.
In the above pseudo code, (*) is the multiplication operator,(x) is the number of pages in one data space; (y) is the number of bytes per page; (m) is the size in pages of the data object; (n) is the number of pages represented by 1 bit; (i) is a counter.
For each SAVE request issued, the data base manager would keep the control block built to issue the request until the request completes. On completion, if the operating system indicates that some of the I/O could not be done due to a hardware problem, the data base manager can take corrective action.
No restrictions are placed on pages that are referenced in a MAPMDISK SAVE
request so that they can be modified after the request is issued even before the SAVE
completion is received. No assumptions is made on what version of the page was written out, but the system will save the latest copy at a later time.
With this invention, the database manager I/O function is bypassed.
Furthermore, page I/O is implemented so that the operating system informs the database manager that a paging I/O
is needed (and will inform it when it is completed) allowing the database manager to perform other tasks in parallel with the I/O. This technique has been proven to result in very significant improvement in data base response time without compromising data integrity.
Claims (6)
1. A method for mapping on demand a page of a data object contained in a database and stored in one or more database storage disks, wherein the database comprises one or more database data objects, wherein the database is accessed via a contiguous data space representation, the contiguous data space being represented by a plurality of concatenated sub-data spaces, wherein each sub-data space is a virtual data space of a maximum size that is addressable by a computer operating system, each sub-data space comprising a plurality of data segments, each data segment comprising a plurality of pages, wherein the pages in the contiguous data space representation are contiguous, each page comprising a known number of addressable storage locations, wherein the contiguous data space, the sub-data spaces, the data segments, and the pages are addressable by a database management system, wherein the address of a page to be mapped has been determined to be placed in a data segment of a sub-data space, the method comprising the steps of:
(1) determining whether the sub-data space has been created for representation in the contiguous data space;
(2) creating the sub-data space for representation in the contiguous data space if step (1) determines that the sub-data space has not been created in the contiguous data space;
(3) creating a segment bit map for the sub-data space if step (1) determines that the sub-data space has not been created in the contiguous data space, wherein said segment bit map comprises a plurality of bits, each bit representing a data segment contained in the sub-data space and indicating whether said data segment has been mapped to the database storage disk;
(4) determining if a bit in said segment bit map is equal to a predetermined value, wherein said bit corresponds to the data segment of the sub-data space containing the page to be mapped, and said predetermined value indicates that the data segment corresponding to said bit has been mapped to the database storage disk;
(5) mapping a data segment of the data object from the database storage disk to the data segment of the sub-data space, wherein said data segment from the database storage disk contains the page to be mapped, and said mapping occurs if step (4) determines that said bit in said segment bit map does not equal said predetermined value; and (6) setting said bit in said segment bit map to said predetermined value, wherein said setting occurs when step (5) is performed.
(1) determining whether the sub-data space has been created for representation in the contiguous data space;
(2) creating the sub-data space for representation in the contiguous data space if step (1) determines that the sub-data space has not been created in the contiguous data space;
(3) creating a segment bit map for the sub-data space if step (1) determines that the sub-data space has not been created in the contiguous data space, wherein said segment bit map comprises a plurality of bits, each bit representing a data segment contained in the sub-data space and indicating whether said data segment has been mapped to the database storage disk;
(4) determining if a bit in said segment bit map is equal to a predetermined value, wherein said bit corresponds to the data segment of the sub-data space containing the page to be mapped, and said predetermined value indicates that the data segment corresponding to said bit has been mapped to the database storage disk;
(5) mapping a data segment of the data object from the database storage disk to the data segment of the sub-data space, wherein said data segment from the database storage disk contains the page to be mapped, and said mapping occurs if step (4) determines that said bit in said segment bit map does not equal said predetermined value; and (6) setting said bit in said segment bit map to said predetermined value, wherein said setting occurs when step (5) is performed.
2. The method of claim 1, wherein the page to be mapped is to be modified, thereby creating a page to be modified, comprising the further steps of:
(7) determining whether said page to be modified requires a new mapping to the database storage disk;
wherein if step (7) determines that said page to be modified requires said new mapping to the database storage disk, the method comprise the further steps of:
(8) copying said page to be modified to another work space in computer memory and modifying said page to be modified in said another work space in computer memory;
(9) allocating a new database storage page in said database storage disk;
(10) determining whether a mapping request to map said page to be modified to said new database storage page in said database storage disk can be combined with a prior mapping request, wherein said prior mapping request maps a second page in the sub-data space to a second database storage page in said database storage disk and said page to be modified is adjacent to said second page in the sub-data space;
(11) issuing said prior mapping request if step (10) determines that said mapping request cannot be combined with said prior mapping request and said prior mapping request exists;
(12) copying said second page of said prior mapping request from said another work space in computer memory if step (11) issues said prior mapping request;
(13) preparing said mapping request if step (11) issues said prior mapping request or said prior mapping request does not exist; and (14) combining said mapping request with said prior mapping request if step (10) determines that said mapping request can be combined with said prior mapping request.
(7) determining whether said page to be modified requires a new mapping to the database storage disk;
wherein if step (7) determines that said page to be modified requires said new mapping to the database storage disk, the method comprise the further steps of:
(8) copying said page to be modified to another work space in computer memory and modifying said page to be modified in said another work space in computer memory;
(9) allocating a new database storage page in said database storage disk;
(10) determining whether a mapping request to map said page to be modified to said new database storage page in said database storage disk can be combined with a prior mapping request, wherein said prior mapping request maps a second page in the sub-data space to a second database storage page in said database storage disk and said page to be modified is adjacent to said second page in the sub-data space;
(11) issuing said prior mapping request if step (10) determines that said mapping request cannot be combined with said prior mapping request and said prior mapping request exists;
(12) copying said second page of said prior mapping request from said another work space in computer memory if step (11) issues said prior mapping request;
(13) preparing said mapping request if step (11) issues said prior mapping request or said prior mapping request does not exist; and (14) combining said mapping request with said prior mapping request if step (10) determines that said mapping request can be combined with said prior mapping request.
3. A computer system for mapping on demand a page of a data object contained in a database and stored in one or more database storage disks, wherein the database comprises one or more database data objects, wherein the database is accessed via a contiguous data space representation, the contiguous data space being represented by a plurality of concatenated sub-data spaces, wherein each sub-data space is a virtual data space of a maximum size that is addressable by a computer operating system, each sub-data space comprising a plurality of data segments, each data segment comprising a plurality of pages, wherein the pages in the contiguous data space representation are contiguous, each page comprising a known number of addressable storage locations, wherein the contiguous data space, the sub-data spaces, the data segments, and the pages are addressable by a database management system, wherein a page to be mapped has been determined to be placed in a data segment of a sub-data space, comprising:
first determining means for determining whether the sub-data space has been created for representation in the contiguous data space;
sub-data space means for creating the sub-data space for representation in the contiguous data space if said first determining means determines that the sub-data space has not been created in the contiguous data space;
bit map means for creating a segment bit map for the sub-data space if said first determining means determines that the sub-data space has not been created in the contiguous data space, wherein said segment bit map comprises a plurality of bits, each bit representing a data segment contained in the sub-data space and indicating whether said data segment has been mapped to the database storage disk;
second determining means for determining if a bit in said segment bit map is equal to a predetermined value, wherein said bit corresponds to the data segment of the sub-data space containing the page to be mapped, and said predetermined value indicates that the data segment corresponding to said bit has been mapped to the database storage disk;
mapping means for mapping a data segment of the data object from the database storage disk to the data segment of the sub-data space, wherein said data segment from the database storage disk contains the page to be mapped, and said mapping occurs if said second determining means determines that said bit in said segment bit map does not equal said predetermined value;
and setting means for setting said bit in said segment bit map to said predetermined value, wherein said setting occurs when said mapping means is performed.
first determining means for determining whether the sub-data space has been created for representation in the contiguous data space;
sub-data space means for creating the sub-data space for representation in the contiguous data space if said first determining means determines that the sub-data space has not been created in the contiguous data space;
bit map means for creating a segment bit map for the sub-data space if said first determining means determines that the sub-data space has not been created in the contiguous data space, wherein said segment bit map comprises a plurality of bits, each bit representing a data segment contained in the sub-data space and indicating whether said data segment has been mapped to the database storage disk;
second determining means for determining if a bit in said segment bit map is equal to a predetermined value, wherein said bit corresponds to the data segment of the sub-data space containing the page to be mapped, and said predetermined value indicates that the data segment corresponding to said bit has been mapped to the database storage disk;
mapping means for mapping a data segment of the data object from the database storage disk to the data segment of the sub-data space, wherein said data segment from the database storage disk contains the page to be mapped, and said mapping occurs if said second determining means determines that said bit in said segment bit map does not equal said predetermined value;
and setting means for setting said bit in said segment bit map to said predetermined value, wherein said setting occurs when said mapping means is performed.
4. The computer system according to claim 3, wherein the page to be mapped is to be modified, thereby creating a page to be modified, further comprising:
third determining means for determining whether said page to be modified requires a new mapping to the database storage disk;
wherein if said third determining means determines that said page to be modified requires said new mapping to the database storage disk, the computer system further comprising:
first copying means for copying said page to be modified to another work space in computer memory and modifying said page to be modified in said another work space in computer memory;
allocating means for allocating a new database storage page in said database storage disk;
fourth determining means for determining whether a mapping request to map said page to be modified to said new database storage page in said database storage disk can be combined with a prior mapping request, wherein said prior mapping request maps a second page in the sub-data space to a second database storage page in said database storage disk and said page to be modified is adjacent to said second page in the sub-data space;
issuing means for issuing said prior mapping request if said fourth determining means determines that said mapping request cannot be combined with said prior mapping request and said prior mapping request exists;
second copying means for copying said second page of said prior mapping request from said another work space in computer memory if said issuing means issues said prior mapping request;
preparing means for preparing said mapping request if said issuing means issues said prior mapping request or said prior mapping request does not exist; and combining means for combining said mapping request with said prior mapping request if said fourth determining means determines that said mapping request can be combined with said prior mapping request.
third determining means for determining whether said page to be modified requires a new mapping to the database storage disk;
wherein if said third determining means determines that said page to be modified requires said new mapping to the database storage disk, the computer system further comprising:
first copying means for copying said page to be modified to another work space in computer memory and modifying said page to be modified in said another work space in computer memory;
allocating means for allocating a new database storage page in said database storage disk;
fourth determining means for determining whether a mapping request to map said page to be modified to said new database storage page in said database storage disk can be combined with a prior mapping request, wherein said prior mapping request maps a second page in the sub-data space to a second database storage page in said database storage disk and said page to be modified is adjacent to said second page in the sub-data space;
issuing means for issuing said prior mapping request if said fourth determining means determines that said mapping request cannot be combined with said prior mapping request and said prior mapping request exists;
second copying means for copying said second page of said prior mapping request from said another work space in computer memory if said issuing means issues said prior mapping request;
preparing means for preparing said mapping request if said issuing means issues said prior mapping request or said prior mapping request does not exist; and combining means for combining said mapping request with said prior mapping request if said fourth determining means determines that said mapping request can be combined with said prior mapping request.
5. A computer program product for use with a database management system, comprising:
a computer usable medium having computer readable program code means embodied in said medium for mapping on demand a page of a data object contained in a database and stored in one or more database storage disks, wherein the database comprises one or more database data objects, wherein the database is accessed via a contiguous data space representation, the contiguous data space being represented by a plurality of concatenated sub-data spaces, wherein each sub-data space is a virtual data space of a maximum size that is addressable by a computer operating system, each sub-data space comprising a plurality of data segments, each data segment comprising a plurality of pages, wherein the pages of the contiguous data space representation are contiguous, each page comprising a known number of addressable storage locations, wherein the locations, wherein the contiguous data space, the sub-data spaces, the data segments, and the pages are addressable by the database management system, wherein the address of a page to be mapped has been determined to be placed in a data segment of a sub-data space, said computer program product having:
first computer readable program code means for determining whether the sub-data space has been created for representation in the contiguous data space;
second computer readable program code means for creating the sub-data space for representation in the contiguous data space if said first computer readable program code means determines that the sub-data space has not been created in the contiguous data space;
third computer readable program code means for creating a segment bit map for the sub-data space if said first computer readable program code means determines that the sub-data space has not been created in the contiguous data space, wherein said segment bit map comprises a plurality of bits, each bit representing a data segment contained in the sub-data space and indicating whether said data segment has been mapped to the database storage disk;
fourth computer readable program code means for determining if a bit in said segment bit map is equal to a predetermined value, wherein said bit corresponds to the data segment of the sub-data space containing the page to be mapped, and said predetermined value indicates that the data segment corresponding to said bit has been mapped to the database storage disk;
fifth computer readable program code means for mapping a data segment of the data object from the database storage disk to the data segment of the sub-data space, wherein said data segment from the database storage disk contains the page to be mapped, and said mapping occurs if said fourth computer readable program code means determines that said bit in said segment bit map does not equal said predetermined value; and sixth computer readable program code means for setting said bit in said segment bit map to said predetermined value, wherein said setting occurs when said fifth computer readable program code means is performed.
a computer usable medium having computer readable program code means embodied in said medium for mapping on demand a page of a data object contained in a database and stored in one or more database storage disks, wherein the database comprises one or more database data objects, wherein the database is accessed via a contiguous data space representation, the contiguous data space being represented by a plurality of concatenated sub-data spaces, wherein each sub-data space is a virtual data space of a maximum size that is addressable by a computer operating system, each sub-data space comprising a plurality of data segments, each data segment comprising a plurality of pages, wherein the pages of the contiguous data space representation are contiguous, each page comprising a known number of addressable storage locations, wherein the locations, wherein the contiguous data space, the sub-data spaces, the data segments, and the pages are addressable by the database management system, wherein the address of a page to be mapped has been determined to be placed in a data segment of a sub-data space, said computer program product having:
first computer readable program code means for determining whether the sub-data space has been created for representation in the contiguous data space;
second computer readable program code means for creating the sub-data space for representation in the contiguous data space if said first computer readable program code means determines that the sub-data space has not been created in the contiguous data space;
third computer readable program code means for creating a segment bit map for the sub-data space if said first computer readable program code means determines that the sub-data space has not been created in the contiguous data space, wherein said segment bit map comprises a plurality of bits, each bit representing a data segment contained in the sub-data space and indicating whether said data segment has been mapped to the database storage disk;
fourth computer readable program code means for determining if a bit in said segment bit map is equal to a predetermined value, wherein said bit corresponds to the data segment of the sub-data space containing the page to be mapped, and said predetermined value indicates that the data segment corresponding to said bit has been mapped to the database storage disk;
fifth computer readable program code means for mapping a data segment of the data object from the database storage disk to the data segment of the sub-data space, wherein said data segment from the database storage disk contains the page to be mapped, and said mapping occurs if said fourth computer readable program code means determines that said bit in said segment bit map does not equal said predetermined value; and sixth computer readable program code means for setting said bit in said segment bit map to said predetermined value, wherein said setting occurs when said fifth computer readable program code means is performed.
6. A program storage device readable by a machine tangibly embodying the computer program product according to claim 5, wherein the page to be mapped is to be modified, thereby creating a page to be modified, further comprising:
seventh computer readable program code means for determining whether said page to be modified requires a new mapping to the database storage disk;
wherein if said seventh computer readable program code means determines that said page to be modified requires said new mapping to the database storage disk, the computer program product still further comprising:
eighth computer readable program code means for copying said page to be modified to another work space in computer memory and modifying said page to be modified in said another work space in computer memory;
ninth computer readable program code means for allocating a new database storage page in the database storage disk;
tenth computer readable program code means for determining whether a mapping request to map said page to be modified to said new database storage page in said database storage disk can be combined with a prior mapping request, wherein said prior mapping request maps a second page in the sub-data space to a second database storage page in said database storage disk and said page to be modified is adjacent to said second page in the sub-data space;
eleventh computer readable program code means for issuing said prior mapping request if said tenth computer readable program code means determines that said mapping request cannot be combined with said prior mapping request and said prior mapping request exists;
twelfth computer readable program code means for copying said second page of said prior mapping request from said another work space in computer memory if said eleventh computer readable program code means issues said prior mapping request;
thirteenth computer program code means for preparing said mapping request if said eleventh computer program code means issues said prior mapping request or said prior mapping request does not exist; and fourteenth computer program code means for combining said mapping request with said prior mapping request if said tenth computer program code means determines that said mapping request can be combined with said prior mapping request.
seventh computer readable program code means for determining whether said page to be modified requires a new mapping to the database storage disk;
wherein if said seventh computer readable program code means determines that said page to be modified requires said new mapping to the database storage disk, the computer program product still further comprising:
eighth computer readable program code means for copying said page to be modified to another work space in computer memory and modifying said page to be modified in said another work space in computer memory;
ninth computer readable program code means for allocating a new database storage page in the database storage disk;
tenth computer readable program code means for determining whether a mapping request to map said page to be modified to said new database storage page in said database storage disk can be combined with a prior mapping request, wherein said prior mapping request maps a second page in the sub-data space to a second database storage page in said database storage disk and said page to be modified is adjacent to said second page in the sub-data space;
eleventh computer readable program code means for issuing said prior mapping request if said tenth computer readable program code means determines that said mapping request cannot be combined with said prior mapping request and said prior mapping request exists;
twelfth computer readable program code means for copying said second page of said prior mapping request from said another work space in computer memory if said eleventh computer readable program code means issues said prior mapping request;
thirteenth computer program code means for preparing said mapping request if said eleventh computer program code means issues said prior mapping request or said prior mapping request does not exist; and fourteenth computer program code means for combining said mapping request with said prior mapping request if said tenth computer program code means determines that said mapping request can be combined with said prior mapping request.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA002055295A CA2055295C (en) | 1991-11-12 | 1991-11-12 | Logical mapping of data objects using data spaces |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002055295A Division CA2055295C (en) | 1991-11-12 | 1991-11-12 | Logical mapping of data objects using data spaces |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2285096A1 CA2285096A1 (en) | 1993-05-13 |
| CA2285096C true CA2285096C (en) | 2000-05-09 |
Family
ID=4148740
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002055295A Expired - Fee Related CA2055295C (en) | 1991-11-12 | 1991-11-12 | Logical mapping of data objects using data spaces |
| CA002285089A Expired - Fee Related CA2285089C (en) | 1991-11-12 | 1991-11-12 | Logical mapping of data objects using data spaces |
| CA002285096A Expired - Fee Related CA2285096C (en) | 1991-11-12 | 1991-11-12 | Logical mapping of data objects using data spaces |
Family Applications Before (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002055295A Expired - Fee Related CA2055295C (en) | 1991-11-12 | 1991-11-12 | Logical mapping of data objects using data spaces |
| CA002285089A Expired - Fee Related CA2285089C (en) | 1991-11-12 | 1991-11-12 | Logical mapping of data objects using data spaces |
Country Status (4)
| Country | Link |
|---|---|
| US (6) | US5652873A (en) |
| EP (1) | EP0542483A1 (en) |
| JP (1) | JP2625619B2 (en) |
| CA (3) | CA2055295C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6647398B1 (en) | 1999-04-01 | 2003-11-11 | Beacon Information Technology Inc. | Data managing method and apparatus, and recording medium |
Families Citing this family (157)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5579474A (en) * | 1992-12-28 | 1996-11-26 | Hitachi, Ltd. | Disk array system and its control method |
| US5918225A (en) * | 1993-04-16 | 1999-06-29 | Sybase, Inc. | SQL-based database system with improved indexing methodology |
| US5794228A (en) * | 1993-04-16 | 1998-08-11 | Sybase, Inc. | Database system with buffer manager providing per page native data compression and decompression |
| US5581765A (en) * | 1994-08-30 | 1996-12-03 | International Business Machines Corporation | System for combining a global object identifier with a local object address in a single object pointer |
| US5732256A (en) * | 1995-08-30 | 1998-03-24 | Microsoft Corporation | CD-ROM optimization and stream splitting |
| US5717919A (en) * | 1995-10-02 | 1998-02-10 | Sybase, Inc. | Database system with methods for appending data records by partitioning an object into multiple page chains |
| GB2312059B (en) * | 1996-04-12 | 2000-11-15 | Sony Uk Ltd | Data storage |
| US5852820A (en) * | 1996-08-09 | 1998-12-22 | Digital Equipment Corporation | Method for optimizing entries for searching an index |
| US6742080B1 (en) | 1996-09-06 | 2004-05-25 | Intel Corporation | Disk block allocation optimization methodology and application |
| US5802593A (en) * | 1996-09-06 | 1998-09-01 | Intel Corporation | Method and apparatus for improving disk drive performance |
| DE69719934T2 (en) * | 1996-12-20 | 2003-11-27 | Ibm | Method and device for fast and secure data collection |
| JPH10333948A (en) * | 1997-04-01 | 1998-12-18 | Kokusai Zunou Sangyo Kk | Virtual data base space system and computer readable recording medium recording data base program |
| US5848405A (en) * | 1997-04-21 | 1998-12-08 | Oracle Corporation | Method and apparatus for identifying new data by address ranges |
| US7031987B2 (en) * | 1997-05-30 | 2006-04-18 | Oracle International Corporation | Integrating tablespaces with different block sizes |
| US6035374A (en) * | 1997-06-25 | 2000-03-07 | Sun Microsystems, Inc. | Method of executing coded instructions in a multiprocessor having shared execution resources including active, nap, and sleep states in accordance with cache miss latency |
| US7581077B2 (en) * | 1997-10-30 | 2009-08-25 | Commvault Systems, Inc. | Method and system for transferring data in a storage operation |
| US6418478B1 (en) * | 1997-10-30 | 2002-07-09 | Commvault Systems, Inc. | Pipelined high speed data transfer mechanism |
| US6118940A (en) * | 1997-11-25 | 2000-09-12 | International Business Machines Corp. | Method and apparatus for benchmarking byte code sequences |
| US6175900B1 (en) * | 1998-02-09 | 2001-01-16 | Microsoft Corporation | Hierarchical bitmap-based memory manager |
| US6438675B1 (en) * | 1998-03-23 | 2002-08-20 | Ati Technologies, Inc. | Variable format memory access device |
| US5995960A (en) * | 1998-04-08 | 1999-11-30 | International Business Machines Corporaiton | Method and system for improving efficiency of programs utilizing databases by exeuting scenarios based on recalled processed information |
| DE19819205A1 (en) * | 1998-04-29 | 1999-11-04 | Siemens Ag | Data retention system for persistent data |
| US6226730B1 (en) * | 1998-06-05 | 2001-05-01 | Intel Corporation | Achieving page hit memory cycles on a virtual address reference |
| FI981355A (en) * | 1998-06-11 | 1999-12-12 | Nokia Mobile Phones Ltd | Electronic file retrieval method and system |
| US9361243B2 (en) | 1998-07-31 | 2016-06-07 | Kom Networks Inc. | Method and system for providing restricted access to a storage medium |
| US8234477B2 (en) | 1998-07-31 | 2012-07-31 | Kom Networks, Inc. | Method and system for providing restricted access to a storage medium |
| US6334134B1 (en) * | 1998-08-18 | 2001-12-25 | International Business Machines Corporation | Insert performance on a multi-system transaction environment |
| US6366902B1 (en) | 1998-09-24 | 2002-04-02 | International Business Machines Corp. | Using an epoch number to optimize access with rowid columns and direct row access |
| US6144970A (en) * | 1998-09-24 | 2000-11-07 | International Business Machines Corporation | Technique for inplace reorganization of a LOB table space |
| US6363389B1 (en) | 1998-09-24 | 2002-03-26 | International Business Machines Corporation | Technique for creating a unique quasi-random row identifier |
| US6694340B1 (en) | 1998-09-24 | 2004-02-17 | International Business Machines Corporation | Technique for determining the age of the oldest reading transaction with a database object |
| US6606617B1 (en) | 1998-09-24 | 2003-08-12 | International Business Machines Corporation | Optimized technique for prefetching LOB table space pages |
| US6470359B1 (en) | 1998-09-24 | 2002-10-22 | International Business Machines Corporation | Fast technique for recovering an index on an auxiliary table |
| US6343293B1 (en) | 1998-09-24 | 2002-01-29 | International Business Machines Corporation | Storing the uncompressed data length in a LOB map to speed substring access within a LOB value |
| US6343286B1 (en) | 1998-09-24 | 2002-01-29 | International Business Machines Corporation | Efficient technique to defer large object access with intermediate results |
| US6393498B1 (en) * | 1999-03-02 | 2002-05-21 | Mentor Arc Inc. | System for reducing processor workloads with memory remapping techniques |
| US6449697B1 (en) * | 1999-04-23 | 2002-09-10 | International Business Machines Corporation | Prestaging data into cache in preparation for data transfer operations |
| US7389305B1 (en) * | 1999-06-01 | 2008-06-17 | Fair Isaac Corporation | System and method for managing a database |
| US7035880B1 (en) | 1999-07-14 | 2006-04-25 | Commvault Systems, Inc. | Modular backup and retrieval system used in conjunction with a storage area network |
| US7395282B1 (en) | 1999-07-15 | 2008-07-01 | Commvault Systems, Inc. | Hierarchical backup and retrieval system |
| US7389311B1 (en) | 1999-07-15 | 2008-06-17 | Commvault Systems, Inc. | Modular backup and retrieval system |
| US6735765B1 (en) * | 1999-12-07 | 2004-05-11 | Storage Technology Corporation | Sharing data between operating systems |
| US6839827B1 (en) * | 2000-01-18 | 2005-01-04 | International Business Machines Corporation | Method, system, program, and data structures for mapping logical blocks to physical blocks |
| US7155481B2 (en) | 2000-01-31 | 2006-12-26 | Commvault Systems, Inc. | Email attachment management in a computer system |
| US7003641B2 (en) | 2000-01-31 | 2006-02-21 | Commvault Systems, Inc. | Logical view with granular access to exchange data managed by a modular data and storage management system |
| US6658436B2 (en) | 2000-01-31 | 2003-12-02 | Commvault Systems, Inc. | Logical view and access to data managed by a modular data and storage management system |
| EP1178407A3 (en) * | 2000-06-02 | 2007-12-12 | Compaq Computer Corporation | Architecture for parallel distributed table driven I/O mapping |
| DE10058391C2 (en) * | 2000-11-24 | 2003-06-18 | Siemens Ag | Object processing device |
| US20020184337A1 (en) * | 2001-05-21 | 2002-12-05 | Anders Hyldahl | Method and computer system for constructing references to virtual data |
| KR100397702B1 (en) * | 2001-08-29 | 2003-09-13 | 주식회사 대우일렉트로닉스 | Holographic digital data storage apparatus and data input/output method thereof |
| ATE321301T1 (en) * | 2001-10-24 | 2006-04-15 | Koninkl Philips Electronics Nv | SAFETY DEVICE FOR MASS STORAGE |
| JP4130076B2 (en) * | 2001-12-21 | 2008-08-06 | 富士通株式会社 | Database management program and recording medium |
| US7159085B2 (en) * | 2002-04-19 | 2007-01-02 | Emc Corporation | Method of allowing point-in-time view of data on a disk |
| US7249118B2 (en) * | 2002-05-17 | 2007-07-24 | Aleri, Inc. | Database system and methods |
| US6988165B2 (en) * | 2002-05-20 | 2006-01-17 | Pervasive Software, Inc. | System and method for intelligent write management of disk pages in cache checkpoint operations |
| US7647354B2 (en) * | 2002-05-24 | 2010-01-12 | Oracle International Corporation | High-performance change capture for data warehousing |
| US8370542B2 (en) | 2002-09-16 | 2013-02-05 | Commvault Systems, Inc. | Combined stream auxiliary copy system and method |
| US7254668B1 (en) | 2002-10-28 | 2007-08-07 | Sandisk Corporation | Method and apparatus for grouping pages within a block |
| US7530067B2 (en) * | 2003-05-12 | 2009-05-05 | International Business Machines Corporation | Filtering processor requests based on identifiers |
| JP4086718B2 (en) * | 2003-06-10 | 2008-05-14 | キヤノン株式会社 | Information processing method and information processing apparatus |
| US7454569B2 (en) | 2003-06-25 | 2008-11-18 | Commvault Systems, Inc. | Hierarchical system and method for performing storage operations in a computer network |
| US7873684B2 (en) | 2003-08-14 | 2011-01-18 | Oracle International Corporation | Automatic and dynamic provisioning of databases |
| US7188228B1 (en) * | 2003-10-01 | 2007-03-06 | Sandisk Corporation | Hybrid mapping implementation within a non-volatile memory system |
| US7076632B2 (en) * | 2003-10-16 | 2006-07-11 | International Business Machines Corporation | Fast paging of a large memory block |
| WO2005050381A2 (en) | 2003-11-13 | 2005-06-02 | Commvault Systems, Inc. | Systems and methods for performing storage operations using network attached storage |
| WO2005065084A2 (en) | 2003-11-13 | 2005-07-21 | Commvault Systems, Inc. | System and method for providing encryption in pipelined storage operations in a storage network |
| US7440982B2 (en) | 2003-11-13 | 2008-10-21 | Commvault Systems, Inc. | System and method for stored data archive verification |
| WO2005048085A2 (en) | 2003-11-13 | 2005-05-26 | Commvault Systems, Inc. | System and method for performing an image level snapshot and for restoring partial volume data |
| US7613748B2 (en) | 2003-11-13 | 2009-11-03 | Commvault Systems, Inc. | Stored data reverification management system and method |
| US7155586B1 (en) | 2003-12-30 | 2006-12-26 | Emc Corporation | Method of allowing point-in-time view of data on a disk using a map on cache disk |
| US7478211B2 (en) * | 2004-01-09 | 2009-01-13 | International Business Machines Corporation | Maintaining consistency for remote copy using virtualization |
| US20050154786A1 (en) * | 2004-01-09 | 2005-07-14 | International Business Machines Corporation | Ordering updates in remote copying of data |
| US8311974B2 (en) | 2004-02-20 | 2012-11-13 | Oracle International Corporation | Modularized extraction, transformation, and loading for a database |
| JP4390618B2 (en) * | 2004-04-28 | 2009-12-24 | 富士通株式会社 | Database reorganization program, database reorganization method, and database reorganization apparatus |
| US8554806B2 (en) | 2004-05-14 | 2013-10-08 | Oracle International Corporation | Cross platform transportable tablespaces |
| US7318168B2 (en) * | 2004-07-28 | 2008-01-08 | International Business Machines Corporation | Bit map write logging with write order preservation in support asynchronous update of secondary storage |
| US7657581B2 (en) * | 2004-07-29 | 2010-02-02 | Archivas, Inc. | Metadata management for fixed content distributed data storage |
| TWI254856B (en) * | 2004-10-29 | 2006-05-11 | Via Tech Inc | Memory interleave method |
| WO2006052872A2 (en) | 2004-11-05 | 2006-05-18 | Commvault Systems, Inc. | System and method to support single instance storage operations |
| US7747835B1 (en) * | 2005-06-10 | 2010-06-29 | American Megatrends, Inc. | Method, system, and apparatus for expanding storage capacity in a data storage system |
| US8078636B2 (en) * | 2005-08-24 | 2011-12-13 | Temporal Dynamics, Inc. | Database heap management system with variable page size and fixed instruction set address resolution |
| US7620710B2 (en) | 2005-12-19 | 2009-11-17 | Commvault Systems, Inc. | System and method for performing multi-path storage operations |
| US7617253B2 (en) * | 2005-12-19 | 2009-11-10 | Commvault Systems, Inc. | Destination systems and methods for performing data replication |
| US7636743B2 (en) | 2005-12-19 | 2009-12-22 | Commvault Systems, Inc. | Pathname translation in a data replication system |
| US7617262B2 (en) | 2005-12-19 | 2009-11-10 | Commvault Systems, Inc. | Systems and methods for monitoring application data in a data replication system |
| US8655850B2 (en) | 2005-12-19 | 2014-02-18 | Commvault Systems, Inc. | Systems and methods for resynchronizing information |
| EP1974296B8 (en) | 2005-12-19 | 2016-09-21 | Commvault Systems, Inc. | Systems and methods for performing data replication |
| US7962709B2 (en) | 2005-12-19 | 2011-06-14 | Commvault Systems, Inc. | Network redirector systems and methods for performing data replication |
| US7543125B2 (en) | 2005-12-19 | 2009-06-02 | Commvault Systems, Inc. | System and method for performing time-flexible calendric storage operations |
| US7651593B2 (en) | 2005-12-19 | 2010-01-26 | Commvault Systems, Inc. | Systems and methods for performing data replication |
| US7606844B2 (en) | 2005-12-19 | 2009-10-20 | Commvault Systems, Inc. | System and method for performing replication copy storage operations |
| US7430553B2 (en) * | 2005-12-30 | 2008-09-30 | Microsoft Corporation | Managing states with delta pager |
| US8726242B2 (en) * | 2006-07-27 | 2014-05-13 | Commvault Systems, Inc. | Systems and methods for continuous data replication |
| US8909599B2 (en) | 2006-11-16 | 2014-12-09 | Oracle International Corporation | Efficient migration of binary XML across databases |
| US8312323B2 (en) | 2006-12-22 | 2012-11-13 | Commvault Systems, Inc. | Systems and methods for remote monitoring in a computer network and reporting a failed migration operation without accessing the data being moved |
| WO2008085989A1 (en) | 2007-01-10 | 2008-07-17 | Richard Garfinkle | A software method for data storage and retrieval |
| US8290808B2 (en) | 2007-03-09 | 2012-10-16 | Commvault Systems, Inc. | System and method for automating customer-validated statement of work for a data storage environment |
| US8006061B1 (en) | 2007-04-13 | 2011-08-23 | American Megatrends, Inc. | Data migration between multiple tiers in a storage system using pivot tables |
| US8370597B1 (en) | 2007-04-13 | 2013-02-05 | American Megatrends, Inc. | Data migration between multiple tiers in a storage system using age and frequency statistics |
| US9449047B2 (en) | 2007-06-19 | 2016-09-20 | Sybase, Inc. | Dynamic modification of schemas in streaming databases |
| US8745012B2 (en) | 2007-08-10 | 2014-06-03 | Sybase, Inc. | Log-structured store for streaming data |
| US20090150355A1 (en) * | 2007-11-28 | 2009-06-11 | Norton Garfinkle | Software method for data storage and retrieval |
| US9495382B2 (en) | 2008-12-10 | 2016-11-15 | Commvault Systems, Inc. | Systems and methods for performing discrete data replication |
| US8204859B2 (en) | 2008-12-10 | 2012-06-19 | Commvault Systems, Inc. | Systems and methods for managing replicated database data |
| US8510271B1 (en) * | 2009-03-30 | 2013-08-13 | Symantec Corporation | Application and file system data virtualization from image backup |
| US8688948B2 (en) * | 2009-10-15 | 2014-04-01 | Freescale Semiconductor, Inc. | Flexible memory controller for autonomous mapping of memory |
| US8504517B2 (en) | 2010-03-29 | 2013-08-06 | Commvault Systems, Inc. | Systems and methods for selective data replication |
| US8352422B2 (en) | 2010-03-30 | 2013-01-08 | Commvault Systems, Inc. | Data restore systems and methods in a replication environment |
| US8504515B2 (en) | 2010-03-30 | 2013-08-06 | Commvault Systems, Inc. | Stubbing systems and methods in a data replication environment |
| US8725698B2 (en) | 2010-03-30 | 2014-05-13 | Commvault Systems, Inc. | Stub file prioritization in a data replication system |
| US9135031B1 (en) * | 2010-04-28 | 2015-09-15 | Netapp, Inc. | System and method for determining storage resources of a virtual machine in a virtual server environment |
| US8589347B2 (en) | 2010-05-28 | 2013-11-19 | Commvault Systems, Inc. | Systems and methods for performing data replication |
| US8812445B2 (en) * | 2010-09-24 | 2014-08-19 | Hitachi Data Systems Corporation | System and method for managing scalability in a distributed database |
| US8452900B2 (en) * | 2010-12-30 | 2013-05-28 | Emc Corporation | Dynamic compression of an I/O data block |
| US9021198B1 (en) | 2011-01-20 | 2015-04-28 | Commvault Systems, Inc. | System and method for sharing SAN storage |
| US8972696B2 (en) | 2011-03-07 | 2015-03-03 | Microsoft Technology Licensing, Llc | Pagefile reservations |
| US8832411B2 (en) | 2011-12-14 | 2014-09-09 | Microsoft Corporation | Working set swapping using a sequentially ordered swap file |
| US9298715B2 (en) | 2012-03-07 | 2016-03-29 | Commvault Systems, Inc. | Data storage system utilizing proxy device for storage operations |
| US9471578B2 (en) | 2012-03-07 | 2016-10-18 | Commvault Systems, Inc. | Data storage system utilizing proxy device for storage operations |
| US9628438B2 (en) * | 2012-04-06 | 2017-04-18 | Exablox | Consistent ring namespaces facilitating data storage and organization in network infrastructures |
| US9342537B2 (en) | 2012-04-23 | 2016-05-17 | Commvault Systems, Inc. | Integrated snapshot interface for a data storage system |
| US9886346B2 (en) | 2013-01-11 | 2018-02-06 | Commvault Systems, Inc. | Single snapshot for multiple agents |
| US9262435B2 (en) | 2013-01-11 | 2016-02-16 | Commvault Systems, Inc. | Location-based data synchronization management |
| US9552382B2 (en) | 2013-04-23 | 2017-01-24 | Exablox Corporation | Reference counter integrity checking |
| WO2014201270A1 (en) | 2013-06-12 | 2014-12-18 | Exablox Corporation | Hybrid garbage collection |
| US10102148B2 (en) | 2013-06-13 | 2018-10-16 | Microsoft Technology Licensing, Llc | Page-based compressed storage management |
| WO2014205286A1 (en) | 2013-06-19 | 2014-12-24 | Exablox Corporation | Data scrubbing in cluster-based storage systems |
| US9934242B2 (en) | 2013-07-10 | 2018-04-03 | Exablox Corporation | Replication of data between mirrored data sites |
| US10248556B2 (en) | 2013-10-16 | 2019-04-02 | Exablox Corporation | Forward-only paged data storage management where virtual cursor moves in only one direction from header of a session to data field of the session |
| US9985829B2 (en) | 2013-12-12 | 2018-05-29 | Exablox Corporation | Management and provisioning of cloud connected devices |
| US9632874B2 (en) | 2014-01-24 | 2017-04-25 | Commvault Systems, Inc. | Database application backup in single snapshot for multiple applications |
| US9639426B2 (en) | 2014-01-24 | 2017-05-02 | Commvault Systems, Inc. | Single snapshot for multiple applications |
| US9753812B2 (en) | 2014-01-24 | 2017-09-05 | Commvault Systems, Inc. | Generating mapping information for single snapshot for multiple applications |
| US9495251B2 (en) | 2014-01-24 | 2016-11-15 | Commvault Systems, Inc. | Snapshot readiness checking and reporting |
| US9774582B2 (en) | 2014-02-03 | 2017-09-26 | Exablox Corporation | Private cloud connected device cluster architecture |
| US9830324B2 (en) | 2014-02-04 | 2017-11-28 | Exablox Corporation | Content based organization of file systems |
| US9684625B2 (en) | 2014-03-21 | 2017-06-20 | Microsoft Technology Licensing, Llc | Asynchronously prefetching sharable memory pages |
| US10042716B2 (en) | 2014-09-03 | 2018-08-07 | Commvault Systems, Inc. | Consolidated processing of storage-array commands using a forwarder media agent in conjunction with a snapshot-control media agent |
| US9774672B2 (en) | 2014-09-03 | 2017-09-26 | Commvault Systems, Inc. | Consolidated processing of storage-array commands by a snapshot-control media agent |
| US9448731B2 (en) | 2014-11-14 | 2016-09-20 | Commvault Systems, Inc. | Unified snapshot storage management |
| US9648105B2 (en) | 2014-11-14 | 2017-05-09 | Commvault Systems, Inc. | Unified snapshot storage management, using an enhanced storage manager and enhanced media agents |
| US9904481B2 (en) | 2015-01-23 | 2018-02-27 | Commvault Systems, Inc. | Scalable auxiliary copy processing in a storage management system using media agent resources |
| US9898213B2 (en) | 2015-01-23 | 2018-02-20 | Commvault Systems, Inc. | Scalable auxiliary copy processing using media agent resources |
| US9632924B2 (en) | 2015-03-02 | 2017-04-25 | Microsoft Technology Licensing, Llc | Using memory compression to reduce memory commit charge |
| US10037270B2 (en) | 2015-04-14 | 2018-07-31 | Microsoft Technology Licensing, Llc | Reducing memory commit charge when compressing memory |
| EP3093773B1 (en) * | 2015-05-13 | 2019-07-10 | Huawei Technologies Co., Ltd. | System and method for creating selective snapshots of a database |
| CN106326144B (en) * | 2015-06-24 | 2019-08-06 | 龙芯中科技术有限公司 | Method for reading data and device based on big page mapping |
| US20170060924A1 (en) | 2015-08-26 | 2017-03-02 | Exablox Corporation | B-Tree Based Data Model for File Systems |
| US10503753B2 (en) | 2016-03-10 | 2019-12-10 | Commvault Systems, Inc. | Snapshot replication operations based on incremental block change tracking |
| US9846553B2 (en) | 2016-05-04 | 2017-12-19 | Exablox Corporation | Organization and management of key-value stores |
| US11249968B2 (en) * | 2016-05-09 | 2022-02-15 | Sap Se | Large object containers with size criteria for storing mid-sized large objects |
| US11010261B2 (en) | 2017-03-31 | 2021-05-18 | Commvault Systems, Inc. | Dynamically allocating streams during restoration of data |
| US10732885B2 (en) | 2018-02-14 | 2020-08-04 | Commvault Systems, Inc. | Block-level live browsing and private writable snapshots using an ISCSI server |
| KR102583810B1 (en) * | 2018-05-15 | 2023-10-05 | 에스케이하이닉스 주식회사 | Memory system and operation method thereof |
| US11042318B2 (en) | 2019-07-29 | 2021-06-22 | Commvault Systems, Inc. | Block-level data replication |
| US11321354B2 (en) * | 2019-10-01 | 2022-05-03 | Huawei Technologies Co., Ltd. | System, computing node and method for processing write requests |
| US11809285B2 (en) | 2022-02-09 | 2023-11-07 | Commvault Systems, Inc. | Protecting a management database of a data storage management system to meet a recovery point objective (RPO) |
Family Cites Families (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3911401A (en) * | 1973-06-04 | 1975-10-07 | Ibm | Hierarchial memory/storage system for an electronic computer |
| US4136385A (en) * | 1977-03-24 | 1979-01-23 | International Business Machines Corporation | Synonym control means for multiple virtual storage systems |
| US4285040A (en) * | 1977-11-04 | 1981-08-18 | Sperry Corporation | Dual mode virtual-to-real address translation mechanism |
| US4315310A (en) * | 1979-09-28 | 1982-02-09 | Intel Corporation | Input/output data processing system |
| US4481573A (en) * | 1980-11-17 | 1984-11-06 | Hitachi, Ltd. | Shared virtual address translation unit for a multiprocessor system |
| AU556499B2 (en) * | 1981-05-22 | 1986-11-06 | Data General Corporation | Data processing system |
| US4525778A (en) * | 1982-05-25 | 1985-06-25 | Massachusetts Computer Corporation | Computer memory control |
| US4577274A (en) * | 1983-07-11 | 1986-03-18 | At&T Bell Laboratories | Demand paging scheme for a multi-ATB shared memory processing system |
| US4686620A (en) * | 1984-07-26 | 1987-08-11 | American Telephone And Telegraph Company, At&T Bell Laboratories | Database backup method |
| US4718008A (en) * | 1986-01-16 | 1988-01-05 | International Business Machines Corporation | Method to control paging subsystem processing in a virtual memory data processing system during execution of critical code sections |
| US4742447A (en) * | 1986-01-16 | 1988-05-03 | International Business Machines Corporation | Method to control I/O accesses in a multi-tasking virtual memory virtual machine type data processing system |
| US5043871A (en) * | 1986-03-26 | 1991-08-27 | Hitachi, Ltd. | Method and apparatus for database update/recovery |
| US4758946A (en) * | 1986-04-09 | 1988-07-19 | Elxsi | Page mapping system |
| US4819156A (en) * | 1986-06-13 | 1989-04-04 | International Business Machines Corporation | Database index journaling for enhanced recovery |
| US4843542A (en) * | 1986-11-12 | 1989-06-27 | Xerox Corporation | Virtual memory cache for use in multi-processing systems |
| US4843541A (en) * | 1987-07-29 | 1989-06-27 | International Business Machines Corporation | Logical resource partitioning of a data processing system |
| US5220657A (en) * | 1987-12-02 | 1993-06-15 | Xerox Corporation | Updating local copy of shared data in a collaborative system |
| US5008811A (en) * | 1988-02-10 | 1991-04-16 | International Business Machines Corp. | Control mechanism for zero-origin data spaces |
| US5021946A (en) * | 1988-06-17 | 1991-06-04 | Modular Computer Systems, Inc. | Mostly contiguous file allocation technique involving file extension |
| US4961134A (en) * | 1988-07-15 | 1990-10-02 | International Business Machines Corporation | Method for minimizing locking and reading in a segmented storage space |
| US5095420A (en) * | 1988-11-21 | 1992-03-10 | International Business Machines | Method and system for performing virtual address range mapping in a virtual storage data processing system |
| US5058003A (en) * | 1988-12-15 | 1991-10-15 | International Business Machines Corporation | Virtual storage dynamic address translation mechanism for multiple-sized pages |
| US5125086A (en) * | 1989-06-29 | 1992-06-23 | Digital Equipment Corporation | Virtual memory paging apparatus with variable size in-page clusters |
| US5075848A (en) * | 1989-12-22 | 1991-12-24 | Intel Corporation | Object lifetime control in an object-oriented memory protection mechanism |
| JPH04506720A (en) * | 1990-03-23 | 1992-11-19 | イーストマン・コダック・カンパニー | Virtual memory management and allocation device for digital data processing systems |
| US5388196A (en) * | 1990-09-07 | 1995-02-07 | Xerox Corporation | Hierarchical shared books with database |
| US5230069A (en) * | 1990-10-02 | 1993-07-20 | International Business Machines Corporation | Apparatus and method for providing private and shared access to host address and data spaces by guest programs in a virtual machine computer system |
| US5317731A (en) * | 1991-02-25 | 1994-05-31 | International Business Machines Corporation | Intelligent page store for concurrent and consistent access to a database by a transaction processor and a query processor |
| WO1992015933A1 (en) * | 1991-03-05 | 1992-09-17 | Zitel Corporation | Cache memory system and method of operating the cache memory system |
| US5269019A (en) * | 1991-04-08 | 1993-12-07 | Storage Technology Corporation | Non-volatile memory storage and bilevel index structure for fast retrieval of modified records of a disk track |
| US5410700A (en) * | 1991-09-04 | 1995-04-25 | International Business Machines Corporation | Computer system which supports asynchronous commitment of data |
| US5428759A (en) * | 1992-03-30 | 1995-06-27 | Unisys Corporation | Associative memory system having segment and page descriptor content-addressable memories |
| US5392415A (en) * | 1992-12-15 | 1995-02-21 | International Business Machines Corporation | System for grouping non-contiguous pages belonging to a storage object for page out |
| US5473777A (en) * | 1993-07-19 | 1995-12-05 | Moeller; Christopher P. | Wrapper for enabling an object otented application to maintain virtual memory using procedural function calls |
-
1991
- 1991-11-12 CA CA002055295A patent/CA2055295C/en not_active Expired - Fee Related
- 1991-11-12 CA CA002285089A patent/CA2285089C/en not_active Expired - Fee Related
- 1991-11-12 CA CA002285096A patent/CA2285096C/en not_active Expired - Fee Related
-
1992
- 1992-11-05 JP JP4295727A patent/JP2625619B2/en not_active Expired - Lifetime
- 1992-11-05 EP EP92310160A patent/EP0542483A1/en not_active Withdrawn
-
1995
- 1995-05-17 US US08/443,371 patent/US5652873A/en not_active Expired - Fee Related
- 1995-05-17 US US08/443,397 patent/US5594881A/en not_active Expired - Lifetime
- 1995-05-17 US US08/442,770 patent/US5561778A/en not_active Expired - Fee Related
- 1995-06-06 US US08/468,095 patent/US5664160A/en not_active Expired - Fee Related
- 1995-06-06 US US08/468,771 patent/US5687343A/en not_active Expired - Fee Related
- 1995-06-06 US US08/468,987 patent/US5579499A/en not_active Expired - Fee Related
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6647398B1 (en) | 1999-04-01 | 2003-11-11 | Beacon Information Technology Inc. | Data managing method and apparatus, and recording medium |
Also Published As
| Publication number | Publication date |
|---|---|
| US5561778A (en) | 1996-10-01 |
| CA2055295C (en) | 2000-05-23 |
| EP0542483A1 (en) | 1993-05-19 |
| US5594881A (en) | 1997-01-14 |
| US5687343A (en) | 1997-11-11 |
| CA2285096A1 (en) | 1993-05-13 |
| US5652873A (en) | 1997-07-29 |
| US5579499A (en) | 1996-11-26 |
| US5664160A (en) | 1997-09-02 |
| CA2055295A1 (en) | 1993-05-13 |
| JP2625619B2 (en) | 1997-07-02 |
| CA2285089C (en) | 2000-05-09 |
| JPH05225065A (en) | 1993-09-03 |
| CA2285089A1 (en) | 1993-05-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2285096C (en) | Logical mapping of data objects using data spaces | |
| EP0370178B1 (en) | Method and system for mapping data in a virtual storage data processing system | |
| US4742447A (en) | Method to control I/O accesses in a multi-tasking virtual memory virtual machine type data processing system | |
| EP0112442B1 (en) | Data storage hierarchy and its use for data storage space management | |
| US5454103A (en) | Method and apparatus for file storage allocation for secondary storage using large and small file blocks | |
| US6587915B1 (en) | Flash memory having data blocks, spare blocks, a map block and a header block and a method for controlling the same | |
| EP1247184B1 (en) | Method and apparatus for pointer relocation optimization for virtual memory mapping and transaction management in a database system | |
| US6112285A (en) | Method, system and computer program product for virtual memory support for managing translation look aside buffers with multiple page size support | |
| US5408654A (en) | Method to reorganize an index file without sorting by changing the physical order of pages to match the logical order determined from the index structure | |
| US5129088A (en) | Data processing method to create virtual disks from non-contiguous groups of logically contiguous addressable blocks of direct access storage device | |
| US5765201A (en) | Changing page size in storage media of computer system | |
| EP0408070B1 (en) | Method for allocating real pages to virtual pages having different page sizes therefrom | |
| GB2377284A (en) | Storage system with a point in time copy function | |
| US20080162863A1 (en) | Bucket based memory allocation | |
| US5963982A (en) | Defragmentation of stored data without pointer indirection | |
| US7058785B1 (en) | Addressing objects in a large persistent storage address space | |
| Li et al. | {ctFS}: Replacing File Indexing with Hardware Memory Translation through Contiguous File Allocation for Persistent Memory | |
| KR19990013934A (en) | Mass memory allocation method and device | |
| US5423013A (en) | System for addressing a very large memory with real or virtual addresses using address mode registers | |
| JPH05274152A (en) | Object management system | |
| EP0370175A2 (en) | Method and system for storing data in and retrieving data | |
| US20040039875A1 (en) | Disk array device and virtual volume management method in disk array device | |
| US5966735A (en) | Array index chaining for tree structure save and restore in a process swapping system | |
| EP0319147B1 (en) | Method for storing pre-organised groups of related information files in a data processing system | |
| EP0250876A2 (en) | Apparatus and method for page replacement in a data-processing system having a virtual memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |