|Publication number||US20080071733 A1|
|Application number||US 11/905,585|
|Publication date||Mar 20, 2008|
|Filing date||Oct 2, 2007|
|Priority date||Mar 6, 2002|
|Also published as||US7287033, US20030204515|
|Publication number||11905585, 905585, US 2008/0071733 A1, US 2008/071733 A1, US 20080071733 A1, US 20080071733A1, US 2008071733 A1, US 2008071733A1, US-A1-20080071733, US-A1-2008071733, US2008/0071733A1, US2008/071733A1, US20080071733 A1, US20080071733A1, US2008071733 A1, US2008071733A1|
|Inventors||Moshe Shadmon, Neal Sample|
|Original Assignee||Ori Software Development Ltd.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Referenced by (12), Classifications (13)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This is a divisional of application Ser. No. 10/379,002, filed Mar. 5, 2003 which claims the benefit of U.S. Provisional Application No. 60/361,700, filed Mar. 6, 2002. The entire disclosure of the prior application is hereby incorporated by reference herein in its entirety.
The present invention is in the general field of traversals over hierarchical data, indexing and accessing semistructured data and hierarchical data.
A discussion on indexing can be found in The Art of Computer Programming, volume 3/Sorting and Searching, by Donald Knuth (©1973, Addison-Wesley). More specifically, a discussion on B-trees can be found on pages 473-479, a trie is discussed on pages 481-490, and a discussion on Patricia structure can be found on pages 490-499.
The invention provides for in a storage medium used by a database file management system executed on data processing system, a data structure comprising: an index, partitioned into blocks, over keys representing hierarchical data; leaf blocks of said index are associated with data records; said hierarchical data is represented by keys of type K(i) as a parent and for each parent 0 or more children of the type K(i)+J(1) . . . K(i)+J(n); in order to satisfy a query for parents data, it is possible to traverse from a link of a parent K(i) to a link of the sequentially next parent K(i+1), skipping over links to the children K(i)+J(1) . . . K(i)+J(n); to satisfy a query for a parent and his children, the index further supports a traversal from K(i) to the children J(1) . . . J(n) of said K(i); said index maintains the key order; said index constitutes an essentially balanced structure of blocks.
The invention further provides in a storage medium used by a database file management system executed on data processing system, that includes: a tree based index, partitioned into blocks, over keys representing hierarchical data;
Still further, the invention provides for a method for encoding hierarchical data using an index partitioned into blocks, over the keys representing the data;
Yet further, the invention provides for a method for encoding semi-structured data and/or hierarchical data, comprising a layered index partitioned into blocks over keys representing the said data;
For a better understanding, the invention will now be provided, by way of example only, with reference to the accompanying drawings, in which:
One of the most popular index schemes is B-tree. Most table-based DBMSs use Btree as one of the major index schemes. However, semi-structured data (such as XML data) is more complicated to index and search than tables based data. A query over semi-structured data (as well as queries over other types of data) is usually evaluated in terms of efficiency. As it was explained in the US Patent Publication US20020120598 A1 published Aug. 29, 2002 and entitled “Encoding Semi-Structured Data For Efficient Search and Browse” (herein the Semi-Structured Publication), whose contents in its entirety is incorporated herein by reference. The first paragraph of the summary of the Semi-Structured Publication (Col. 5 ) defines efficiency as:
“efficient access means the ability to locate relevant data in a short amount of time or with only a small number (e.g., several) disk accesses, even though the data items stored may number in the millions or more, or even though the total size of the data items is such that they cannot all fit in the memory available to the search process. Efficient access refers to the overall or typical behavior of the system. In other words, while individual accesses may sometimes be inefficient, the access behavior observed over time is efficient.”
As was further shown in the Semi-Structured Publication, a semi-structured data (such as the invoice data of FIG. 1 of the Semi-Structured Publication) can be described hierarchically using designators and the subordination relationship (such as the structure of FIG. 7 of the Semi-Structured Publication).
Thus, it is stated on page 11  of the Semi-Structured Publication:
“The usage of designated data and subordination relationships for semi-structured data can be exemplified by a) using a set of rules and/or knowledge and/or formula to map the semi-structured data to designated data records where each such designated record can be a subordinated record; b) combining the designated records into strings that can represent structural and non-structural information that is either explicit or not explicit in the original data item; and c) creating a designated index in which part or all of each string is treated as a key, thereby enabling search over this set of designated records. In a preferred embodiment such index would be a layered index. The designated records being an example of data items.”
Note that an index such as B-tree organizes the keys by their lexicographical order. It does not consider hierarchical organization. For example, consider the keys “AAA” and “BBB”. If the key “AAA” has hierarchically related key “AAA111” (“AAA111” is prefixed by the key “AAA” and is considered a child of “AAA”, “AAA111” can also be described as subordinated to “AAA”), The B-tree index would organize the keys sequentially as—“AAA”, “AAA111”, “BBB”. A query for all the parent data would be performed by scanning the complete index due to the lack of differentiation between a parent and a child within the index. In this example, the traversal would include the child key “AAA111” hence there is no way for the traversal mechanism to know that the next key after the key “AAA” is a child.
An index such as a trie is more sensitive to the hierarchical nature of the indexed elements. A query for the parent data can use the trie nodes to traverse the trie only to the depth that addresses a parent skipping over sub-trees that address keys of children data. Indexing the above keys using a trie, if the trie nodes represent byte offsets, a trie-based index can conceptually (and practically) ignore sub-trees rooted from nodes with the value 3 (or higher), (assuming position 0 for the first byte of the key and that the key of the child starts as position 3).
Similar to a B-tree structure, also trie maintains the lexicographical order of the keys. However, a trie structure is not balanced and thus usually fails to efficiently support large and disk based data sets.
The semi-structured data does not have a strict schema and does not have to conform to a known format. This made the semi-structured data hard to maintain, search, and index. The need to handle semi-structured data was linked to SGML (Standard Generalized Markup Language) to form a way to describe the semi-structured data. SGML (and the industry specific subsets such as XML, HTML, CML etc.) describes the semi-structured data as a set of hierarchical parent child relationships whereas each element in the set is attached with a tag that encapsulates structural and description of the data element. This resulted in a hierarchical description of the semi-structured data whereas each data element in a data set has a parent except the first one.
Describing the data as sets of hierarchies does not solve the problem of efficient search. In order to allow efficient search, index structures are build to address the different data elements in the hierarchies. The most (or one of the most) commonly used index structure in today's DBMSs is a Btree index. The original incarnation of Btrees allowed for pointers to data at any level of the tree. This was very problematic for range queries because it meant that every level of the Btree, not just the leaves, had to be searched when looking for a range of data. This was a problem in terms of both search performance and search complexity.
To find a range of data items in a regular Btree requires that a large portion of the index (sometimes all of the index) be examined. For example, consider the following SQL-like query:
SELECT department WHERE company=‘rightorder’;
This query finds all departments that belong to the company “rightorder”. Assuming a standard Btree,
The original Btree was not always efficient in terms of storage because of the data pointers that appeared near the root, and searching the Btree was a non-trivial process. To alleviate these difficulties, a new index was proposed, the B+tree (pronounced “bee plus tree”). A B+tree is the same as a Btree, but there is the restriction that pointers to data can only appear in the leaves. This has two advantages. First, search in the index is straightforward since all data pointers are in the leaves. The second advantage is that potentially fewer blocks have to be examined during a range query.
It should be noted that a typical B+tree (as exemplified above) is built over uniform data. This is a simplifying assumption that makes it simple to maintain the B+tree, but has a significant impact on performance with certain queries. Since a typical B+tree treats all of the data indexed in a uniform manner, the data is not clustered based on its type; it is only ordered based on value. This is a feature that confers to queries of this type an advantage when run using the Layered Index instead of a B+tree. This advantage will be further shown below.
The Layered Index is described in U.S. Pat. No. 6,175,835 B1 whose content in its entirety is incorporated here by reference. With the Layered Index there is a further improvement beyond that of the B+tree. A Layered Index can maintain a Patricia-based leaf layer that is similar to the original Btree. An explanation of the Patricia structure can be found at Donald Knuth, The Art of Computer Programming, Vol. III, Sorting and Searching, Third Edition, pages 490-493, 497-499, 501-504 and the Patricia implementation is referenced on column 10 line 5 of the above U.S. Pat. No. 6,175,835 B1 patent
The general search approach of the Layered Index (by a particular key) can be described as a search using an index arranged in blocks, each block of said index contains at least compressed information identifying the keys accessible from such block and links such that data records can be found according to their key. The process includes reading a series of blocks composed of a root block and additional 0 or more blocks and 0 or more data records addressed by a leaf block from the said series of blocks. The link to each additional block in the series or to the data record is found by applying a formula based on the compressed information in each previously read block and the key of the searched data record. Since the information in each block can be sparse, a wrong data record might be accessed. Thus, when a data record is read, it is possible to decide that the searched data record does not exist or to initiate a new search by adding the information found in the accessed data record to the search formula. This last step of initiating a new search is repeated until the data record is found or it is possible to decide that the searched data record does not exist. A general explanation of the Layered Index is provided in the above U.S. Pat. No. 6,175,835 B1 e.g. from line 23 of column 8 to line 57 of column 13. With the Layered Index, pointers to data elements appear throughout the blocks in the leaf layer. This would be a disadvantage, in terms of search efficiency, except that getting to a block in the Btree-like leaf layer is done in a B+tree fashion thus for a Layered Index the pointers to data appear at the leaf layer.
The horizontal layers of the Layered Index capture all the advantages of the B+tree in terms of navigation and space efficiency: pointers to data appear only in the leaf layer of the Layered Index. However, the leaf layer of the Layered Index is itself a tree.
This hybrid approach leaves the Layered Index with the best of both worlds. As explained and exemplified in the U.S. Pat. No. 6,175,835 B1 patent (for example from line 49 of Col. 8 to line 64 of the same Col.), searching for keys is balanced and simple. Once at the leaf layer of the Layered Index, the Btree properties that are performance killers in a non-layered index prove to be an advantage for range queries.
In a particular embodiment of the Layered Index, the leaf layer of the index maintains a tree structure over the keys of data records. Said leaf layer is based on a Patricia trie. For each parent element, links to its children elements appear in a sub-tree in the leaf layer, the root of the sub-tree includes a link to the parent. The search by a particular key in the index is performed in a balanced manner by traversing to the relevant block in the leaf layer using the non-leafs layers (rather than the trie of the leaf layer).
As explained in U.S. Pat. No. 6,175,835 B1, for example, from line 4 of column 10 to line 37 of the same column, some search traversals can result with an error, which is followed by a process to fix the search error. However, these errors are relatively rare and do not affect the overall performance and efficiency of the search and update processes. Because of a mistake, a particular search (or update) can be more expensive than a mistake free search (or update), however these mistakes are distributed among the search (and update) keys regardless of their hierarchy having a similar effect on the average efficiency of the search (and update) of keys in each level of hierarchy. Thus, with the Layered Index (as will be demonstrated in the examples to follow), the index maintains both—the hierarchical nature of the information (as children are subordinated to parents) in the trie of the leaf layer with the ability to access any data element (a parent or a child) in the same level of efficiency (i.e. with the same number of I/Os).
Note that the Layered Index is a self-organizing index. This means that related elements tend to be stored in the same region. Similar objects tend to be clustered together in the leaf layer of the index. What is needed is a process that allows the search for related items efficiently. This can be a significant advantage for range queries, without adversely affecting point (“rifle-shot”) queries. With the query from above:
SELECT department WHERE company=‘rightorder’;
This query finds all departments that belong to the company “rightorder”. For this query over a regular Btree,
The index of
Thus the Layered Index maintains a balanced structure of blocks, the organization of the leaf layer blocks and their internal representation in a particular implementation is such that if the key of record A is the prefix of the key of record B, the link to record A appears in the root of a sub-tree that includes a link to record B. In a different implementation the link to record A can be found from the root of a sub-tree that includes a link to record B.
The following is a more detailed description of a search process to retrieve XML and/or hierarchical data indexed by a Layered Index. Each layer is an index over the representative keys of the next lower level of the index; layer n is an index over layer n−1; the leaf layer (layer 0) is an index over the keys of the data records. In this example, each layer is based on a sparse trie. A definition of representative keys is available e.g. at the paragraph on lines 23-67 of Col. 8 of the U.S. Pat. No. 6,175,835 B1.
Assuming a hierarchy similar to that seen in
This data is semi-structured, as it's structure is not known in advance. Additional elements can appear in any place in the hierarchy. One way to describe a particular source data is by using XML notations to be:
<Companies> <Company> <Name>leftorder</Name> <Department> <Name>accounting</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> <Department> <Name>engineering</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> <Department> <Name>finance</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> </Company> <Company> <Name>order</Name> <Department> <Name>accounting</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> <Department> <Name>engineering</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> <Department> <Name>finance</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> </Company> <Company> <Name>rightorder</Name> <Department> <Name>accounting</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> <Department> <Name>engineering</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> <Department> <Name>finance</Name> <Employee> <Name>jane</Name> <Name>robert</Name> <Name>sally</Name> </Employee> </Department> </Company> </Companies>
This XML/hierarchical information can be represented with the following 39 strings (in sorted order, the number to the left of each string is not part of the string and is for convenience only):
1. leftorder 2. leftorder accounting 3. leftorder accounting jane 4. leftorder accounting robert 5. leftorder accounting sally 6. leftorder engineering 7. leftorder engineering jane 8. leftorder engineering robert 9. leftorder engineering sally 10. leftorder finance 11. leftorder finance jane 12. leftorder finance robert 13. leftorder finance sally 14. order 15. order accounting 16. order accounting jane 17. order accounting robert 18. order accounting sally 19. order engineering 20. order engineering jane 21. order engineering robert 22. order engineering sally 23. order finance 24. order finance jane 25. order finance robert 26. order finance sally 27. rightorder 28. rightorder accounting 29. rightorder accounting jane 30. rightorder accounting robert 31. rightorder accounting sally 32. rightorder engineering 33. rightorder engineering jane 34. rightorder engineering robert 35. rightorder engineering sally 36. rightorder finance 37. rightorder finance jane 38. rightorder finance robert 39. rightorder finance sally
Designators to the different record types are defined such that companies are designated with a string of bits equivalent to the character A, departments with the string of bits equivalent to the character B and employees with the string of bits equivalent to the character C. The hierarchical nature of the data and parent child relationships are treated as subordination of data records. For example, the record whose key is #2 “leftorder accounting” is considered subordinated to the record whose key is #1 “leftorder”. A description of designators and subordination of data records can be found, e.g. in U.S. Pat. No. 6,175,835 B1 from line 1 of Col. 14 to line 19 of Col. 15. An index over the designated strings is called a designated index.
The procedure to map the XML data to designated strings is exemplified throughout the Semi-Structured Publication and for example with reference to FIGS. 7 and 8 of the Semi-Structured Publication.
In order to simplify the example, it is assumed that the key including the designator in each level is 20 bytes long and that a null value is appended to the end of each string and forms part of the key.
For example, the key #4—“leftorder . . . accounting . . . robert” is designated to be “Aleftorder . . . Baccounting . . . Crobert”. The designator “A” is considered to be in position 0 of the key string; “B” and “C” are in positions 20 and 40 respectively. In addition, the last character of the key string (position 60) is Null. In order to search in the trie for key #4 (614 in
Although tries, such as the trie of
If more data is add to the trie of
“Aleftorder . . . Baccounting . . . Csandra” representing a new employee in the accounting department of leftorder would create a leaf node with a value 43 to differentiate between the new key and key #5 (615).
As explained in the U.S. Pat. No. 6,175,835 B1, in order to solve these problems, (to allow efficient and balanced access to data), the original trie is partitioned into blocks and a Layered Index is formed.
Using the Layered Index, the search path to record #4 (714) would start at the trie of the root block #1 (701), use the link labeled “l” (707) from the root node (706) to the (leftmost) node labeled 20 (708) and follow the far link labeled “B” (709) to reach Block #3 (703). Once block #3 is available, the search by the key #4 starting at the root node of block #3 (710) would end at the record whose key is #4 (714).
In the same manner, a search by the key #2 “Aleftorder . . . Baccounting . . . ” would follow the path 706, 707, 708, 709 (in block #1 (701)) to block #3 (by examining positions 1 and 20 in the key), and the path from node 710 to key #2 (731) (by examining positions 21 and 40 in the key).
And in a similar manner, the search by the key #1 “Aleftorder . . . ” would follow node 706 and link 707 to node 708 in block #1 (701). Since the node 708 has a value of 20 and a link with a null value is not available from this node (null is appended to the key #1 at position 20), the search process backtracks to the direct link 720 to read block #2 (702) and navigate to the key #1 (725) using nodes and links 721, 722, 723 and 724.
An important feature of the invention was demonstrated with reference to
There follows now a more detailed description of an efficient range search:
Thus, the index of
Keys representing parents, children and parent-child relationship of the XML/hierarchical data are such that the prefix of the key of the children records is identical to the key of the parent record (omitting the null value at the end of the key of the parent). Thus, for example, the prefix of the key of “Aleftorder . . . Baccounting . . . ” is the key of the parent record (Aleftorder . . . ”). Searching for the children of “Aleftorder . . . ” is possible by searching by the key “Aleftorder . . . ” to find the sub-tree that represents the records that include Aleftorder . . . ” at their key prefix. As explained above (and now with greater detail with reference to
The second example is a search procedure to retrieve all the companies (without their children). This query can be described by a pseudo-SQL:
SELECT company WHERE company=‘*’;
And would use the key “A” (being the designator of companies) as the search key. The search would use the direct link (720) from the root node of block #1 (706) to block #2 (702) hence the value of the root node of block #1 (701) relates to a position beyond the search key. Then, the search would follow the links “l” (722) “o” (727) “r” (728) to find the nodes (723, 729, 730) that are linked to the companies and read the data records #1, #14 and #47.
The search traversed the sub-tree rooted by node 721 (using the rule depth first post order) to find all the companies. If the search reach a node representing a position beyond the company key or a position representing the last bits of the key—such as the nodes with the value 20 at block #2 (723, 729, 730), the search is able to retrieve the searched data record (in this example by using the null link) and ignoring afterwards the sub-tree rooted from that node. This process yields the following traversal: 721, 722, 723 to collect the key #1 (Aleftorder), backtracking to the node 721 and using 727 and 729 to collect key #14 (Aorder), backtracking to node 721 and using 728 and 730 to collect key #27 (Arightorder).
The above examples and discussion demonstrated a tree based index, partitioned into blocks, over keys representing hierarchical data (the data set 1-39); keys of the dataset include keys of parent records (such as the key “Aleftorder . . . ”); one or more parent records have children such that a key of a child record is prefixed by the key of its parent (for example, departments are prefixed by the key of their company); The index includes a sub-index such that the sub-index is an index over the children keys of a parent (for example, the data indexed by the sub-tree rooted at node 710 index the children of “leftorder”). The index maintains a balanced structure of blocks as navigation to any of the leaf blocks (#2, #3, #4 and #5) by any particular indexed key is through the non-leaf layers (block #1 (701)).
Thus, it is possible to search in the index by a key of the parent. For example searching by the key “Aleftorder” to find the parent 725 (using the non-leaf layer). In addition, it is possible to continue the search for the children of “leftorder” by traversal using the link 726 to the sub-tree (rooted at node 710) to retrieve the children. Or it is possible to backtrack from node 723 to node 721 and then traverse to node 729 to retrieve the next parent #14 (rather than retrieving the children).
In this example, in order to physically cluster the links to the companies (and thus being able to retrieve the companies with minimal I/Os), the preferred split positions were those who disconnected a parent from his children rather than positions that disconnected a parent including his children. For example, the split link 726 disconnected the parent “leftorder” (725) from his children (such as “accounting” (731)).
However, since specific implementations maintain a link to the parent in the data record of the child, it is not needed to physically cluster the link of a parent with that of a child in the index. (A detailed description is available e.g. with reference to FIG. 13E in U.S. Pat. No. 6,175,835 B1. In the particular embodiment described there, every child data record (subordinated data record) maintains a link to its immediate parent).
Alternatively, if the split is between a parent and its children, the link to the parent can be maintained once in the disconnected block (for example, in the header of the block) hence all the data elements addressed by the disconnected block are children (at some level) of that parent. For example a link to the parent “leftorder” (725) can be maintained in the header of the block #3 (703) or attached to the root node 710. A search to a department (such as “accounting” (731)) is able to retrieve the parent (“leftorder” (725)) from the block that address the department (if such link is maintained in the block) even if a split process disconnected the part of the trie that addressed the company (node 723) from the sub-trie that address its children.
Thus, the invention in accordance with one aspect provides for a method for encoding XML and or semi-structured data and/or hierarchical data, such that: Providing an index (exemplified by the index of
The index allows to navigate from the link of K(i) to the link of the sequentially next K(i+1) skipping over the links to K(i)+J(1) . . . K(i)+J(n). This was demonstrated above by the traversal to the different companies #1, #14 and #27 of
In addition, the said index allows to navigate from the link K(i) to all of its children J(1) . . . J(n) by the order of the key. As exemplified above by navigating to the subordinated information of the company “leftorder” (725 of
In the particular example, the splits links (such as link 726) disconnected the block that maintained the link to the parent from the block that maintained the links to the children (for example, the parent key #1 (725) is addressed from the block #2 (702) whereas its children such as the key #2 (731) is addressed from block #3 (703)) allowing for the links to the parents to be clustered (meaning that the companies are addressed from small number of blocks and in this example, from a single block #2 (702)). Thus, for every 2 consecutive K keys (K(i),K(i+1)), whereas the first K(i) key has children such that the size in the index to represent K(i) and his children is equal or larger than a block, it is possible to partition the index such that K(i) and the next consecutive K(i+1) reside in the same block;
As exemplified, the index of
An alternative approach to the clustering in the Layered Index is the partitioned B-tree, referred to elsewhere as an Index-Organized Table (IOT). IOTs are used for example in Oracle 9i. Additional information on IOTs can be found in: Oracle 9i Index-Organized Tables Technical Whitepaper, September 2001, http://otn.oracle.com/products/oracle9i/pdf/iot twp.pdf. The IOT has performance characteristics similar to the Layered Index for the range query mentioned above.
Also, in the known per se IOT implementations, only one type of key can exist in each layer, just like a regular relational table. The Layered Index does not suffer this limitation due to the use of designators. As explained and referenced above, designators are strings of bits that are added to the keys. One of the benefits of using designators is that different types of keys of the same level in the hierarchy can be differentiated. For example, a “manager name” that would be added to department (as immediate child/subordinated to department), would be reflected as a new key in the sub-trie that maintains the department information. Using the designator M to identify managers and inserting a key #40 with the value “Aleftorder . . . Baccounting . . . Mbob . . . ” to the index of
By utilizing the index layering techniques, access to even IOT structures can be balanced. Before looking at a technique to balance access, the following reviews the imbalance issue with IOTs:
The IOT is a special form of Btree, where the partitioning into layers is based on subordinate fields within the data. The root layer of the IOT represents one portion of a data row, and each subsequent layer represents an addition portion of a data row. This means that particular data fields (a.k.a., “attributes,” “columns,” etc.) within an IOT will have very different access times. Those near the root of the IOT will be quick to access, while those near the leaves will take much more time.
The Btree does not provide balanced access, but a B+tree does. However, the B+tree is not good for the range query. Using the layering technique, it is possible to combine these two indexes to provide balanced access, and to simultaneously support efficient hierarchical queries.
Note that the preferred implementation of the Layered Index performs the same balancing act by building multiple indexes to provide balance. The vertical layer of the Layered Index is built from a compressed Patricia tree. The horizontal layer is built from a second Patricia tree. If the second Patricia is also imbalanced, another may be built to provide balanced access. This elegant recursion occurs repeatedly until there is a layer that contains a Patricia that fits in a single disk block. A description of this structure can be found in the above U.S. Pat. No. 6,175,835 B1 for example from line 22 of Col. 8 to line 32 of Col. 9.
Note that the key strings of the data represented with the hierarchy of
The following demonstrates a range query in the B+tree, the regular Btree, and the IOT. The query in pseudo-SQL is identical to that previously presented:
SELECT department WHERE company=‘rightorder’;
This query finds all departments that belong to the company ‘rightorder’.
Assuming the B+tree shown above,
There are six blocks examined. Two internal blocks to get to the right place in the leaf layer (130 and 131), then four blocks scanned in the leaf to get the appropriate data (132-135).
The B+tree is not capable of skipping over blocks with keys irrelevant to the query. For example, the last key examined in block 134 is “rightorder finance”. Block 135 has no department keys, however, the B+tree algorithm needs to read block 135 since it may include a department. For example, if “rightorder” had a “research” department it would reside in block 135.
Assuming a standard Btree,
Note that the Btree (as the B+tree) can't determine if a block is relevant for the search without examining the content of the block. For example, block 143 is not needed to satisfy the query, however, it is read since it can potentially include a department. Block 146 is not needed to satisfy the query but is read since it can potentially include a department (such as “rightorder research”).
Assuming an IOT Btree,
However, the IOT is limited in its applicability to efficiently support key lookups—as can be seen from
The second limitation of the IOT is not being able to support different types of information within a particular level of the hierarchy. For example, if it is needed to update the list of managers for each department, there is no mechanism in the IOT to include the managers in the third layer of the tree hence this layer is used to store the keys of the employees.
Assuming a Layered Index,
The search in a (designated) Layered Index would use the key “Arightorder . . . ” to search for the sub-tree with the related (children) information. The search process starts in the root node (1601) of the root block, continues using the link labeled “r” (1602) standing for the first character of “rightorder”, the node with the value 20 (1603), the link labeled “B” (1604) standing for the designator “B” for departments. This link address the block #5 (1605), thus the search continues with node 21 (1606) that differentiates between the different departments of “rightorder”. Now, the search continues depth first post order to examine the departments. The node with the value 40 (1607) that leads to department accounting (1610) is accessed first. The search skips the link labeled “C” (1613) as it leads to data subordinated to accounting and thus not relevant to the query. This is known since the node with the value (40) (1607) stands for the end position of the department string (in another application the sub-tree being skipped, rotted with the node with the value 41 (1616) extends over multiple blocks). The search now continues to the node with the value 40 (1608) that leads to department engineering (1611). Again, the sub-tree is being skipped and the search ends at the node with the value 40 (1609) and the department finance (1612) which is accessed by link 1615 (labeled null). The related sub-tree to the children is ignored. This procedure allowed to satisfy the query with minimal block accesses, the traversal in the leaf layer was based on the trie (Patricia trie in the example), the subordinated departments (1610, 1611, 1612) were retrieved by the order of their keys, whereas the structure of the Layered Index is balanced (from the root block 1624 to the leafs 1621, 1622, 1623, and 1605). In order to maintain the departments (of each company) clustered in the same block (or in minimal number of blocks) even if more information is added to each department, splits would be done at the sub-trees related to each department, such as at the link 1613 (assuming more data was updated for department accounting (1610) rather than (for example) the link 1614 that would separate between the department “accounting” and the other departments).
As can be seen in this example, the blocks #2, #3, #4, and #5 (1621, 1622, 1623, and 1605 respectively), stand for the leaf layer (whereas block #1 (1624) is a non-leaf layer block). These leaf layer blocks maintain an index (a Patricia index) over the keys of the data elements (similar to the Patricia index of
It can also be noted in this example that the key of the company (“Arightorder . . . ”) is the prefix of the key of its children, for example, the key of the department accounting is: “Arightorder . . . Baccounting . . . ”.
As have been explained and demonstrated, the Layered Index does not suffer from the limitations of the IOT: first, key lookups are satisfied using the additional index layers thus every search by a particular key is done with the same number of IOs (regardless of its hierarchical position) resulting with a similar efficiency for a search of a parent or a child, and second, any type of information can be added at any level (as exemplified, for example, in
The present invention has been described with a certain degree of particularity, but those versed in the art will readily appreciate that various alterations and modifications can be carried out without departing from the scope of the following Claims:
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7515588 *||Mar 18, 2004||Apr 7, 2009||Intel Corporation||Method and apparatus to support a large internet protocol forwarding information base|
|US7765208 *||Jun 6, 2005||Jul 27, 2010||Microsoft Corporation||Keyword analysis and arrangement|
|US8117182 *||Apr 23, 2007||Feb 14, 2012||International Business Machines Corporation||Apparatus and method for optimizing descendant path evaluation in xpath/xquery|
|US8166074||Jan 28, 2010||Apr 24, 2012||Pettovello Primo M||Index data structure for a peer-to-peer network|
|US8176080 *||Mar 6, 2009||May 8, 2012||Hewlett-Packard Development Company, L.P.||Desensitizing character strings|
|US8631028||Oct 29, 2010||Jan 14, 2014||Primo M. Pettovello||XPath query processing improvements|
|US8762359 *||Aug 22, 2008||Jun 24, 2014||Neustring Fze||Method of analyzing data traffic in a telecommunication network|
|US20050207409 *||Mar 18, 2004||Sep 22, 2005||Naik Uday R||Method and apparatus to support a large internet protocol forwarding information base|
|US20060064432 *||Sep 22, 2005||Mar 23, 2006||Pettovello Primo M||Mtree an Xpath multi-axis structure threaded index|
|US20060242169 *||Apr 25, 2005||Oct 26, 2006||Microsoft Corporation||Storing and indexing hierarchical data spatially|
|US20100228783 *||Mar 6, 2009||Sep 9, 2010||Castellanos Maria G||Desensitizing Character Strings|
|US20110184961 *||Aug 22, 2008||Jul 28, 2011||Neustring Fzco||Method of analyzing data traffic in a telecommunication network|
|U.S. Classification||1/1, 707/E17.012, 707/E17.058, 707/E17.083, 707/999.002|
|Cooperative Classification||Y10S707/99943, Y10S707/99942, G06F17/3061, G06F17/30911|
|European Classification||G06F17/30Z1T, G06F17/30T, G06F17/30X1|