WO2004077277A2 - System and methods for implementing an xml - Google Patents
System and methods for implementing an xml Download PDFInfo
- Publication number
- WO2004077277A2 WO2004077277A2 PCT/US2004/006104 US2004006104W WO2004077277A2 WO 2004077277 A2 WO2004077277 A2 WO 2004077277A2 US 2004006104 W US2004006104 W US 2004006104W WO 2004077277 A2 WO2004077277 A2 WO 2004077277A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- query
- xml
- token stream
- context
- stream representation
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/83—Querying
- G06F16/835—Query processing
- G06F16/8358—Query translation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99932—Access augmentation or optimizing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99934—Query formulation, input preparation, or translation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99943—Generating database or data structure, e.g. via user interface
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99944—Object-oriented database structure
- Y10S707/99945—Object-oriented database structure processing
Definitions
- the present invention relates to the querying of XML data.
- XQuery is a standard query language, published by the W3C (World Wide Web Consortium), that uses XML (Extensible Markup Language) notation to define query requests and handle query results.
- XQuery is designed to be an efficient language in which queries are concise and easily understood.
- XQuery is flexible enough to query a broad spectrum of data sources, including relational databases, XML documents, Web services, packaged applications, and legacy systems.
- the XQuery language is derived from various sources, including SQL. It even provides a For-Let-Where-Return (FLWR - pronounced "flower") expression. Developers familiar with SQL will find XQuery very easy to learn.
- Figure 1 is a diagram of exemplary datatype hierarchy showing complex data types directly by their structure, eventually reducing to a structured composition of simple types and their derivatives.
- An XQRL implementation can be based on a token stream representation, which allows large XML documents to be processed without having to first read the entire document.
- An XQRL implementation can also utilize a streaming XQuery engine.
- a set of algorithms and a set of rewrite rules can be used for the implementation. Given a query, a set of rules can be applied to that query to transform the query into another query. Some of these rules allow the resulting query to be streamable.
- a standard query that would otherwise require an XML document to be read into memory can now be transformed into a query that can work on an XML stream.
- a transformation can be applied to a query such that the entire query process is streamable.
- a user could generate a query where data is to be sorted in a particular some order. Later, that query might be used in a larger context where the order is not important.
- An XQRL implementation can analyze the query, and can determine that it would be an unnecessary use of resources to sort the later result. Sorting typically requires all the data to be read into memory and stored until all elements are known and can be sorted accordingly. If the sorting step is eliminated from the later query, it is possible to simply process the XML on an XML stream.
- a user or developer can add extensions or annotations to the XQuery language in order to address certain aspects that are not yet covered in the draft specification.
- a graphical tool, or graphical user interface can be used with an XQRL implementation that allows a user to build data transformations.
- the transformation tools in the interface in at least one embodiment are sitting on top of the XQuery engines.
- a token stream is an XML representation that can be used throughout an XQuery language implementation, such as XQRL.
- a token stream can be compact and efficient.
- the tokenized format can represent the structure of an XML document with a minimal memory footprint, even in presence of repetitive elements.
- a token stream can be designed for the internal data representation of a processor, the stream can also be a convenient XML format for application interchange.
- a token stream can be used to easily manage XML fragments, and is easily serializable on both network and disk. If an application uses XQuery as a principal means for XML processing, intermediate results may be better kept in a format directly available for data processing.
- An XQuery processor such as XQRL can process XML data in a streaming fashion.
- a data transformation operation, or query, in XQuery can be decomposed in a series of successive elementary operations on the data itself.
- An XQuery expression can be transformed by the XQRL processor in a sequence of elementary operators, with each operation applying a transformation on its input and feeding with its output the next operator.
- each operator can consume a minimal amount of data and can immediately begin to produce results, which can be directly available to the next operator.
- a streaming implementation can be consumer directed, such as may utilize a 'pull' mode.
- a consumer of the result of the query can demand data tokens from the outmost operator composing the query.
- the outmost operator can, in turn, ask for data from the previous queries, and so on, up to the input source.
- the data can be represented in a format that is granular enough for this processing, such as the token stream.
- a token stream is a tokenized representation of an XML data fragment. Repeating structural elements can be replaced by tokens, which can retain the semantics of the original data element, discarding any uimecessary infonnation accompanying those elements.
- An XQRL processor can be implemented in the Java programming language, which can have tokens represented by objects, or the addresses of those objects. Repeating elements in the stream can be repetitions of the same pointer to a static object. Tokens in a token stream can be immutable objects. Data transformations can be performed on the same input tokens coming from an XML parser or tokenizer. The output of the query can be made of the same exact tokens as the input, and possibly some generated by element constructors. Such an approach can keep the rate of object creation to a bare minimum during all the different phases of data transformation.
- the tokenizer and the schema validator can transform the previous data fragment in the following sequence of tokens:
- the first token [Begin Element] marks the beginning of the element judgement. Following is a QName token indicating the fully qualified name for the element, and another QName specifying the type of the element as it is defined in the schema.
- [Begin Attribute] marks the beginning of the attribute, following the QName with its name and type.
- a typed value token (represented by int : 11) gives the value of the attribute as an integer.
- [End Attribute] marks the end of the attribute.
- the type system of the XQuery language can be based on XML Schema.
- XML Schema has a number of built-in (more or less) basic types (elements in the Figure 1 below: duration, dateTime, time, date, gYearMonth, gYear, gMonthDay, gpay, gMonth, Boolean, base64Binary, hexBinary, float, double, anyURI, QName, NOTATION, string, decimal) which exist as typed tokens in the Token Stream representation.
- Each basic type can have derived types, which are represented by typed tokens inheriting from the base type and having an additional field specifying the name of the derived type.
- the physical data representation and the default behavior is instead shared with the parent, the derivation being essentially a restriction of the parent type.
- a second example shows an entire XML document and its tokenized representation.
- the [Begin Document] and [End Document] tokens can be seen, as well as the name space declarations and a few additional data types.
- XQuery programs can execute in a Context.
- the context can holds the variable bindings, namespaces, operators, and default parameters.
- the context can hold (static) references to an ID manager (IdManager) in charge of creating node IDs in XML documents, a type manager used by the static type inference engine and run-time type validation, and a schema manager or database of XML schemas.
- IdManager ID manager
- the context can be entirely defined at the query compilation or preparation time, and may never be modified by the query execution.
- the compiler can also resolve all the elements of the context to direct pointer references in the compiled query. Consequently, a query execution engine may never need to resolve variables by their name at run time.
- a new context can be created holding the newly (implicitly or explicitly) scoped variables.
- This new context can be linked to its parent. All variables of the parent can be accessible from the context, possibly except for those that are redefined by the new context shadowing their parent counterpart.
- a base context can be used to hold the default environment, which can include standard namespaces, built-in functions and operators, and default behaviors.
- Queries can reuse pre-initialized contexts, but they may not be able to directly modify those contexts.
- a new scratch context can always be created for query execution, which can inherit directly from the base context or from another explicitly-given context.
- Special queries can be used to pre-initialize contexts to be reutilized by subsequent queries, reducing the cost of context creation.
- Execution objects can be opaque objects embedded in Connection objects, which are provided in one embodiment by an XDBC interface.
- a Connection object can be created through an interface such as a DriverManager getConnection interface, and can be used to prepare statements which are subsequently populated using a special query prologue, such as a populateContext method of Connection.
- Contexts can be saved for later use associating them to a URI, such as by using a publish method of a DriverManager, such that other contexts can be subsequently created inheriting from the saved ones.
- Connection c DriverManager.getConnection()
- Connection c DriverManager.getConnection(
- Tokenlterator doc ds.executeQueryQ
Abstract
Systems and methods in accordance with the present invention provide for an implementation of the XQuery standard, and such an implementation will be referred to herein as an XQRL (XML Query language) implementation. An XQRL implementation can be based on a token stream representation, which allows large XML documents to be processed without having to first read the entire document. An XQRL implementation can also utilize a streaming XQuery engine. A set of algorithms and a set of rewrite rules can be used for the implementation. Given a query, a set of rules can be applied to that query to transform the query into another query. Some of these rules allow the resulting query to be streamable. In other words, a standard query that would otherwise require an XML document to be read into memory can now be transformed into a query that can work on an XML stream. The type system of the XQuery language can be based on XML Schema, and the basic types of XML Schema can have derived types. The XQuery programs can execute in a context comprising of variable bindings, namespaces, operators, default parameters etc. This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.
Description
SYSTEMS AND METHODS FOR IMPLEMENTING AN XML QUERY LANGUAGE
Inventor: Fabio Riccardi
CLAIM OF PRIORITY
This application claims priority to U.S. Provisional Patent Application No. 60/450,381 filed February 27, 2003, entitled "SYSTEMS AND METHODS FOR IMPLEMENTING AN XML QUERY LANGUAGE" Attorney Docket No. BEAS-01447US0); and
U.S. Patent Application No. filed February 26, 2004, entitled
"SYSTEMS AND METHODS FOR IMPLEMENTING AN XML QUERY LANGUAGE" Attorney Docket No. BEAS-01447US1), which are hereby incorporated herein by reference.
COPYRIGHT NOTICE
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document of the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
FIELD OF THE INVENTION
The present invention relates to the querying of XML data.
BACKGROUND
XQuery is a standard query language, published by the W3C (World Wide Web Consortium), that uses XML (Extensible Markup Language) notation to
define query requests and handle query results. XQuery is designed to be an efficient language in which queries are concise and easily understood. XQuery is flexible enough to query a broad spectrum of data sources, including relational databases, XML documents, Web services, packaged applications, and legacy systems.
The XQuery language is derived from various sources, including SQL. It even provides a For-Let-Where-Return (FLWR - pronounced "flower") expression. Developers familiar with SQL will find XQuery very easy to learn.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 is a diagram of exemplary datatype hierarchy showing complex data types directly by their structure, eventually reducing to a structured composition of simple types and their derivatives.
DETAILED DESCRIPTION
Systems and methods in accordance with the present invention provide for an implementation of the XQuery standard, and such an implementation will be referred to herein as an XQRL (XML Query language) implementation. An XQRL implementation can be based on a token stream representation, which allows large XML documents to be processed without having to first read the entire document. An XQRL implementation can also utilize a streaming XQuery engine. A set of algorithms and a set of rewrite rules can be used for the implementation. Given a query, a set of rules can be applied to that query to transform the query into another query. Some of these rules allow the resulting query to be streamable. In other words, a standard query that would otherwise require an XML document to be read into memory can now be transformed into a query that can work on an XML stream.
In many cases, such a transformation can be applied to a query such that the entire query process is streamable. For example, a user could generate a query where data is to be sorted in a particular some order. Later, that query might be used in a larger context where the order is not important. An XQRL implementation can analyze the query, and can determine that it would be an unnecessary use of resources to sort the later result. Sorting typically requires all the data to be read into memory and stored until all elements are known and can be sorted accordingly. If the sorting step is eliminated from the later query, it is possible to simply process the XML on an XML stream.
A user or developer can add extensions or annotations to the XQuery language in order to address certain aspects that are not yet covered in the draft specification. A graphical tool, or graphical user interface, can be used with an XQRL implementation that allows a user to build data transformations. The transformation tools in the interface in at least one embodiment are sitting on top of the XQuery engines.
Token Stream
A token stream (TS) is an XML representation that can be used throughout an XQuery language implementation, such as XQRL. A token stream can be compact and efficient. The tokenized format can represent the structure of an XML document with a minimal memory footprint, even in presence of repetitive elements. Although a token stream can be designed for the internal data representation of a processor, the stream can also be a convenient XML format for application interchange. A token stream can be used to easily manage XML fragments, and is easily serializable on both network and disk. If an application uses XQuery as a principal means for XML processing, intermediate results may be better kept in a format directly available for data processing.
An XQuery processor such as XQRL can process XML data in a streaming fashion. A data transformation operation, or query, in XQuery can be decomposed in a series of successive elementary operations on the data itself. An
XQuery expression can be transformed by the XQRL processor in a sequence of elementary operators, with each operation applying a transformation on its input and feeding with its output the next operator.
Instead of completely consuming input before performing the transformation, each operator can consume a minimal amount of data and can immediately begin to produce results, which can be directly available to the next operator.
Using such an approach, an operator may not need to bufferize or materialize large amounts of data, thereby reducing memory consumption and processing time. Also, the result of such a query can be consumed and made available as soon as possible, often well before the input data has been completely consumed.
A streaming implementation can be consumer directed, such as may utilize a 'pull' mode. A consumer of the result of the query can demand data tokens from the outmost operator composing the query. The outmost operator can, in turn, ask for data from the previous queries, and so on, up to the input source.
To make such a flow of data possible, the data can be represented in a format that is granular enough for this processing, such as the token stream. As the name implies, a token stream is a tokenized representation of an XML data fragment. Repeating structural elements can be replaced by tokens, which can retain the semantics of the original data element, discarding any uimecessary infonnation accompanying those elements.
An XQRL processor can be implemented in the Java programming language, which can have tokens represented by objects, or the addresses of those objects. Repeating elements in the stream can be repetitions of the same pointer to a static object.
Tokens in a token stream can be immutable objects. Data transformations can be performed on the same input tokens coming from an XML parser or tokenizer. The output of the query can be made of the same exact tokens as the input, and possibly some generated by element constructors. Such an approach can keep the rate of object creation to a bare minimum during all the different phases of data transformation.
For example, consider the following XML data fragment:
Schema:
<xsd:complexType name="vote"> <xsd: simpleContent>
<xsd:extension base="xsd:float">
<xsd: attribute name- 'index" type="xsd:int'7> </xsd: extension> </xsd: simpleContent> </xsd:complexType>
<xsd:element name- 'judgement" type="vote"/>
Data Instance:
•^judgement index— '11 ">43.5</judgement>
The tokenizer and the schema validator can transform the previous data fragment in the following sequence of tokens:
[Begin Element]
QName : judgement@http://www.xqrl.com/examplel]
QName : [vote@http://www.xqrl.com/examplel]
[Begin Attribute]
QName : [index]
QName : [int@http://www.w3.org/2001/XMLSchema] t : 11
[End Attribute]
[Begin Text]
CharData : "43.5"
[End Text] float : 43.5
[End Element]
In the example, the first token [Begin Element] marks the beginning of the element judgement. Following is a QName token indicating the fully qualified name for the element, and another QName specifying the type of the element as it is defined in the schema. [Begin Attribute] marks the beginning of the attribute, following the QName with its name and type. A typed value token (represented by int : 11) gives the value of the attribute as an integer. [End Attribute] marks the end of the attribute.
According to the XQuery specification, the original lexical representation for each element should be available for processing, thereby the [Begin Text] CharData : "43.5" [End Text] tokens, followed by the value token carrying the typed float representation of the element's data float : 43.5 finally the [End Element] token marks the end of the element judgment.
The type system of the XQuery language can be based on XML Schema. XML Schema has a number of built-in (more or less) basic types (elements in the Figure 1 below: duration, dateTime, time, date, gYearMonth, gYear, gMonthDay, gpay, gMonth, Boolean, base64Binary, hexBinary, float, double, anyURI, QName, NOTATION, string, decimal) which exist as typed tokens in the Token Stream representation. Each basic type can have derived types, which are represented by typed tokens inheriting from the base type and having an additional field specifying the name of the derived type. The physical data representation and the default behavior is instead shared with the parent, the derivation being essentially a restriction of the parent type.
As seen in the previous example, complex data types can be represented directly by their structure, eventually reducing to a structured composition of simple types and their derivatives. An exemplary datatype hierarchy is given by Figure 1.
A second example shows an entire XML document and its tokenized representation. In the token stream the [Begin Document] and [End Document]
tokens can be seen, as well as the name space declarations and a few additional data types.
The Schema:
<?xml version-" 1.0" encoding="UTF-8"?> <xsd:schema xmlns:xsd-http://www.w3.org/2001/XMLSchema' xmlns="http://www.xqrl.com/examplel" targetNamespace- 'http://www.xqrl.com/examplel" elementForrnDefault- 'qualified" >
<xsd:complexType name="rootType"> <xsd:sequence>
<xsd:element name="judgement" type- 'vote" minOccurs="l" maxOccurs— '5"/> </xsd: sequence> </xsd:complexType>
<xsd:complexType name="vote"> <xsd: simpleContent>
<xsd: extension base="grade"> <xsd: attribute name="index" type="xsd:int" default="0"/> </xsd:extension> </xsd: simpleContent> </xsd:complexType>
<xsd:simpleType name- 'grade"> <xsd:union>
<xsd:simpleType>
<xsd:restriction base="xsd:date"/> </xsd:simpleType> <xsd:simpleType>
<xsd:restriction base- 'xsd:int7> </xsd: simpleType> <xsd:simpleType>
<xsd:restriction base:="xsd:float"/> </xsd: simpleType> <xsd:simpleType>
<xsd:restriction base="xsd:double"/> </xsd: simpleType> <xsd:simpleType>
<xsd:restriction base="xsd: string"/> </xsd:simpleType> </xsd:union> </xsd:simpleType>
<xsd:element name- 'root" type="rootType"/> </xsd:schema>
A Data Instance:
<?xml version="1.0" encoding="UTF-8"?> <root xmlns="http://www.xqrl.com/exampler' xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.xqrl.com/examplel tests/schema/example l.xsd" > judgement index- ' 11 ">43.5</judgement> <judgement>3.14</judgement> <judgement>pizza</judgement> <judgement>2002-06- 14</judgement> </root>
The token stream:
[Begin Document]
[Begin Element]
QName : [root@http://www.xqrl.com/examplel]
QName : [iOθtType@http://www.xqιτ.com/examplel]
[Begin Namespace]
QName : [@http://www.xqrl.com/examplel]
[End Namespace]
[Begin Namespace]
QName : [xsi@http://www.w3.org/2001/XMLSchema-instance]
[End Namespace]
[Begin Attribute]
QName : [schemaLocation@http://www .w3.org/2001/XMLSchema- instance]
QName : [anySimpleType@http://www .w3.org/2001/XMLSchema]
CharData : "http://www.xqrl.com/examplel tests/schema/example l.xsd"
[End Attribute]
[Begin Text]
CharData : "\n\n "
[End Text]
[Begin Element]
QName : [judgement@http://www.xqrl.com/examplel]
QName : [vote@http://www.xqrl. com/example 1]
[Begin Attribute]
QName : [index]
QName : [int@http://www. w3.org/2001/XMLSchema]
int : 11
[End Attribute]
[Begin Text]
CharData : "43.5"
[End Text] float : 43.5
[End Element]
[Begin Text]
CharData : "\n "
[End Text]
[Begin Element]
QName : [judgement@http://www.xqrl.com/examplel]
QName : [vote@http://www.xqrl.com/examplel]
[Begin Attribute]
QName : [index]
QName : [int@http://www.w3.org/2001/XMLSchema] int : 0
[End Attribute]
[Begin Text]
CharData : "3.14"
[End Text] float : 3.14
[End Element]
[Begin Text]
CharData : "\n "
[End Text]
[Begin Element]
QName : [judgement@http://www.xqrl.com/examplel]
QName : [vote@http://www.xqrl. com/example 1]
[Begin Attribute]
QName : [index]
QName : [int@http://www.w3.org/2001/XMLSchema] int : 0
[End Attribute]
[Begin Text]
CharData : "pizza"
[End Text] string : pizza
[End Element]
[Begin Text]
CharData : "\n "
[End Text]
[Begin Element]
QName : [judgement@http://www.xqrl.com/examplel]
QName : [vote@http://www.xqrl.com/examplel]
[Begin Attribute]
QName : [index]
QName : [int@http://www. w3.org/2001/XMLSchema] int : 0
[End Attribute]
[Begin Text]
CharData : "2002-06-14"
[End Text] date : 2002-6-14T0:0:0.0D
[End Element]
[Begin Text]
CharData : "\n"
[End Text]
[End Element]
[End Document]
Context
XQuery programs can execute in a Context. The context can holds the variable bindings, namespaces, operators, and default parameters. Furthermore, the context can hold (static) references to an ID manager (IdManager) in charge of creating node IDs in XML documents, a type manager used by the static type inference engine and run-time type validation, and a schema manager or database of XML schemas.
The context can be entirely defined at the query compilation or preparation time, and may never be modified by the query execution. The compiler can also resolve all the elements of the context to direct pointer references in the compiled query. Consequently, a query execution engine may never need to resolve variables by their name at run time.
To implement the semantics of FLWR expressions and LET statements, implicit variables defined in [ ] xpath expressions, and sortby dots, a new context can be created holding the newly (implicitly or explicitly) scoped variables. This new context can be linked to its parent. All variables of the parent can be accessible from the context, possibly except for those that are redefined by the new context shadowing their parent counterpart.
To factorize things and to reduce start-up time, a base context can be used to hold the default environment, which can include standard namespaces, built-in functions and operators, and default behaviors.
Queries can reuse pre-initialized contexts, but they may not be able to directly modify those contexts. A new scratch context can always be created for query execution, which can inherit directly from the base context or from another explicitly-given context. Special queries can be used to pre-initialize contexts to be reutilized by subsequent queries, reducing the cost of context creation.
Given to their critical nature, execution contexts may not be directly accessible to the user. Execution objects can be opaque objects embedded in Connection objects, which are provided in one embodiment by an XDBC interface.
A Connection object can be created through an interface such as a DriverManager getConnection interface, and can be used to prepare statements which are subsequently populated using a special query prologue, such as a populateContext method of Connection.
Contexts can be saved for later use associating them to a URI, such as by using a publish method of a DriverManager, such that other contexts can be subsequently created inheriting from the saved ones.
Examples
// derive a connection from the Base Context (default) Connection c = DriverManager.getConnection();
// populate it with someXQuery prologue c.populateContext(prologue);
//publish it! DriverManager.publish("www.xqrl.com/contexts/stuffed", c);
// some other time, some other place, ...
// derive a Connection from the previously saved one
Connection c = DriverManager.getConnection(
"www.xqrl.com/contexts/stuffed", Connection.Type.INHERITED);
// use it to execute your favorite query! PreparedStatement ds =
(PreparedStatement) c.prepareStatement(dataQuery);
Tokenlterator doc = ds.executeQueryQ;
The foregoing description of preferred embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations will be apparent to one of ordinary skill in the art. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to understand the invention for various embodiments and with various modifications that are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalence.
Claims
1. A method for an implementation of XML Query language, comprising the steps of: analyzing a standard XML query; creating a token stream representation of at least one XML document required to process said standard XML query; transforming said standard XML query into a second query that can work on said token stream representation; and, executing said second query on said token stream representation.
2. A method according to claim 1, wherein: said implementation is programmed in Java language.
3. A method according to claim 1, wherein: said step of transforming includes utilizing a streaming XML query engine that uses a set of algorithms and a set of rewrite rules.
4. A method according to claim 3, wherein: said set of rewrite rules comprises rules that allow the second query to be streamable.
5. A method according to claim 1, wherein: said implementation further allows a user to add extensions and annotations to said XML Query language.
6. A method according to claim 1, further comprising: using a graphical tool that allows a user to build data transformations.
7. A method according to claim 1, wherein: said token stream representation is serializable.
8. A method according to claim 1, wherein: said transforming step further comprises converting said standard XML query into a sequence of elementary operators, with each operator applying a transformation of the operator input and feeding the operator output to next operator.
9. A method according to claim 1, wherein: said token stream representation comprises tokens that are immutable objects.
10. A method according to claim 1, wherein: said transforming step further comprises keeping the rate of object creation to a minimum during all phases of data transformation by using the exact tokens of said token stream representation for the output of said standard XML query.
11. A method according to claim 1 , wherein: said token stream representation comprises typed tokens, wherein each typed token specifies the type of XML element of the XML document represented.
12. A method according to claim 11 , wherein: said typed tokens have derived types, wherein said derived types share physical data representation and default behavior with said typed tokens.
13. A method according to claim 11 , wherein: said typed tokens are based on XML Schema.
14. A method according to claim 1, further comprising: executing said XML query in a context, wherein said context comprises of resources required for said XML query execution.
15. A method according to claim 14, wherein: said context is defined at said XML query preparation time.
16. A method according to claim 14, wherein: said context is defined at said XML query compilation time.
17. A method according to claim 14, wherein: said context is saved for later use, such that a second context is subsequently created inheriting from saved said context.
18. A method according to claim 14, wherein: said context is used to create a new context, wherein said new context inherits from said context.
19. A method according to claim 14, further comprising: creating new context for holding the newly scoped variables defined by expressions and statements in said XML query.
20. A method according to claim 1, further comprising: providing an interface for creating connection object to execute said XML query.
21. A system for an implementation of XML Query language, comprising: a token stream processor that can create a token stream representation of at least one XML document required to process a standard XML query; a query engine that can transform said standard XML query into a second query that can work on said token stream representation; and a query processor that can execute said second query on said token stream representation.
22. A system according to claim 21, wherein: said implementation is programmed in Java language.
23. A system according to claim 21, wherein: said query engine uses a set of algorithms and a set of rewrite rules to transform said standard XML query into said second query that can work on said token stream representation.
24. A system according to claim 23, wherein: said set of rewrite rules comprise of rules that allow the second query to be streamable.
25. A system according to claim 21, wherein: said implementation further allows a user to add extensions and annotations to said XML Query language.
26. A system according to claim 21, further comprising: a graphical tool that allows a user to build data transformations for said XML query.
27. A system according to claim 21, wherein: said token stream representation is serializable.
28. A system according to claim 21, wherein: ' said query engine further converts said standard XML query into a sequence of elementary operators, with each operator applying a transformation of the operator input and feeding the operator output to next operator.
29. A system according to claim 21, wherein: said token stream representation comprises tokens that are immutable objects.
30. A system according to claim 21, wherein: said query engine keeps the rate of object creation to a minimum during all phases of data transformation by using the exact tokens of said token stream for the output of said standard XML query.
31. A system according to claim 21 , wherein: said token stream representation comprises typed tokens, wherein each typed token specifies the type of XML element of the XML document represented.
32. A system according to claim 31, wherein: said typed tokens have derived types, wherein said derived types share physical data representation and default behavior with said typed tokens.
33. A system according to claim 31 , wherein: said typed tokens are based on XML Schema.
34. A system according to claim 1, further comprising: a context for executing said XML query, wherein said context comprises of resources required for said XML query execution.
35. A system according to claim 34, wherein: said context is defined at said XML query preparation time.
36. A system according to claim 34, wherein: said context is defined at said XML query compilation time.
37. A system according to claim 34, wherein: said context is saved for later use, such that a second context is subsequently created inheriting from saved said context.
38. A system according to claim 34, wherein: said context is used to create a new context, wherein the new context inherits from said context.
39. A system according to claim 34, further comprising: a new context for holding the newly scoped variables defined by expressions and statements in said XML query.
40. A system according to claim 31 , further comprising: an interface for creating connection object to execute said XML query.
41. A computer system comprising: a processor; object code executed by said processor, said object code configured to: create a token stream representation of at least one XML document required for processing a standard XML query; transform said standard XML query into a second query that can work on said token stream representation; and execute said second query on said token stream representation.
42. A computer data signal embodied in a transmission medium, comprising: a code segment including instructions to create a token stream representation of at least one XML document required for processing a standard XML query; a code segment including instructions to transform said standard XML query into a second query that can work on said token stream representation; and a code segment including instructions to execute said second query on said token stream representation.
43. A method to implement an XML Query language, comprising the steps of: creating a token stream representation of at least one XML document in order to process an XML query; and transforming said XML query into a second query that can work on said token stream representation.
44. The method of claim 43 including the step of using a standard XML query in the creating step.
45. The method of claim 43 wherein said transforming step uses a set of algorithms.
46. The method of claim 43 wherein said transforming step uses a set of rewrite rules.
47. A system to implement an XML Query language comprising: a creation module that can create a token stream representation of at least one XML document required to precess an XML query; a transform module that can transform said XML query into a second query that can work on said token stream representation; and an execution module that can execute said second query on said token stream representation.
48. The system of claim 43 wherein said transform module uses a set of algorithms.
49 The system of claim 43 wherein said transform module uses a set of rewrite rules.
50. A system for an implementation of XML Query language, comprising: means for creating token stream representation of at least one XML document required to process a standard XML query; means for transforming said standard XML query into a second query that can work on said token stream representation; and means for executing said second query on said token stream representation.
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US45038103P | 2003-02-27 | 2003-02-27 | |
US60/450,381 | 2003-02-27 | ||
US10/787,714 | 2004-02-26 | ||
US10/787,714 US7194462B2 (en) | 2003-02-27 | 2004-02-26 | Systems and methods for implementing an XML query language |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2004077277A2 true WO2004077277A2 (en) | 2004-09-10 |
WO2004077277A3 WO2004077277A3 (en) | 2005-01-06 |
Family
ID=32930552
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2004/006104 WO2004077277A2 (en) | 2003-02-27 | 2004-02-27 | System and methods for implementing an xml |
Country Status (2)
Country | Link |
---|---|
US (2) | US7194462B2 (en) |
WO (1) | WO2004077277A2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2008039311A2 (en) * | 2006-09-22 | 2008-04-03 | Raytheon Company | Sytem and method for converting a query for retrieval of information from a data source |
US7779396B2 (en) | 2005-08-10 | 2010-08-17 | Microsoft Corporation | Syntactic program language translation |
Families Citing this family (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7930277B2 (en) * | 2004-04-21 | 2011-04-19 | Oracle International Corporation | Cost-based optimizer for an XML data repository within a database |
US7509631B2 (en) * | 2004-05-21 | 2009-03-24 | Bea Systems, Inc. | Systems and methods for implementing a computer language type system |
US8583694B2 (en) * | 2005-05-09 | 2013-11-12 | Atlas Development Corporation | Health-care related database middleware |
CA2646996C (en) * | 2005-05-09 | 2014-11-04 | Atlas Development Corporation | Health-care related database middleware |
US7814065B2 (en) * | 2005-08-16 | 2010-10-12 | Oracle International Corporation | Affinity-based recovery/failover in a cluster environment |
US8073841B2 (en) * | 2005-10-07 | 2011-12-06 | Oracle International Corporation | Optimizing correlated XML extracts |
US20070250471A1 (en) * | 2006-04-25 | 2007-10-25 | International Business Machines Corporation | Running XPath queries over XML streams with incremental predicate evaluation |
US7853573B2 (en) * | 2006-05-03 | 2010-12-14 | Oracle International Corporation | Efficient replication of XML data in a relational database management system |
US20080154860A1 (en) * | 2006-06-14 | 2008-06-26 | Nec Laboratories America, Inc. | Efficient processing of tree pattern queries over xml documents |
US20080097959A1 (en) * | 2006-06-14 | 2008-04-24 | Nec Laboratories America, Inc. | Scalable xml filtering with bottom up path matching and encoded path joins |
US20070294217A1 (en) * | 2006-06-14 | 2007-12-20 | Nec Laboratories America, Inc. | Safety guarantee of continuous join queries over punctuated data streams |
US7730080B2 (en) * | 2006-06-23 | 2010-06-01 | Oracle International Corporation | Techniques of rewriting descendant and wildcard XPath using one or more of SQL OR, UNION ALL, and XMLConcat() construct |
US7801856B2 (en) * | 2006-08-09 | 2010-09-21 | Oracle International Corporation | Using XML for flexible replication of complex types |
US8397226B2 (en) * | 2006-08-14 | 2013-03-12 | Microsoft Corporation | Pattern matching over structured literals |
US7958112B2 (en) * | 2008-08-08 | 2011-06-07 | Oracle International Corporation | Interleaving query transformations for XML indexes |
US9128895B2 (en) * | 2009-02-19 | 2015-09-08 | Oracle International Corporation | Intelligent flood control management |
US8266172B2 (en) * | 2009-04-03 | 2012-09-11 | Microsoft Corporation | Data parallel query analysis |
US8713426B2 (en) * | 2009-06-25 | 2014-04-29 | Oracle International Corporation | Technique for skipping irrelevant portions of documents during streaming XPath evaluation |
US8539457B2 (en) * | 2009-11-06 | 2013-09-17 | Microsoft Corporation | Partial on-demand lazy semantic analysis |
US9165086B2 (en) * | 2010-01-20 | 2015-10-20 | Oracle International Corporation | Hybrid binary XML storage model for efficient XML processing |
US8326821B2 (en) | 2010-08-25 | 2012-12-04 | International Business Machines Corporation | Transforming relational queries into stream processing |
US20120079583A1 (en) * | 2010-09-23 | 2012-03-29 | Microsoft Corporation | Offload reads and writes |
US9092149B2 (en) | 2010-11-03 | 2015-07-28 | Microsoft Technology Licensing, Llc | Virtualization and offload reads and writes |
US9146765B2 (en) | 2011-03-11 | 2015-09-29 | Microsoft Technology Licensing, Llc | Virtual disk storage techniques |
US20120246154A1 (en) * | 2011-03-23 | 2012-09-27 | International Business Machines Corporation | Aggregating search results based on associating data instances with knowledge base entities |
US9959326B2 (en) | 2011-03-23 | 2018-05-01 | International Business Machines Corporation | Annotating schema elements based on associating data instances with knowledge base entities |
US9817582B2 (en) | 2012-01-09 | 2017-11-14 | Microsoft Technology Licensing, Llc | Offload read and write offload provider |
US9071585B2 (en) | 2012-12-12 | 2015-06-30 | Microsoft Technology Licensing, Llc | Copy offload for disparate offload providers |
US9251201B2 (en) | 2012-12-14 | 2016-02-02 | Microsoft Technology Licensing, Llc | Compatibly extending offload token size |
US11909707B2 (en) * | 2022-04-15 | 2024-02-20 | Red Hat, Inc. | Message schema migration in messaging systems |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6366934B1 (en) * | 1998-10-08 | 2002-04-02 | International Business Machines Corporation | Method and apparatus for querying structured documents using a database extender |
US6654734B1 (en) * | 2000-08-30 | 2003-11-25 | International Business Machines Corporation | System and method for query processing and optimization for XML repositories |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7493603B2 (en) * | 2002-10-15 | 2009-02-17 | International Business Machines Corporation | Annotated automaton encoding of XML schema for high performance schema validation |
-
2004
- 2004-02-26 US US10/787,714 patent/US7194462B2/en active Active
- 2004-02-27 WO PCT/US2004/006104 patent/WO2004077277A2/en active Application Filing
- 2004-11-30 US US10/999,746 patent/US7571160B2/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6366934B1 (en) * | 1998-10-08 | 2002-04-02 | International Business Machines Corporation | Method and apparatus for querying structured documents using a database extender |
US6654734B1 (en) * | 2000-08-30 | 2003-11-25 | International Business Machines Corporation | System and method for query processing and optimization for XML repositories |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7779396B2 (en) | 2005-08-10 | 2010-08-17 | Microsoft Corporation | Syntactic program language translation |
WO2008039311A2 (en) * | 2006-09-22 | 2008-04-03 | Raytheon Company | Sytem and method for converting a query for retrieval of information from a data source |
WO2008039311A3 (en) * | 2006-09-22 | 2008-09-18 | Raytheon Co | Sytem and method for converting a query for retrieval of information from a data source |
US7747569B2 (en) | 2006-09-22 | 2010-06-29 | Raytheon Company | Systems, methods, and language for selection and retrieval of information from databases |
Also Published As
Publication number | Publication date |
---|---|
US20050131875A1 (en) | 2005-06-16 |
WO2004077277A3 (en) | 2005-01-06 |
US7571160B2 (en) | 2009-08-04 |
US7194462B2 (en) | 2007-03-20 |
US20050114328A1 (en) | 2005-05-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7194462B2 (en) | Systems and methods for implementing an XML query language | |
US7634515B2 (en) | Data model and schema evolution | |
US7774746B2 (en) | Generating a format translator | |
US7219102B2 (en) | Method, computer program product, and system converting relational data into hierarchical data structure based upon tagging trees | |
Pal et al. | XQuery implementation in a relational database system | |
EP1686495A1 (en) | Mapping web services to ontologies | |
WO2002031690A2 (en) | A universal device to construct outputs for xml queries | |
WO2003077142A1 (en) | Method, apparatus, and system for data modeling and processing | |
US6735598B1 (en) | Method and apparatus for integrating data from external sources into a database system | |
US20070294268A1 (en) | Apparatus and method for processing data corresponding to multiple cobol data record schemas | |
US7016915B2 (en) | Method for processing XML queries over relational data and meta-data using a relational database system | |
Tsinaraki et al. | XS2OWL: a formal model and a system for enabling XML schema applications to interoperate with OWL-DL domain knowledge and semantic web tools | |
Chen et al. | Automatic generation of XQuery view definitions from ORA-SS views | |
Robie | XML processing and data integration with XQuery | |
Qu et al. | Integrating XQuery-enabled SCORM XML Metadata Repositories into an RDF-based E-Learning P2P Network | |
Barbosa et al. | Declarative generation of synthetic XML data | |
Pal et al. | Managing collections of XML schemas in Microsoft SQL Server 2005 | |
May | A Logic-Based Approach to XML Data Integration | |
van Zwol et al. | Dxl: Data exchange language | |
van Zwol et al. | Data Exchange over Web-based Applications with DXL. | |
Wu et al. | Design and implementation of data-driven based universal data editing framework | |
Chen et al. | A Case Tool for Designing XML Views. | |
Fousteris et al. | Multidimensional xpath | |
Groppe et al. | Using XSLT Stylesheets to Transform XPath Queries. | |
Stefanova et al. | SPARQL queries to RDFS views of Topic Maps |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): BW GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
122 | Ep: pct application non-entry in european phase |