US 20060095250 A1
A computer-implemented method for developing a parser is provided. The method includes accessing a corpus of sentences and parsing the sentences to generate a structural description of each sentence. The parser is trained based on the structural description of each sentence.
1. A computer-implemented method for developing a parser, comprising:
accessing a corpus of sentences;
parsing the sentences to generate a structural description of each sentence using an existing parser; and
training another parser based on the structural description of each sentence.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
9. A computer-readable medium having instructions that developing a parser for use in natural language processing, the instructions comprising:
a corpus of sentences;
an existing parser adapted to parse the sentences to generate a structural description of each; and
a training module adapted to train the parser based on the structural description of each sentence.
10. The computer-readable medium of
11. The computer-readable medium of
12. The computer-readable medium of
13. The computer-readable medium of
14. The computer-readable medium of
15. The computer-readable medium of
16. The computer-readable medium of
The present invention generally relates to natural language processing. More particularly, the present invention relates to developing a parser for use in natural language processing.
Parsing assigns a structural description, typically in the form of a parse tree or a dependency tree, to a given sentence. A parsed sentence can be used in a variety of natural language processing applications including information retrieval, writing assistance, machine translation, questioning/answering, searching, and the like. Current parsers are trained using statistical or machine learning approaches on hand-annotated corpora. Hand-annotation of corpora requires a large amount of time and expense. Since hand-annotated corpora are expensive to generate, developing a parser based on automatic generation of training data would provide a parser that is less costly to develop and/or maintain.
A computer-implemented method for developing a parser is provided. The method includes accessing a corpus of sentences and parsing the sentences using a pre-existing parser to generate a structural description of each sentence. Subsequently, there can be a deterministic method which does not rely on any human input or labor to transform the resulting structural descriptions into training data. This training data is used directly to train the parser.
The present invention deals with developing a parser for natural language processing. More specifically, the present invention deals with utilizing output from a first parser to train a second parser. However, prior to discussing the present invention in greater detail, one exemplary environment in which the present invention can be used will be discussed.
The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
With reference to
Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 100. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier WAV or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, FR, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.
The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation,
The computer 110 may also include other removable/non- removable volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media discussed above and illustrated in
A user may enter commands and information into the computer 110 through input devices such as a keyboard 162, a microphone 163, and a pointing device 161, such as a mouse, trackball or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 120 through a user input interface 160 that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor 191 or other type of display device is also connected to the system bus 121 via an interface, such as a video interface 190. In addition to the monitor, computers may also include other peripheral output devices such as speakers 197 and printer 196, which may be connected through an output peripheral interface 190.
The computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be a personal computer, a hand-held device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 110. The logical connections depicted in
When used in a LAN networking environment, the computer 110 is connected to the LAN 171 through a network interface or adapter 170. When used in a WAN networking environment, the computer 110 typically includes a modem 172 or other means for establishing communications over the WAN 173, such as the Internet. The modem 172, which may be internal or external, may be connected to the system bus 121 via the user-input interface 160, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
It should be noted that the present invention can be carried out on a computer system such as that described with respect to
There are several different options for developing a parser. One option is to develop a parser as a rule-based or statistical parser. A rule-based parser is typically based on a human generated grammar. A statistical parser is typically based on a human annotation of a corpus of parse trees. Unlike rule-based parsers, statistical parsers have been shown to be more accurate when parsing sentences from complex domains such as newspaper text. However, the statistical parser generally provides less detail about the corresponding sentence structure. Another option is to develop a full or shallow parser. A full parser explores all parse trees while a shallow parser explores a subset of parse trees. The shallow parser is faster, but the output is less detailed.
In accordance with the present invention, a shallow statistical parser is developed and trained using parsed data that is output from a pre-existing parser. This parser in itself requires less human labor than the more prevalent alternative of having a human hand-parse the data. The shallow parser in question can be referred to as a “skeleton” parser, because output from the parser is restricted to being either subject-verb or verb-object relationships. We define “skeleton relations” as these kinds of relationships. They are valuable in determining importance of words and subject matter of a sentence. Furthermore, limiting the skeleton parser to output only skeleton relations makes it more accurate, because it is trained directly on the output of a pre-existing parser, and pre-existing parsers generally output skeleton relations with higher accuracy than other kinds of relations.
EZParse is a hand-crafted parser with a statistical component. Hand-crafted components include an augmented context-free grammar and a knowledge base with morphological, lexicographic, and semantic information. The statistical component guides a search algorithm that is used by the parser. The input to EZParse is parsed in a cascaded manner with morphological, syntactic chunk, surface syntactic, and semantic information.
Using the parsed data, data is transformed at step 256. Transformation is further described with respect to
Next, at step 286, verb groups are extracted from the portrait output. A verb group includes a head word of a clause and any pre-modifiers. For example, preceding auxiliary verbs and adverbs are included in verb groups. Detecting these verb groups can rely on part-of-speech information as well as a hand-coded list of auxiliary verbs.
At step 288, skeleton relations from the logical form are extracted. The skeleton relation is a word pair consisting of a predicate head word and its corresponding argument head word, and also an associated label that is the relation between the pair. Some relations are extracted verbatim from the EZParse logical form output. Others can be transformed into relations that are defined as skeleton relations. One transformation includes replacing a direct link between a subject and an object in a predicative constructions with two links, wherein one links the subject and the other links the object with a verb. For example,
At step 308, a determination is made as to whether each candidate word pair is a skeleton relation. This determination can be made using a so-called maximum entropy (ME) model. The model makes this determination by using features describing the candidate predicate and argument. These features include features from the input sentence as well as features from part-of-speech tagging and base NP chunking from step 304. For example, features include the predicate headword and its part of speech and the argument headword and its part of speech. Additionally, there are features specifying headwords and part-of-speech tags surrounding the candidate predicate and argument, for example, a distance between headwords or a number of chunks to a headword. Other features consider the predicate and argument together, for example, whether the predicate or argument occurs first, the number of punctuations between the predicate and argument and the sequence of parts-of-speech of chunk headword between the predicate and the argument.
The ME model is an exponential model that can learn to find skeleton relations given the output of the part-of-speech tagger and base noun phrase chunker of the skeleton parser. The exponential model that is used is shown below as equation 1:
In equation 1, h is a candidate word pair and its surrounding context, t is a prediction, and f1, . . . , fk are characteristic functions each corresponding to a possible feature value holding in h and value of t. p(h,t) is the probability of h and t as determined by the ME model. It will further be appreciated that alternative components can be used for the ME model, such as different part-of-speech tags, distances between predicate and argument, etc. in accordance with the present invention. Moreover, the ME model may be replaced with other machine learning models, also in accordance with the present invention.
A central aspect of the skeleton parser described above is that the parser is trained on the output of a pre-existing parser, currently embodied by EZParse, a hand-crafted parser. A key advantage of this approach is that training data can be obtained without costly human-annotation. When the skeleton parser is trained with enough (easily-obtainable) data, the resulting parser has increased accuracy over the original EZParse parser. Use of the skeleton parser in applications that need skeleton relations is preferable to use of EZParse because the former is more than five times as fast.
Furthermore, learning more complex features rather than less complex ones from noisy data can increase the skeleton parser's accuracy. For example, using complex part-of-speech tags as features, though noisier, can be more effective than using simple part-of-speech tags. Additionally, a ME model can achieve high accuracy in finding grammatical relations. Also, experiments in selecting which features are to be used in the ME model can enhance the development of the skeleton parser. Features related to the candidate predicate and argument by their relative positions and by feature type, either word or part-of-speech, can serve to increase model accuracy.
Although the present invention has been described with reference to particular embodiments, workers skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the invention.