CA2614416C - Processing collocation mistakes in documents - Google Patents
Processing collocation mistakes in documents Download PDFInfo
- Publication number
- CA2614416C CA2614416C CA2614416A CA2614416A CA2614416C CA 2614416 C CA2614416 C CA 2614416C CA 2614416 A CA2614416 A CA 2614416A CA 2614416 A CA2614416 A CA 2614416A CA 2614416 C CA2614416 C CA 2614416C
- Authority
- CA
- Canada
- Prior art keywords
- sentence
- word
- queries
- query
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/253—Grammatical analysis; Style critique
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99936—Pattern matching access
Abstract
A sentence is accessed and at least one query is generated based on the sentence. At least one query can be compared to text within a collection of documents, for example using a web search engine. Collocation errors in the sentence can be detected and/or corrected based on the comparison of the at least one query and the text within the collection of documents.
Description
2 PROCESSING COLLOCATION MISTAKES IN
DOCUMENTS
BACKGROUND
The discussion below is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.
With an ever increasing global economy, and with the rapid development of the Internet, people all over the world are becoming increasingly familiar with writing in a language which is not their native language.
Unfortunately, for some societies that possess significantly different cultures and writing styles, the ability to write in some non-native languages is an ever-present barrier. When writing in a non-native language (for example English), language usage mistakes are frequently made by non-native speakers (for example, people who speak Chinese, Japanese, Korean or other non-English languages). These kind of mistakes can include both grammatical mistakes and improper usage of collocations such as verb-object, adjective-noun, adverb-verb, etc.
Many people have the ability to write in a non-native language using proper grammar, but they still may struggle with mistakes in collocations between two words. Still others struggle with both grammar and other mistakes such as collocations between two words. While spell checking and grammar checking programs are useful in correcting grammatical mistakes, detection and/or correction of mistakes in collocations between two words can be difficult, particularly since these mistakes can be otherwise grammatically correct. Therefore, grammar checkers typically provide very little assistance, if any, in detecting mistakes relating to the collocation between words. English is used as an example of the non-native language in the following discussion, but these problems persist across other language boundaries.
For example, consider the following sentences that contain collocation mistakes which cause the sentences to not be native-like English, even if otherwise grammatically correct.
1. Open the light.
2. Everybody hates the crowded traffic on weekends.
DOCUMENTS
BACKGROUND
The discussion below is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.
With an ever increasing global economy, and with the rapid development of the Internet, people all over the world are becoming increasingly familiar with writing in a language which is not their native language.
Unfortunately, for some societies that possess significantly different cultures and writing styles, the ability to write in some non-native languages is an ever-present barrier. When writing in a non-native language (for example English), language usage mistakes are frequently made by non-native speakers (for example, people who speak Chinese, Japanese, Korean or other non-English languages). These kind of mistakes can include both grammatical mistakes and improper usage of collocations such as verb-object, adjective-noun, adverb-verb, etc.
Many people have the ability to write in a non-native language using proper grammar, but they still may struggle with mistakes in collocations between two words. Still others struggle with both grammar and other mistakes such as collocations between two words. While spell checking and grammar checking programs are useful in correcting grammatical mistakes, detection and/or correction of mistakes in collocations between two words can be difficult, particularly since these mistakes can be otherwise grammatically correct. Therefore, grammar checkers typically provide very little assistance, if any, in detecting mistakes relating to the collocation between words. English is used as an example of the non-native language in the following discussion, but these problems persist across other language boundaries.
For example, consider the following sentences that contain collocation mistakes which cause the sentences to not be native-like English, even if otherwise grammatically correct.
1. Open the light.
2. Everybody hates the crowded traffic on weekends.
3. This is a check of US$ 500.
4. I congratulate you for your success.
The native-like English versions of these sentences should be like:
1. Turn on the light.
2. Everybody hates the heavy traffic on weekends.
3. This is a check for US$ 500.
4. I congratulate you on your success.
As an example of the barriers faced by non-native English speaking peoples, consider the plight of the Chinese user. By culture, background and thinking habits, Chinese people often produce English sentences which may be grammatical, but not natural. For example, Chinese people tend to directly translate subjects in Chinese into subjects in English, and do the same with objects and verbs. When writing in English, Chinese people often experience difficulty in deciding the collocations between verbs and prepositions, adjectives and nouns, verbs and nouns, etc. Moreover, in specific domains like the business domain, special writing skills and styles are needed.
Common dictionaries are mainly used by non-native speakers for the purpose of reading (a kind of decoding process), but these dictionaries do not provide enough support for writing (a kind of encoding process). They only provide the explanation of a single word, and they typically do not provide sufficient information to explain relevant phrases and collocations. Moreover, there is no easy way to get this kind of information from dictionaries, even if some of the information is provided in the dictionaries. On the other hand, current widely used grammar checking tools have some limited ability in detecting apt-to-make grammatical mistakes, but are not able to detect the collocation mistakes.
SUMMARY
This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
, , A sentence is accessed and at least one query is generated based on the sentence. At least one query can be compared to text within a collection of documents, for example using a web search engine. Collocation errors in the sentence can be detected and/or corrected based on the comparison of the at least one query and the text within the collection of documents.
According to one aspect of the present invention, there is provided a computer-implemented method of correcting collocation errors in a document, comprising: accessing a sentence of text in the document, using a processor; detecting one of a plurality of different checking word types in the sentence of text, the detected checking word type being indicative of a particular collocation error type; identifying the particular collocation error type based on the detected checking word type; accessing, in a memory, a set of query generation rules to select a set of rules, from a plurality of different sets of rules, for query generation based on the identified particular collocation error type indicated by the checking word type detected; generating a particular set of queries for the accessed sentence, using the selected set of query generation rules, the particular set of queries including a plurality of queries based on the sentence, each of the plurality of queries including a different set of text extracted from the sentence of text in the document wherein generating a set of queries comprises generating a query template that includes words from the extracted text with a placeholder for the word corresponding to the checking word type, and wherein generating the plurality of queries includes, a sentence query including the sentence without the identified word, a chunk query including a chunk of the sentence without , - 4a -the identified word, and a word query including a pair of words in the sentence without the identified word; submitting the plurality of queries to a search module that searches a collection of documents to obtain results; receiving the results from the search module that include text corresponding to text in the plurality of queries wherein the placeholder in the query template is matched by any word; comparing, in order, beginning with a first of the plurality of queries to a last of the plurality of queries, the text in the queries to the text in the results, only until a match occurs between the text in a query being compared and the text in the results; if the last of the plurality of queries is compared, without a match occurring, then detecting a collocation error based on the comparison of the text in the queries to the text in the results by identifying portions of the text in the queries that do not match word-for-word, with the text in the results; and providing an indication of the detected collocation error in the sentence of text and possible corrections for the detected collocation error.
According to another aspect of the present invention, there is provided a computer-implemented method of identifying and correcting collocation errors comprising: accessing, with a processor, a sentence of text in a document including a word that is identified as creating a given type of collocation error; identifying query generation rules, in memory, that indicate a set of queries to be generated, based on the given type of collocation error; generating a query template for each of the set of queries using the identified query generation rules, the set of queries including a plurality of queries including portions of the sentence without the identified word with a placeholder in the template in place of the identified - 4b -word, wherein generating the plurality of queries includes, a sentence query including the sentence without the identified word, a chunk query including a chunk of the sentence without the identified word, and a word query including a pair of words in the sentence without the identified word; submitting the plurality of queries to a search module to obtain search results corresponding to the portions of the sentence without the word; comparing the search results to the plurality of queries wherein the placeholder in the template matches any word in the search results; identifying at least one candidate replacement word for the identified word in the sentence based on a comparison of the queries with the search results; and providing an indication of the at least one candidate replacement word, for correction of the collocation error.
According to still another aspect of the present invention, there is provided a computer-implemented method of identifying collocation errors in text in the document, the method comprising: accessing a sentence of the text in the document using a processor; identifying a word having an error word type in the sentence that is indicative of a given type of collocation error; parsing the sentence to identify parts of speech contained therein; applying a set of query generation rules, chosen based on the given type of collocation error, to generate a plurality of queries based on the identified parts of speech, wherein the plurality of queries include a sentence query including the sentence, a chunk query, containing a plurality of words including a chunk of the sentence, and a word query including a pair of words in the sentence, by generating a query template, for each query, that includes words in the sentence adjacent to the error word and a placeholder for the error word; submitting the query template - 4c -to a search module to generate search results by searching for text that matches the words in the query template wherein the placeholder is matched by any word; comparing the plurality of queries, one-by-one, to the search results until text in a query template for a query being compared matches text in the search results; if the text in the query being compared matches text in the search results, then identifying a word in the search results that matches the placeholder as a replacement word; and outputting an indication that a collocation error exists, of the given collocation error type, and alternative text including the replacement word that likely corrects the collocation error.
According to another aspect of the present invention, there is provided a computer readable storage medium having computer executable instructions stored thereon for execution by one or more computers, that when executed implement a method as described above or below.
Other embodiments of the invention provide computer readable media having computer executable instructions stored thereon for execution by one or more computers, that when executed implement a method as summarized above or as detailed below.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of a general computing environment.
FIG. 2 is a flow diagram of a system for detecting and correcting collocation errors.
FIG. 3 is a flow diagram of a method for detecting and correcting collocation errors.
- 4d -FIG. 4 is a block diagram of a query generation module.
FIG. 5 is a flow diagram of a method for detecting collocation errors.
FIG. 6 is a flow diagram of a method for presenting candidate collocation corrections.
DETAILED DESCRIPTION
FIG. 1 illustrates an example of a suitable computing system environment 100 on which the invention may be implemented. The computing system environment 100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing environment 100 be interpreted as having any dependency or requirement relating to any one
The native-like English versions of these sentences should be like:
1. Turn on the light.
2. Everybody hates the heavy traffic on weekends.
3. This is a check for US$ 500.
4. I congratulate you on your success.
As an example of the barriers faced by non-native English speaking peoples, consider the plight of the Chinese user. By culture, background and thinking habits, Chinese people often produce English sentences which may be grammatical, but not natural. For example, Chinese people tend to directly translate subjects in Chinese into subjects in English, and do the same with objects and verbs. When writing in English, Chinese people often experience difficulty in deciding the collocations between verbs and prepositions, adjectives and nouns, verbs and nouns, etc. Moreover, in specific domains like the business domain, special writing skills and styles are needed.
Common dictionaries are mainly used by non-native speakers for the purpose of reading (a kind of decoding process), but these dictionaries do not provide enough support for writing (a kind of encoding process). They only provide the explanation of a single word, and they typically do not provide sufficient information to explain relevant phrases and collocations. Moreover, there is no easy way to get this kind of information from dictionaries, even if some of the information is provided in the dictionaries. On the other hand, current widely used grammar checking tools have some limited ability in detecting apt-to-make grammatical mistakes, but are not able to detect the collocation mistakes.
SUMMARY
This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
, , A sentence is accessed and at least one query is generated based on the sentence. At least one query can be compared to text within a collection of documents, for example using a web search engine. Collocation errors in the sentence can be detected and/or corrected based on the comparison of the at least one query and the text within the collection of documents.
According to one aspect of the present invention, there is provided a computer-implemented method of correcting collocation errors in a document, comprising: accessing a sentence of text in the document, using a processor; detecting one of a plurality of different checking word types in the sentence of text, the detected checking word type being indicative of a particular collocation error type; identifying the particular collocation error type based on the detected checking word type; accessing, in a memory, a set of query generation rules to select a set of rules, from a plurality of different sets of rules, for query generation based on the identified particular collocation error type indicated by the checking word type detected; generating a particular set of queries for the accessed sentence, using the selected set of query generation rules, the particular set of queries including a plurality of queries based on the sentence, each of the plurality of queries including a different set of text extracted from the sentence of text in the document wherein generating a set of queries comprises generating a query template that includes words from the extracted text with a placeholder for the word corresponding to the checking word type, and wherein generating the plurality of queries includes, a sentence query including the sentence without the identified word, a chunk query including a chunk of the sentence without , - 4a -the identified word, and a word query including a pair of words in the sentence without the identified word; submitting the plurality of queries to a search module that searches a collection of documents to obtain results; receiving the results from the search module that include text corresponding to text in the plurality of queries wherein the placeholder in the query template is matched by any word; comparing, in order, beginning with a first of the plurality of queries to a last of the plurality of queries, the text in the queries to the text in the results, only until a match occurs between the text in a query being compared and the text in the results; if the last of the plurality of queries is compared, without a match occurring, then detecting a collocation error based on the comparison of the text in the queries to the text in the results by identifying portions of the text in the queries that do not match word-for-word, with the text in the results; and providing an indication of the detected collocation error in the sentence of text and possible corrections for the detected collocation error.
According to another aspect of the present invention, there is provided a computer-implemented method of identifying and correcting collocation errors comprising: accessing, with a processor, a sentence of text in a document including a word that is identified as creating a given type of collocation error; identifying query generation rules, in memory, that indicate a set of queries to be generated, based on the given type of collocation error; generating a query template for each of the set of queries using the identified query generation rules, the set of queries including a plurality of queries including portions of the sentence without the identified word with a placeholder in the template in place of the identified - 4b -word, wherein generating the plurality of queries includes, a sentence query including the sentence without the identified word, a chunk query including a chunk of the sentence without the identified word, and a word query including a pair of words in the sentence without the identified word; submitting the plurality of queries to a search module to obtain search results corresponding to the portions of the sentence without the word; comparing the search results to the plurality of queries wherein the placeholder in the template matches any word in the search results; identifying at least one candidate replacement word for the identified word in the sentence based on a comparison of the queries with the search results; and providing an indication of the at least one candidate replacement word, for correction of the collocation error.
According to still another aspect of the present invention, there is provided a computer-implemented method of identifying collocation errors in text in the document, the method comprising: accessing a sentence of the text in the document using a processor; identifying a word having an error word type in the sentence that is indicative of a given type of collocation error; parsing the sentence to identify parts of speech contained therein; applying a set of query generation rules, chosen based on the given type of collocation error, to generate a plurality of queries based on the identified parts of speech, wherein the plurality of queries include a sentence query including the sentence, a chunk query, containing a plurality of words including a chunk of the sentence, and a word query including a pair of words in the sentence, by generating a query template, for each query, that includes words in the sentence adjacent to the error word and a placeholder for the error word; submitting the query template - 4c -to a search module to generate search results by searching for text that matches the words in the query template wherein the placeholder is matched by any word; comparing the plurality of queries, one-by-one, to the search results until text in a query template for a query being compared matches text in the search results; if the text in the query being compared matches text in the search results, then identifying a word in the search results that matches the placeholder as a replacement word; and outputting an indication that a collocation error exists, of the given collocation error type, and alternative text including the replacement word that likely corrects the collocation error.
According to another aspect of the present invention, there is provided a computer readable storage medium having computer executable instructions stored thereon for execution by one or more computers, that when executed implement a method as described above or below.
Other embodiments of the invention provide computer readable media having computer executable instructions stored thereon for execution by one or more computers, that when executed implement a method as summarized above or as detailed below.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of a general computing environment.
FIG. 2 is a flow diagram of a system for detecting and correcting collocation errors.
FIG. 3 is a flow diagram of a method for detecting and correcting collocation errors.
- 4d -FIG. 4 is a block diagram of a query generation module.
FIG. 5 is a flow diagram of a method for detecting collocation errors.
FIG. 6 is a flow diagram of a method for presenting candidate collocation corrections.
DETAILED DESCRIPTION
FIG. 1 illustrates an example of a suitable computing system environment 100 on which the invention may be implemented. The computing system environment 100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing environment 100 be interpreted as having any dependency or requirement relating to any one
-5-or combination of components illustrated in the exemplary operating environment 100.
The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, telephony systems, distributed computing environments that include any of the above systems or devices, and the like.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices. Tasks performed by the programs and modules are described below and with the aid of figures.
Those skilled in the art can implement the description and figures as processor executable instructions, which can be written on any form of a computer readable medium.
The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, telephony systems, distributed computing environments that include any of the above systems or devices, and the like.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices. Tasks performed by the programs and modules are described below and with the aid of figures.
Those skilled in the art can implement the description and figures as processor executable instructions, which can be written on any form of a computer readable medium.
-6-With reference to FIG. 1, an exemplary system for implementing the invention includes a general-purpose computing device in the form of a computer 110.
Components of computer 110 may include, but are not limited to, a processing unit 120, a system memory 130, and a system bus 121 that couples various system components including the system memory to the processing unit 120. The system bus 121 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.
Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes,
Components of computer 110 may include, but are not limited to, a processing unit 120, a system memory 130, and a system bus 121 that couples various system components including the system memory to the processing unit 120. The system bus 121 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.
Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes,
-7-magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 110. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.
The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation, FIG. 1 illustrates operating system 134, application programs 135, other program modules 136, and program data 137.
The computer 110 may also include other removable/non-removable volatile/nonvolatile computer
The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation, FIG. 1 illustrates operating system 134, application programs 135, other program modules 136, and program data 137.
The computer 110 may also include other removable/non-removable volatile/nonvolatile computer
-8-storage media. By way of example only, FIG. 1 illustrates a hard ,disk drive 141 that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive 151 that reads from or writes to a removable, nonvolatile magnetic disk 152, and an optical disk drive 155 that reads from or writes to a removable, nonvolatile optical disk 156 such as a CD ROM or other optical media.
Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive 141 is typically connected to the system bus 121 through a non-removable memory interface such as interface 140, and magnetic disk drive 151 and optical disk drive 155 are typically connected to the system bus 121 by a removable memory interface, such as interface 150.
The drives and their associated computer storage media discussed above and illustrated in FIG. 1, provide storage of computer readable instructions, data structures, program modules and other data for the computer 110. In FIG. 1, for example, hard disk drive 141 is illustrated as storing operating system 144, application programs 145, other program modules 146, and program data 147. Note that these components can either be the same as or different from operating system 134, application programs 135, other program modules 136, and program data 137. Operating system 144, application programs 145, other program modules 146, and program data 147 are given different numbers here to illustrate that, at a minimum, they are
Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive 141 is typically connected to the system bus 121 through a non-removable memory interface such as interface 140, and magnetic disk drive 151 and optical disk drive 155 are typically connected to the system bus 121 by a removable memory interface, such as interface 150.
The drives and their associated computer storage media discussed above and illustrated in FIG. 1, provide storage of computer readable instructions, data structures, program modules and other data for the computer 110. In FIG. 1, for example, hard disk drive 141 is illustrated as storing operating system 144, application programs 145, other program modules 146, and program data 147. Note that these components can either be the same as or different from operating system 134, application programs 135, other program modules 136, and program data 137. Operating system 144, application programs 145, other program modules 146, and program data 147 are given different numbers here to illustrate that, at a minimum, they are
-9-different copies.
A user may enter commands and information into the computer 110 through input devices such as a .keyboard 162, a microphone 163, and a pointing device 161, such as a mouse, trackball or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 120 through a user input interface 160 that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor 191 or other type of display device is also connected to the system bus 121 via an interface, such as a video interface 190. In addition to the monitor, computers may also include other peripheral output devices such as speakers 197 and printer 196, which may be connected through an output peripheral interface 190.
The computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be a personal computer, a hand-held device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 110. The logical connections depicted in FIG. 1 include a local area network (LAN) 171 and a wide area network (WAN) 173, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.
When used in a LAN networking environment, the
A user may enter commands and information into the computer 110 through input devices such as a .keyboard 162, a microphone 163, and a pointing device 161, such as a mouse, trackball or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 120 through a user input interface 160 that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor 191 or other type of display device is also connected to the system bus 121 via an interface, such as a video interface 190. In addition to the monitor, computers may also include other peripheral output devices such as speakers 197 and printer 196, which may be connected through an output peripheral interface 190.
The computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be a personal computer, a hand-held device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 110. The logical connections depicted in FIG. 1 include a local area network (LAN) 171 and a wide area network (WAN) 173, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.
When used in a LAN networking environment, the
-10-computer 110 is connected to the LAN 171 through a network interface or adapter 170. When used in a WAN
networking environment, the computer 110 typically includes a modem 172 or other means for establishing communications over the WAN 173, such as the Internet.
The modem 172, which may be internal or external, may be connected to the system bus 121 via the user input interface 160, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation, FIG. 1 illustrates remote application programs 185 as residing on remote computer 180. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.
FIG. 2 is a flow diagram of a system 200 for detecting and correcting collocation errors within text.
There are many types of collocation errors. In one aspect of system 200, four types of collocation errors are detected. The collocation error types include:
1. verb-noun (VN, e.g. *learn/acquire knowledge), 2, preposition-noun (PN, e.g. *on/in the morning), 3. adjective-noun (AN, e.g. *social/socialist country), and 4. verb-adverb (VA, e.g. situations change *largely/greatly).
Preprocessing module 202 processes text to provide part of speech tagging and parsing of the text. Many
networking environment, the computer 110 typically includes a modem 172 or other means for establishing communications over the WAN 173, such as the Internet.
The modem 172, which may be internal or external, may be connected to the system bus 121 via the user input interface 160, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation, FIG. 1 illustrates remote application programs 185 as residing on remote computer 180. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.
FIG. 2 is a flow diagram of a system 200 for detecting and correcting collocation errors within text.
There are many types of collocation errors. In one aspect of system 200, four types of collocation errors are detected. The collocation error types include:
1. verb-noun (VN, e.g. *learn/acquire knowledge), 2, preposition-noun (PN, e.g. *on/in the morning), 3. adjective-noun (AN, e.g. *social/socialist country), and 4. verb-adverb (VA, e.g. situations change *largely/greatly).
Preprocessing module 202 processes text to provide part of speech tagging and parsing of the text. Many
-11-different types of parsers can be used to process the text. Below is an example sentence:
I have recognized this person for years.
Preprocessing module 202 tags this sentence and chunks the sentence as follows:
(NP I/PRP][VP have/VBP recognized/VBNUNP this/DT
person/NN) [PP for/INJ(NP years. </s> INNS) Using the processed text, query generation module 204 constructs queries. In one example, four sets of queries are generated for each type of collocation error type identified above. For example, the collocation error types can be verb-noun, preposition-noun, adjective-noun and verb-adverb. The queries generated can include the full text of the sentence as well as a reduced portion of the sentence where auxiliaries are removed. Example reduced queries for the sentence above can include, "have recognized this person", 'have recognized", 'this person" and 'recognized person".
The queries are submitted to a search module 206. In one embodiment, -the search module can be a web based search engine such as MSN Search (search.msn.com), Google (www.google.com) and/or Yahoo! (www.yahoo.com). It is known to those skilled in the art that web based search engines search a collection of documents to obtain results therefrom. Portions of text similar to the queries are retrieved from the search module based on results from the query submission. Since the web includes a vast amount of text, it can be an inexpensive resource to detect collocation. errors. Error detection module 208 compares the queries generated by query generation module 204 to results obtained by search module 206. Error correction module 210 provides candidate corrections for the errors identified by error detection module 208.
FIG. 3 is a flow diagram of a method 220 that can be implemented in system 200 illustrated in FIG. 2. At step 222, a sentence is accessed. The sentence may include
I have recognized this person for years.
Preprocessing module 202 tags this sentence and chunks the sentence as follows:
(NP I/PRP][VP have/VBP recognized/VBNUNP this/DT
person/NN) [PP for/INJ(NP years. </s> INNS) Using the processed text, query generation module 204 constructs queries. In one example, four sets of queries are generated for each type of collocation error type identified above. For example, the collocation error types can be verb-noun, preposition-noun, adjective-noun and verb-adverb. The queries generated can include the full text of the sentence as well as a reduced portion of the sentence where auxiliaries are removed. Example reduced queries for the sentence above can include, "have recognized this person", 'have recognized", 'this person" and 'recognized person".
The queries are submitted to a search module 206. In one embodiment, -the search module can be a web based search engine such as MSN Search (search.msn.com), Google (www.google.com) and/or Yahoo! (www.yahoo.com). It is known to those skilled in the art that web based search engines search a collection of documents to obtain results therefrom. Portions of text similar to the queries are retrieved from the search module based on results from the query submission. Since the web includes a vast amount of text, it can be an inexpensive resource to detect collocation. errors. Error detection module 208 compares the queries generated by query generation module 204 to results obtained by search module 206. Error correction module 210 provides candidate corrections for the errors identified by error detection module 208.
FIG. 3 is a flow diagram of a method 220 that can be implemented in system 200 illustrated in FIG. 2. At step 222, a sentence is accessed. The sentence may include
-12-text that has been input into a word processor, for example Microsoft Word available from Microsoft Corporation of Redmond, Washington. At step 224, the sentence is parsed into chunks and parts of speech within the sentence are identified. Then, queries are generated based on the parse at step 226. At step 228, the queries are submitted to a search engine, for example MSN Search, Google and/or Yahoo!. Collocation errors in the sentence are detected at step 230 by comparing the queries and results from the search engine. After detecting errors, ranked candidates for .alternatives to the collocation errors are presented to the user at step 232.
FIG. 4 is a block diagram of query generation module 204 of FIG. 2.
Query generation module 204 accepts a parsed sentence 240, for example a parsed sentence received from preprocessing module 202. Based on parsed sentence 240, query generation module 204 generates sentence queries 242, chunk queries 244 and word queries 246. Given the types of potential collocation errors identified above, a checking word (i.e. a word that potentially causes a collocation error) is detected as follows: verb in type VN, preposition in type PN, adjective in type AN, and adverb in type VA. Depending on the type, query generation module 204 generates a different set of queries as follows:
1. Sentence queries 242: original sentence and reduced sentence (by removing auxiliaries pre-defined for each type), called S-Query, 2. Chunk queries 242: corresponding chunk pairs in the sentence, called C-Query, and 3. Word queries 246: corresponding headword pairs in the sentence, called W-Query.
FIG. 4 is a block diagram of query generation module 204 of FIG. 2.
Query generation module 204 accepts a parsed sentence 240, for example a parsed sentence received from preprocessing module 202. Based on parsed sentence 240, query generation module 204 generates sentence queries 242, chunk queries 244 and word queries 246. Given the types of potential collocation errors identified above, a checking word (i.e. a word that potentially causes a collocation error) is detected as follows: verb in type VN, preposition in type PN, adjective in type AN, and adverb in type VA. Depending on the type, query generation module 204 generates a different set of queries as follows:
1. Sentence queries 242: original sentence and reduced sentence (by removing auxiliaries pre-defined for each type), called S-Query, 2. Chunk queries 242: corresponding chunk pairs in the sentence, called C-Query, and 3. Word queries 246: corresponding headword pairs in the sentence, called W-Query.
-13-Example queries for the sentence "I have recognized this person for years" for type VN detection are presented below, where - means that two adjacent words can be adjacent or one-word away from each other.
S-Query: ["I have recognized this person for years"]
S-Query: ["have recognized this person") C-Query: ["have recognized" - "this person"]
W-Query: ["recognized" - "person"]
Example rules for generating queries of each type are as follows.
= VN: S-Queries, one C-Query V-N, and one W-Query Vh-Nh (Nh denotes a headword of a corresponding noun chunk).
= PN: one C-Query of PN, which contains the preposition;
= AN: one C-Query of AN, which contains the AN pair;
and = VA: C-Queries that contain VA pairs and W-Queries that contains VA headwords.
FIG. 5 is a flow diagram of a method 250 of detecting errors in a sentence.
Queries generated by query generation module 204 are submitted to the search module 206 at step 251. Search results obtained by the search module 206 are compared to the queries. In one example, the results include a summary of text for a document retrieved using a web search engine. At step 252, S-queries 242 from query generation module 204 are compared to results from the search module. Then, at step 254, a determination is made as to whether one or more of the S-queries 242 match the search module results. If one or more of the S-queries match the
S-Query: ["I have recognized this person for years"]
S-Query: ["have recognized this person") C-Query: ["have recognized" - "this person"]
W-Query: ["recognized" - "person"]
Example rules for generating queries of each type are as follows.
= VN: S-Queries, one C-Query V-N, and one W-Query Vh-Nh (Nh denotes a headword of a corresponding noun chunk).
= PN: one C-Query of PN, which contains the preposition;
= AN: one C-Query of AN, which contains the AN pair;
and = VA: C-Queries that contain VA pairs and W-Queries that contains VA headwords.
FIG. 5 is a flow diagram of a method 250 of detecting errors in a sentence.
Queries generated by query generation module 204 are submitted to the search module 206 at step 251. Search results obtained by the search module 206 are compared to the queries. In one example, the results include a summary of text for a document retrieved using a web search engine. At step 252, S-queries 242 from query generation module 204 are compared to results from the search module. Then, at step 254, a determination is made as to whether one or more of the S-queries 242 match the search module results. If one or more of the S-queries match the
-14-search module results, it is determined that no collocation error exists at step 256.
However, if a match does not exist, method 250 proceeds to step 258, wherein C-queries 244 are compared to the search module results. At step 260, it is determined whether one or more of the C-queries closely match the search module results and if a score for the comparison is greater than a threshold. In one example, the score is computed by dividing a number of times the chunk of the C-query appears in the search results by the number of times words in the C-query co-occur in the search results. If the score is greater than the threshold, it is determined that no collocation error exists at step 256.
If the score is less than the threshold, method 250 proceeds to step 262, wherein W-Queries are compared to the search engine data. Step 264 determines whether there is a close match between the W-queries and the search engine data and whether a score for the comparison is greater than a threshold. If the score is greater than the threshold, it is determined that no collocation error exists at step 256. The score for the comparison can be similar to the C-Query comparison score.
Thus, the W-Query comparison score can be calculated by dividing the number of times the W-Query occurs in the search results by the total number of times the pair of words in the W-Query co-occurs. If the score is less than the threshold, method 250 proceeds to step 266, wherein the user is notified of a potential collocation error.
FIG. 6 is a flow diagram of a method 270 for presenting potential corrected collocations to a user.
However, if a match does not exist, method 250 proceeds to step 258, wherein C-queries 244 are compared to the search module results. At step 260, it is determined whether one or more of the C-queries closely match the search module results and if a score for the comparison is greater than a threshold. In one example, the score is computed by dividing a number of times the chunk of the C-query appears in the search results by the number of times words in the C-query co-occur in the search results. If the score is greater than the threshold, it is determined that no collocation error exists at step 256.
If the score is less than the threshold, method 250 proceeds to step 262, wherein W-Queries are compared to the search engine data. Step 264 determines whether there is a close match between the W-queries and the search engine data and whether a score for the comparison is greater than a threshold. If the score is greater than the threshold, it is determined that no collocation error exists at step 256. The score for the comparison can be similar to the C-Query comparison score.
Thus, the W-Query comparison score can be calculated by dividing the number of times the W-Query occurs in the search results by the total number of times the pair of words in the W-Query co-occurs. If the score is less than the threshold, method 250 proceeds to step 266, wherein the user is notified of a potential collocation error.
FIG. 6 is a flow diagram of a method 270 for presenting potential corrected collocations to a user.
-15-At step 272 a query template is generated. The query templates are generated based on a word that has been identified as an error (i.e. the checking word above includes a collocation error as determined by method 250 in FIG. 5). Query templates are derived from the input sentence after the checking word causing the collocation error has been replaced with a "+". In the sentence above, "recognized" has been identified as the checking word and thus query templates are developed based on the word. For example, the query templates of the sentence "I have recognized this person for years", for VN
detection, are as follows, where + denotes any word.
S-QT: ["I have + this person for years"]
S-QT: PI have + this person"]
S-QT: ["have + this person for years"]
S-QT: ["I have + this person"]
C-QT: ["+ this person for years"]
C-QT: ["+ this person"]
Example rules for generating query templates can be as follows.
= VN: S-QT, C-QT (where the verb has been replaced with +).
= PN: S-QT, C-QT (where the preposition has been replaced with +);
= AN: S-QT, C-QT (where the adjective has been replaced with +); and = VA: S-QT, C-QT (where the adverb has been replaced with +).
detection, are as follows, where + denotes any word.
S-QT: ["I have + this person for years"]
S-QT: PI have + this person"]
S-QT: ["have + this person for years"]
S-QT: ["I have + this person"]
C-QT: ["+ this person for years"]
C-QT: ["+ this person"]
Example rules for generating query templates can be as follows.
= VN: S-QT, C-QT (where the verb has been replaced with +).
= PN: S-QT, C-QT (where the preposition has been replaced with +);
= AN: S-QT, C-QT (where the adjective has been replaced with +); and = VA: S-QT, C-QT (where the adverb has been replaced with +).
-16-At step 274, the query template is submitted to a search module, herein a search engine. At step 276, strings from the search engine results are retrieved.
The strings can include summaries of text that have words of surrounding context. Strings that match the query templates, where the position of can be any one word, are identified as strings candidates. Candidates that do not contain the collocation (which is formed by a word replacing + and another word in the string according to the collocation type) are removed at step 278. Remaining candidates are ranked according to a score based on a corresponding weight of query template that matched the string candidate. For example, the weight of the query template can be based on the number of words in the query template. The score for each candidate is calculated by taking a sum for weights across all summaries containing the candidates. The score for query templates (QT,) that retrieve the candidate can be expressed by:
Score(candidate) = Weight(QT) A ranked list of candidates is then presented to a user at step 280. For example, a pop-up menu can be used to present the ranked list. A user can choose one of the selections from the list to correct the collocation error.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts
The strings can include summaries of text that have words of surrounding context. Strings that match the query templates, where the position of can be any one word, are identified as strings candidates. Candidates that do not contain the collocation (which is formed by a word replacing + and another word in the string according to the collocation type) are removed at step 278. Remaining candidates are ranked according to a score based on a corresponding weight of query template that matched the string candidate. For example, the weight of the query template can be based on the number of words in the query template. The score for each candidate is calculated by taking a sum for weights across all summaries containing the candidates. The score for query templates (QT,) that retrieve the candidate can be expressed by:
Score(candidate) = Weight(QT) A ranked list of candidates is then presented to a user at step 280. For example, a pop-up menu can be used to present the ranked list. A user can choose one of the selections from the list to correct the collocation error.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts
-17-described above are disclosed as example forms of implementing the claims.
Claims (14)
1. A computer-implemented method of correcting collocation errors in a document, comprising:
accessing a sentence of text in the document, using a processor;
detecting one of a plurality of different checking word types in the sentence of text, the detected checking word type being indicative of a particular collocation error type;
identifying the particular collocation error type based on the detected checking word type;
accessing, in a memory, a set of query generation rules to select a set of rules, from a plurality of different sets of rules, for query generation based on the identified particular collocation error type indicated by the checking word type detected;
generating a particular set of queries for the accessed sentence, using the selected set of query generation rules, the particular set of queries including a plurality of queries based on the sentence, each of the plurality of queries including a different set of text extracted from the sentence of text in the document wherein generating a set of queries comprises generating a query template that includes words from the extracted text with a placeholder for the word corresponding to the checking word type, and wherein generating the plurality of queries includes, a sentence query including the sentence without the identified word, a chunk query including a chunk of the sentence without the identified word, and a word query including a pair of words in the sentence without the identified word;
submitting the plurality of queries to a search module that searches a collection of documents to obtain results; receiving the results from the search module that include text corresponding to text in the plurality of queries wherein the placeholder in the query template is matched by any word;
comparing, in order, beginning with a first of the plurality of queries to a last of the plurality of queries, the text in the queries to the text in the results, only until a match occurs between the text in a query being compared and the text in the results;
if the last of the plurality of queries is compared, without a match occurring, then detecting a collocation error based on the comparison of the text in the queries to the text in the results by identifying portions of the text in the queries that do not match word-for-word, with the text in the results; and providing an indication of the detected collocation error in the sentence of text and possible corrections for the detected collocation error.
accessing a sentence of text in the document, using a processor;
detecting one of a plurality of different checking word types in the sentence of text, the detected checking word type being indicative of a particular collocation error type;
identifying the particular collocation error type based on the detected checking word type;
accessing, in a memory, a set of query generation rules to select a set of rules, from a plurality of different sets of rules, for query generation based on the identified particular collocation error type indicated by the checking word type detected;
generating a particular set of queries for the accessed sentence, using the selected set of query generation rules, the particular set of queries including a plurality of queries based on the sentence, each of the plurality of queries including a different set of text extracted from the sentence of text in the document wherein generating a set of queries comprises generating a query template that includes words from the extracted text with a placeholder for the word corresponding to the checking word type, and wherein generating the plurality of queries includes, a sentence query including the sentence without the identified word, a chunk query including a chunk of the sentence without the identified word, and a word query including a pair of words in the sentence without the identified word;
submitting the plurality of queries to a search module that searches a collection of documents to obtain results; receiving the results from the search module that include text corresponding to text in the plurality of queries wherein the placeholder in the query template is matched by any word;
comparing, in order, beginning with a first of the plurality of queries to a last of the plurality of queries, the text in the queries to the text in the results, only until a match occurs between the text in a query being compared and the text in the results;
if the last of the plurality of queries is compared, without a match occurring, then detecting a collocation error based on the comparison of the text in the queries to the text in the results by identifying portions of the text in the queries that do not match word-for-word, with the text in the results; and providing an indication of the detected collocation error in the sentence of text and possible corrections for the detected collocation error.
2. The method of claim 1 and further comprising parsing the sentence to identify parts of speech contained therein and wherein generating the plurality of queries is based on the identified parts of speech.
3. The method of claim 1 wherein providing possible corrections comprises presenting candidates of alternative text, that are alternatives to the portion of text in the query that includes the detected collocation error.
4. The method of claim 3 and further comprising ranking the candidates.
5. The method of claim 1 wherein the type of the collocation error includes at least one of verb-noun, preposition-noun, adjective-noun and verb-adverb.
6. A computer-implemented method of identifying and correcting collocation errors comprising:
accessing, with a processor, a sentence of text in a document including a word that is identified as creating a given type of collocation error;
identifying query generation rules, in memory, that indicate a set of queries to be generated, based on the given type of collocation error;
generating a query template for each of the set of queries using the identified query generation rules, the set of queries including a plurality of queries including portions of the sentence without the identified word with a placeholder in the template in place of the identified word, wherein generating the plurality of queries includes, a sentence query including the sentence without the identified word, a chunk query including a chunk of the sentence without the identified word, and a word query including a pair of words in the sentence without the identified word;
submitting the plurality of queries to a search module to obtain search results corresponding to the portions of the sentence without the word;
comparing the search results to the plurality of queries wherein the placeholder in the template matches any word in the search results;
identifying at least one candidate replacement word for the identified word in the sentence based on a comparison of the queries with the search results; and providing an indication of the at least one candidate replacement word, for correction of the collocation error.
accessing, with a processor, a sentence of text in a document including a word that is identified as creating a given type of collocation error;
identifying query generation rules, in memory, that indicate a set of queries to be generated, based on the given type of collocation error;
generating a query template for each of the set of queries using the identified query generation rules, the set of queries including a plurality of queries including portions of the sentence without the identified word with a placeholder in the template in place of the identified word, wherein generating the plurality of queries includes, a sentence query including the sentence without the identified word, a chunk query including a chunk of the sentence without the identified word, and a word query including a pair of words in the sentence without the identified word;
submitting the plurality of queries to a search module to obtain search results corresponding to the portions of the sentence without the word;
comparing the search results to the plurality of queries wherein the placeholder in the template matches any word in the search results;
identifying at least one candidate replacement word for the identified word in the sentence based on a comparison of the queries with the search results; and providing an indication of the at least one candidate replacement word, for correction of the collocation error.
7. The method of claim 6 and further comprising identifying a list of candidate replacement words and ranking the list.
8. The method of claim 6 wherein the search module is a web based search engine.
9. The method of claim 6 wherein generating a plurality of queries includes generating a template of queries based on the sentence, chunks of the sentence and a pair of words in the sentence.
10. A computer-implemented method of identifying collocation errors in text in a document, the method comprising:
accessing a sentence of the text in the document using a processor;
identifying a word having an error word type in the sentence that is indicative of a given type of collocation error;
parsing the sentence to identify parts of speech contained therein;
applying a set of query generation rules, chosen based on the given type of collocation error, to generate a plurality of queries based on the identified parts of speech, wherein the plurality of queries include a sentence query including the sentence, a chunk query, containing a plurality of words including a chunk of the sentence, and a word query including a pair of words in the sentence, by generating a query template, for each query, that includes words in the sentence adjacent to the error word and a placeholder for the error word;
submitting the query template to a search module to generate search results by searching for text that matches the words in the query template wherein the placeholder is matched by any word;
comparing the plurality of queries, one-by-one, to the search results until text in a query template for a query being compared matches text in the search results;
if the text in the query being compared matches text in the search results, then identifying a word in the search results that matches the placeholder as a replacement word; and outputting an indication that a collocation error exists, of the given collocation error type, and alternative text including the replacement word that likely corrects the collocation error.
accessing a sentence of the text in the document using a processor;
identifying a word having an error word type in the sentence that is indicative of a given type of collocation error;
parsing the sentence to identify parts of speech contained therein;
applying a set of query generation rules, chosen based on the given type of collocation error, to generate a plurality of queries based on the identified parts of speech, wherein the plurality of queries include a sentence query including the sentence, a chunk query, containing a plurality of words including a chunk of the sentence, and a word query including a pair of words in the sentence, by generating a query template, for each query, that includes words in the sentence adjacent to the error word and a placeholder for the error word;
submitting the query template to a search module to generate search results by searching for text that matches the words in the query template wherein the placeholder is matched by any word;
comparing the plurality of queries, one-by-one, to the search results until text in a query template for a query being compared matches text in the search results;
if the text in the query being compared matches text in the search results, then identifying a word in the search results that matches the placeholder as a replacement word; and outputting an indication that a collocation error exists, of the given collocation error type, and alternative text including the replacement word that likely corrects the collocation error.
11. The method of claim 10 wherein parsing the sentence includes identifying at least one of a verb-noun pair, a preposition-noun pair, an adjective-noun pair and a verb-adverb pair.
12. The method of claim 10 wherein identifying an error word type comprises identifying an error word in the sentence that causes the collocation error.
13. The method of claim 10 and further comprising:
determining a score based on a comparison of at least one of the chunk query and the word query with the search results; and determining whether a collocation error occurs in the sentence based on the score.
determining a score based on a comparison of at least one of the chunk query and the word query with the search results; and determining whether a collocation error occurs in the sentence based on the score.
14. A computer readable storage medium having computer executable instructions stored thereon for execution by one or more computers, that when executed implement a method according to any one of claims 1 to 13.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11/177,136 | 2005-07-08 | ||
| US11/177,136 US7574348B2 (en) | 2005-07-08 | 2005-07-08 | Processing collocation mistakes in documents |
| PCT/US2006/026012 WO2007008492A2 (en) | 2005-07-08 | 2006-06-30 | Processing collocation mistakes in documents |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2614416A1 CA2614416A1 (en) | 2007-01-18 |
| CA2614416C true CA2614416C (en) | 2014-05-27 |
Family
ID=37619276
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2614416A Expired - Fee Related CA2614416C (en) | 2005-07-08 | 2006-06-30 | Processing collocation mistakes in documents |
Country Status (10)
| Country | Link |
|---|---|
| US (1) | US7574348B2 (en) |
| EP (1) | EP1899835B1 (en) |
| JP (1) | JP5362353B2 (en) |
| KR (1) | KR20080023341A (en) |
| CN (1) | CN101218573A (en) |
| AU (1) | AU2006269494A1 (en) |
| CA (1) | CA2614416C (en) |
| MX (1) | MX2008000176A (en) |
| NO (1) | NO20080112L (en) |
| WO (1) | WO2007008492A2 (en) |
Families Citing this family (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100837750B1 (en) * | 2006-08-25 | 2008-06-13 | 엔에이치엔(주) | Method for searching chinese language using tone signs and system for executing the method |
| US7774193B2 (en) * | 2006-12-05 | 2010-08-10 | Microsoft Corporation | Proofing of word collocation errors based on a comparison with collocations in a corpus |
| CA2679094A1 (en) * | 2007-02-23 | 2008-08-28 | 1698413 Ontario Inc. | System and method for delivering content and advertisements |
| KR100978581B1 (en) * | 2008-05-08 | 2010-08-27 | 엔에이치엔(주) | Method and system for providing convenient dictionary service while a user is reading a webpage |
| US8473278B2 (en) * | 2008-07-24 | 2013-06-25 | Educational Testing Service | Systems and methods for identifying collocation errors in text |
| US20100082324A1 (en) * | 2008-09-30 | 2010-04-01 | Microsoft Corporation | Replacing terms in machine translation |
| US8484014B2 (en) * | 2008-11-03 | 2013-07-09 | Microsoft Corporation | Retrieval using a generalized sentence collocation |
| TWI403911B (en) * | 2008-11-28 | 2013-08-01 | Inst Information Industry | Chinese dictionary constructing apparatus and methods, and storage media |
| US8250072B2 (en) * | 2009-03-06 | 2012-08-21 | Dmitri Asonov | Detecting real word typos |
| CN101930594B (en) * | 2010-04-14 | 2012-05-23 | 山东山大鸥玛软件有限公司 | Rapid correction method for scanning document image |
| WO2011137386A1 (en) * | 2010-04-30 | 2011-11-03 | Orbis Technologies, Inc. | Systems and methods for semantic search, content correlation and visualization |
| US10496714B2 (en) * | 2010-08-06 | 2019-12-03 | Google Llc | State-dependent query response |
| US9262397B2 (en) | 2010-10-08 | 2016-02-16 | Microsoft Technology Licensing, Llc | General purpose correction of grammatical and word usage errors |
| US8855997B2 (en) | 2011-07-28 | 2014-10-07 | Microsoft Corporation | Linguistic error detection |
| US9015080B2 (en) | 2012-03-16 | 2015-04-21 | Orbis Technologies, Inc. | Systems and methods for semantic inference and reasoning |
| US8484017B1 (en) | 2012-09-10 | 2013-07-09 | Google Inc. | Identifying media content |
| US20140074466A1 (en) | 2012-09-10 | 2014-03-13 | Google Inc. | Answering questions using environmental context |
| US9189531B2 (en) | 2012-11-30 | 2015-11-17 | Orbis Technologies, Inc. | Ontology harmonization and mediation systems and methods |
| CN103365838B (en) * | 2013-07-24 | 2016-04-20 | 桂林电子科技大学 | Based on the english composition grammar mistake method for automatically correcting of diverse characteristics |
| US9298695B2 (en) | 2013-09-05 | 2016-03-29 | At&T Intellectual Property I, Lp | Method and apparatus for managing auto-correction in messaging |
| CN103678714B (en) * | 2013-12-31 | 2017-05-10 | 北京百度网讯科技有限公司 | Construction method and device for entity knowledge base |
| US20160087929A1 (en) * | 2014-09-24 | 2016-03-24 | Zoho Corporation Private Limited | Methods and apparatus for document creation via email |
| US10691709B2 (en) | 2015-10-28 | 2020-06-23 | Open Text Sa Ulc | System and method for subset searching and associated search operators |
| US10241716B2 (en) | 2017-06-30 | 2019-03-26 | Microsoft Technology Licensing, Llc | Global occupancy aggregator for global garbage collection scheduling |
| US11556527B2 (en) | 2017-07-06 | 2023-01-17 | Open Text Sa Ulc | System and method for value based region searching and associated search operators |
| US10824686B2 (en) * | 2018-03-05 | 2020-11-03 | Open Text Sa Ulc | System and method for searching based on text blocks and associated search operators |
| US11551006B2 (en) * | 2019-09-09 | 2023-01-10 | International Business Machines Corporation | Removal of personality signatures |
Family Cites Families (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH083815B2 (en) * | 1985-10-25 | 1996-01-17 | 株式会社日立製作所 | Natural language co-occurrence relation dictionary maintenance method |
| GB8625468D0 (en) | 1986-10-24 | 1987-04-15 | Smiths Industries Plc | Speech recognition apparatus |
| US4868750A (en) * | 1987-10-07 | 1989-09-19 | Houghton Mifflin Company | Collocational grammar system |
| US5251129A (en) * | 1990-08-21 | 1993-10-05 | General Electric Company | Method for automated morphological analysis of word structure |
| US5541836A (en) * | 1991-12-30 | 1996-07-30 | At&T Corp. | Word disambiguation apparatus and methods |
| US5383120A (en) * | 1992-03-02 | 1995-01-17 | General Electric Company | Method for tagging collocations in text |
| US5617488A (en) * | 1995-02-01 | 1997-04-01 | The Research Foundation Of State University Of New York | Relaxation word recognizer |
| US5887120A (en) * | 1995-05-31 | 1999-03-23 | Oracle Corporation | Method and apparatus for determining theme for discourse |
| US5680511A (en) * | 1995-06-07 | 1997-10-21 | Dragon Systems, Inc. | Systems and methods for word recognition |
| US5721938A (en) | 1995-06-07 | 1998-02-24 | Stuckey; Barbara K. | Method and device for parsing and analyzing natural language sentences and text |
| US5907839A (en) * | 1996-07-03 | 1999-05-25 | Yeda Reseach And Development, Co., Ltd. | Algorithm for context sensitive spelling correction |
| US6173298B1 (en) * | 1996-09-17 | 2001-01-09 | Asap, Ltd. | Method and apparatus for implementing a dynamic collocation dictionary |
| CN1193779A (en) * | 1997-03-13 | 1998-09-23 | 国际商业机器公司 | Method for dividing sentences in Chinese language into words and its use in error checking system for texts in Chinese language |
| GB2329047A (en) | 1997-09-05 | 1999-03-10 | Sharp Kk | A method of identifying collocates |
| KR980004126A (en) | 1997-12-16 | 1998-03-30 | 양승택 | Query Language Conversion Apparatus and Method for Searching Multilingual Web Documents |
| GB2334115A (en) | 1998-01-30 | 1999-08-11 | Sharp Kk | Processing text eg for approximate translation |
| US6216123B1 (en) * | 1998-06-24 | 2001-04-10 | Novell, Inc. | Method and system for rapid retrieval in a full text indexing system |
| GB9821787D0 (en) * | 1998-10-06 | 1998-12-02 | Data Limited | Apparatus for classifying or processing data |
| JP2001101186A (en) * | 1999-09-30 | 2001-04-13 | Oki Electric Ind Co Ltd | Machine translating device |
| GB0006721D0 (en) | 2000-03-20 | 2000-05-10 | Mitchell Thomas A | Assessment methods and systems |
| US7860706B2 (en) * | 2001-03-16 | 2010-12-28 | Eli Abir | Knowledge system method and appparatus |
| US20020152219A1 (en) | 2001-04-16 | 2002-10-17 | Singh Monmohan L. | Data interexchange protocol |
| US7269546B2 (en) * | 2001-05-09 | 2007-09-11 | International Business Machines Corporation | System and method of finding documents related to other documents and of finding related words in response to a query to refine a search |
| US7003444B2 (en) * | 2001-07-12 | 2006-02-21 | Microsoft Corporation | Method and apparatus for improved grammar checking using a stochastic parser |
| US7246060B2 (en) * | 2001-11-06 | 2007-07-17 | Microsoft Corporation | Natural input recognition system and method using a contextual mapping engine and adaptive user bias |
| US20030154071A1 (en) | 2002-02-11 | 2003-08-14 | Shreve Gregory M. | Process for the document management and computer-assisted translation of documents utilizing document corpora constructed by intelligent agents |
| KR100530154B1 (en) | 2002-06-07 | 2005-11-21 | 인터내셔널 비지네스 머신즈 코포레이션 | Method and Apparatus for developing a transfer dictionary used in transfer-based machine translation system |
| US7031911B2 (en) | 2002-06-28 | 2006-04-18 | Microsoft Corporation | System and method for automatic detection of collocation mistakes in documents |
| US7171351B2 (en) * | 2002-09-19 | 2007-01-30 | Microsoft Corporation | Method and system for retrieving hint sentences using expanded queries |
| US7249012B2 (en) * | 2002-11-20 | 2007-07-24 | Microsoft Corporation | Statistical method and apparatus for learning translation relationships among phrases |
| US7689412B2 (en) | 2003-12-05 | 2010-03-30 | Microsoft Corporation | Synonymous collocation extraction using translation information |
| US7707039B2 (en) * | 2004-02-15 | 2010-04-27 | Exbiblio B.V. | Automatic modification of web pages |
| US20060282255A1 (en) * | 2005-06-14 | 2006-12-14 | Microsoft Corporation | Collocation translation from monolingual and available bilingual corpora |
| US20070016397A1 (en) * | 2005-07-18 | 2007-01-18 | Microsoft Corporation | Collocation translation using monolingual corpora |
-
2005
- 2005-07-08 US US11/177,136 patent/US7574348B2/en not_active Expired - Fee Related
-
2006
- 2006-06-30 WO PCT/US2006/026012 patent/WO2007008492A2/en active Application Filing
- 2006-06-30 CN CNA2006800248782A patent/CN101218573A/en active Pending
- 2006-06-30 MX MX2008000176A patent/MX2008000176A/en not_active Application Discontinuation
- 2006-06-30 AU AU2006269494A patent/AU2006269494A1/en not_active Abandoned
- 2006-06-30 EP EP06774479.7A patent/EP1899835B1/en active Active
- 2006-06-30 KR KR1020087000528A patent/KR20080023341A/en not_active IP Right Cessation
- 2006-06-30 CA CA2614416A patent/CA2614416C/en not_active Expired - Fee Related
- 2006-06-30 JP JP2008520339A patent/JP5362353B2/en not_active Expired - Fee Related
-
2008
- 2008-01-08 NO NO20080112A patent/NO20080112L/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2007008492A2 (en) | 2007-01-18 |
| CN101218573A (en) | 2008-07-09 |
| US20070010992A1 (en) | 2007-01-11 |
| EP1899835A2 (en) | 2008-03-19 |
| WO2007008492A3 (en) | 2007-06-21 |
| EP1899835A4 (en) | 2017-10-25 |
| NO20080112L (en) | 2008-02-01 |
| JP5362353B2 (en) | 2013-12-11 |
| EP1899835B1 (en) | 2019-06-26 |
| MX2008000176A (en) | 2008-04-02 |
| US7574348B2 (en) | 2009-08-11 |
| JP2009500754A (en) | 2009-01-08 |
| AU2006269494A1 (en) | 2007-01-18 |
| KR20080023341A (en) | 2008-03-13 |
| CA2614416A1 (en) | 2007-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2614416C (en) | Processing collocation mistakes in documents | |
| US6424983B1 (en) | Spelling and grammar checking system | |
| US7970600B2 (en) | Using a first natural language parser to train a second parser | |
| US8321201B1 (en) | Identifying a synonym with N-gram agreement for a query phrase | |
| Rayson | Matrix: A statistical method and software tool for linguistic analysis through corpus comparison | |
| US7788085B2 (en) | Smart string replacement | |
| US8285541B2 (en) | System and method for handling multiple languages in text | |
| US20090157386A1 (en) | Diagnostic evaluation of machine translators | |
| US20100332217A1 (en) | Method for text improvement via linguistic abstractions | |
| JP2005267638A (en) | System and method for improved spell checking | |
| Erdmann et al. | Improving the extraction of bilingual terminology from Wikipedia | |
| KR20150017507A (en) | Method and system for Context-sensitive Spelling Correction Rules using Korean WordNet | |
| Way et al. | wEBMT: developing and validating an example-based machine translation system using the world wide web | |
| Vilares et al. | Studying the effect and treatment of misspelled queries in Cross-Language Information Retrieval | |
| Vilares et al. | Managing misspelled queries in IR applications | |
| Darwish | Probabilistic methods for searching OCR-degraded Arabic text | |
| Lazarinis et al. | Current research issues and trends in non-English Web searching | |
| Koleva et al. | An automatic part-of-speech tagger for Middle Low German | |
| Melero et al. | Holaaa!! writin like u talk is kewl but kinda hard 4 NLP | |
| Nwesri | Effective retrieval techniques for Arabic text | |
| JP2003323425A (en) | Parallel translation dictionary creating device, translation device, parallel translation dictionary creating program, and translation program | |
| Naemi et al. | Informal-to-formal word conversion for persian language using natural language processing techniques | |
| Wang | Matching meaning for cross-language information retrieval | |
| Vasuki et al. | English to Tamil machine translation system using parallel corpus | |
| Cucerzan | A case study of using web search statistics: case restoration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |
Effective date: 20190702 |