CA1288871C - Method for verifying spelling of compound words - Google Patents
Method for verifying spelling of compound wordsInfo
- Publication number
- CA1288871C CA1288871C CA000557768A CA557768A CA1288871C CA 1288871 C CA1288871 C CA 1288871C CA 000557768 A CA000557768 A CA 000557768A CA 557768 A CA557768 A CA 557768A CA 1288871 C CA1288871 C CA 1288871C
- Authority
- CA
- Canada
- Prior art keywords
- word
- dictionary
- words
- stated
- compound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/191—Automatic line break hyphenation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/232—Orthographic correction, e.g. spell checking or vowelisation
Abstract
METHOD FOR VERIFYING SPELLING OF COMPOUND WORDS
Abstract This invention describes a method for automatically verifying spelling of compound words in many natural languages such as German, Danish, Swedish, Norwegian, Dutch, Icelandic, Afrikaans, Swiss German, etc. The basic technology of looking up words in a dictionary is supplemented by the association of component flags with each word and by the application of powerful tree-scanning techniques that isolate the components of compound words and determine their correctness in isolation and in association with each other. The technique can be used in word processing systems to support spelling verification, to hyphenate text, and to unhyphenate text.
Abstract This invention describes a method for automatically verifying spelling of compound words in many natural languages such as German, Danish, Swedish, Norwegian, Dutch, Icelandic, Afrikaans, Swiss German, etc. The basic technology of looking up words in a dictionary is supplemented by the association of component flags with each word and by the application of powerful tree-scanning techniques that isolate the components of compound words and determine their correctness in isolation and in association with each other. The technique can be used in word processing systems to support spelling verification, to hyphenate text, and to unhyphenate text.
Description
~ ~9-86-011 Description METHOD FO~ VERIFYING SPELLING OF COMPOUND WORDS
Backqround of the Invention 1. Technical Field The invention disclosed broadly relates to data processing and more particularly relates to linguistic applications in data processing.
Backqround of the Invention 1. Technical Field The invention disclosed broadly relates to data processing and more particularly relates to linguistic applications in data processing.
2. Backqround_Art Text processing word processing systems have been developed for both stand-alone applications and distributed processing applications. The terms text processing and word processing will be used interchangeably herein to refer to data processing systems primarily used for the creation, editing, communication, and/or printing of alphanumeric character strings composing written text. A particular distributed processing system for word processing is disclosed in the copending Canadian Application No. 509,372, filed June 16, 1986 entitled "Multilingual Processing f~r Screen Image Build and Command Decode in a Word Processor, with Full Command, Message and Help Support," by K. W. Borgendale, et al.
~. ` ' . .
Three techniques for verifying spelling of compound words have been used by word processing systems. One prior art technique contains all the compound words that the system was able to verify stored in a dictionary data base. Verification consists of checking the dictionary for a match. An obvious limitation of this technique is the enormous amount of storage required to obtain passable coverage. Comprehensive coverage is impossible, particularly in the Germanic languages, because word compounding is used so extensively that a dictionary of all meaningful compounds cannot be constructed.
A second prior art technique described in the copending U.~. Patent No. 4,672~571, issued June 9, 1987 and assigned to the IBM Corporation, consists of parsing, or separating, the constituent words of the compound and then checking them against the words stored in the dictionary data ~ ~8~
base. This technique is the only practical way of obtaining adequate verification of compound words, but the approach is prone to problems such as false coordination of components and imprecise determination of the "joints" between the word components. In this technique, certain letter pairs which had :
a high probability of being the "joints" between components were used-as clues for breaking the wor~s and then verifying the parts against the dictionary. In terms of system performance, since any unrecognized word must be parsed before it can be marked as misspelled, the parser must have a large number of break points in order to verify correct compounds.
Thus, the identification of incorrect words is slowed down and degrades the performance of the system. Also, since languages that use compound words have longer average word lengths than non-compounding languages, the wasted computer time to try all the combinations allowed by the list of "joint" letter pairs can be considerable. As mentioned above, this second parsing technique suffers from false coordination errors. That is, a misspelled word consisting of two correctly spelled components will be considered correct. For example, if the word "overtime" is misspelled as "evertime" the word would be considered correct by this technique since "ever" and "time"
are both correctly spelled components. Similarly, run-on words such as "thatis" will be verified as "correct" compounds.
Ambiguities in identifying the components of a compound can lead to incorrect hyphenation. For example, "snakeskin" may be interpreted as "snakes-kin."
7~
~A9--86--011--3--The shortcomings of the prior art techniques have made it necessary to develop a more efficient algorithm and a comprehensive set of compound codes to handle agglutinative languages in computerized applications adequately.
Objects of the Invention It is therefore an object of the invention to provide an improved technique for verifying the spelling of compound words.
It is yet a further object of the invention to provide an improved technique for verifying the spelling of compound words, which also accounts for letter elisions.
It is still a further object of the invention to provide an improved technique for verifying the spelling of compound words which i9 more efficient and comprehensive in the handling of agglutinative languages, than has been available in the prior art.
Summary of the Invention These and other objects, features and advantages of the invention are achieved by the method for verifying spelling of compound words disclosed herein~ This invention describes a method for automatically verifying spelling of compound words in many natural languages such as German, Danish, Swedish, Norwegian, Dutch, Icelandic, Afrikaans, Swiss German, etc. The basic technology of looking up words in a dictionary is ~9-86-011 37~
supplemented by the association of component flags with each word and by the application of powerful tree-scanning techniques that isolate the components of compound words and determine their correctness in isolation and in association with each other. The technique can be used in word processing systems to support spelling verification, to hyphenate text, and to unhyphenate text.
Brief Description of the Drawings These and other objects, features and advantages of the invention will be more fully appreciated with reference to the accompanying figures.
Fig. 1 illustrates an example of the various candidate substrings possible in the word "isomorphically."
Fig. 2 is a tree diagram of the analysis of the word "isomorphically" i]lustrated in Fig. 1.
Fig. 3 is a flow diagram of the invention.
Fig. 4 is an example of tree scanning.
Description of the Best Mode for Carrying Out the Invention The invention is a technique for parsing compound words by matching the input word against dictionary words to obtain those that match a portion of the input word and have appropriate compounding codes. When the input word does not match any dictionary word exactly, the process is repeated with the remaining portion of the input word until either a match is ~ound and all the components have been identified, or all possibilities have been explored and none is satis~actory.
Two aspects of the invention are a comprehensive set of compound flags associated with each dictionary entry and an efficient method for examining the input word systematically.
These two aspects are described in detail.
The Compound Flags Linguistic and theoretical analysis of compound words has shown that each component may have any of four independent characteristics relative to compound formation: 1) it can occur as an isolated word, 2) it can be a front component or prefix, 3) it can be a middle component of a word, and 4) it can be a back component or suffix. These four characteristics can occur in any combination for a total of 15 possibilities, the case where none of the characteristics apply never being used. The word "sun," for example, might be tagged with ~9-86-011 _5_ 7~
characteristics 1 and 2 since it can occur as an isolated word or as a prefix in words like ~sunglasses~ or suntan." The word "glasses," on the other hand, would have characteristics 1 and 4 since it can be used either isolated or as a suffix. A word like "pseudo" would be marked with characteristic 2 since it :
can only be a prefix. The word "intangible" would have characteristic 1 because it only occurs as an isolated word.
For some German language examples, the combination of features 1~3+4 can be associated with the word "Sandale" which can occur alone or as a middle or back component of a word:
Example of word "Sandale" with features 1+3+4:
Leder ~ Sandale + Reparatur = Ledersandalereparatur Example of word "ig" with features 3+4:
Kurz + Bein + ig ~ keit - Kurzbeinigkeit Additional codes may be used to indicate the type of morphological transformation that a word can undergo.
The dictionary containing the compounding characteristics for each word can be generated either manually or automatically. The automatic process requires the decomposition of correct compound words into their components using the parsing algorithm given below while keeping track of the position of each component within the word. The segmented words can be reviewed manually to assure that no compounding codes for incorrectly segmented words are admitted into the dictionary.
The Parsing Algorithm The parsing algorithm is a pxocess for systematically exploring all possible word breaks efficiently. The algorithm is based on "preorder" scanning of trees. Although it is not obvious that this type of scanning is applicable to word parsing, the application is made possible by recognizing that the components of a compound word can be ordered so as to map "isomorphically" with a tree of variable depth.
Consider, for example, the word "isomorphically" as a compound word. If we examine this word for the substrings that can match against the dictionary we will find the words that are capitalized in Fig. 1.
~ 8~ 7~
The list of words in Fig 1 can be represented as a tree of the form indicated in Fig. 2. The tree in Fig. 2 can be scanned in preorder sequence to examine all the possibilities that can be used for word formation. However, some branches of the tree can be eliminated from consideration at the outset since the words forming the branches will have inappropriate com~ound flags. For example, it is not even necessary to check beyond the word "IS" since it can only occur as an isolated ~ord.
In practice, at each branching point in the tree, the dictionary is examined for all words starting with those particular letters to optimize dictionary access. The candidate words are considered by applying the algorithm recursively or by placing the words in a stack that is processed iteratively. Either of these two approaches will produce equivalent results.
A "stack," in computer terminology, is an array where information is stored in a prescribed sequence. Generally, the last item stored in the stack is the first one retrieved (analogous to a stack of coins on a table). The implementation of the invention with a stack requires that as the dictionary is scanned for substrings that match the word, the position and length of each matching dictionary word is stored in the stack.
~t is not necessary to save the matching dictionary word, since it is a substring of the compound word. When all suitable matches have been found (as determined by examination of the flags associated with the dictionary entries), the characters of the compound word not included in the last word added to the stack are matched against the dictionary again, and the process is repeated until a suitable match that encompasses all the remaining characters of the compound word are found. However, if no suitable matches are found, the unsuccessful words at the top of the stack are removed and another attempt is made with the next word in the stack. If the stack is empty before a suitable match is found, the process finishes without identifying any components.
If certain language-specific characteristics are present, morphological adaptation procedures are applied to analyze for elision of consonants (as for "schiffahrt") or to convert multiple letter combinations such as "ue" into a "u" with an umlaut depending on the requirements of the language. In a German hyphenated word like "zuk-ker" the removal of the hyphen also changes the spelling to "zucker."
A stack-based implementation of the algorithm is given 4~ below in a pseudo-code similar to PL/1:
~2~
TABLE I
TREE SCANNING PROCEDURE
FOR PARSING COMPOUND WORDS
__________________________________~____________________________. _____ ___ _________________________________________________________________________ /" GENERAL PARAMETERS */
DCI, WORD ~ CHAR(MXLENGTH);/* Word to be decompounded */
DCL MXLENGTH FIXED BIN(15); /* Maxlmum word length */
~CL WORDLEN FIXED BIN(15); /* Length of WORD */
DCL SWORD CHAR(MXLENGTH);/* Subs~ring of word */
/* STACK FOR BACKTRACKING */
DCL 1 STACK, 2 TABNOMAX FIXED BIN(15), /* Maximum no. of components */
2 TAGMAX FIXED BIN(15), /* Maximum no. of substrings for */
/* a component level */
2 TABNO FIXED BIN(15), /* No. of current component level */
TABLE(TABNOMAXJ
~. ` ' . .
Three techniques for verifying spelling of compound words have been used by word processing systems. One prior art technique contains all the compound words that the system was able to verify stored in a dictionary data base. Verification consists of checking the dictionary for a match. An obvious limitation of this technique is the enormous amount of storage required to obtain passable coverage. Comprehensive coverage is impossible, particularly in the Germanic languages, because word compounding is used so extensively that a dictionary of all meaningful compounds cannot be constructed.
A second prior art technique described in the copending U.~. Patent No. 4,672~571, issued June 9, 1987 and assigned to the IBM Corporation, consists of parsing, or separating, the constituent words of the compound and then checking them against the words stored in the dictionary data ~ ~8~
base. This technique is the only practical way of obtaining adequate verification of compound words, but the approach is prone to problems such as false coordination of components and imprecise determination of the "joints" between the word components. In this technique, certain letter pairs which had :
a high probability of being the "joints" between components were used-as clues for breaking the wor~s and then verifying the parts against the dictionary. In terms of system performance, since any unrecognized word must be parsed before it can be marked as misspelled, the parser must have a large number of break points in order to verify correct compounds.
Thus, the identification of incorrect words is slowed down and degrades the performance of the system. Also, since languages that use compound words have longer average word lengths than non-compounding languages, the wasted computer time to try all the combinations allowed by the list of "joint" letter pairs can be considerable. As mentioned above, this second parsing technique suffers from false coordination errors. That is, a misspelled word consisting of two correctly spelled components will be considered correct. For example, if the word "overtime" is misspelled as "evertime" the word would be considered correct by this technique since "ever" and "time"
are both correctly spelled components. Similarly, run-on words such as "thatis" will be verified as "correct" compounds.
Ambiguities in identifying the components of a compound can lead to incorrect hyphenation. For example, "snakeskin" may be interpreted as "snakes-kin."
7~
~A9--86--011--3--The shortcomings of the prior art techniques have made it necessary to develop a more efficient algorithm and a comprehensive set of compound codes to handle agglutinative languages in computerized applications adequately.
Objects of the Invention It is therefore an object of the invention to provide an improved technique for verifying the spelling of compound words.
It is yet a further object of the invention to provide an improved technique for verifying the spelling of compound words, which also accounts for letter elisions.
It is still a further object of the invention to provide an improved technique for verifying the spelling of compound words which i9 more efficient and comprehensive in the handling of agglutinative languages, than has been available in the prior art.
Summary of the Invention These and other objects, features and advantages of the invention are achieved by the method for verifying spelling of compound words disclosed herein~ This invention describes a method for automatically verifying spelling of compound words in many natural languages such as German, Danish, Swedish, Norwegian, Dutch, Icelandic, Afrikaans, Swiss German, etc. The basic technology of looking up words in a dictionary is ~9-86-011 37~
supplemented by the association of component flags with each word and by the application of powerful tree-scanning techniques that isolate the components of compound words and determine their correctness in isolation and in association with each other. The technique can be used in word processing systems to support spelling verification, to hyphenate text, and to unhyphenate text.
Brief Description of the Drawings These and other objects, features and advantages of the invention will be more fully appreciated with reference to the accompanying figures.
Fig. 1 illustrates an example of the various candidate substrings possible in the word "isomorphically."
Fig. 2 is a tree diagram of the analysis of the word "isomorphically" i]lustrated in Fig. 1.
Fig. 3 is a flow diagram of the invention.
Fig. 4 is an example of tree scanning.
Description of the Best Mode for Carrying Out the Invention The invention is a technique for parsing compound words by matching the input word against dictionary words to obtain those that match a portion of the input word and have appropriate compounding codes. When the input word does not match any dictionary word exactly, the process is repeated with the remaining portion of the input word until either a match is ~ound and all the components have been identified, or all possibilities have been explored and none is satis~actory.
Two aspects of the invention are a comprehensive set of compound flags associated with each dictionary entry and an efficient method for examining the input word systematically.
These two aspects are described in detail.
The Compound Flags Linguistic and theoretical analysis of compound words has shown that each component may have any of four independent characteristics relative to compound formation: 1) it can occur as an isolated word, 2) it can be a front component or prefix, 3) it can be a middle component of a word, and 4) it can be a back component or suffix. These four characteristics can occur in any combination for a total of 15 possibilities, the case where none of the characteristics apply never being used. The word "sun," for example, might be tagged with ~9-86-011 _5_ 7~
characteristics 1 and 2 since it can occur as an isolated word or as a prefix in words like ~sunglasses~ or suntan." The word "glasses," on the other hand, would have characteristics 1 and 4 since it can be used either isolated or as a suffix. A word like "pseudo" would be marked with characteristic 2 since it :
can only be a prefix. The word "intangible" would have characteristic 1 because it only occurs as an isolated word.
For some German language examples, the combination of features 1~3+4 can be associated with the word "Sandale" which can occur alone or as a middle or back component of a word:
Example of word "Sandale" with features 1+3+4:
Leder ~ Sandale + Reparatur = Ledersandalereparatur Example of word "ig" with features 3+4:
Kurz + Bein + ig ~ keit - Kurzbeinigkeit Additional codes may be used to indicate the type of morphological transformation that a word can undergo.
The dictionary containing the compounding characteristics for each word can be generated either manually or automatically. The automatic process requires the decomposition of correct compound words into their components using the parsing algorithm given below while keeping track of the position of each component within the word. The segmented words can be reviewed manually to assure that no compounding codes for incorrectly segmented words are admitted into the dictionary.
The Parsing Algorithm The parsing algorithm is a pxocess for systematically exploring all possible word breaks efficiently. The algorithm is based on "preorder" scanning of trees. Although it is not obvious that this type of scanning is applicable to word parsing, the application is made possible by recognizing that the components of a compound word can be ordered so as to map "isomorphically" with a tree of variable depth.
Consider, for example, the word "isomorphically" as a compound word. If we examine this word for the substrings that can match against the dictionary we will find the words that are capitalized in Fig. 1.
~ 8~ 7~
The list of words in Fig 1 can be represented as a tree of the form indicated in Fig. 2. The tree in Fig. 2 can be scanned in preorder sequence to examine all the possibilities that can be used for word formation. However, some branches of the tree can be eliminated from consideration at the outset since the words forming the branches will have inappropriate com~ound flags. For example, it is not even necessary to check beyond the word "IS" since it can only occur as an isolated ~ord.
In practice, at each branching point in the tree, the dictionary is examined for all words starting with those particular letters to optimize dictionary access. The candidate words are considered by applying the algorithm recursively or by placing the words in a stack that is processed iteratively. Either of these two approaches will produce equivalent results.
A "stack," in computer terminology, is an array where information is stored in a prescribed sequence. Generally, the last item stored in the stack is the first one retrieved (analogous to a stack of coins on a table). The implementation of the invention with a stack requires that as the dictionary is scanned for substrings that match the word, the position and length of each matching dictionary word is stored in the stack.
~t is not necessary to save the matching dictionary word, since it is a substring of the compound word. When all suitable matches have been found (as determined by examination of the flags associated with the dictionary entries), the characters of the compound word not included in the last word added to the stack are matched against the dictionary again, and the process is repeated until a suitable match that encompasses all the remaining characters of the compound word are found. However, if no suitable matches are found, the unsuccessful words at the top of the stack are removed and another attempt is made with the next word in the stack. If the stack is empty before a suitable match is found, the process finishes without identifying any components.
If certain language-specific characteristics are present, morphological adaptation procedures are applied to analyze for elision of consonants (as for "schiffahrt") or to convert multiple letter combinations such as "ue" into a "u" with an umlaut depending on the requirements of the language. In a German hyphenated word like "zuk-ker" the removal of the hyphen also changes the spelling to "zucker."
A stack-based implementation of the algorithm is given 4~ below in a pseudo-code similar to PL/1:
~2~
TABLE I
TREE SCANNING PROCEDURE
FOR PARSING COMPOUND WORDS
__________________________________~____________________________. _____ ___ _________________________________________________________________________ /" GENERAL PARAMETERS */
DCI, WORD ~ CHAR(MXLENGTH);/* Word to be decompounded */
DCL MXLENGTH FIXED BIN(15); /* Maxlmum word length */
~CL WORDLEN FIXED BIN(15); /* Length of WORD */
DCL SWORD CHAR(MXLENGTH);/* Subs~ring of word */
/* STACK FOR BACKTRACKING */
DCL 1 STACK, 2 TABNOMAX FIXED BIN(15), /* Maximum no. of components */
2 TAGMAX FIXED BIN(15), /* Maximum no. of substrings for */
/* a component level */
2 TABNO FIXED BIN(15), /* No. of current component level */
TABLE(TABNOMAXJ
3 POS FIXED BIN(15), /* Starting position of substring */
3 NUM FIXED BIN(15), /* No. of substrings for current */
/* level */
3 TAG FIXED BIN(15~, /* Current substring number */
3 LEN(TAGMAX) FIXED BIN(15), /* Length of substrings in word */
~* PSEUDOCODE FOR DECOMPOUNDING */
/* Set up first table entry (first level */
TABNO = l;
TAGtTABNO) = l;
POS(TABNO) = l; /* Point to first character of WORD */
SWORD = NORD; /* Set SWORD equal to WORD */
FORWARD:
/* Find the leading substrings of SWORD that exist in the */
/* dictionariesO Store number of substrings found and the */
/* length of each substring in the stack. */
CALL FIND SUBSTRINGS(SWORD,STACK~; -/*~If no substring is found, backup to the previous level */
IF NUM(TABNO) = 0 THEN GOTO BACKWARD; */
FLAG_CHECK:
/* Examine the compound flag of the substring for validity */
CALL FLAGCHECK(WORD,WORDLEN,STACK,FLAGRC);
/* If the flag is valid (FLAGRC=0), then check if the */
/* substring exhausts the word. If it does, we have found */
/* a valid compound word. If it does not then go to the next *~
/* level and finding the next subword to be decompounded. */
IF FLAGRC = 0 THEN DO;
IF POS(TABNO) ~ LEN(TABNO,TAG) 1 = WORDLEN
THEN GOTO SUCCESS; /* Complete word OK */
POS(TABNO+l) = P09(TABNO) + /* Start of new substring */
LEN(TABNO,TAG~;
TABNO = TABNO +l; /* Increment level */
/* SWORD contains remaining substring of the word. */
SWORD = SUBSTR(WORD,POS(TABNO),WORDLEN)-POS(TABNO));
TAG(TABNO) = l; /* Set TAG=l *~
GOTO FORWARD; /* Try new candidate */
~9-86-011 :~%~
END;
/* If the flag is not valid (FLAGRC=l), then if there is */
~* another substring at this level, examine the flag for */
/* this substring. Otherwise, go back to the previous level */
/* after checking morphological possibilities. */
IF TAG(TABNO) < NUM(TABNO) THEN DO; /* More substr.this level? */
TAG(TABNO~ = TAG(TABNO) + l; /* Next substr.this level */
GOTO FLAG CHECK; /* Check its flag */
END;
/* No more substrings at .his level */
/* ~pply language-specific morphological procedures. */
CALL MORPH(WORD,WORDLEN,STACK);
~ACKWARD:
/* Calculate the previous level. If the resulting level is */
/* zero, we have failed to find a valid word */
TABNO = TABNO -1;
IF TABNO = 0 THEN GOTO FAIL;
/* If there is another substring at this level, try -this */
/* substring. Otherwise, go back to the previous level. */
IF TAG(TABNO) < NUM(TABNO) THEN DO;
TAG(TABNO) = TAG(TABNO) + l;
GOTO FORWARD:
END;
~OTO BACKWARD;
~* END OF TREE SCANNING PROCEDURE */
_g_ ~ ~8~
Letter Elisions:
There are circumstances when letters are elided for linguistic reasons when two words are combined and the resulting compound word is not strictly a juxtaposition of the components. The invention is able to process these cases for :
certain general cases. In German, for example, a letter is elided when a word ending in a double consonant is combined with another word that starts with the same consonant. For example, the word "Schiffahrt" (German for "boat trip") is constructed from "Schiff" ("boat") and "Fahrt" ("trip") by eliding an "f." Therefore, as part of the morphological transformations performed by the invention, when no suitable dictionary entries can be found for the remainder of the word (in this case for "ahrt"), a check is made to see if the previous component ends in a double consonant and the next character a vowel. When these conditions are found, the starting character of the remaining string is set to last character of the previous word. In this way the program checks for the word "fahrt."
Letter Insertions:
A case analogous to letter elisions is the case of letter insertions. In German this happens with the genitive case where the letter "s" is added to the initial component. The invention handles this problem by checking to see if the remaining component starts with "s" whenever no suitable match can be found. If an "s" is present, it is skipped and a match is attempted on the remaining portion of the compound word.
Capitalization:
Just as English requires proper nouns and adjectives derived from them to be capitalized (e.g., American), 50 German grammar requires that all nouns be capitalized. Thus, a German dictionary contains many words starting with upper case. This presents a problem when matching a compound word, since only its first character may be capitalized and any internal components will start in lower case. This is solved by the invention by ordering the dictionary in alphabetical order without regard to case (but indicating case as an additional attribute) and by converting the dictionary words and the compound word to a common character set at the time of matching.
Specific Morphological Transformation:
In addition to the features that characterize the usage of a component in a compound word, codes specifying specific transformations can be associated with each word in the dictionary. For example, the Afrikaans language has a large number of words for which the combining non-back form of the genitive singular has the same spelling dS the nominative plural, both adding "s" to the nominative singular. When such a word is compounded with a word starting with "s," one of them must be elided. This can be indicated by associating a special code with such words. For example, such a code associated with the word "mans" would result in elision o~ an "s" when combined with "skool:" mans + skool = manskool. Such specific transformations are represented by codes in the dictionary and enable the decomposition of the compound word into its components. Notice that this technique relies on the specific encoding of the attribute within the dictionary rather than as a part of the algorithm as indicated for elisions above.
Support of Hyphenation:
One of the by-products of compound word analysis as described in this invention is the state of the stack after a compound has been processed. The stack contains the information that identifies the components of the compound, and hence, the major hyphenation points of the compound. These hyphenation points may be supplemented by hyphenation information stored for each component. In addition, the fact that the invention identifies letter elisions makes it possible to hyphenate correctly such words. For example, the German word "Schiffahrt" would be correctly hyphenated as "Schiff-fahrt" by restoring the elided letter.
The flow diagram of Fig. 3 is implemented with a stack.
Step 20 says to get the text word. This is a either text from an input buffer or it can be supplied by another computer program. In Step 22, we clear the stack. That means we set the stack to empty and we set the level equal to one. The procedure by which we scan the stack is basically a pre-order form and that means that you set a chain as in Fig.4, starting with the node "a" which splits off into nodes "b" and "c." The nodes "b" and "c" are at the level 2 and then if "b" splits into nodes "d," "e' and "f," this is the third level. I~ node "c" splits into nodes "g" and "h," again this is the third level. The tree would be scanned by going from "a" to "b" to MA9-8~-011 d~
"d," then "e," then "~," back to ~b,~ back to "a" and then to "c" to "g" and then "h," back to "c" and back to "a." That would terminate the scan of the tree. The disclosed invention does the equivalent function for trying the various possibilities of finding substrings within a word for verification. We now proceed to the label "Forward." At this point, we get the text word in Step 24; in this case, there is no change in the text word because it has not been altered. It is the same as that which was originally input. In subsequent 1~ looping through the flow chart, it would be a modified version of the word. Then we proceed to Step 26 labeled "Check dictionary for matching substrings." At this stage, we check the dictionary and any dictionary entries which match exactly against the initial portion of the string are identified. If none of the words match, we go to the label "sackward" where we are going to decrement the levels and try other possibilities by going to other branches of the chain. But if there are some matching entries, then this creates a new level, so we increment the level in Step 28. In order to account for upper case, we have to check if the level is greater than one in Step 30. That means it is not an initial substring of the word, and we disregard the upper case that may be in the word stored in the dictionary. At this point, we come to Step 32 labeled "Store number of substrings at current level in their length in stack." Basically this is just a mechanism for storing the strings for future reference. Itis storing the place where the string starts and its length for the current level. By having stored this information, we proceed to check the flag of the substrings in Step 32. If we find an unsuitable flag, then we go to Step 36 marked "More substrings at the current level."
We continue checking those substrings until we either do not find any substrings at the current level or until we find a substring with a suitable flag. When we find the substring with the suitable flag, we proceed to Step 38 "Apply transformation specified by flag or code, if any." In this step, transformations requiring additions of letters are performed. After these transformations are applied, then we check to see if the substring is at the end of the word in Step 40. If it is, then we can say that the word verified because all the components have been checked and they have appropriate flags, so at this point, we exitat Step 42. However, if the substring is not at the end of the word, now we remove from the word that initial substring in Step 44 and keep the remainder 7~L
(the right part) of the substring and then go back to the label "Go forward" to repeat the process with the remaining substring. In this way, each segment of the word is iteratively processed until the whole word has been recognized.
In Step 46 labeled "Apply morphological transformationS," ~ -whenever there are no more substrings at the current level, the operator must take into consideration whether ~here were any transformations applied between the words. At this point, those transformations are applied. If the transformation is successful, again we go to the label "Forward," continue processing the string. If we cannot transform it, then we go back to the label "Backward." "Backward" is really the process of recovering from a failure along the branch and if we can explore the branch of a tree which does not lead to a complete recognition of the word, then we decrement the level in Step 50 after we have tried all the possibilities at that level (Step 52). If the level is zero in Step 54, then we know that the word does not verify (Step 56) because we have tried all the possibilities. If the level is not zero and there are more substrings at the current level, we continue with "Forward"
until we exhaust all the possibilities.
Although a specific embodiment of the invention has been disclosed, it will be understood by those of skill in the art that changes can be made to the invention without departing from the spirit and the scope of the invention.
.~ . .
3 NUM FIXED BIN(15), /* No. of substrings for current */
/* level */
3 TAG FIXED BIN(15~, /* Current substring number */
3 LEN(TAGMAX) FIXED BIN(15), /* Length of substrings in word */
~* PSEUDOCODE FOR DECOMPOUNDING */
/* Set up first table entry (first level */
TABNO = l;
TAGtTABNO) = l;
POS(TABNO) = l; /* Point to first character of WORD */
SWORD = NORD; /* Set SWORD equal to WORD */
FORWARD:
/* Find the leading substrings of SWORD that exist in the */
/* dictionariesO Store number of substrings found and the */
/* length of each substring in the stack. */
CALL FIND SUBSTRINGS(SWORD,STACK~; -/*~If no substring is found, backup to the previous level */
IF NUM(TABNO) = 0 THEN GOTO BACKWARD; */
FLAG_CHECK:
/* Examine the compound flag of the substring for validity */
CALL FLAGCHECK(WORD,WORDLEN,STACK,FLAGRC);
/* If the flag is valid (FLAGRC=0), then check if the */
/* substring exhausts the word. If it does, we have found */
/* a valid compound word. If it does not then go to the next *~
/* level and finding the next subword to be decompounded. */
IF FLAGRC = 0 THEN DO;
IF POS(TABNO) ~ LEN(TABNO,TAG) 1 = WORDLEN
THEN GOTO SUCCESS; /* Complete word OK */
POS(TABNO+l) = P09(TABNO) + /* Start of new substring */
LEN(TABNO,TAG~;
TABNO = TABNO +l; /* Increment level */
/* SWORD contains remaining substring of the word. */
SWORD = SUBSTR(WORD,POS(TABNO),WORDLEN)-POS(TABNO));
TAG(TABNO) = l; /* Set TAG=l *~
GOTO FORWARD; /* Try new candidate */
~9-86-011 :~%~
END;
/* If the flag is not valid (FLAGRC=l), then if there is */
~* another substring at this level, examine the flag for */
/* this substring. Otherwise, go back to the previous level */
/* after checking morphological possibilities. */
IF TAG(TABNO) < NUM(TABNO) THEN DO; /* More substr.this level? */
TAG(TABNO~ = TAG(TABNO) + l; /* Next substr.this level */
GOTO FLAG CHECK; /* Check its flag */
END;
/* No more substrings at .his level */
/* ~pply language-specific morphological procedures. */
CALL MORPH(WORD,WORDLEN,STACK);
~ACKWARD:
/* Calculate the previous level. If the resulting level is */
/* zero, we have failed to find a valid word */
TABNO = TABNO -1;
IF TABNO = 0 THEN GOTO FAIL;
/* If there is another substring at this level, try -this */
/* substring. Otherwise, go back to the previous level. */
IF TAG(TABNO) < NUM(TABNO) THEN DO;
TAG(TABNO) = TAG(TABNO) + l;
GOTO FORWARD:
END;
~OTO BACKWARD;
~* END OF TREE SCANNING PROCEDURE */
_g_ ~ ~8~
Letter Elisions:
There are circumstances when letters are elided for linguistic reasons when two words are combined and the resulting compound word is not strictly a juxtaposition of the components. The invention is able to process these cases for :
certain general cases. In German, for example, a letter is elided when a word ending in a double consonant is combined with another word that starts with the same consonant. For example, the word "Schiffahrt" (German for "boat trip") is constructed from "Schiff" ("boat") and "Fahrt" ("trip") by eliding an "f." Therefore, as part of the morphological transformations performed by the invention, when no suitable dictionary entries can be found for the remainder of the word (in this case for "ahrt"), a check is made to see if the previous component ends in a double consonant and the next character a vowel. When these conditions are found, the starting character of the remaining string is set to last character of the previous word. In this way the program checks for the word "fahrt."
Letter Insertions:
A case analogous to letter elisions is the case of letter insertions. In German this happens with the genitive case where the letter "s" is added to the initial component. The invention handles this problem by checking to see if the remaining component starts with "s" whenever no suitable match can be found. If an "s" is present, it is skipped and a match is attempted on the remaining portion of the compound word.
Capitalization:
Just as English requires proper nouns and adjectives derived from them to be capitalized (e.g., American), 50 German grammar requires that all nouns be capitalized. Thus, a German dictionary contains many words starting with upper case. This presents a problem when matching a compound word, since only its first character may be capitalized and any internal components will start in lower case. This is solved by the invention by ordering the dictionary in alphabetical order without regard to case (but indicating case as an additional attribute) and by converting the dictionary words and the compound word to a common character set at the time of matching.
Specific Morphological Transformation:
In addition to the features that characterize the usage of a component in a compound word, codes specifying specific transformations can be associated with each word in the dictionary. For example, the Afrikaans language has a large number of words for which the combining non-back form of the genitive singular has the same spelling dS the nominative plural, both adding "s" to the nominative singular. When such a word is compounded with a word starting with "s," one of them must be elided. This can be indicated by associating a special code with such words. For example, such a code associated with the word "mans" would result in elision o~ an "s" when combined with "skool:" mans + skool = manskool. Such specific transformations are represented by codes in the dictionary and enable the decomposition of the compound word into its components. Notice that this technique relies on the specific encoding of the attribute within the dictionary rather than as a part of the algorithm as indicated for elisions above.
Support of Hyphenation:
One of the by-products of compound word analysis as described in this invention is the state of the stack after a compound has been processed. The stack contains the information that identifies the components of the compound, and hence, the major hyphenation points of the compound. These hyphenation points may be supplemented by hyphenation information stored for each component. In addition, the fact that the invention identifies letter elisions makes it possible to hyphenate correctly such words. For example, the German word "Schiffahrt" would be correctly hyphenated as "Schiff-fahrt" by restoring the elided letter.
The flow diagram of Fig. 3 is implemented with a stack.
Step 20 says to get the text word. This is a either text from an input buffer or it can be supplied by another computer program. In Step 22, we clear the stack. That means we set the stack to empty and we set the level equal to one. The procedure by which we scan the stack is basically a pre-order form and that means that you set a chain as in Fig.4, starting with the node "a" which splits off into nodes "b" and "c." The nodes "b" and "c" are at the level 2 and then if "b" splits into nodes "d," "e' and "f," this is the third level. I~ node "c" splits into nodes "g" and "h," again this is the third level. The tree would be scanned by going from "a" to "b" to MA9-8~-011 d~
"d," then "e," then "~," back to ~b,~ back to "a" and then to "c" to "g" and then "h," back to "c" and back to "a." That would terminate the scan of the tree. The disclosed invention does the equivalent function for trying the various possibilities of finding substrings within a word for verification. We now proceed to the label "Forward." At this point, we get the text word in Step 24; in this case, there is no change in the text word because it has not been altered. It is the same as that which was originally input. In subsequent 1~ looping through the flow chart, it would be a modified version of the word. Then we proceed to Step 26 labeled "Check dictionary for matching substrings." At this stage, we check the dictionary and any dictionary entries which match exactly against the initial portion of the string are identified. If none of the words match, we go to the label "sackward" where we are going to decrement the levels and try other possibilities by going to other branches of the chain. But if there are some matching entries, then this creates a new level, so we increment the level in Step 28. In order to account for upper case, we have to check if the level is greater than one in Step 30. That means it is not an initial substring of the word, and we disregard the upper case that may be in the word stored in the dictionary. At this point, we come to Step 32 labeled "Store number of substrings at current level in their length in stack." Basically this is just a mechanism for storing the strings for future reference. Itis storing the place where the string starts and its length for the current level. By having stored this information, we proceed to check the flag of the substrings in Step 32. If we find an unsuitable flag, then we go to Step 36 marked "More substrings at the current level."
We continue checking those substrings until we either do not find any substrings at the current level or until we find a substring with a suitable flag. When we find the substring with the suitable flag, we proceed to Step 38 "Apply transformation specified by flag or code, if any." In this step, transformations requiring additions of letters are performed. After these transformations are applied, then we check to see if the substring is at the end of the word in Step 40. If it is, then we can say that the word verified because all the components have been checked and they have appropriate flags, so at this point, we exitat Step 42. However, if the substring is not at the end of the word, now we remove from the word that initial substring in Step 44 and keep the remainder 7~L
(the right part) of the substring and then go back to the label "Go forward" to repeat the process with the remaining substring. In this way, each segment of the word is iteratively processed until the whole word has been recognized.
In Step 46 labeled "Apply morphological transformationS," ~ -whenever there are no more substrings at the current level, the operator must take into consideration whether ~here were any transformations applied between the words. At this point, those transformations are applied. If the transformation is successful, again we go to the label "Forward," continue processing the string. If we cannot transform it, then we go back to the label "Backward." "Backward" is really the process of recovering from a failure along the branch and if we can explore the branch of a tree which does not lead to a complete recognition of the word, then we decrement the level in Step 50 after we have tried all the possibilities at that level (Step 52). If the level is zero in Step 54, then we know that the word does not verify (Step 56) because we have tried all the possibilities. If the level is not zero and there are more substrings at the current level, we continue with "Forward"
until we exhaust all the possibilities.
Although a specific embodiment of the invention has been disclosed, it will be understood by those of skill in the art that changes can be made to the invention without departing from the spirit and the scope of the invention.
.~ . .
Claims (8)
1. A computer method for parsing a compound word composed of a plurality of word components, comprising the steps of:
storing a dictionary of stored word components and associating with each of said components a flag indicating whether said word component can be an independent word, a prefix of a word, a middle element of a word, a suffix, or a code specifying the type of morphological transformation it can undergo;
inputting an input word stream which includes a compound word which is to be parsed;
selecting all words from the dictionary that match an initial substring of the input word and retaining only the dictionary words which have a component flag indicating that said dictionary word can be a prefix element;
processing all the remaining portions of the input word, selecting all words from the dictionary that match an initial substring of said remaining portion of the input word and retaining only the dictionary words which have a component flag indicating that the word can be a middle element of a word;
processing all the remaining portions of the input word, selecting all words from the dictionary that exactly match said remaining portion of the input word and retaining only the dictionary words which have a component flag indicating that the word can be a word suffix element;
applying morphological rules during the search for middle or suffix elements if no suitable dictionary candidates can be found.
storing a dictionary of stored word components and associating with each of said components a flag indicating whether said word component can be an independent word, a prefix of a word, a middle element of a word, a suffix, or a code specifying the type of morphological transformation it can undergo;
inputting an input word stream which includes a compound word which is to be parsed;
selecting all words from the dictionary that match an initial substring of the input word and retaining only the dictionary words which have a component flag indicating that said dictionary word can be a prefix element;
processing all the remaining portions of the input word, selecting all words from the dictionary that match an initial substring of said remaining portion of the input word and retaining only the dictionary words which have a component flag indicating that the word can be a middle element of a word;
processing all the remaining portions of the input word, selecting all words from the dictionary that exactly match said remaining portion of the input word and retaining only the dictionary words which have a component flag indicating that the word can be a word suffix element;
applying morphological rules during the search for middle or suffix elements if no suitable dictionary candidates can be found.
2. The process as stated in claim 1 where only prefix and suffix elements are considered.
3. The process as stated in claim 1 where the search for matches against the dictionary is controlled by a tree-scanning procedure implemented with a stack.
4. The process as stated in claim 1 where the morphological rule applied during the search for middle and suffix elements consists of identifying whether the remaining portion of the input word under consideration starts with a vowel and the preceding element ends in a double consonant, in which case a copy of said consonant is inserted ahead of said vowel.
5. The process as stated in claim 1 where the morphological rule applied during the search for middle and suffix elements consists of identifying whether the remaining portion of the input word under consideration starts with the letter "S," in which case said letter is removed.
6. The process as stated in claim 1 where the morphological rule applied during the search for middle or suffix elements consists of identifying whether the dictionary term is capitalized, in which case the initial letter of said dictionary term is made lower case.
7. The process as stated in claim 1 where the morphological rule applied during the search for middle or suffix elements consists of a morphological transformation specified by the code associated in the dictionary with the preceding element.
8. The process as stated in claim 1 where the components identified by said process are stored as reference to provide hyphenation points for the input compound word.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US025,041 | 1987-03-12 | ||
| US07/025,041 US4777617A (en) | 1987-03-12 | 1987-03-12 | Method for verifying spelling of compound words |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA1288871C true CA1288871C (en) | 1991-09-10 |
Family
ID=21823726
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA000557768A Expired - Fee Related CA1288871C (en) | 1987-03-12 | 1988-01-29 | Method for verifying spelling of compound words |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US4777617A (en) |
| EP (1) | EP0281742B1 (en) |
| JP (1) | JPS63231569A (en) |
| CA (1) | CA1288871C (en) |
| DE (1) | DE3865321D1 (en) |
Families Citing this family (44)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4974195A (en) * | 1986-06-20 | 1990-11-27 | Canon Kabushiki Kaisha | Document processing apparatus |
| US4868743A (en) * | 1986-11-25 | 1989-09-19 | Hitachi, Ltd. | Traversal method of processing tree structure information and apparatus using the same |
| US4864503A (en) * | 1987-02-05 | 1989-09-05 | Toltran, Ltd. | Method of using a created international language as an intermediate pathway in translation between two national languages |
| US4873634A (en) * | 1987-03-27 | 1989-10-10 | International Business Machines Corporation | Spelling assistance method for compound words |
| EP0287713B1 (en) * | 1987-04-23 | 1994-06-22 | Océ-Nederland B.V. | A text processing system and methods for checking in a text processing system the correct and consistent use of units or chemical formulae |
| US5551026A (en) * | 1987-05-26 | 1996-08-27 | Xerox Corporation | Stored mapping data with information for skipping branches while keeping count of suffix endings |
| US5754847A (en) * | 1987-05-26 | 1998-05-19 | Xerox Corporation | Word/number and number/word mapping |
| US5560037A (en) * | 1987-12-28 | 1996-09-24 | Xerox Corporation | Compact hyphenation point data |
| JP2664915B2 (en) * | 1988-01-12 | 1997-10-22 | 株式会社日立製作所 | Information retrieval system |
| US5295069A (en) * | 1991-06-05 | 1994-03-15 | International Business Machines Corporation | Computer method for ranked hyphenation of multilingual text |
| DE4135261C1 (en) * | 1991-10-25 | 1993-03-18 | International Business Machines Corp., Armonk, N.Y., Us | |
| US5625554A (en) * | 1992-07-20 | 1997-04-29 | Xerox Corporation | Finite-state transduction of related word forms for text indexing and retrieval |
| EP0579873B1 (en) * | 1992-07-20 | 1999-05-06 | Océ-Technologies B.V. | Method of reproducing text on a raster output device |
| US5867812A (en) * | 1992-08-14 | 1999-02-02 | Fujitsu Limited | Registration apparatus for compound-word dictionary |
| DE4323241A1 (en) * | 1993-07-12 | 1995-02-02 | Ibm | Method and computer system for finding incorrect character strings in a text |
| JPH0877173A (en) * | 1994-09-01 | 1996-03-22 | Fujitsu Ltd | System and method for correcting character string |
| US5799299A (en) * | 1994-09-14 | 1998-08-25 | Kabushiki Kaisha Toshiba | Data processing system, data retrieval system, data processing method and data retrieval method |
| DE19526264A1 (en) * | 1995-07-19 | 1997-04-10 | Daimler Benz Ag | Process for creating descriptors for the classification of texts |
| JP3605682B2 (en) * | 1996-05-16 | 2004-12-22 | 本田技研工業株式会社 | Voice input control device |
| JP3272288B2 (en) * | 1997-12-24 | 2002-04-08 | 日本アイ・ビー・エム株式会社 | Machine translation device and machine translation method |
| US6760887B1 (en) | 1998-12-31 | 2004-07-06 | International Business Machines Corporation | System and method for highlighting of multifont documents |
| US6813747B1 (en) | 1998-12-31 | 2004-11-02 | International Business Machines Corporation | System and method for output of multipart documents |
| US7103532B1 (en) | 1998-12-31 | 2006-09-05 | International Business Machines Corp. | System and method for evaluating character in a message |
| US7039637B2 (en) | 1998-12-31 | 2006-05-02 | International Business Machines Corporation | System and method for evaluating characters in an inputted search string against a character table bank comprising a predetermined number of columns that correspond to a plurality of pre-determined candidate character sets in order to provide enhanced full text search |
| US7031002B1 (en) | 1998-12-31 | 2006-04-18 | International Business Machines Corporation | System and method for using character set matching to enhance print quality |
| US6539118B1 (en) | 1998-12-31 | 2003-03-25 | International Business Machines Corporation | System and method for evaluating character sets of a message containing a plurality of character sets |
| US6718519B1 (en) | 1998-12-31 | 2004-04-06 | International Business Machines Corporation | System and method for outputting character sets in best available fonts |
| US7191114B1 (en) | 1999-08-27 | 2007-03-13 | International Business Machines Corporation | System and method for evaluating character sets to determine a best match encoding a message |
| US6626960B1 (en) * | 1999-09-01 | 2003-09-30 | International Business Machines Corporation | Method, system, and program for generating a table to determine boundaries between characters |
| US6671856B1 (en) | 1999-09-01 | 2003-12-30 | International Business Machines Corporation | Method, system, and program for determining boundaries in a string using a dictionary |
| SE519636C2 (en) * | 2000-07-06 | 2003-03-25 | Hapax Information Systems Ab | Method and apparatus for analyzing compound words |
| US7092871B2 (en) * | 2000-07-20 | 2006-08-15 | Microsoft Corporation | Tokenizer for a natural language processing system |
| US7610189B2 (en) * | 2001-10-18 | 2009-10-27 | Nuance Communications, Inc. | Method and apparatus for efficient segmentation of compound words using probabilistic breakpoint traversal |
| US20040205675A1 (en) * | 2002-01-11 | 2004-10-14 | Thangaraj Veerappan | System and method for determining a document language and refining the character set encoding based on the document language |
| GB0413743D0 (en) * | 2004-06-19 | 2004-07-21 | Ibm | Method and system for approximate string matching |
| US7698128B2 (en) | 2006-01-13 | 2010-04-13 | Research In Motion Limited | Handheld electronic device and method for disambiguation of compound text input and that employs N-gram data to limit generation of low-probability compound language solutions |
| US7538692B2 (en) * | 2006-01-13 | 2009-05-26 | Research In Motion Limited | Handheld electronic device and method for disambiguation of compound text input and for prioritizing compound language solutions according to quantity of text components |
| JP4720570B2 (en) * | 2006-03-27 | 2011-07-13 | カシオ計算機株式会社 | Information display control device and information display control program |
| US8630841B2 (en) | 2007-06-29 | 2014-01-14 | Microsoft Corporation | Regular expression word verification |
| US8176419B2 (en) * | 2007-12-19 | 2012-05-08 | Microsoft Corporation | Self learning contextual spell corrector |
| JP4993319B2 (en) * | 2009-02-13 | 2012-08-08 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Apparatus and method for supporting verification of software internationalization |
| CN102859515B (en) * | 2010-02-12 | 2016-01-13 | 谷歌公司 | Compound word splits |
| US9081762B2 (en) * | 2012-07-13 | 2015-07-14 | Enyuan Wu | Phrase-based dictionary extraction and translation quality evaluation |
| JP6753401B2 (en) * | 2015-07-24 | 2020-09-09 | 富士通株式会社 | Coding programs, coding devices, and coding methods |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE2555792A1 (en) * | 1975-12-11 | 1977-06-23 | Eichhorn Friedrich Prof Dr | PROCEDURE FOR QUALITY ASSURANCE OF THE WELDED JOINTS DURING ELECTRIC RESISTANCE SPOT WELDING |
| US4342085A (en) * | 1979-01-05 | 1982-07-27 | International Business Machines Corporation | Stem processing for data reduction in a dictionary storage file |
| US4355371A (en) * | 1980-03-25 | 1982-10-19 | International Business Machines Corporation | Instantaneous alpha content prescan method for automatic spelling error correction |
| JPS57113187A (en) * | 1980-12-19 | 1982-07-14 | Ibm | Method of processing text |
| US4499553A (en) * | 1981-09-30 | 1985-02-12 | Dickinson Robert V | Locating digital coded words which are both acceptable misspellings and acceptable inflections of digital coded query words |
| US4456969A (en) * | 1981-10-09 | 1984-06-26 | International Business Machines Corporation | System for automatically hyphenating and verifying the spelling of words in a multi-lingual document |
| US4597057A (en) * | 1981-12-31 | 1986-06-24 | System Development Corporation | System for compressed storage of 8-bit ASCII bytes using coded strings of 4 bit nibbles |
| JPS59127138A (en) * | 1982-12-28 | 1984-07-21 | Fujitsu Ltd | Sentence fault detector |
| JPS6126176A (en) * | 1984-07-17 | 1986-02-05 | Nec Corp | Dictionary for processing language |
| US4672571A (en) * | 1984-10-24 | 1987-06-09 | International Business Machines Corporation | Compound word suitability for spelling verification |
| US4701851A (en) * | 1984-10-24 | 1987-10-20 | International Business Machines Corporation | Compound word spelling verification |
| JPS61128365A (en) * | 1984-11-28 | 1986-06-16 | Toshiba Corp | Spelling check system of word processing system |
| JPS61173379A (en) * | 1985-01-29 | 1986-08-05 | Brother Ind Ltd | Spelling checking device |
| JPS61175848A (en) * | 1985-01-31 | 1986-08-07 | Toshiba Corp | Spelling checking system in word processor |
-
1987
- 1987-03-12 US US07/025,041 patent/US4777617A/en not_active Expired - Lifetime
- 1987-12-18 JP JP62319132A patent/JPS63231569A/en active Pending
-
1988
- 1988-01-22 EP EP88100948A patent/EP0281742B1/en not_active Expired - Lifetime
- 1988-01-22 DE DE8888100948T patent/DE3865321D1/en not_active Expired - Fee Related
- 1988-01-29 CA CA000557768A patent/CA1288871C/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| DE3865321D1 (en) | 1991-11-14 |
| EP0281742A3 (en) | 1988-11-23 |
| JPS63231569A (en) | 1988-09-27 |
| US4777617A (en) | 1988-10-11 |
| EP0281742B1 (en) | 1991-10-09 |
| EP0281742A2 (en) | 1988-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA1288871C (en) | Method for verifying spelling of compound words | |
| EP0283685B1 (en) | A spelling assistance method for compound words | |
| US5610812A (en) | Contextual tagger utilizing deterministic finite state transducer | |
| US7809744B2 (en) | Method and system for approximate string matching | |
| US7191177B2 (en) | Keyword extracting device | |
| US5890103A (en) | Method and apparatus for improved tokenization of natural language text | |
| JP4544674B2 (en) | A system that provides information related to the selected string | |
| US6816830B1 (en) | Finite state data structures with paths representing paired strings of tags and tag combinations | |
| JPH0724055B2 (en) | Word division processing method | |
| KR20010040499A (en) | Automatic segmentation of a text | |
| US5560037A (en) | Compact hyphenation point data | |
| Pal et al. | OCR error correction of an inflectional indian language using morphological parsing | |
| JPH0211934B2 (en) | ||
| US8041556B2 (en) | Chinese to english translation tool | |
| US6385569B1 (en) | Translator, translating method and recording medium having translating program recorded thereon | |
| JP3309174B2 (en) | Character recognition method and device | |
| KR100420474B1 (en) | Apparatus and method of long sentence translation using partial sentence frame | |
| JP2807236B2 (en) | Morphological analysis method | |
| US20220004708A1 (en) | Methods and apparatus to improve disambiguation and interpretation in automated text analysis using structured language space and transducers applied on automatons | |
| JP2827066B2 (en) | Post-processing method for character recognition of documents with mixed digit strings | |
| Lin et al. | A Simple and Practical Approach to Improve Misspellings in OCR Text | |
| JP2608384B2 (en) | Machine translation apparatus and method | |
| KR20000038079A (en) | Device and method for extracting example of use statistically for large amount language batch | |
| Mostafa et al. | A novel approach for detecting and correcting segmentation and recognition errors in Arabic OCR systems | |
| Srinivas et al. | Exploiting supertag representation for fast coreference resolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MKLA | Lapsed |