WO1996001453A1 - Multiple pen stroke character set and handwriting recognition system - Google Patents

Multiple pen stroke character set and handwriting recognition system Download PDF

Info

Publication number
WO1996001453A1
WO1996001453A1 PCT/US1995/008113 US9508113W WO9601453A1 WO 1996001453 A1 WO1996001453 A1 WO 1996001453A1 US 9508113 W US9508113 W US 9508113W WO 9601453 A1 WO9601453 A1 WO 9601453A1
Authority
WO
WIPO (PCT)
Prior art keywords
character
input
glyphs
modifier
recognition
Prior art date
Application number
PCT/US1995/008113
Other languages
French (fr)
Inventor
Jeffrey C. Hawkins
Joseph K. Sipher
Ron Ii Marianetti
Original Assignee
Palm Computing, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Palm Computing, Inc. filed Critical Palm Computing, Inc.
Priority to DE69533479T priority Critical patent/DE69533479T2/en
Priority to JP8503905A priority patent/JPH10510639A/en
Priority to EP95924077A priority patent/EP0769175B9/en
Publication of WO1996001453A1 publication Critical patent/WO1996001453A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/32—Digital ink
    • G06V30/36—Matching; Classification
    • G06V30/373—Matching; Classification using a special pattern or subpattern alphabet
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/26—Techniques for post-processing, e.g. correcting the recognition result
    • G06V30/262—Techniques for post-processing, e.g. correcting the recognition result using context analysis, e.g. lexical, syntactic or semantic context
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition

Definitions

  • This invention relates to computer input systems in general, and more specifically to an apparatus and handwriting alphabet for use in a handwritten input and recognition system used in personal computing systems such as "palm-top” computers.
  • FIGURE 1 An example of a prior art pen-based hand-held computer is shown in FIGURE 1.
  • the illustrated hand-held computer 1 is typically about 4 inches by 6.5 inches, with the majority of one surface comprising a touch- sensitive display screen 2.
  • the display screen 2 is typically a liquid crystal display (LCD) having a resolution of 256x320 pixels (although larger or smaller pixel arrays could be used) .
  • LCD liquid crystal display
  • Various technologies can be used to sense the location of a pen or stylus 3 touched against the surface of the LCD screen 2 to indicate to the computer's operating system the X-Y coordinates of the touch.
  • Various hardware buttons 4 may be provided to control different functions, and/or to turn power on or off to the unit.
  • a variety of software buttons or icons 5 may be provided, in known fashion, to indicate such functions as, for example, word processing or a delete function.
  • Computer-generated information is typically shown on the display screen 2 as ASCII charac ⁇ ters 6.
  • One such hand-held computer is available a ⁇ the "Zoomer" from Casio Corporation.
  • Ink comprises a series or trail of pixels changed (e.g., darkened or lightened) as a pen 3 is moved across the display screen 2 by a user, thus mimicking the application of real ink to paper.
  • the user writes text without boxes in a free form manner.
  • the computer applies special algorithms to separate the ink strokes into characters and then recognize each ink character as an ASCII character. It then replaces the ink representation of the characters drawn by the user with the standardized ASCII representation of those characters.
  • these systems require less input area than boxed input systems, they are still difficult to implement on a palmtop computer having a small display.
  • the computer has the additional burden of figuring out where one character ends and the next begins. This leads to recognition errors.
  • prior art handwriting recognition systems typically require between 2 to 5 seconds after the user writes the ink character on the input tablet to recognize and display the ASCII character on a display device.
  • the prior art systems require the user to write a few words and then wait several seconds for the computer to start the recognition process.
  • some systems e.g., the "Newton" from Apple Computer
  • the words are still recognized several seconds after they are written. In all cases, the user cannot immediately realize when a recognition mistake has occurred.
  • an improved pen data entry solution is needed which can accurately and efficiently recognize text on a small display. It has become evident that one crucial characteristic of such an improved solution is the ability to instantaneously (i.e., with little or no perceptible delay) recognize and display input text, similar to the response of currently available personal computers using keyboard input devices. Palm-top computers having the ability to instantly recognize and display text offer the user the opportunity to quickly recognize and correct mistakes. Instant recognition also permits the use of smaller input areas because the input area can be reused for writing subsequent characters.
  • One of the major impediments facing "instant" handwritten text recognition systems is presented by the multiple stroke (multi-stroke) characteristic of many English text characters. That is, many characters comprise more than one pen stroke.
  • a single pen stroke is defined as a continuous movement of a pen while maintaining contact with a writing tablet.
  • the letters "T", “H,” and “E” typically comprise multiple pen strokes, while the letters “S” and “0” typically comprise a single pen stroke.
  • Prior art recognition systems have had difficulty in achieving essentially “instantaneous" recognition due to the fact that characters may comprise more than one pen stroke.
  • any given input character might be a multi-stroke character
  • a vertical line might represent the letter "I” or it could represent the first stroke in the multi- stroke letters "T", "H” or "E” .
  • recognition systems have solved this ambiguity by waiting until the user stopped writing, or by having a fixed delay period after which characters were recognized, or by detecting the start of a next stroke sufficiently far from prior strokes as to indicate a new character.
  • Each of these approaches are deficient due to the recognition time delay ⁇ introduced.
  • the first approach is offered by Sharp Electronics of Japan in their PVF1 handheld computer sy ⁇ tem, which provides "immediate" recognition of both English and Japanese characters.
  • the Sharp system uses a modified boxed input method. It displays several adjacent boxes on a screen for text input. Every text character is written into one of the boxes. Recognition timing delays are reduced because the system knows to begin recognizing a character previously written into a first entry box as soon as the user begins writing into an another entry box. The recognized character is subsequently displayed upon the ⁇ creen (not in the box) as soon as the recognition process completes. Expert users can quickly enter multiple characters by alternating between two adjacent boxes.
  • Xerox teaches a method whereby every character that a user wishe ⁇ to write is repre ⁇ ented by a ⁇ ingle ⁇ troke glyph. Because every character is represented using a single stroke, recognition commences as soon as the user lifts the pen from the writing tablet.
  • the sy ⁇ tem provide ⁇ improved recognition speeds over the Sharp approach and avoids the problems associated with the writing boxes used in the Sharp system.
  • the Xerox method suffers from two major disadvantages. First, the Xerox approach is difficult to learn becau ⁇ e it require ⁇ the u ⁇ er to memorize an entirely new alphabet for entering text.
  • the alphabet i ⁇ ⁇ pecially de ⁇ igned to maximize the recognition abilitie ⁇ of the computer, not to maximize ease of learning.
  • the Xerox disclosure recognizes this difficulty yet submits that the inefficiency of learning the alphabet is compensated by the improved recognition speeds once the user becomes an expert.
  • the present invention provides such a handwritten text recognition system.
  • the present invention uses a pen or stylu ⁇ a ⁇ an input device to a pen-based computer handwriting recognition system capable of interpreting a ⁇ pecial pre ⁇ defined ⁇ et of character strokes or glyphs.
  • the invention teaches a system which provides true immediate character recognition, yet allows characters to be written with any number of ⁇ troke ⁇ , thus making it natural to use and easy to learn.
  • the present invention defines three different categories of pen strokes: (1) pre-character modifier strokes, (2) character or symbol strokes, and (3) post-character modifier ⁇ troke ⁇ .
  • Pre-character modifier strokes precede character stroke ⁇ and inform the present recognition sy ⁇ tem that ⁇ ubsequently entered character strokes are to be modified by the pre-character modifier stroke in a defined manner. They function primarily to control the interpretation of a sub ⁇ equently entered character ⁇ troke. For example, a pre-modifier control ⁇ troke may indicate that the next character stroke is to be interpreted a ⁇ a punctuation character. Pre-character modifier ⁇ trokes may or may not cause an immediate visible display change. In the preferred embodiment of the invention, pre-character modifier stroke ⁇ do re ⁇ ult in a display change (by either changing a status indicator or by displaying a temporary character) , so the user knows the pre-character modifier stroke was successfully entered.
  • Character strokes always cau ⁇ e a letter or other symbol to be displayed the moment the stroke is input on the writing tablet, interpreted in accordance with any pre-character modifier strokes previously entered. Any ⁇ tatus indicators or temporary characters displayed due to earlier pre-character modifier strokes are removed upon recognizing a character stroke.
  • Post-character modifier ⁇ trokes cause the recognition sy ⁇ tem to modify, in a defined manner, a character or ⁇ ymbol which was previously entered and displayed.
  • a post-character modifier may be used to add a diacritical mark to a character.
  • An important advantage of the present invention is its ability to recognize characters consisting of multiple pen stroke ⁇ yet ⁇ till provide in ⁇ tantaneou ⁇ recognition and di ⁇ play of the recognized character.
  • ⁇ troke ⁇ By combining mutually exclu ⁇ ive pre-character modifier ⁇ troke ⁇ , character strokes, and post-character modifier strokes, a myriad of alpha, numeric, punctuation, and accented characters may be entered with natural and easy to learn styles.
  • the present invention is flexible and can be u ⁇ ed to enter not only Engli ⁇ h and other Roman character-ba ⁇ ed languages, but other written alphabets, such as Japanese hiragana and katakana.
  • FIGURE 1 is a front left-side per ⁇ pective drawing ⁇ howing a prior art pen-ba ⁇ ed hand-held computer.
  • FIGURE 2 i ⁇ a flow chart de ⁇ cribing the preferred embodiment of the handwriting recognition system of the present invention.
  • FIGURE 3 shows the pen strokes used to represent the 26 letters of the ordinary English alphabet in a prior art sy ⁇ tem taught by Xerox.
  • FIGURE 4a shows the pen ⁇ troke ⁇ used to represent the 26 letters of the ordinary English alphabet in the preferred embodiment of the pre ⁇ ent invention.
  • FIGURE 4b shows the pen stroke ⁇ u ⁇ ed to repre ⁇ ent the 10 number ⁇ of the Arabic number system in the preferred embodiment of the present invention.
  • FIGURE 5a shows the pen strokes used as character strokes to represent three common "non-printing" characters (the ⁇ pace, the back ⁇ pace, and the carriage return) in the preferred embodiment of the present invention.
  • FIGURE 5b shows the pen stroke ⁇ u ⁇ ed as pre- character modifier strokes to create capitalized character ⁇ , punctuation, and extended characters in the preferred embodiment of the present invention.
  • FIGURE 5c show ⁇ the pen ⁇ troke ⁇ u ⁇ ed to repre ⁇ ent common punctuation symbol ⁇ when preceded by the pre- character modifier stroke for punctuation in the preferred embodiment of the pre ⁇ ent invention.
  • FIGURE 6 ⁇ how ⁇ the pen ⁇ troke ⁇ u ⁇ ed to repre ⁇ ent several extended symbols when preceded by the pre- character modifier stroke for extended symbol ⁇ in the preferred embodiment of the pre ⁇ ent invention.
  • FIGURE 7a how ⁇ the pen strokes used in the preferred embodiment of the present invention as post- character modifier strokes to add accents to letters created with a character stroke and none or more pre- character modifier ⁇ troke ⁇ .
  • FIGURE 7b how ⁇ several examples of writing accented characters using multiple pen strokes used in the preferred embodiment of the present invention.
  • FIGURE 7c how ⁇ an example of writing an upper case accented character using a pre-character modifier stroke, a character stroke, and a post-character modifier stroke.
  • FIGURE 8 show ⁇ the dictionary mapping used to enter katakana characters. Each of these entries consi ⁇ t ⁇ of either a ⁇ ingle character stroke, a pre- character modifier stroke combined with a character ⁇ troke, or two pre-character modifier strokes combined with a character stroke. This mapping follows the well known romaji input sy ⁇ tem.
  • FIGURE 9 ⁇ how ⁇ the sequence of stroke ⁇ and the re ⁇ ulting di ⁇ play when entering a three ⁇ troke katakana character.
  • FIGURE 10 shows the dictionary mapping used to enter ⁇ pecial two katakana character sequences. Each of these entrie ⁇ consists of two pre-character modifier strokes and one character ⁇ troke. This mapping follows the well known romaji input system.
  • FIGURE 11a shows named pen strokes useful for defining various symbol ⁇ in the preferred embodiment of the pre ⁇ ent invention.
  • FIGURE lib shows variations of the pen strokes that can be used to repre ⁇ ent the 26 letters of the ordinary English alphabet in the preferred embodiment of the pre ⁇ ent invention.
  • FIGURE lie ⁇ hows the pen strokes used as character stroke ⁇ to represent common "non-printing" characters in the preferred embodiment of the present invention.
  • FIGURE lid shows the pen strokes used as character strokes to repre ⁇ ent common punctuation characters in the preferred embodiment of the present invention.
  • FIGURE lie shows the pen strokes used as character strokes to represent additional punctuation characters in the preferred embodiment of the present invention.
  • FIGURE llf shows the pen strokes used as character strokes to represent extended characters in the preferred embodiment of the present invention.
  • FIGURE llg shows the pen strokes used as character ⁇ trokes to represent non-accented foreign characters in the preferred embodiment of the present invention.
  • Like reference numbers and designations in the various drawings refer to like elements.
  • the pre ⁇ ent invention i ⁇ preferably implemented as a computer program operating on a pen-based computer system such as the "Zoomer” and “Newton” products de ⁇ cribed above.
  • the computer program is stored on a storage media or device readable by a computer, and configure ⁇ and operate ⁇ the computer when the ⁇ torage media or device i ⁇ read by the computer, the computer being operated to determine, recognize, cla ⁇ ify, and sometimes display handwritten strokes.
  • the logic functions of ⁇ uch a computer program may be implemented a ⁇ an electronic ⁇ yste , such as a firmware programmed, microcoded, or hardwired logic ⁇ ystem.
  • FIGURE 2 is a flow chart that describe ⁇ the ba ⁇ ic proce ⁇ of the invention.
  • a stroke is a movement by a u ⁇ er of a pen or equivalent input device on a tablet, pad, or digitizer.
  • a stroke begins when the pen touches the tablet, and ends when the user removes the pen from the tablet.
  • the position and movement of the pen during the stroke is converted into a series of X-Y coordinates. This pen-position data is received from the pen input device and pas ⁇ ed to the glyph recognizer portion of the character input ⁇ ystem.
  • the glyph recognizer logic recognizes the pen- ⁇ troke data a ⁇ being a rendition of a particular glyph based on the movement and shape of the input ⁇ troke.
  • the recognition procedure used by the present invention is es ⁇ entially that di ⁇ clo ⁇ ed in U.S. Patent
  • each glyph Associated with each glyph is a definition, which includes the current classification of the glyph a ⁇ a pre-character modifier ⁇ troke, a character (meaning any symbol) stroke, a post-character modifier ⁇ troke, or a ⁇ a currently una ⁇ signed stroke.
  • the definition of each glyph also includes what character or other indicator, if any, is to be output in respon ⁇ e to a ⁇ troke that matche ⁇ that glyph. It al ⁇ o include ⁇ a specification of what changes if any, are to be made to any glyph definitions in respon ⁇ e to a ⁇ troke that matche ⁇ that glyph.
  • Steps 103 and 104 are decision steps. Different actions are taken depending on whether the current ⁇ troke represents a pre-character modifier stroke, a character stroke, a post-character modifier stroke, or an unassigned ⁇ troke. In the case of pre-character modifier stroke ⁇ , control passes ⁇ to ⁇ tep 200.
  • the proce ⁇ ing logic cau ⁇ e ⁇ an indication to be made that a particular pre-character modifier ⁇ troke has been input. While the pre-character modifier stroke does not result in a character being output from the character recognizer, it is nevertheles ⁇ desirable to display to the user an indicator of what pre-character modifier stroke was received. In the preferred embodiment of the invention a character i ⁇ di ⁇ played which is representative of the pre-character modifier ⁇ troke.
  • step 201 the definition in the glyph dictionary of one or more glyphs is modified according to the input pre-character modifier ⁇ troke, a ⁇ further described below.
  • step 300 if the clas ⁇ of the current ⁇ troke i ⁇ "character”, then any previously displayed indicators or characters representing pre-character modifier strokes are deleted and removed from the display.
  • the definition of the corresponding glyph in the dictionary includes the character that is to be output in response to receiving strokes matching that glyph.
  • step 301 the proces ⁇ ing logic output ⁇ that character for display.
  • step 302 the definition in the glyph dictionary of one or more glyphs is modified according to the character ⁇ troke entered and the current state of the dictionary, as further described below.
  • step 400 if the class of the current pen stroke is "post-character modifier", then the processing logic cause ⁇ the mo ⁇ t recently output character to be removed and replaced with a new or modified character.
  • the new or modified character is determined by the glyph definition corresponding to the post-character modifier stroke.
  • step 401 the definition in the glyph dictionary of one or more glyphs is modified according to the post-character modifier stroke entered and the current state of the dictionary, as further de ⁇ cribed below.
  • the glyph dictionary ⁇ pecifies exactly one interpretation for each input glyph that the ⁇ ystem knows how to recognize. It is possible for that interpretation to be to ignore that glyph.
  • the glyph which looks like a pair of eyegla ⁇ e ⁇ (see FIGURE 5c) is not active in the initial dictionary definitions.
  • the stroke is recognized, but ignored.
  • the eyeglas ⁇ e ⁇ glyph will not be ignored.
  • the eyegla ⁇ e ⁇ glyph become ⁇ active as a post-character modifier stroke and corresponds to the umlaut accent.
  • the eyeglas ⁇ e ⁇ glyph is then input, the original "u” or "a” input character will be changed to " ⁇ " or "a”.
  • the eyeglas ⁇ es glyph is defined a ⁇ a character ⁇ troke repre ⁇ enting the percent "%" character.
  • each stroke that i ⁇ recognized must be a pre-character modifier stroke, a character stroke, a post-character modifier stroke, or unassigned. It cannot at a any point in time have multiple clas ⁇ ification ⁇ . Nevertheless, any stroke, when recognized, may modify the definition of any or all glyphs so as to reclassify them.
  • a particular glyph may correspond sometimes to a character ⁇ troke, and at other times to a post-character or pre-character modifier ⁇ troke.
  • Step ⁇ 201, 302, and 401 all modify the dictionary to reassign the definitions of glyphs.
  • the dictionary can be organized to allow for this modification.
  • the dictionary is implemented as a tree data structure.
  • the root level of the tree contain ⁇ definitions for all the pre-character modifier strokes and character stroke ⁇ defined in the initial ⁇ tate of the system. When one of these strokes is entered, there may be a subsequent branch defined for that stroke which contains new definitions for all the glyphs.
  • Each ⁇ troke leads to a new point in the tree data structure, thus modifying the dictionary.
  • Implementing the dictionary a ⁇ a tree data ⁇ tructure i ⁇ flexible and allows the dictionary to be located in ROM. Other methods for organizing the dictionary would be pos ⁇ ible, as known in the art.
  • FIGURE 3 show ⁇ the glyph ⁇ taught in the prior art referenced earlier from Xerox for entering the Engli ⁇ h alphabet (in thi ⁇ and subsequent figures, a dot at one end of a stroke indicates that the stroke is drawn starting at that end.)
  • This system achieves immediate recognition by forcing every character to be input with only a single stroke.
  • This rigid unistroke approach makes it difficult to extend the character set significantly beyond the base alphabet.
  • Xerox does not teach any method for punctuation, accented characters, extended characters, upper ca ⁇ e letters, numbers, or other non-Roman alphabets.
  • the Xerox alphabet was also de ⁇ igned for speed of entry and simplicity of recognition. This result ⁇ in a glyph ⁇ et that look ⁇ unfamiliar and i ⁇ difficult to learn.
  • FIGURE 11a shows ⁇ named pen ⁇ troke ⁇ useful for defining various symbols in the preferred embodiment of the present invention. These ⁇ troke ⁇ , variation ⁇ of the ⁇ e strokes, and other ⁇ troke ⁇ , a ⁇ character ⁇ troke ⁇ , pre-character modifier ⁇ troke ⁇ , and po ⁇ t-character modifier ⁇ trokes, can be defined to represent es ⁇ entially any character or ⁇ ymbol.
  • FIGURE 4a ⁇ hows the glyph set used in the preferred embodiment of the present invention for entering the English alphabet. This alphabet wa ⁇ chosen to be as familiar as pos ⁇ ible and ea ⁇ y to learn, yet adhere to the principles taught in this invention.
  • FIGURE 4a The alphabet ⁇ hown in FIGURE 4a provide ⁇ near 100% recognition accuracy, yet is easy to learn and use due to it ⁇ obviou ⁇ ⁇ imilarity to natural handwriting styles.
  • FIGURE lib shows that there are actually multiple ways (different ⁇ trokes) which can be used to write many of these letters, making the system even easier to learn and u ⁇ e.
  • the recognition ⁇ ystem simply maps the input strokes for these letters to the same output symbol in the glyph dictionary. For example, there are two ways to write a "Y": the glyph shown in FIGURE 4a, and a shape similar to a lower case ⁇ cripted "Y" a ⁇ ⁇ hown in FIGURE lib.
  • FIGURE 4b shows the glyph set used in the preferred embodiment of the pre ⁇ ent invention for entering the digit ⁇ 0 through 9. Many of these glyphs are also used to enter letters of the alphabet.
  • One method of overcoming this ambiguity problem is to have a separate numeric mode where a user only can enter digits. This numeric mode can be entered by pressing a button on the display of the computer, by writing a "num-lock" glyph, or other means.
  • a u ⁇ er can enter and exit a numeric mode by either tapping an on- ⁇ creen icon or by writing a "forward ⁇ la ⁇ h" stroke (a slanted line written from bottom left to upper right) . Testing has shown that occasionally a user forgets to exit numeric mode after writing several digits. A refinement of the present invention helps fix this problem by automatically exiting numeric mode when the user writes a character stroke which can only be interpreted as a letter or when the user writes the pre-character modifier ⁇ troke for capital letter shift.
  • FIGURE 5a shows the glyph set used in the preferred embodiment of the present invention for entering three common "non-printing" characters: the space, the backspace, and the carriage return.
  • a recognition sy ⁇ tem with immediate re ⁇ ponse operate ⁇ in many ways more like a keyboard than a conventional handwriting recognition ⁇ ystem. For example, because user ⁇ ⁇ ee results instantly, they find it more natural to "backspace" over an incorrect character than write over it or delete it with a gesture. Therefore, simple character strokes may be provided for space, backspace, carriage return, and other keyboard equivalents.
  • FIGURE lie details other non-printing keyboard equivalents.
  • FIGURE 5b shows the glyph set used in the preferred embodiment of the present invention as pre- character modifier strokes for the English language.
  • the three strokes are used to respectively indicate that the sub ⁇ equent character ⁇ troke repre ⁇ ents a capital letter, a punctuation character, or an extended character.
  • other pre-character modifier ⁇ trokes may be defined as "sticky" shift ⁇ trokes that take effect until reset. For example, a "caps lock" ⁇ troke would cau ⁇ e all subsequent ⁇ troke ⁇ to be interpreted a ⁇ if the pre- character modifier "cap ⁇ " stroke had been entered before each sub ⁇ equent stroke.
  • FIGURE 5c shows the glyph set used in the preferred embodiment of the present invention for entering common punctuation symbol ⁇ . All of these glyph ⁇ are cla ⁇ ified a ⁇ character ⁇ troke ⁇ when preceded by a pre-character modifier ⁇ troke repre ⁇ enting punctuation character ⁇ . A ⁇ ⁇ hown in FIGURE 5b, a "dot" ⁇ troke i ⁇ u ⁇ ed in the preferred embodiment of the invention to indicate a punctuation character. A more complete list of preferred punctuation character ⁇ is shown in FIGURE lid and FIGURE lie.
  • FIGURE 6 shows the glyph set used in the preferred embodiment of the present invention for entering several extended symbols.
  • All of the ⁇ e glyph ⁇ are classified as character stroke ⁇ when preceded by a pre-character modifier ⁇ troke repre ⁇ enting extended symbol characters.
  • a " ⁇ lash" stroke is used in the preferred embodiment of the invention to indicate an extended symbol character.
  • FIGURE 7a shows ⁇ the glyph ⁇ et u ⁇ ed in the preferred embodiment of the present invention for adding accents or diacritical marks to letters.
  • These glyphs are classified as po ⁇ t-character modifier strokes after entering a letter which i ⁇ capable of being accented.
  • FIGURE 7b shows several examples of how a user would write an accented character using a two stroke combination of a character ⁇ troke and a po ⁇ t-character modifier ⁇ troke.
  • Writing these accented character ⁇ u ⁇ ing the present invention is very similar to how the user would write them on paper.
  • the user writes a base letter, then adds an accent.
  • the base letter stroke After writing the base letter stroke, the corresponding character is immediately output to the display.
  • the accent post- character modifier stroke the base letter i ⁇ erased and replaced with the correct accented letter.
  • the system achieves an immediate recognition respon ⁇ e on a multiple ⁇ troke character even though it i ⁇ initially unknown whether the u ⁇ er is writing a single stroke letter or a multiple stroke accented letter.
  • FIGURE 7c shows an example of how a user would write an accented upper ca ⁇ e character u ⁇ ing a three ⁇ troke combination of a pre-character modifier stroke, a character stroke, and a post-character modifier stroke.
  • the strokes are numbered in the order they are written.
  • the user writes a pre-character modifier stroke indicating that the following character ⁇ hould be capitalized.
  • Thi ⁇ causes the display of a temporary character indicating the acceptance of the stroke, in this case an "up arrow" character.
  • the user writes a character stroke.
  • the temporary character is removed and replaced by the appropriate upper case letter.
  • the user writes a post-character modifier ⁇ troke, causing the base upper case letter to be erased and replaced with the correct upper case accented letter.
  • FIGURE 8 illu ⁇ trates one method of how the invention can be adapted to quickly and easily enter the Japanese katakana alphabet.
  • Japan a common method for entering katakana with a keyboard i ⁇ called the -"romaji" method.
  • a u ⁇ er types the English phonetic equivalent of the katakana characters.
  • the user types " ⁇ u” which re ⁇ ult ⁇ in the di ⁇ play of the corre ⁇ ponding katakana symbol.
  • the user types "shi" which results in the display of the corresponding katakana symbol.
  • All of the approximately 90 katakana characters can be input with combinations of one, two, or three typed letters.
  • the present invention can duplicate this method ⁇ imply with a change to the dictionary of glyph mapping ⁇ .
  • the new dictionary assigns strokes representing the consonant ⁇ "BCDGHKMNPRSTXYZ" a ⁇ pre-character modifier ⁇ trokes and as ⁇ igns strokes representing the vowels "AEIOU" as character strokes.
  • Some katakana character ⁇ are entered with ju ⁇ t one character ⁇ troke.
  • Some katakana character ⁇ are entered with one pre-character modifier stroke and one character stroke.
  • Some katakana characters are entered with two pre-character modifier strokes and one character ⁇ troke.
  • two temporary indicator characters are preferably displayed, representing the two pre- character modifier stoke ⁇ . Both temporary character ⁇ are deleted and replaced with the final katakana character upon entering the character ⁇ troke.
  • Thi ⁇ sequence is illustrated in FIGURE 9.
  • FIGURE 10 shows ⁇ how ⁇ pecial double katakana ⁇ ymbol combinations can be entered with three stroke combinations of two pre-character modifier strokes and one character ⁇ troke.
  • a character ⁇ troke or post-character modifier stroke can result in the output and display of any length sequence of character ⁇ .
  • the stroke representing the letter "N” is initially a pre-character modifier ⁇ troke. Once entered it is reas ⁇ igned as a character stroke. This permits the entry of the "NN" katakana symbol, which is an exception to the general consonant-vowel pairing for katakana.
  • FIGURE llg shows the pen strokes u ⁇ ed a ⁇ character ⁇ troke ⁇ to repre ⁇ ent non-accented foreign character ⁇ in the preferred embodiment of the pre ⁇ ent invention.
  • the principles of the present invention can be extended to other characters sets, such as Japane ⁇ e hiragana, Chinese kanji, or Korean hangul.
  • the concepts of the present invention also can be extended to provide greater flexibility for user ⁇ .
  • the computer system could be programmed to allow a user to define new input strokes, and/or to as ⁇ ociate ⁇ ymbols, characters, or even complete words or phrase ⁇ , to a combination of input ⁇ troke ⁇ .
  • a user-maintained glossary could be built where the user could define the sequences of characters — or symbols, text, or program functions — to be as ⁇ ociated with a ⁇ troke, a multi- ⁇ troke combination, or ⁇ equence of multiple ⁇ troke combination ⁇ .
  • the u ⁇ er could al ⁇ o define new ⁇ troke ⁇ within a table (or other data structure) and assign context to each such stroke.
  • the present recognition ⁇ y ⁇ tem provides several improvements over prior art sy ⁇ tems by achieving immediate recognition of multiple stroke characters without u ⁇ ing boxes for input. It improves over prior art sy ⁇ tem ⁇ by immediately di ⁇ playing every character written a ⁇ ⁇ oon as the user finishes the character. No unneces ⁇ ary delay ⁇ are incurred, nor are additional action ⁇ required of a u ⁇ er to translate input. The immediate respon ⁇ e help ⁇ a user to quickly identify mistake ⁇ and correct them.
  • the pre ⁇ ent system accomplishe ⁇ thi ⁇ immediate re ⁇ pon ⁇ e while at the ⁇ ame time accommodating characters which are more easily learned and written using multiple stroke ⁇ .
  • Example ⁇ provided above include accented characters, punctuation characters, extended symbols, the letter "X”, and the katakana character set. Defining character ⁇ with multiple ⁇ troke ⁇ make ⁇ the pre ⁇ ent invention much easier to use and learn than prior art systems requiring single stroke only characters. Another advantage provided by the present recognition system i ⁇ that large sets of characters can be represented without relying on a large set of unique strokes or glyphs. For example, accented letters use the same base character stroke as their unaccented counterpart letters. Similarly, punctuation marks and capitalized letters are realized using a combination of pre-modifier control ⁇ trokes and character ⁇ troke ⁇ .
  • the present invention provides an additional advantage over prior art systems because the present system does not require multiple writing boxes or other large on-screen gadgetry. Valuable display space is thereby saved, allowing the pre ⁇ ent system to be used on very small devices.
  • the present ⁇ y ⁇ tem al ⁇ o teaches an alphabet for inputing Roman-character based languages which, although not identical to a user's conventional writing ⁇ tyle, i ⁇ very ⁇ imilar and therefore ea ⁇ y to learn. A user can achieve near 100% recognition accuracy when using this alphabet, yet it is very easy to learn and use because it is very ⁇ imilar to a natural writing style.

Abstract

A pen-based computer text input system capable of interpreting a special predefined set of single stroke glyphs (101). Each input stroke is identified with one of three characters: (1) pre-character modifier strokes, (2) character or symbol strokes, or (3) post-character modifier strokes. Pre-character modifier strokes precede character strokes and inform the present recognition system that a subsequently entered character stroke is to be modified by the pre-character modifier stroke in a pre-defined manner. Character strokes cause a character or symbol to be displayed on the display device the moment it is input on the writing tablet (200), interpreted in accordance with any pre-character modifier stroke. A post-character modifier stroke causes the recognition system to modify, in a pre-defined manner, a character or symbol which was previously entered and displayed (400). An important advantage of the present invention is its ability to provide immediate recognition of multiple stroke characters without using boxed input.

Description

MULTIPLE PEN STROKE CHARACTER SET AND HANDWRITING RECOGNITION SYSTEM
Field of the Invention
This invention relates to computer input systems in general, and more specifically to an apparatus and handwriting alphabet for use in a handwritten input and recognition system used in personal computing systems such as "palm-top" computers.
Description of Related Art As computers have become increasingly popular for various applications, portable computers have been developed for a wide variety of uses. While many such portable computers use a traditional keyboard for input, for smaller computers, particularly including hand-held computers, the use of "pens" as an interface has been introduced as a way of making a small computer easier to use. With a pen interface, a user can place a pen or stylus directly on a touch-sensitive screen of the computer to control the software running on the computer and to input information. For many people, controlling a computer and entering text with a pen is more natural than using a keyboard. An example of a prior art pen-based hand-held computer is shown in FIGURE 1. The illustrated hand-held computer 1 is typically about 4 inches by 6.5 inches, with the majority of one surface comprising a touch- sensitive display screen 2. The display screen 2 is typically a liquid crystal display (LCD) having a resolution of 256x320 pixels (although larger or smaller pixel arrays could be used) . Various technologies can be used to sense the location of a pen or stylus 3 touched against the surface of the LCD screen 2 to indicate to the computer's operating system the X-Y coordinates of the touch. Various hardware buttons 4 may be provided to control different functions, and/or to turn power on or off to the unit. In addition, a variety of software buttons or icons 5 may be provided, in known fashion, to indicate such functions as, for example, word processing or a delete function. Computer-generated information is typically shown on the display screen 2 as ASCII charac¬ ters 6. One such hand-held computer is available aε the "Zoomer" from Casio Corporation.
A common characteristic of such pen-based computers is the use of electronic "ink". "Ink" comprises a series or trail of pixels changed (e.g., darkened or lightened) as a pen 3 is moved across the display screen 2 by a user, thus mimicking the application of real ink to paper.
Some prior art system designers suggest the use of unrecognized handwritten ink input. Although this approach works well for recording notes for personal use, it is not always suitable for data entry into a file which needs to be searched at a later date. In addition, ink requires considerably more storage space than ASCII characters. Accordingly, practical pen-based computers need a method of inputing text which usually includes some form of recognition system.
Various methods of recognizing handwriting are well known. One prior art approach is to provide a series of boxes in the input area (which is usually the display area) for entering character information. These systems use boxes for entry of text in an attempt to improve accuracy of recognition and to separate one character from the next. In these systems, an array of boxes is displayed and the user writes one character in each box. Although the boxes aid in improving the accuracy of recognition, most people find it awkward to write in boxes. Additionally, due to the number of boxes necessary to capture even a short sentence, these systems are not very practical in a palm-top computer having a reduced data input area. Another character recognition system is described in U.S. Patent No. 5,125,039, entitled "Object Recognition System", by the inventor of the present invention. In such a system, the user writes text without boxes in a free form manner. After a user inputs several ink characters, the computer applies special algorithms to separate the ink strokes into characters and then recognize each ink character as an ASCII character. It then replaces the ink representation of the characters drawn by the user with the standardized ASCII representation of those characters. Although these systems require less input area than boxed input systems, they are still difficult to implement on a palmtop computer having a small display. In addition, the computer has the additional burden of figuring out where one character ends and the next begins. This leads to recognition errors.
One additional major difficulty presented by prior art handwriting recognition systems is the delay time between text input and text recognition. The prior art systems typically require between 2 to 5 seconds after the user writes the ink character on the input tablet to recognize and display the ASCII character on a display device. In typical use, the prior art systems require the user to write a few words and then wait several seconds for the computer to start the recognition process. Alternatively, some systems (e.g., the "Newton" from Apple Computer) perform recognition without the user stopping and waiting. But in these systems, the words are still recognized several seconds after they are written. In all cases, the user cannot immediately realize when a recognition mistake has occurred. This type of handwritten text recognition system makes error correction difficult because the user must constantly look at the display for words which the user input several seconds before in order to be sure that text was correctly entered and correctly interpreted. Moreover, once a user detects an error, error correction is difficult because the user has to first select the word or characters which need to be corrected. In summary, three of the major problems with current handwriting recognition systems are the delay from writing to recognition, the limited writing area of palmtop computers, and the difficulty of accurately recognizing separate characters in non-boxed entry systems.
Therefore, an improved pen data entry solution is needed which can accurately and efficiently recognize text on a small display. It has become evident that one crucial characteristic of such an improved solution is the ability to instantaneously (i.e., with little or no perceptible delay) recognize and display input text, similar to the response of currently available personal computers using keyboard input devices. Palm-top computers having the ability to instantly recognize and display text offer the user the opportunity to quickly recognize and correct mistakes. Instant recognition also permits the use of smaller input areas because the input area can be reused for writing subsequent characters. One of the major impediments facing "instant" handwritten text recognition systems is presented by the multiple stroke (multi-stroke) characteristic of many English text characters. That is, many characters comprise more than one pen stroke. In thiε context, a single pen stroke is defined as a continuous movement of a pen while maintaining contact with a writing tablet. For example, the letters "T", "H," and "E" typically comprise multiple pen strokes, while the letters "S" and "0" typically comprise a single pen stroke. Prior art recognition systems have had difficulty in achieving essentially "instantaneous" recognition due to the fact that characters may comprise more than one pen stroke.
For example, due to the possibility that any given input character might be a multi-stroke character, it has been difficult to determine when a user has completed writing a one stroke (unistroke) character, or when the user is continuing to write a multi-stroke character. For example, a vertical line might represent the letter "I" or it could represent the first stroke in the multi- stroke letters "T", "H" or "E" . In the past, recognition systems have solved this ambiguity by waiting until the user stopped writing, or by having a fixed delay period after which characters were recognized, or by detecting the start of a next stroke sufficiently far from prior strokes as to indicate a new character. Each of these approaches are deficient due to the recognition time delayε introduced.
Recently, two approaches have been attempted for immediate recognition of handwritten text. Neither of these two approaches has proven wholly satisfactory. The first approach is offered by Sharp Electronics of Japan in their PVF1 handheld computer syεtem, which provides "immediate" recognition of both English and Japanese characters. The Sharp system uses a modified boxed input method. It displays several adjacent boxes on a screen for text input. Every text character is written into one of the boxes. Recognition timing delays are reduced because the system knows to begin recognizing a character previously written into a first entry box as soon as the user begins writing into an another entry box. The recognized character is subsequently displayed upon the εcreen (not in the box) as soon as the recognition process completes. Expert users can quickly enter multiple characters by alternating between two adjacent boxes. This is different from previous boxed input systems where the user wrote characters in a left to right fashion in a multitude of boxes. The Sharp approach achieves faster recognition response on a reduced display area than previous systems. However, it suffers from several disadvantages. Although the Sharp system uses fewer boxes (aε little aε two will εuffice) , the boxeε still occupy a significant amount of εcreen area. In addition, aε with all boxed input εyεtems, the user has to be careful to always write within the box. If one stroke of a multi- stroke character falls outside the box, the character will be recognized incorrectly. This requires the user to carefully look at the screen at all times while writing. Another, and more seriouε drawback, iε that the recognition of characterε is not completely "instant". In this system, recognition of one character does not commence until the user starts writing a subsequent character. Although this system representε an improvement over the prior art systemε where recognition delays were longer, recognition is still delayed. So, when the user writes just one character, or when the user writeε the laεt character in a εequence, that character iε not recognized until after a pre-determined time-out delay. This delay after writing a single character makes it frustrating and therefore impractical to make quick editing changes such as writing a "backspace" character, or to insert a single character.
A second approach at immediate recognition of handwritten text was recently described by Xerox Corporation of Palo Alto, CA. Xerox teaches a method whereby every character that a user wisheε to write is repreεented by a εingle εtroke glyph. Because every character is represented using a single stroke, recognition commences as soon as the user lifts the pen from the writing tablet. The syεtem provideε improved recognition speeds over the Sharp approach and avoids the problems associated with the writing boxes used in the Sharp system. However, the Xerox method suffers from two major disadvantages. First, the Xerox approach is difficult to learn becauεe it requireε the uεer to memorize an entirely new alphabet for entering text. The alphabet iε εpecially deεigned to maximize the recognition abilitieε of the computer, not to maximize ease of learning. The Xerox disclosure recognizes this difficulty yet submits that the inefficiency of learning the alphabet is compensated by the improved recognition speeds once the user becomes an expert.
Second, the Xerox approach iε difficult to implement with a full εet of characterε. The user must learn a single stroke representation for every possible character. Although this task may be feasible when representing only the 26 letters of the English alphabet in one case (upper or lower) , there are many more characters requiring representation and recognition. For example, both upper and lower case English characters must be recognized. European languages have multiple accented characters as well as many other unique characters. In addition, a myriad of punctuation marks and mathematical symbols require representation. Assigning each of these characters to a unique εingle stroke glyph requires inventing many strange and novel glyphs that are non-intuitive and therefore difficult to learn by the average user. Compounding this difficulty is the problem of similarly looking accented characters (for example, A, A, A, A, and A) . Aεεigning unique glyphs for these characters would make the extended alphabet especially non-intuitive and difficult to learn. The limitations of a uniεtroke alphabet as taught by Xerox are magnified when trying to create an immediate recognition system for Asian languages. For example, it is nearly imposεible to define single stroke alphabetε for Aεian εymbolε, εuch as Japanese katakana or hiragana, Chineεe kanji, or Korean hangul, due to the large number of εymbolε that need to be repreεented. Accordingly, there is a need for an improved handwritten text recognition εyεte capable of inεtantaneouεly and accurately recognizing handwritten text entrieε. There iε alεo a need for an improved handwritten text entry and recognition εystem which is user-friendly, easy to learn, and easy to implement.
The present invention provides such a handwritten text recognition system.
SUMMARY OF THE INVENTION The present invention uses a pen or styluε aε an input device to a pen-based computer handwriting recognition system capable of interpreting a εpecial pre¬ defined εet of character strokes or glyphs. The invention teaches a system which provides true immediate character recognition, yet allows characters to be written with any number of εtrokeε, thus making it natural to use and easy to learn. The present invention defines three different categories of pen strokes: (1) pre-character modifier strokes, (2) character or symbol strokes, and (3) post-character modifier εtrokeε. Pre-character modifier strokes precede character strokeε and inform the present recognition syεtem that εubsequently entered character strokes are to be modified by the pre-character modifier stroke in a defined manner. They function primarily to control the interpretation of a subεequently entered character εtroke. For example, a pre-modifier control εtroke may indicate that the next character stroke is to be interpreted aε a punctuation character. Pre-character modifier εtrokes may or may not cause an immediate visible display change. In the preferred embodiment of the invention, pre-character modifier strokeε do reεult in a display change (by either changing a status indicator or by displaying a temporary character) , so the user knows the pre-character modifier stroke was successfully entered.
Character strokes always cauεe a letter or other symbol to be displayed the moment the stroke is input on the writing tablet, interpreted in accordance with any pre-character modifier strokes previously entered. Any εtatus indicators or temporary characters displayed due to earlier pre-character modifier strokes are removed upon recognizing a character stroke.
Post-character modifier εtrokes cause the recognition syεtem to modify, in a defined manner, a character or εymbol which was previously entered and displayed. For example, a post-character modifier may be used to add a diacritical mark to a character.
An important advantage of the present invention is its ability to recognize characters consisting of multiple pen strokeε yet εtill provide inεtantaneouε recognition and diεplay of the recognized character. By combining mutually excluεive pre-character modifier εtrokeε, character strokes, and post-character modifier strokes, a myriad of alpha, numeric, punctuation, and accented characters may be entered with natural and easy to learn styles.
The use of the three different types of strokeε guaranteeε that the εyεte alwayε knowε whether the uεer is starting a new character, has completed a character, or is modifying a previously recognized character. This enables the system to provide the immediate response that is desired.
It will be shown that the present invention is flexible and can be uεed to enter not only Engliεh and other Roman character-baεed languages, but other written alphabets, such as Japanese hiragana and katakana.
The detailε of the preferred embodiment of the preεent invention are set forth in the accompanying drawings and the description below. Once the details of the invention are known, numerous additional innovations and changes will become obvious to one skilled in the art.
BRIEF DESCRIPTION OF THE DRAWINGS FIGURE 1 is a front left-side perεpective drawing εhowing a prior art pen-baεed hand-held computer.
FIGURE 2 iε a flow chart deεcribing the preferred embodiment of the handwriting recognition system of the present invention.
FIGURE 3 shows the pen strokes used to represent the 26 letters of the ordinary English alphabet in a prior art syεtem taught by Xerox.
FIGURE 4a εhowε the pen εtrokeε used to represent the 26 letters of the ordinary English alphabet in the preferred embodiment of the preεent invention. FIGURE 4b shows the pen strokeε uεed to repreεent the 10 numberε of the Arabic number system in the preferred embodiment of the present invention.
FIGURE 5a shows the pen strokes used as character strokes to represent three common "non-printing" characters (the εpace, the backεpace, and the carriage return) in the preferred embodiment of the present invention.
FIGURE 5b shows the pen strokeε uεed as pre- character modifier strokes to create capitalized characterε, punctuation, and extended characters in the preferred embodiment of the present invention.
FIGURE 5c showε the pen εtrokeε uεed to repreεent common punctuation symbolε when preceded by the pre- character modifier stroke for punctuation in the preferred embodiment of the preεent invention.
FIGURE 6 εhowε the pen εtrokeε uεed to repreεent several extended symbols when preceded by the pre- character modifier stroke for extended symbolε in the preferred embodiment of the preεent invention.
FIGURE 7a εhowε the pen strokes used in the preferred embodiment of the present invention as post- character modifier strokes to add accents to letters created with a character stroke and none or more pre- character modifier εtrokeε.
FIGURE 7b εhowε several examples of writing accented characters using multiple pen strokes used in the preferred embodiment of the present invention.
FIGURE 7c εhowε an example of writing an upper case accented character using a pre-character modifier stroke, a character stroke, and a post-character modifier stroke.
FIGURE 8 showε the dictionary mapping used to enter katakana characters. Each of these entries consiεtε of either a εingle character stroke, a pre- character modifier stroke combined with a character εtroke, or two pre-character modifier strokes combined with a character stroke. This mapping follows the well known romaji input syεtem. FIGURE 9 εhowε the sequence of strokeε and the reεulting diεplay when entering a three εtroke katakana character.
FIGURE 10 εhows the dictionary mapping used to enter εpecial two katakana character sequences. Each of these entrieε consists of two pre-character modifier strokes and one character εtroke. This mapping follows the well known romaji input system.
FIGURE 11a shows named pen strokes useful for defining various symbolε in the preferred embodiment of the preεent invention.
FIGURE lib shows variations of the pen strokes that can be used to repreεent the 26 letters of the ordinary English alphabet in the preferred embodiment of the preεent invention. FIGURE lie εhows the pen strokes used as character strokeε to represent common "non-printing" characters in the preferred embodiment of the present invention.
FIGURE lid shows the pen strokes used as character strokes to repreεent common punctuation characters in the preferred embodiment of the present invention.
FIGURE lie shows the pen strokes used as character strokes to represent additional punctuation characters in the preferred embodiment of the present invention.
FIGURE llf shows the pen strokes used as character strokes to represent extended characters in the preferred embodiment of the present invention.
FIGURE llg shows the pen strokes used as character εtrokes to represent non-accented foreign characters in the preferred embodiment of the present invention. Like reference numbers and designations in the various drawings refer to like elements.
DETAILED DESCRIPTION OF THE INVENTION Throughout this description, the preferred embodiment and examples shown should be conεidered aε exemplars, rather than as limitations on the present invention. Overview
The preεent invention iε preferably implemented as a computer program operating on a pen-based computer system such as the "Zoomer" and "Newton" products deεcribed above. The computer program is stored on a storage media or device readable by a computer, and configureε and operateε the computer when the εtorage media or device iε read by the computer, the computer being operated to determine, recognize, claεεify, and sometimes display handwritten strokes. However, as is known in the art, the logic functions of εuch a computer program may be implemented aε an electronic εyste , such as a firmware programmed, microcoded, or hardwired logic εystem. FIGURE 2 is a flow chart that describeε the baεic proceεε of the invention. Operation εtartε at εtep 100, where a pen εtroke iε received. A stroke is a movement by a uεer of a pen or equivalent input device on a tablet, pad, or digitizer. A stroke begins when the pen touches the tablet, and ends when the user removes the pen from the tablet. The position and movement of the pen during the stroke is converted into a series of X-Y coordinates. This pen-position data is received from the pen input device and pasεed to the glyph recognizer portion of the character input εystem.
In step 101, the glyph recognizer logic recognizes the pen-εtroke data aε being a rendition of a particular glyph based on the movement and shape of the input εtroke. The recognition procedure used by the present invention is esεentially that diεcloεed in U.S. Patent
No. 5,125,039, entitled Object Recognition Syεtem, issued to Jeffrey Hawkins, one of the present inventors. This patent iε incorporated herein by reference.
In εtep 102, a dictionary iε consulted to look up how the current glyph is to be interpreted. Associated with each glyph is a definition, which includes the current classification of the glyph aε a pre-character modifier εtroke, a character (meaning any symbol) stroke, a post-character modifier εtroke, or aε a currently unaεsigned stroke. The definition of each glyph also includes what character or other indicator, if any, is to be output in responεe to a εtroke that matcheε that glyph. It alεo includeε a specification of what changes if any, are to be made to any glyph definitions in responεe to a εtroke that matcheε that glyph.
Steps 103 and 104 are decision steps. Different actions are taken depending on whether the current εtroke represents a pre-character modifier stroke, a character stroke, a post-character modifier stroke, or an unassigned εtroke. In the case of pre-character modifier strokeε, control passeε to εtep 200.
In εtep 200, the proceεεing logic cauεeε an indication to be made that a particular pre-character modifier εtroke has been input. While the pre-character modifier stroke does not result in a character being output from the character recognizer, it is neverthelesε desirable to display to the user an indicator of what pre-character modifier stroke was received. In the preferred embodiment of the invention a character iε diεplayed which is representative of the pre-character modifier εtroke. For example, if a εimple "tap" or "dot" pre-character modifier εtroke iε uεed to interpret the next character εtroke aε a punctuation εymbol, then a bullet or "dot" character could be diεplayed indicating that the dot εtroke waε entered successfully. These characters are temporary and will be removed in step 300.
In step 201, the definition in the glyph dictionary of one or more glyphs is modified according to the input pre-character modifier εtroke, aε further described below. In step 300, if the clasε of the current εtroke iε "character", then any previously displayed indicators or characters representing pre-character modifier strokes are deleted and removed from the display. The definition of the corresponding glyph in the dictionary includes the character that is to be output in response to receiving strokes matching that glyph. In step 301, the procesεing logic outputε that character for display. In step 302, the definition in the glyph dictionary of one or more glyphs is modified according to the character εtroke entered and the current state of the dictionary, as further described below.
In step 400, if the class of the current pen stroke is "post-character modifier", then the processing logic causeε the moεt recently output character to be removed and replaced with a new or modified character. In the preferred embodiment, the prior character iε removed by εending a "backspace" character to the display syεtem. The new or modified character is determined by the glyph definition corresponding to the post-character modifier stroke.
In step 401, the definition in the glyph dictionary of one or more glyphs is modified according to the post-character modifier stroke entered and the current state of the dictionary, as further deεcribed below.
At any point in time, the glyph dictionary εpecifies exactly one interpretation for each input glyph that the εystem knows how to recognize. It is possible for that interpretation to be to ignore that glyph. For example, the glyph which looks like a pair of eyeglaεεeε (see FIGURE 5c) is not active in the initial dictionary definitions. When a stroke corresponding to that glyph is received when that glyph iε not active, the stroke is recognized, but ignored. At other times however, the eyeglasεeε glyph will not be ignored. For example, after the recognition of a character εtroke for a "u" or "a", the eyeglaεεeε glyph becomeε active as a post-character modifier stroke and corresponds to the umlaut accent. If the eyeglasεeε glyph is then input, the original "u" or "a" input character will be changed to "ϋ" or "a". Similarly, after a pre-character modifier stroke for punctuation, the eyeglasεes glyph is defined aε a character εtroke repreεenting the percent "%" character. At any point in time, each stroke that iε recognized must be a pre-character modifier stroke, a character stroke, a post-character modifier stroke, or unassigned. It cannot at a any point in time have multiple clasεificationε. Nevertheless, any stroke, when recognized, may modify the definition of any or all glyphs so as to reclassify them. Thus, a particular glyph may correspond sometimes to a character εtroke, and at other times to a post-character or pre-character modifier εtroke.
Stepε 201, 302, and 401 all modify the dictionary to reassign the definitions of glyphs. There are numerous ways the dictionary can be organized to allow for this modification. In the preferred embodiment of the present invention, the dictionary is implemented as a tree data structure. The root level of the tree containε definitions for all the pre-character modifier strokes and character strokeε defined in the initial εtate of the system. When one of these strokes is entered, there may be a subsequent branch defined for that stroke which contains new definitions for all the glyphs. Each εtroke leads to a new point in the tree data structure, thus modifying the dictionary. Implementing the dictionary aε a tree data εtructure iε flexible and allows the dictionary to be located in ROM. Other methods for organizing the dictionary would be posεible, as known in the art.
FIGURE 3 showε the glyphε taught in the prior art referenced earlier from Xerox for entering the Engliεh alphabet (in thiε and subsequent figures, a dot at one end of a stroke indicates that the stroke is drawn starting at that end.) This system achieves immediate recognition by forcing every character to be input with only a single stroke. This rigid unistroke approach makes it difficult to extend the character set significantly beyond the base alphabet. Xerox does not teach any method for punctuation, accented characters, extended characters, upper caεe letters, numbers, or other non-Roman alphabets. The Xerox alphabet was also deεigned for speed of entry and simplicity of recognition. This resultε in a glyph εet that lookε unfamiliar and iε difficult to learn.
FIGURE 11a εhowε named pen εtrokeε useful for defining various symbols in the preferred embodiment of the present invention. These εtrokeε, variationε of theεe strokes, and other εtrokeε, aε character εtrokeε, pre-character modifier εtrokeε, and poεt-character modifier εtrokes, can be defined to represent esεentially any character or εymbol. FIGURE 4a εhows the glyph set used in the preferred embodiment of the present invention for entering the English alphabet. This alphabet waε chosen to be as familiar as posεible and eaεy to learn, yet adhere to the principles taught in this invention. Most of these alphabet characterε are written with a εingle character εtroke. However, the preεent invention teaches a method of achieving immediate recognition with multiple strokes. It waε found through uεer testing that most people had difficulty learning any εingle stroke "X" . Therefore, in the base alphabet, "X" is written with two εequential εtrokes. The first εtroke going from the upper left to the lower right iε a pre-character modifier stroke, and the second εtroke going from the upper right to the lower left iε a character εtroke. Uεer teεting haε εhown that thiε two εtroke combination is far easier to write than any one εtroke "X" .
The alphabet εhown in FIGURE 4a provideε near 100% recognition accuracy, yet is easy to learn and use due to itε obviouε εimilarity to natural handwriting styles. FIGURE lib shows that there are actually multiple ways (different εtrokes) which can be used to write many of these letters, making the system even easier to learn and uεe. The recognition εystem simply maps the input strokes for these letters to the same output symbol in the glyph dictionary. For example, there are two ways to write a "Y": the glyph shown in FIGURE 4a, and a shape similar to a lower case εcripted "Y" aε εhown in FIGURE lib. Uεer teεting has εhown that εome uεerε prefer one method and εome prefer the other. FIGURE 4b shows the glyph set used in the preferred embodiment of the preεent invention for entering the digitε 0 through 9. Many of these glyphs are also used to enter letters of the alphabet. One method of overcoming this ambiguity problem is to have a separate numeric mode where a user only can enter digits. This numeric mode can be entered by pressing a button on the display of the computer, by writing a "num-lock" glyph, or other means. In the preferred embodiment of the preεent invention, a uεer can enter and exit a numeric mode by either tapping an on-εcreen icon or by writing a "forward εlaεh" stroke (a slanted line written from bottom left to upper right) . Testing has shown that occasionally a user forgets to exit numeric mode after writing several digits. A refinement of the present invention helps fix this problem by automatically exiting numeric mode when the user writes a character stroke which can only be interpreted as a letter or when the user writes the pre-character modifier εtroke for capital letter shift. FIGURE 5a εhows the glyph set used in the preferred embodiment of the present invention for entering three common "non-printing" characters: the space, the backspace, and the carriage return. A recognition syεtem with immediate reεponse operateε in many ways more like a keyboard than a conventional handwriting recognition εystem. For example, because userε εee results instantly, they find it more natural to "backspace" over an incorrect character than write over it or delete it with a gesture. Therefore, simple character strokes may be provided for space, backspace, carriage return, and other keyboard equivalents. FIGURE lie details other non-printing keyboard equivalents.
FIGURE 5b shows the glyph set used in the preferred embodiment of the present invention as pre- character modifier strokes for the English language. The three strokes are used to respectively indicate that the subεequent character εtroke repreεents a capital letter, a punctuation character, or an extended character. If desired, other pre-character modifier εtrokes may be defined as "sticky" shift εtrokes that take effect until reset. For example, a "caps lock" εtroke would cauεe all subsequent εtrokeε to be interpreted aε if the pre- character modifier "capε" stroke had been entered before each subεequent stroke. FIGURE 5c shows the glyph set used in the preferred embodiment of the present invention for entering common punctuation symbolε. All of these glyphε are claεεified aε character εtrokeε when preceded by a pre-character modifier εtroke repreεenting punctuation characterε. Aε εhown in FIGURE 5b, a "dot" εtroke iε uεed in the preferred embodiment of the invention to indicate a punctuation character. A more complete list of preferred punctuation characterε is shown in FIGURE lid and FIGURE lie. FIGURE 6 shows the glyph set used in the preferred embodiment of the present invention for entering several extended symbols. All of theεe glyphε are classified as character strokeε when preceded by a pre-character modifier εtroke repreεenting extended symbol characters. As shown in FIGURE 5b, a "εlash" stroke is used in the preferred embodiment of the invention to indicate an extended symbol character. A more complete list of preferred extended symbolε iε εhown in FIGURE llf. FIGURE 7a εhowε the glyph εet uεed in the preferred embodiment of the present invention for adding accents or diacritical marks to letters. These glyphs are classified as poεt-character modifier strokes after entering a letter which iε capable of being accented. FIGURE 7b shows several examples of how a user would write an accented character using a two stroke combination of a character εtroke and a poεt-character modifier εtroke. Writing these accented characterε uεing the present invention is very similar to how the user would write them on paper. First, the user writes a base letter, then adds an accent. After writing the base letter stroke, the corresponding character is immediately output to the display. Upon writing the accent post- character modifier stroke the base letter iε erased and replaced with the correct accented letter. Thus, the system achieves an immediate recognition responεe on a multiple εtroke character even though it iε initially unknown whether the uεer is writing a single stroke letter or a multiple stroke accented letter.
FIGURE 7c shows an example of how a user would write an accented upper caεe character uεing a three εtroke combination of a pre-character modifier stroke, a character stroke, and a post-character modifier stroke. The strokes are numbered in the order they are written. First, the user writes a pre-character modifier stroke indicating that the following character εhould be capitalized. Thiε causes the display of a temporary character indicating the acceptance of the stroke, in this case an "up arrow" character. Next, the user writes a character stroke. The temporary character is removed and replaced by the appropriate upper case letter.
Laεtly, the user writes a post-character modifier εtroke, causing the base upper case letter to be erased and replaced with the correct upper case accented letter.
The present invention is quite flexible and can accommodate many different types of languages, alphabets, and symbol systemε. FIGURE 8 illuεtrates one method of how the invention can be adapted to quickly and easily enter the Japanese katakana alphabet. In Japan, a common method for entering katakana with a keyboard iε called the -"romaji" method. In the romaji method, a uεer types the English phonetic equivalent of the katakana characters. For example, to enter the katakana for "sushi", the user types "εu" which reεultε in the diεplay of the correεponding katakana symbol. Then the user types "shi", which results in the display of the corresponding katakana symbol. All of the approximately 90 katakana characters can be input with combinations of one, two, or three typed letters. The present invention can duplicate this method εimply with a change to the dictionary of glyph mappingε. In the preferred embodiment of a katakana stroke recognition syεtem in accordance with the preεent invention, the new dictionary assigns strokes representing the consonantε "BCDGHKMNPRSTXYZ" aε pre-character modifier εtrokes and asεigns strokes representing the vowels "AEIOU" as character strokes. Some katakana characterε are entered with juεt one character εtroke. Some katakana characterε are entered with one pre-character modifier stroke and one character stroke. Some katakana characters are entered with two pre-character modifier strokes and one character εtroke. In the latter caεe, two temporary indicator characters are preferably displayed, representing the two pre- character modifier stokeε. Both temporary characterε are deleted and replaced with the final katakana character upon entering the character εtroke. Thiε sequence is illustrated in FIGURE 9.
FIGURE 10 εhowε how εpecial double katakana εymbol combinations can be entered with three stroke combinations of two pre-character modifier strokes and one character εtroke. Thiε mapping εtill followε the romaji method common in Japan. It illuεtrates the flexibility of the present invention by showing how a character εtroke can result in the display of more than one character or symbol. In principle, a character εtroke or post-character modifier stroke can result in the output and display of any length sequence of characterε.
There are many fine pointε in this particular implementation which are not detailed here but would be obvious to anyone experienced in the romaji input syεtem and familiar with the preεent invention. For example, the stroke representing the letter "N" is initially a pre-character modifier εtroke. Once entered it is reasεigned as a character stroke. This permits the entry of the "NN" katakana symbol, which is an exception to the general consonant-vowel pairing for katakana.
FIGURE llg shows the pen strokes uεed aε character εtrokeε to repreεent non-accented foreign characterε in the preferred embodiment of the preεent invention.
The principles of the present invention can be extended to other characters sets, such as Japaneεe hiragana, Chinese kanji, or Korean hangul. The concepts of the present invention also can be extended to provide greater flexibility for userε. For example, the computer system could be programmed to allow a user to define new input strokes, and/or to asεociate εymbols, characters, or even complete words or phraseε, to a combination of input εtrokeε. Thus, a user-maintained glossary could be built where the user could define the sequences of characters — or symbols, text, or program functions — to be asεociated with a εtroke, a multi-εtroke combination, or εequence of multiple εtroke combinationε. Alternatively, the uεer could alεo define new εtrokeε within a table (or other data structure) and assign context to each such stroke. Summary
The present recognition εyεtem provides several improvements over prior art syεtems by achieving immediate recognition of multiple stroke characters without uεing boxes for input. It improves over prior art syεtemε by immediately diεplaying every character written aε εoon as the user finishes the character. No unnecesεary delayε are incurred, nor are additional actionε required of a uεer to translate input. The immediate responεe helpε a user to quickly identify mistakeε and correct them. The preεent system accomplisheε thiε immediate reεponεe while at the εame time accommodating characters which are more easily learned and written using multiple strokeε. Exampleε provided above include accented characters, punctuation characters, extended symbols, the letter "X", and the katakana character set. Defining characterε with multiple εtrokeε makeε the preεent invention much easier to use and learn than prior art systems requiring single stroke only characters. Another advantage provided by the present recognition system iε that large sets of characters can be represented without relying on a large set of unique strokes or glyphs. For example, accented letters use the same base character stroke as their unaccented counterpart letters. Similarly, punctuation marks and capitalized letters are realized using a combination of pre-modifier control εtrokes and character εtrokeε. The present invention provides an additional advantage over prior art systems because the present system does not require multiple writing boxes or other large on-screen gadgetry. Valuable display space is thereby saved, allowing the preεent system to be used on very small devices. The present εyεtem alεo teaches an alphabet for inputing Roman-character based languages which, although not identical to a user's conventional writing εtyle, iε very εimilar and therefore eaεy to learn. A user can achieve near 100% recognition accuracy when using this alphabet, yet it is very easy to learn and use because it is very εimilar to a natural writing style.
A number of embodiments of the preεent invention have been described. Nevertheless, it will be understood that variouε modificationε may be made without departing from the εpirit and εcope of the invention. For example, while particular εtrokes and asεociations for such strokes have been discloεed, the invention encompaεεeε other εtrokes, combinationε, and aεεociationε. Further, as noted above, a stroke may be associated with any context, such as character(ε) , symbol(s), text, or program functions. The term "character" εhould thus be underεtood to encompass any of these contexts. Accordingly, it iε to be understood that the invention is not to be limited by the specific illustrated embodiment, but only by the scope of the appended claims.

Claims

1. An improved electronic handwriting recognition εystem for interpreting input strokes and displaying corresponding characters, the εyεtem being of the type having an input writing surface, a device for inputing strokes on the input writing surface, and a display for displaying characters, the improvement comprising:
(a) recognition logic for recognizing and clasεifying each individual input εtroke after input as a member of one of a plurality of εetε of glyphε, the sets comprising at least:
(1) a εet of character glyphs;
(2) a set of post-character modifier glyphε; (b) proceεsing logic, coupled to the recognition logic, for outputing to the display a character corresponding to a character glyph, wherein the procesεing logic modifieε the displayed character in a pre-defined manner in response to any immediately succeeding post-character modifier glyphs.
2. An improved electronic handwriting recognition system for interpreting input strokeε and diεplaying correεponding characters, the syεtem being of the type having an input writing εurface, a device for inputing εtrokes on the input writing εurface, and a diεplay for diεplaying characters, the improvement comprising:
(a) recognition logic for recognizing and clasεifying each individual input stroke after input as a member of one of a plurality of setε of glyphε, the sets comprising at least:
(1) a set of pre-character modifier glyphε; (2) a εet of character glyphs;
(3) a εet of poεt-character modifier glyphs; (b) processing logic, coupled to the recognition logic, for outputing to the display a character corresponding to a character glyph, wherein the processing logic modifies the character to be displayed in a pre-defined manner in responεe to any immediately preceding pre-character modifier glyphε, and wherein the processing logic modifies the displayed character in a pre-defined manner in response to any immediately succeeding post-character modifier glyphs.
3. An automated method for interpreting input strokeε and diεplaying correεponding characters on an electronic handwriting recognition syεtem, the system comprising an input writing εurface, a device for inputing εtrokes on the input writing εurface, and a diεplay for diεplaying characterε, the method comprising the stepε of:
(a) recognizing and classifying each individual input stroke after input aε a member of one of a plurality of sets of glyphs, the sets compriεing at least: (1) a set of character glyphε;
(2) a εet of post-character modifier glyphs;
(b) outputing to the display a character corresponding to a character glyph, wherein the proceεεing logic modifieε the diεplayed character in a pre-defined manner in response to any immediately succeeding post-character modifier glyphs.
4. An automated method for interpreting input strokeε and displaying correεponding characters on an electronic handwriting recognition system, the syεtem comprising an input writing surface, a device for inputing strokes on the input writing surface, and a display for displaying characters, the method comprising the steps of:
(1) recognizing and classifying each individual input stroke after input as a member of one of a plurality of sets of glyphs, the sets compriεing at least:
(1) a εet of pre-character modifier glyphε;
(2) a εet of character glyphε;
(3) a set of post-character modifier glyphs; (b) outputing to the display a character correεponding to a character glyph, wherein the proceεεing logic modifies the character to be displayed in a pre-defined manner in responεe to any immediately preceding pre- character modifier glyphs, and wherein the processing logic modifies the displayed character in a pre-defined manner in responεe to any immediately εucceeding poεt-character modifier glyphε.
5. A εtorage media readable by a programmable computer when coupled to the εtorage media, the εtorage media containing a control program tangibly εtored thereon, εuch that the computer iε operated by the control program when the storage media is read by the computer, the computer being operated to determine, recognize, and classify handwritten strokeε, the control program being configured to operate the computer to perform the functionε of: (a) recognizing and claεεifying each individual input εtroke after input as a member of one of a plurality of sets of glyphε, the εets comprising a set of character glyphs and at least one of:
(1) a set of pre-character modifier glyphs;
(2) a set of post-character modifier glyphs;
(b) outputing to the diεplay a character correεponding to a character glyph, wherein the proceεεing logic modifieε the character to be displayed in a pre-defined manner in response to any immediately preceding pre- character modifier glyphs, and wherein the processing logic modifies the displayed character in a pre-defined manner in responεe to any immediately succeeding post-character modifier glyphs.
6. A control program tangibly stored on a storage media readable by a programmable computer, such that the computer is operated by the control program when the storage media is read by the computer, the computer being operated to determine, recognize, and classify handwritten strokes, such functions being performed by the combination of the control program and the computer performing the functions of:
(a) recognizing and classifying each individual input stroke after input as a member of one of a plurality of εets of glyphs, the setε compriεing a εet of character glyphε and at least one of:
(1) a set of pre-character modifier glyphε;
(2) a εet of post-character modifier glyphs;
(b) outputing to the display a character corresponding to a character glyph, wherein the processing logic modifies the character to be displayed in a pre-defined manner in reεponεe to any immediately preceding pre- character modifier glyphs, and wherein the procesεing logic modifieε the diεplayed character in a pre-defined manner in response to any immediately succeeding post-character modifier glyphs.
7. The invention of claims 1 or 2, wherein the recognition logic and procesεing logic are embodied in a pen-based portable computer.
8. The invention of claim 2, wherein the processing logic outputs an indicator to the display in responεe to recognition and classification of a pre- character modifier glyph, and removes the indicator in response to recognition and classification of a character glyph.
9. The invention of claims 1, 2, 3, 4, 5, or 6, wherein modification of the displayed character is accomplished by outputing a "backspace" character to delete the diεplayed character and then outputing a modified character to replace the deleted character.
10. The invention of claimε 1, 2, 3, 4, 5, or 6, wherein recognition of a εtroke modifies the clasεification of at least one subεequent εtroke.
11. The invention of claimε 2, 4, 5, or 6, wherein recognition of a pre-character modifier εtroke modifieε the definition of the character corresponding to at leaεt one εubsequently input character glyph.
12. The invention of claim 4, further including the εteps of:
(a) outputing an indicator to the display in responεe to recognition and claεεification of a pre-character modifier glyph; and
(b) removing the indicator in response to recognition and claεεification of a character glyph.
13. The invention of claimε 5 or 6, further including the functions of:
(a) outputing an indicator to the display in response to recognition and classification of a pre-character modifier glyph; and
(b) removing the indicator in response to recognition and classification of a character glyph.
14. A εet of glyphs for use in conjunction with an electronic handwriting recognition syεtem, the εyεtem comprising an input writing εurface, a device for inputing εtrokes on the input writing surface, and a display for displaying characterε, wherein each stroke glyph correspondε to a unique alphabetic character, the εet of glyphε and correεponding characterε being subεtantially as shown in FIGURE 4a.
15. A εet of glyphε for uεe in conjunction with an electronic handwriting recognition εyεtem, the εystem comprising an input writing εurface, a device for inputing εtrokeε on the input writing εurface, and a diεplay for displaying characters, wherein each εtroke glyph corresponds to a unique alphabetic vowel character, the set of glyphε and corresponding vowel characters being substantially as shown in FIGURE 4a.
16. A set of glyphs for use in conjunction with an electronic handwriting recognition syεtem, the syεtem comprising an input writing surface, a device for inputing strokes on the input writing surface, and a display for displaying characters, wherein each stroke glyph correspondε to a unique numeric character, the set of glyphs and corresponding characters being substantially as shown in FIGURE 4b.
17. A εet of glyphε for uεe in conjunction with an electronic handwriting recognition εyεtem, the εyεtem comprising an input writing surface, a device for inputing strokes on the input writing surface, and a display for displaying characters, wherein each εtroke glyph corresponds to a unique non-printing character, the set of glyphs and corresponding characters being substantially as shown in FIGURE 5a.
18. A set of glyphs for use in conjunction with an electronic handwriting recognition εyεtem, the εyεtem compriεing an input writing εurface, a device for inputing strokes on the input writing surface, and a display for displaying characterε, wherein each εtroke glyph correεponds to a unique pre-character modifier stroke, the set of glyphs and corresponding modifier strokes being substantially as shown in FIGURE 5b.
19. A εet of glyphs for uεe in conjunction with an electronic handwriting recognition εyεtem, the εystem comprising an input writing εurface, a device for inputing strokes on the input writing surface, and a display for displaying characters, wherein each stroke glyph correεpondε to a unique poεt-character modifier stroke, the set of glyphε and corresponding modifier strokes being subεtantially aε εhown in FIGURE 7a.
20. An improved electronic handwriting recognition syεtem for interpreting input strokeε and displaying corresponding non-Roman characters, the system being of the type having an input writing surface, a device for inputing εtrokes on the input writing surface, and a display for displaying characters, the improvement comprising:
(a) recognition logic for recognizing and classifying each individual input stroke after input as a member of one of a plurality of sets of glyphs, the sets comprising at leaεt:
(1) a εet of pre-character modifier glyphε;
(2) a set of character glyphs; (b) processing logic, coupled to the recognition logic, for outputing to the display a non- Roman character corresponding to a character glyph and from zero to two pre-character modifier glyphs.
21. An improved electronic handwriting recognition system for interpreting input strokes and displaying corresponding katakana characters, the system being of the type having an input writing εurface, a device for inputing strokes on the input writing surface, and a display for displaying characters, the improvement comprising:
(a) recognition logic for recognizing and classifying each individual input stroke after input as a member of one of a plurality of sets of glyphs, the sets comprising at least:
(1) a εet of pre-character modifier glyphs;
(2) a set of character glyphs; (b) procesεing logic, coupled to the recognition logic, for outputing to the diεplay a katakana character corresponding to a character glyph and from zero to two pre- character modifier glyphs.
22. An improved electronic handwriting recognition εyεtem for interpreting input εtrokeε and diεplaying correεponding katakana characters, the syεtem being of the type having an input writing εurface, a device for inputing strokes on the input writing surface, and a display for displaying characters, the improvement comprising:
(a) recognition logic for recognizing and clasεifying each individual input stroke after input aε a member of one of a plurality of εetε of glyphs, the sets comprising at least:
(1) a set of pre-character modifier glyphs corresponding to Roman character consonants;
(2) a εet of character glyphs correεponding to Roman character vowels;
(b) processing logic, coupled to the recognition logic, for outputing to the display a katakana character corresponding to a character glyph and from zero to two pre- character modifier glyphs.
PCT/US1995/008113 1994-07-01 1995-06-29 Multiple pen stroke character set and handwriting recognition system WO1996001453A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
DE69533479T DE69533479T2 (en) 1994-07-01 1995-06-29 CHARACTER SET WITH CHARACTERS FROM MULTIPLE BARS AND HANDWRITING IDENTIFICATION SYSTEM
JP8503905A JPH10510639A (en) 1994-07-01 1995-06-29 Multi pen stroke character set and handwritten document recognition system
EP95924077A EP0769175B9 (en) 1994-07-01 1995-06-29 Multiple pen stroke character set and handwriting recognition system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US27002594A 1994-07-01 1994-07-01
US08/270,025 1994-07-01

Publications (1)

Publication Number Publication Date
WO1996001453A1 true WO1996001453A1 (en) 1996-01-18

Family

ID=23029579

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1995/008113 WO1996001453A1 (en) 1994-07-01 1995-06-29 Multiple pen stroke character set and handwriting recognition system

Country Status (5)

Country Link
US (2) US6493464B1 (en)
EP (1) EP0769175B9 (en)
JP (1) JPH10510639A (en)
DE (1) DE69533479T2 (en)
WO (1) WO1996001453A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0917040A2 (en) * 1997-11-11 1999-05-19 Hewlett-Packard Company Method for providing a cue in a computer system
WO2000073890A1 (en) * 1999-05-27 2000-12-07 Alexei Vladimirovich Afanasiev A method of inputting information into a computer device, a sticker keyboard and a computer device using this method
US6647145B1 (en) 1997-01-29 2003-11-11 Co-Operwrite Limited Means for inputting characters or commands into a computer
US7406666B2 (en) 2002-08-26 2008-07-29 Palm, Inc. User-interface features for computers with contact-sensitive displays
US8031212B2 (en) 2000-05-18 2011-10-04 Hewlett-Packard Development Company, L.P. Reorienting display on portable computing device
US8259075B2 (en) 2009-01-06 2012-09-04 Hewlett-Packard Development Company, L.P. Secondary key group layout for keyboard
US8976108B2 (en) 2001-06-11 2015-03-10 Qualcomm Incorporated Interface for processing of an alternate symbol in a computer device
US9549056B2 (en) 2001-06-11 2017-01-17 Qualcomm Incorporated Integrated personal digital assistant device
US20180095654A1 (en) * 2016-09-30 2018-04-05 Atlassian Pty Ltd Creating tables using gestures

Families Citing this family (223)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6366697B1 (en) * 1993-10-06 2002-04-02 Xerox Corporation Rotationally desensitized unistroke handwriting recognition
GB2282906B (en) 1993-10-13 1996-11-06 Dataquill Ltd Data enty systems
JPH10510639A (en) * 1994-07-01 1998-10-13 パーム コンピューティング，インコーポレーテッド Multi pen stroke character set and handwritten document recognition system
US5889888A (en) 1996-12-05 1999-03-30 3Com Corporation Method and apparatus for immediate response handwriting recognition system that handles multiple character sets
US7286115B2 (en) 2000-05-26 2007-10-23 Tegic Communications, Inc. Directional input system with automatic correction
US7610194B2 (en) 2002-07-18 2009-10-27 Tegic Communications, Inc. Dynamic database reordering system
US7750891B2 (en) 2003-04-09 2010-07-06 Tegic Communications, Inc. Selective input system based on tracking of motion parameters of an input device
US6970599B2 (en) * 2002-07-25 2005-11-29 America Online, Inc. Chinese character handwriting recognition system
US7821503B2 (en) 2003-04-09 2010-10-26 Tegic Communications, Inc. Touch screen and graphical user interface
JP4519381B2 (en) 1999-05-27 2010-08-04 テジック コミュニケーションズ インク Keyboard system with automatic correction
US7030863B2 (en) 2000-05-26 2006-04-18 America Online, Incorporated Virtual keyboard system with automatic correction
US7564995B1 (en) * 2000-03-07 2009-07-21 Apple Inc. Method and apparatus for acquiring and organizing ink information in pen-aware computer systems
US8645137B2 (en) 2000-03-16 2014-02-04 Apple Inc. Fast, language-independent method for user authentication by voice
US20020054026A1 (en) * 2000-04-17 2002-05-09 Bradley Stevenson Synchronized transmission of recorded writing data with audio
US6782974B2 (en) * 2000-05-08 2004-08-31 Richard Leifer Server call system
US6809725B1 (en) * 2000-05-25 2004-10-26 Jishan Zhang On screen chinese keyboard
US6812939B1 (en) 2000-05-26 2004-11-02 Palm Source, Inc. Method and apparatus for an event based, selectable use of color in a user interface display
US7075555B1 (en) 2000-05-26 2006-07-11 Palmsource, Inc. Method and apparatus for using a color table scheme for displaying information on either color or monochrome display
US6785417B1 (en) 2000-08-22 2004-08-31 Microsoft Corp Method and system for searching for words in ink word documents
US6836759B1 (en) 2000-08-22 2004-12-28 Microsoft Corporation Method and system of handling the selection of alternates for recognized words
US6771817B1 (en) 2000-08-22 2004-08-03 Microsoft Corporation Method and system for extending ink word data structures while maintaining version compatibility
US6754386B1 (en) 2000-08-22 2004-06-22 Microsft Corporation Method and system of matching ink processor and recognizer word breaks
US6801660B1 (en) 2000-08-22 2004-10-05 Microsoft Corporation Method and system for maintaining alternates in association with recognized words
EP1220140A1 (en) * 2000-12-27 2002-07-03 Asulab S.A. Method for recognising characters drawn manually on a capture area and electronic device for implementing the method
US7142195B2 (en) 2001-06-04 2006-11-28 Palm, Inc. Interface for interaction with display visible from both sides
US20030014239A1 (en) * 2001-06-08 2003-01-16 Ichbiah Jean D. Method and system for entering accented and other extended characters
US6973214B1 (en) * 2001-07-30 2005-12-06 Mobigence, Inc. Ink display for multi-stroke hand entered characters
JP4261145B2 (en) * 2001-09-19 2009-04-30 株式会社リコー Information processing apparatus, information processing apparatus control method, and program for causing computer to execute the method
US7158675B2 (en) * 2002-05-14 2007-01-02 Microsoft Corporation Interfacing with ink
US20030214553A1 (en) * 2002-05-14 2003-11-20 Microsoft Corporation Ink regions in an overlay control
US20030214531A1 (en) * 2002-05-14 2003-11-20 Microsoft Corporation Ink input mechanisms
US8166388B2 (en) 2002-05-14 2012-04-24 Microsoft Corporation Overlaying electronic ink
US7925987B2 (en) * 2002-05-14 2011-04-12 Microsoft Corporation Entry and editing of electronic ink
US7228501B2 (en) * 2002-11-01 2007-06-05 Microsoft Corporation Method for selecting a font
US7284200B2 (en) * 2002-11-10 2007-10-16 Microsoft Corporation Organization of handwritten notes using handwritten titles
US7634729B2 (en) * 2002-11-10 2009-12-15 Microsoft Corporation Handwritten file names
US7729542B2 (en) * 2003-04-04 2010-06-01 Carnegie Mellon University Using edges and corners for character input
US20050068322A1 (en) * 2003-09-29 2005-03-31 Falcioni Richard A. Generating alphanumeric characters
CN1275124C (en) * 2003-10-14 2006-09-13 中国科学院合肥智能机械研究所 Handwrite input platform of computer and information fetching method
US8074184B2 (en) * 2003-11-07 2011-12-06 Mocrosoft Corporation Modifying electronic documents with recognized content or other associated data
US7620244B1 (en) 2004-01-06 2009-11-17 Motion Computing, Inc. Methods and systems for slant compensation in handwriting and signature recognition
US20050152600A1 (en) * 2004-01-14 2005-07-14 International Business Machines Corporation Method and apparatus for performing handwriting recognition by analysis of stroke start and end points
US7650568B2 (en) * 2004-01-30 2010-01-19 Microsoft Corporation Implementing handwritten shorthand in a computer system
US7580029B2 (en) * 2004-04-02 2009-08-25 Nokia Corporation Apparatus and method for handwriting recognition
WO2005096217A1 (en) * 2004-04-02 2005-10-13 Nokia Corporation Apparatus and method for handwriting recognition
US7814024B2 (en) * 2004-05-14 2010-10-12 Ching Peter N Multi-way transactions related data exchange apparatus and methods
US11017097B2 (en) 2004-05-14 2021-05-25 Peter N. Ching Systems and methods for prevention of unauthorized access to resources of an information system
JP2005346415A (en) * 2004-06-03 2005-12-15 Hitachi Ltd Editing system using electronic pen, editing method and program therefor
DE112005001422T5 (en) * 2004-06-18 2007-05-03 Microth, Inc. Bar-based data entry device, system and method
US7979795B2 (en) * 2004-08-02 2011-07-12 Hewlett-Packard Development Company, L.P. System and method for inputting syllables of a phonetic script into a computer
US8160363B2 (en) * 2004-09-25 2012-04-17 Samsung Electronics Co., Ltd Device and method for inputting characters or drawings in a mobile terminal using a virtual screen
US7443386B2 (en) * 2004-11-01 2008-10-28 Nokia Corporation Mobile phone and method
US7490033B2 (en) * 2005-01-13 2009-02-10 International Business Machines Corporation System for compiling word usage frequencies
US20060202865A1 (en) * 2005-03-04 2006-09-14 Nguyen Mitchell V Text entry coding system and handheld computing device
US8229252B2 (en) * 2005-03-18 2012-07-24 The Invention Science Fund I, Llc Electronic association of a user expression and a context of the expression
US7809215B2 (en) 2006-10-11 2010-10-05 The Invention Science Fund I, Llc Contextual information encoded in a formed expression
US7672512B2 (en) 2005-03-18 2010-03-02 Searete Llc Forms for completion with an electronic writing device
US8599174B2 (en) * 2005-03-18 2013-12-03 The Invention Science Fund I, Llc Verifying a written expression
US7826687B2 (en) 2005-03-18 2010-11-02 The Invention Science Fund I, Llc Including contextual information with a formed expression
US20060212430A1 (en) * 2005-03-18 2006-09-21 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Outputting a saved hand-formed expression
US8290313B2 (en) 2005-03-18 2012-10-16 The Invention Science Fund I, Llc Electronic acquisition of a hand formed expression and a context of the expression
US8340476B2 (en) * 2005-03-18 2012-12-25 The Invention Science Fund I, Llc Electronic acquisition of a hand formed expression and a context of the expression
US8640959B2 (en) * 2005-03-18 2014-02-04 The Invention Science Fund I, Llc Acquisition of a user expression and a context of the expression
US8232979B2 (en) * 2005-05-25 2012-07-31 The Invention Science Fund I, Llc Performing an action with respect to hand-formed expression
US20070273674A1 (en) * 2005-03-18 2007-11-29 Searete Llc, A Limited Liability Corporation Machine-differentiatable identifiers having a commonly accepted meaning
US20060221056A1 (en) * 2005-03-31 2006-10-05 Inventec Corporation Method and system for inputting single-stroke code
DE102005020142A1 (en) * 2005-04-29 2006-11-02 Siemens Ag Character inputting method for e.g. portable computer, involves activating multiple section mode for entering character with multiple locus curve sections, and deactivating mode and reactivating timer after last section is detected
US20060279559A1 (en) * 2005-06-10 2006-12-14 Wang Kongqiao Mobile communications terminal and method therefore
US7428516B2 (en) * 2005-06-23 2008-09-23 Microsoft Corporation Handwriting recognition using neural networks
KR100656523B1 (en) * 2005-07-27 2006-12-11 삼성전자주식회사 Method and system using human body motion
US7567711B2 (en) * 2005-08-01 2009-07-28 Microsoft Corporation Cleaning up of handwriting intra-stroke and inter-stroke overtracing
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US20070174916A1 (en) * 2005-10-28 2007-07-26 Ching Peter N Method and apparatus for secure data transfer
US7808480B2 (en) * 2005-10-28 2010-10-05 Sap Ag Method and system for secure input
TW200809660A (en) * 2006-03-01 2008-02-16 Zi Decuma Ab A method for additive character recognition and an apparatus thereof
US7502509B2 (en) * 2006-05-12 2009-03-10 Velosum, Inc. Systems and methods for digital pen stroke correction
US7817857B2 (en) * 2006-05-31 2010-10-19 Microsoft Corporation Combiner for improving handwriting recognition
US20080129712A1 (en) * 2006-07-31 2008-06-05 Mitchell Van Nguyen Pen-based computer system having variable automatic scroll
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US8201087B2 (en) 2007-02-01 2012-06-12 Tegic Communications, Inc. Spell-check for a keyboard system with automatic correction
US8225203B2 (en) 2007-02-01 2012-07-17 Nuance Communications, Inc. Spell-check for a keyboard system with automatic correction
US20080244511A1 (en) * 2007-03-30 2008-10-02 Microsoft Corporation Developing a writing system analyzer using syntax-directed translation
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US7903002B2 (en) * 2007-05-17 2011-03-08 Sony Ericsson Mobile Communications Ab Electronic device having vibration input recognition and method
US7971152B2 (en) * 2007-12-14 2011-06-28 Microsoft Corporation Direction-based data entry system
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US8996376B2 (en) 2008-04-05 2015-03-31 Apple Inc. Intelligent text-to-speech conversion
US10496753B2 (en) 2010-01-18 2019-12-03 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US20100020033A1 (en) * 2008-07-23 2010-01-28 Obinna Ihenacho Alozie Nwosu System, method and computer program product for a virtual keyboard
US20100030549A1 (en) 2008-07-31 2010-02-04 Lee Michael M Mobile device having human language translation capability with positional feedback
US8769427B2 (en) 2008-09-19 2014-07-01 Google Inc. Quick gesture input
US8373703B2 (en) * 2008-09-24 2013-02-12 Penny Beckwith System and method for incorporating a personalized writing style into a word program
US9959870B2 (en) 2008-12-11 2018-05-01 Apple Inc. Speech recognition involving a mobile device
US8819597B2 (en) * 2009-04-10 2014-08-26 Google Inc. Glyph entry on computing device
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US10255566B2 (en) 2011-06-03 2019-04-09 Apple Inc. Generating and processing task items that represent tasks to perform
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US9858925B2 (en) 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US8681106B2 (en) 2009-06-07 2014-03-25 Apple Inc. Devices, methods, and graphical user interfaces for accessibility using a touch-sensitive surface
US9431006B2 (en) 2009-07-02 2016-08-30 Apple Inc. Methods and apparatuses for automatic speech recognition
US10553209B2 (en) 2010-01-18 2020-02-04 Apple Inc. Systems and methods for hands-free notification summaries
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10705794B2 (en) 2010-01-18 2020-07-07 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10679605B2 (en) 2010-01-18 2020-06-09 Apple Inc. Hands-free list-reading by intelligent automated assistant
US8977584B2 (en) 2010-01-25 2015-03-10 Newvaluexchange Global Ai Llp Apparatuses, methods and systems for a digital conversation management platform
US8682667B2 (en) 2010-02-25 2014-03-25 Apple Inc. User profiling for selecting user specific voice input processing information
US8554280B2 (en) * 2010-03-23 2013-10-08 Ebay Inc. Free-form entries during payment processes
US20110291964A1 (en) * 2010-06-01 2011-12-01 Kno, Inc. Apparatus and Method for Gesture Control of a Dual Panel Electronic Device
US9104306B2 (en) * 2010-10-29 2015-08-11 Avago Technologies General Ip (Singapore) Pte. Ltd. Translation of directional input to gesture
US10762293B2 (en) 2010-12-22 2020-09-01 Apple Inc. Using parts-of-speech tagging and named entity recognition for spelling correction
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US20120310642A1 (en) 2011-06-03 2012-12-06 Apple Inc. Automatically creating a mapping between text data and audio data
US8989492B2 (en) * 2011-06-03 2015-03-24 Apple Inc. Multi-resolution spatial feature extraction for automatic handwriting recognition
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
KR20120136900A (en) * 2011-06-10 2012-12-20 삼성전자주식회사 Apparatus and method for inputting braille in device with touch screen
US8994660B2 (en) 2011-08-29 2015-03-31 Apple Inc. Text correction processing
US8286104B1 (en) * 2011-10-06 2012-10-09 Google Inc. Input method application for a touch-sensitive user interface
US8478777B2 (en) * 2011-10-25 2013-07-02 Google Inc. Gesture-based search
KR20130059495A (en) * 2011-11-29 2013-06-07 삼성전자주식회사 Method for processing a ui control element in wireless terminal
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US9483461B2 (en) 2012-03-06 2016-11-01 Apple Inc. Handling speech synthesis of content for multiple languages
CA2772554A1 (en) * 2012-03-19 2013-09-19 Corel Corporation Method and system for interactive font feature access
US8504842B1 (en) 2012-03-23 2013-08-06 Google Inc. Alternative unlocking patterns
US8881269B2 (en) 2012-03-31 2014-11-04 Apple Inc. Device, method, and graphical user interface for integrating recognition of handwriting gestures with a screen reader
US9280610B2 (en) 2012-05-14 2016-03-08 Apple Inc. Crowd sourcing information to fulfill user requests
US9721563B2 (en) 2012-06-08 2017-08-01 Apple Inc. Name recognition system
US9495129B2 (en) 2012-06-29 2016-11-15 Apple Inc. Device, method, and user interface for voice-activated navigation and browsing of a document
US9576574B2 (en) 2012-09-10 2017-02-21 Apple Inc. Context-sensitive handling of interruptions by intelligent digital assistant
US9547647B2 (en) 2012-09-19 2017-01-17 Apple Inc. Voice-based media searching
TWM448713U (en) * 2012-11-12 2013-03-11 shun-fu Luo Trajectory input and recognition operation control devices
KR101997447B1 (en) * 2012-12-10 2019-07-08 엘지전자 주식회사 Mobile terminal and controlling method thereof
EP2954514B1 (en) 2013-02-07 2021-03-31 Apple Inc. Voice trigger for a digital assistant
US9368114B2 (en) 2013-03-14 2016-06-14 Apple Inc. Context-sensitive handling of interruptions
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
AU2014233517B2 (en) 2013-03-15 2017-05-25 Apple Inc. Training an at least partial voice command system
WO2014144579A1 (en) 2013-03-15 2014-09-18 Apple Inc. System and method for updating an adaptive speech recognition model
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
WO2014197336A1 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for detecting errors in interactions with a voice-based digital assistant
WO2014197334A2 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
WO2014197335A1 (en) 2013-06-08 2014-12-11 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
WO2014200728A1 (en) 2013-06-09 2014-12-18 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
AU2014278595B2 (en) 2013-06-13 2017-04-06 Apple Inc. System and method for emergency calls initiated by voice command
CN103294403A (en) * 2013-06-21 2013-09-11 苏州鼎富软件科技有限公司 Handwriting input method identification method
KR101749009B1 (en) 2013-08-06 2017-06-19 애플 인크. Auto-activating smart responses based on activities from remote devices
JP2015049603A (en) * 2013-08-30 2015-03-16 株式会社東芝 Electronic apparatus
US9424823B2 (en) * 2014-02-10 2016-08-23 Myscript Method and apparatus for recognising music symbols
WO2015141260A1 (en) * 2014-03-17 2015-09-24 株式会社河合楽器製作所 Handwritten music notation recognition device and program
US9620105B2 (en) 2014-05-15 2017-04-11 Apple Inc. Analyzing audio input for efficient speech and music recognition
US10592095B2 (en) 2014-05-23 2020-03-17 Apple Inc. Instantaneous speaking of content on touch devices
US9502031B2 (en) 2014-05-27 2016-11-22 Apple Inc. Method for supporting dynamic grammars in WFST-based ASR
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US10289433B2 (en) 2014-05-30 2019-05-14 Apple Inc. Domain specific language for encoding assistant dialog
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9734193B2 (en) 2014-05-30 2017-08-15 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US10078631B2 (en) 2014-05-30 2018-09-18 Apple Inc. Entropy-guided text prediction using combined word and character n-gram language models
US9966065B2 (en) 2014-05-30 2018-05-08 Apple Inc. Multi-command single utterance input method
US9760559B2 (en) 2014-05-30 2017-09-12 Apple Inc. Predictive text input
US9842101B2 (en) 2014-05-30 2017-12-12 Apple Inc. Predictive conversion of language input
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9785630B2 (en) 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US10659851B2 (en) 2014-06-30 2020-05-19 Apple Inc. Real-time digital assistant knowledge updates
EP2966585B1 (en) * 2014-07-11 2019-09-11 Unify GmbH & Co. KG Method and system for initiating a login of a user
US10446141B2 (en) 2014-08-28 2019-10-15 Apple Inc. Automatic speech recognition based on user feedback
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US10789041B2 (en) 2014-09-12 2020-09-29 Apple Inc. Dynamic thresholds for always listening speech trigger
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US9646609B2 (en) 2014-09-30 2017-05-09 Apple Inc. Caching apparatus for serving phonetic pronunciations
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US9886432B2 (en) 2014-09-30 2018-02-06 Apple Inc. Parsimonious handling of word inflection via categorical stem + suffix N-gram language models
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US10552013B2 (en) 2014-12-02 2020-02-04 Apple Inc. Data detection
US9711141B2 (en) 2014-12-09 2017-07-18 Apple Inc. Disambiguating heteronyms in speech synthesis
US10089291B2 (en) 2015-02-27 2018-10-02 Microsoft Technology Licensing, Llc Ink stroke editing and manipulation
US9865280B2 (en) 2015-03-06 2018-01-09 Apple Inc. Structured dictation using intelligent automated assistants
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US9899019B2 (en) 2015-03-18 2018-02-20 Apple Inc. Systems and methods for structured stem and suffix language models
US9842105B2 (en) 2015-04-16 2017-12-12 Apple Inc. Parsimonious continuous-space phrase representations for natural language processing
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10127220B2 (en) 2015-06-04 2018-11-13 Apple Inc. Language identification from short strings
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US10101822B2 (en) 2015-06-05 2018-10-16 Apple Inc. Language input correction
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US10255907B2 (en) 2015-06-07 2019-04-09 Apple Inc. Automatic accent detection using acoustic models
US10186254B2 (en) 2015-06-07 2019-01-22 Apple Inc. Context-based endpoint detection
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US9697820B2 (en) 2015-09-24 2017-07-04 Apple Inc. Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
FR3041792B1 (en) * 2015-09-29 2018-10-26 Melissa Raffin METHOD AND DEVICE FOR DIGITAL WRITING USING A TOUCH SCREEN
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
DK179309B1 (en) 2016-06-09 2018-04-23 Apple Inc Intelligent automated assistant in a home environment
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
DK179049B1 (en) 2016-06-11 2017-09-18 Apple Inc Data driven natural language event detection and classification
DK179343B1 (en) 2016-06-11 2018-05-14 Apple Inc Intelligent task discovery
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
DK201770439A1 (en) 2017-05-11 2018-12-13 Apple Inc. Offline personal assistant
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK201770432A1 (en) 2017-05-15 2018-12-21 Apple Inc. Hierarchical belief states for digital assistants
DK201770431A1 (en) 2017-05-15 2018-12-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. Far-field extension for digital assistant services
US10455115B2 (en) 2017-05-19 2019-10-22 Microsoft Technology Licensing, Llc Adaptive stylus position interpolation

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3439341A (en) * 1965-08-09 1969-04-15 Lockheed Aircraft Corp Hyphenation machine
US4400697A (en) * 1981-06-19 1983-08-23 Chyron Corporation Method of line buffer loading for a symbol generator

Family Cites Families (73)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NL230090A (en) 1957-08-14
JPS6249483A (en) 1985-08-28 1987-03-04 Hitachi Ltd Character inputting system for real time handwritten character recognition
US3145367A (en) 1959-07-27 1964-08-18 Stanford Research Inst Character recognition circuit
US3199078A (en) 1960-02-05 1965-08-03 Ibm Character identification device
US3462548A (en) 1965-05-13 1969-08-19 Robert M Rinder Combined writing device and computer input
US3626368A (en) 1969-01-13 1971-12-07 Hsing Chu Lee Character-reading apparatus including improved character set sensing structure
US3699518A (en) * 1971-01-18 1972-10-17 Commerce Usa Method and machine for reading handwritten cursive characters
US3784982A (en) 1971-08-16 1974-01-08 Isotec Inc Electro-optical handwritten character reader
US3835453A (en) 1972-06-01 1974-09-10 S Narayanan Code generating and receiving apparatus
US3903502A (en) 1973-08-29 1975-09-02 Creative Ventures Character recording system
US3930229A (en) 1974-01-31 1975-12-30 Stanford Research Inst Handwriting system
CH599599B5 (en) 1974-09-04 1978-05-31 Centre Electron Horloger
US3996557A (en) 1975-01-14 1976-12-07 MI2 Corporation Character recognition system and method
US4156911A (en) 1975-11-18 1979-05-29 Stanford Research Institute Dynamic re-creation of signatures
US4074114A (en) 1976-03-12 1978-02-14 Monarch Marking Systems, Inc. Bar code and method and apparatus for interpreting the same
US4241409A (en) 1977-12-30 1980-12-23 Nolf Jean Marie Hand held pen-size calculator
US4262281A (en) 1978-05-31 1981-04-14 Quest Automation Limited Processing of information
US4365235A (en) 1980-12-31 1982-12-21 International Business Machines Corporation Chinese/Kanji on-line recognition system
DE3126886A1 (en) 1981-07-08 1983-01-27 Olympia Werke Ag DEVICE FOR TEXT PROCESSING AND TEXT PROCESSING
JPS5856188A (en) 1981-09-30 1983-04-02 Fujitsu Ltd One-stroke character real-time recognition system for capital alphabet, number, and special symbol
JPS5935277A (en) 1982-08-23 1984-02-25 Nippon Telegr & Teleph Corp <Ntt> Controlling method by real-time recognition of handwritten character pattern
JPS5975375A (en) 1982-10-21 1984-04-28 Sumitomo Electric Ind Ltd Character recognizer
US4561105A (en) 1983-01-19 1985-12-24 Communication Intelligence Corporation Complex pattern recognition method and system
US4718102A (en) * 1983-01-19 1988-01-05 Communication Intelligence Corporation Process and apparatus involving pattern recognition
US4573196A (en) 1983-01-19 1986-02-25 Communications Intelligence Corporation Confusion grouping of strokes in pattern recognition method and system
US4531231A (en) 1983-01-19 1985-07-23 Communication Intelligence Corporation Method for distinguishing between complex character sets
EP0135615A3 (en) 1983-09-09 1987-03-25 Communication Intelligence Corporation Process and apparatus involving pattern recognition
JPS60131041U (en) 1984-02-08 1985-09-02 シャープ株式会社 Handwritten input information processing device
JPS60205686A (en) 1984-03-30 1985-10-17 Hitachi Ltd Handwritten character and graphic recognizing system
US4608658A (en) 1984-04-13 1986-08-26 Pencept, Inc. Method and apparatus for removing noise at the ends of a stroke caused by retracing
US4562304A (en) 1984-05-23 1985-12-31 Pencept, Inc. Apparatus and method for emulating computer keyboard input with a handprint terminal
US4751741A (en) 1984-07-19 1988-06-14 Casio Computer Co., Ltd. Pen-type character recognition apparatus
US4985929A (en) 1984-09-18 1991-01-15 Chizuko Tsuyama System for use in processing a speech by the use of stenographs
US4795349A (en) 1984-10-24 1989-01-03 Robert Sprague Coded font keyboard apparatus
US4724285A (en) 1985-11-07 1988-02-09 Digitext, Inc. Stenographic translation system
US4972496A (en) * 1986-07-25 1990-11-20 Grid Systems Corporation Handwritten keyboardless entry computer system
US5194852A (en) 1986-12-01 1993-03-16 More Edward S Electro-optic slate for direct entry and display and/or storage of hand-entered textual and graphic information
US4839634A (en) 1986-12-01 1989-06-13 More Edward S Electro-optic slate for input/output of hand-entered textual and graphic information
US4856820A (en) 1987-01-21 1989-08-15 Wright Line Inc. Coding system
US5022081A (en) 1987-10-01 1991-06-04 Sharp Kabushiki Kaisha Information recognition system
US5140645A (en) 1987-10-08 1992-08-18 Whitaker Ranald O Computer compatible character for reliable reading by photoreader
US4953225A (en) 1987-10-16 1990-08-28 Sharp Kabushiki Kaisha Handwritten character-recognizing apparatus for automatically generating and displaying character frames
US4883291A (en) 1988-05-11 1989-11-28 Telesis Controls Corporation Dot matrix formed security fonts
US5109352A (en) 1988-08-09 1992-04-28 Dell Robert B O System for encoding a collection of ideographic characters
US5022086A (en) 1988-12-20 1991-06-04 Sri International, Inc. Handwriting apparatus for information collection based on force and position
JPH02263275A (en) 1989-04-03 1990-10-26 Kiyadeitsukusu:Kk Preparing system for register pattern of handwritten character
US5133076A (en) 1989-06-12 1992-07-21 Grid Systems Corporation Hand held computer
US5125039A (en) 1989-06-16 1992-06-23 Hawkins Jeffrey C Object recognition system
CN1023916C (en) 1989-06-19 1994-03-02 张道政 Chinese keyboard entry technique with both simplified and original complex form of Chinese character root and its keyboard
US5049862A (en) 1989-10-06 1991-09-17 Communication Intelligence Corporation ("Cic") Keyless flat panel portable computer--computer aided notebook
US5038382A (en) 1989-10-13 1991-08-06 International Business Machines Corporation Multi-scale recognizer for hand drawn strokes
JP2865210B2 (en) 1989-12-04 1999-03-08 ソニー株式会社 Character recognition device
US5005205A (en) 1990-01-12 1991-04-02 International Business Machines Corporation Handwriting recognition employing pairwise discriminant measures
US5050232A (en) 1990-03-28 1991-09-17 Bgk Finishing Systems, Inc. Movable heat treating apparatus utilizing proportionally controlled infrared lamps
US5121441A (en) 1990-09-21 1992-06-09 International Business Machines Corporation Robust prototype establishment in an on-line handwriting recognition system
US5148155A (en) 1990-11-13 1992-09-15 Wang Laboratories, Inc. Computer with tablet input to standard programs
US5491495A (en) 1990-11-13 1996-02-13 Wang Laboratories, Inc. User interface having simulated devices
JPH05189617A (en) 1991-04-15 1993-07-30 Microsoft Corp Method and apparatus for arc segmentation in handwritten-character recognition
US5313527A (en) 1991-06-07 1994-05-17 Paragraph International Method and apparatus for recognizing cursive writing from sequential input information
US5550930A (en) 1991-06-17 1996-08-27 Microsoft Corporation Method and system for training a handwriting recognizer at the time of misrecognition
US5315667A (en) 1991-10-31 1994-05-24 International Business Machines Corporation On-line handwriting recognition using a prototype confusability dialog
US5848187A (en) * 1991-11-18 1998-12-08 Compaq Computer Corporation Method and apparatus for entering and manipulating spreadsheet cell data
IL100198A (en) * 1991-11-29 1999-10-28 Art Advanced Recognition Tech Character recognition method
US5333209A (en) 1992-03-24 1994-07-26 At&T Bell Laboratories Method of recognizing handwritten symbols
US5592566A (en) * 1992-05-27 1997-01-07 Apple Computer, Incorporated Method and apparatus for computerized recognition
JP2585922B2 (en) 1992-05-29 1997-02-26 日立ソフトウエアエンジニアリング株式会社 Electronic blackboard device
US5341438A (en) 1992-07-22 1994-08-23 Eastman Kodak Company Method and apparatus for segmenting and classifying unconstrained handwritten characters
US5649027A (en) * 1992-07-24 1997-07-15 Microsoft Corporation Recognition of handwritten words
WO1994010678A1 (en) 1992-11-03 1994-05-11 Palm Computing, Inc. Data input system for pen-based computers
JP2979109B2 (en) * 1992-12-03 1999-11-15 日本アイ・ビー・エム 株式会社 Recognition character information creating method and apparatus
US5500937A (en) * 1993-09-08 1996-03-19 Apple Computer, Inc. Method and apparatus for editing an inked object while simultaneously displaying its recognized object
US5596656B1 (en) 1993-10-06 2000-04-25 Xerox Corp Unistrokes for computerized interpretation of handwriting
JPH10510639A (en) * 1994-07-01 1998-10-13 パーム コンピューティング，インコーポレーテッド Multi pen stroke character set and handwritten document recognition system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3439341A (en) * 1965-08-09 1969-04-15 Lockheed Aircraft Corp Hyphenation machine
US4400697A (en) * 1981-06-19 1983-08-23 Chyron Corporation Method of line buffer loading for a symbol generator

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP0769175A4 *

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6647145B1 (en) 1997-01-29 2003-11-11 Co-Operwrite Limited Means for inputting characters or commands into a computer
EP0917040A3 (en) * 1997-11-11 2000-08-02 Hewlett-Packard Company Method for providing a cue in a computer system
EP0917040A2 (en) * 1997-11-11 1999-05-19 Hewlett-Packard Company Method for providing a cue in a computer system
WO2000073890A1 (en) * 1999-05-27 2000-12-07 Alexei Vladimirovich Afanasiev A method of inputting information into a computer device, a sticker keyboard and a computer device using this method
US8031212B2 (en) 2000-05-18 2011-10-04 Hewlett-Packard Development Company, L.P. Reorienting display on portable computing device
US10326871B2 (en) 2001-06-11 2019-06-18 Qualcomm Incorporated Integrated personal digital assistant device
US8976108B2 (en) 2001-06-11 2015-03-10 Qualcomm Incorporated Interface for processing of an alternate symbol in a computer device
US9549056B2 (en) 2001-06-11 2017-01-17 Qualcomm Incorporated Integrated personal digital assistant device
US9696905B2 (en) 2001-06-11 2017-07-04 Qualcomm Incorporated Interface for processing of an alternate symbol in a computer device
US10097679B2 (en) 2001-06-11 2018-10-09 Qualcomm Incorporated Integrated personal digital assistant device
US7406666B2 (en) 2002-08-26 2008-07-29 Palm, Inc. User-interface features for computers with contact-sensitive displays
US8259075B2 (en) 2009-01-06 2012-09-04 Hewlett-Packard Development Company, L.P. Secondary key group layout for keyboard
US20180095654A1 (en) * 2016-09-30 2018-04-05 Atlassian Pty Ltd Creating tables using gestures
US10620824B2 (en) * 2016-09-30 2020-04-14 Atlassian Pty Ltd Creating tables using gestures
US10990277B2 (en) 2016-09-30 2021-04-27 Atlassian Pty Ltd. Creating tables using gestures
US11275501B2 (en) 2016-09-30 2022-03-15 Atlassian Pty Ltd. Creating tables using gestures
US11693556B2 (en) 2016-09-30 2023-07-04 Atlassian Pty Ltd. Creating tables using gestures

Also Published As

Publication number Publication date
EP0769175A1 (en) 1997-04-23
US20020196978A1 (en) 2002-12-26
DE69533479D1 (en) 2004-10-14
EP0769175A4 (en) 1999-01-20
DE69533479T2 (en) 2005-09-22
EP0769175B9 (en) 2005-01-12
US6839464B2 (en) 2005-01-04
EP0769175B1 (en) 2004-09-08
JPH10510639A (en) 1998-10-13
US6493464B1 (en) 2002-12-10

Similar Documents

Publication Publication Date Title
WO1996001453A1 (en) Multiple pen stroke character set and handwriting recognition system
US5187480A (en) Symbol definition apparatus
US6567549B1 (en) Method and apparatus for immediate response handwriting recognition system that handles multiple character sets
US7257528B1 (en) Method and apparatus for Chinese character text input
US5319386A (en) Ideographic character selection method and apparatus
JPH05233630A (en) Method for describing japanese and chinese
US20040243389A1 (en) Method and apparatus for selecting symbols in ideographic languages
US20090132231A1 (en) Using input of rhyming characters for computer text entry of Chinese characters
WO2000043861A1 (en) Method and apparatus for chinese character text input
JP4030575B2 (en) Touch type key input device, touch type key input method and program
EP1546997B1 (en) Presenting recognised handwritten symbols
GB2259386A (en) Text processing
JPH08190603A (en) Character recognition device and its candidate character display method
JP2922513B2 (en) Display control method and device
JPS60217483A (en) Recognizer of character
AU665293B2 (en) Apparatus for encoding and defining symbols and assembling text in ideographic languages
AU2002322159B2 (en) Method of and apparatus for selecting symbols in ideographic languages
KR20040099937A (en) Device for inputting letter by using touch-screen and method therefor
JPH03502841A (en) character recognition device
JPH06139409A (en) Handwriting input device
JPH10124611A (en) Character recognizing device
AU2002322159A1 (en) Method of and apparatus for selecting symbols in ideographic languages
JPH06208660A (en) On-line character recognizing device
JPH05159106A (en) Handwritten character recognizing device
JPH07168909A (en) Character processing method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): JP KR

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 1995924077

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1995924077

Country of ref document: EP

WWG Wipo information: grant in national office

Ref document number: 1995924077

Country of ref document: EP