US 20030006956 A1
A data entry device having an integral input element capable of recording input movement in two dimensions (including Chinese strokes and characters, Roman letters and Arabic numerals) and delivering resultant signals to a processor. The processor is programmed for identifying a handwriting input represented by the signals.
1. A data entry device comprising:
an array of switching elements capable of recording input movement between at least four discrete points arranged in two dimensions and providing a series of discrete inputs;
a memory having stored therein handwriting input identifiers and at least one series of discrete inputs for each handwriting input identifier;
a processor coupled to the array of switching elements and the memory and programmed for searching the handwriting input identifiers with a series of discrete inputs received from the array of switching elements for identifying a handwriting input represented by the series of discrete inputs.
2. The data entry device of
3. The data entry device of
4. The data entry device of
5. The data entry device of
6. The data entry device of
7. The data entry device of
8. The data entry device of
9. The data entry device of
10. The data entry device of
11. The data entry device of
12. The data entry device of
13. The data entry device of
14. A method of data entry comprising:
providing a data entry device capable of recording input movement between at least four discrete points arranged in two dimensions;
receiving from the data entry device a series of discrete inputs identifying a series of discrete points of the data entry device;
storing in memory a table of characters having at least one series of discrete inputs for each character;
indexing the table of characters with a series of discrete inputs received, thereby identifying a character represented by the series of discrete inputs.
15. The method of
16. The method of
17. The method of
18. The method of
19. A data entry device comprising:
a two-dimensional input device comprising:
a joystick element;
analog elements mounted in first and second dimensions providing analog movement signals responsive to movement of the joystick element in first and second dimensions of a horizontal plane;
an analog-to-digital converter coupled to the input device; and
a processor coupled to the analog-to-digital converter for receiving digitized inputs from the input device, the processor having instructions that cause the processor to perform pre-processing operations for segmenting into strokes digitized inputs received from the input device, and to perform recognition operations recognizing the strokes as handwritten input and generating character identifications therefrom.
20. The data entry device of
21. The data entry device of claim 20, wherein the processor is responsive to the push switch for segmenting into strokes digitized input received while the switch is pushed.
 This invention relates to a method which permits a user to input data (including Chinese strokes and characters, Roman letters and Arabic numerals) using a compact input device in a novel and convenient manner.
 Many proposals and designs exist for using keypad on small hand-held devices for entry of Chinese strokes as part of character input process. In operation, a user simply types in the strokes by pressing the keys in a sequence according to the natural writing order of the strokes of the character he wishes to generate.
 Generally, strokes of Chinese characters can be classified into certain number of basic stroke types. A typical example of that number is 26. We refer to such a number as N. These N basic strokes can be further grouped into 5 to 9 stroke categories according to various defining criteria. From now onwards, we refer to the stroke categories simply as strokes.
 The prevailing method for entry of Chinese strokes on a keypad requires that each kind of stroke be assigned to a specific key on the keypad. A user types in a stroke by pressing the corresponding key. One example of such a scheme can be found in US patent application No. 09/220,308 of Guo et al., filed on Dec. 23, 1998 and assigned to the assignee of the present application, where 9 strokes are defined and assigned to 9 corresponding keys on the phone keypad respectively. Such an example can also be found in Motorola's CD928C cellular telephone.
 In operation, a user enters the strokes of a character by pressing on the corresponding keys one at a time. A set of candidate characters is generated and presented on a display as matching alternatives. The displayed set of candidate characters is updated with every entry of a stroke. The user selects the character he or she wants from the set of candidate characters.
 Two disadvantages with the above method lie in, first, the separately defined keys and second, the less intuitive relationship between the stroke writing process and the key pressing process. Due to the first constraint, for every intended stroke a user has to select a specific key among several corresponding keys. When fast or blind typing of strokes is required, a user may encounter many misfires when doing the key selecting and pressing at a fast pace or in a dark environment. The method requires the conversion from the natural stroke writing process to the key pressing process. The relationship between these two is not very straightforward nor highly intuitive. Therefore, in addition to decoding characters into strokes, a user's mind also needs to constantly engage in matching strokes to associated keys. For a casual user, in order to be assured of a correct result, the user's eyes have to monitor not only the set of candidate characters, but also the key pressing on the keypad. It makes the whole character entry process inefficient and stressful.
 There also exists a method for inputting Roman letters on a small keypad, as can be found on many fixed-line phones or cell phones. In this method, generally, nine of the ten numerical keys are employed to input Roman letters, with each key assigned to a certain number of Roman letters. Typically, numerical key 1 is assigned to ABC, key 2 is assigned to DEF, . . . , key 8 is assigned to WXYZ. In operation, a user presses key 1 once to enter A, twice to enter B, three times to enter C. The user presses on key 2 to enter D or E or F, on key 3 to enter G or H or I, and so on.
 This method involves two steps when inputting a letter. First, a specific key is to be selected from the keypad for the intended letter. Second, the user needs to press that particular key for an exact number of times in order to retrieve the letter. The latter step can be avoided by using a disambiguation scheme such as that described in Arnott, J. L et al, “Probabalistic Character Disambiguation for Reduced Keyboards Using Small Text Samples”, Augmentative and Alternative Communication, 1992.
 Although this method is fairly straightforward, with almost no learning curve needed, it has two disadvantages. First, for a casual user, it requires intensive visual checking on the keypad when doing key selecting and pressing. Second, it is neither compatible to the way a user types on a normal size keyboard nor analogous to the way the user writes on paper. Hence, it affects the efficiency in user's content composing and results in slower input speed.
 Other methods of data entry involve writing characters on a tablet using a stylus and performing hand recognition on the input penstrokes. Devices manufactured for this method require a tablet of significant area and generally require a special stylus. The tablet area does not permit use of the method on very small devices such as small mobile telephones. The stylus is an inconvenient additional element, as it can be lost. Two-handed operation is necessary: one hand to hold the device and the other hand to operate the stylus. Such a method is not optimal for a busy user.
 There is therefore a demand for an improved method for data entry where fewer keys are needed, and where there is an intuitive relationship between the writing and the key pressing process, so as to achieve better ergonomic efficiency.
 According to one aspect of the present invention, a data entry device is provided comprising an array of switching elements capable of recording input movement between at least four discrete points arranged in two dimensions and providing a series of discrete inputs. The array of switching elements may be touch-sensitive or proximity-sensitive switches or pushbuttons and may be activated with a pen, finger or stylus, or they may be discrete points of operation of a joystick, trackball, mouse or similar device. They may be arranged at points of a compass or in a two-dimensional matrix array.
 A table of handwriting input identifiers (e.g. strokes, character components, characters or pseudo-characters) is stored in memory with at least one series of discrete inputs for each handwriting input identifier. The table of handwriting input identifiers is indexed (e.g. by a look-up operation or a search operation) with a series of discrete inputs received from the array of switching elements. In this manner, a handwriting input represented by the series of discrete inputs is identified and may be displayed to the user or stored or otherwise used as original data entry.
FIG. 1 is a block diagram illustrating an embodiment of a data entry device in accordance with the invention.
FIGS. 2a-2 f are diagrams showing different examples of stroke inputs using a four-switch input device.
FIG. 3 is a flow diagram illustrating operation of the program controlling the microprocessor of FIG. 1.
FIGS. 4, 5 and 6 are a front view, rear view and elevation view respectively of a joystick-type device for use in place of the input device of FIG. 1.
FIG. 7 is a diagram of examples of Roman letter input using a four-switch input device.
FIG. 8 is a block diagram of an alternative embodiment of the invention.
 Referring to FIG. 1, a data entry device 10 (such as a cellular telephone, a wireless messaging communicator, a personal digital assistant, a memo-writer or other device) is shown comprising a microprocessor 11, an input device 12, a display 13 (or other output device such as a RF or IR link), a program memory 14 and a data memory 15.
 The input device may take a number of forms, any of which is capable of recording input movement between at least four discrete points arranged in two dimensions. In the preferred embodiment it comprises four push keys 20 in a matrix or square of about 1.5 cm in height and 1.5 cm width, with one key at each corner. For convenience's sake, the top-left key is numbered 1, the top-right 2, the bottom-left 3, and the bottom-right 4. Each key can be sensitive to pressure or sensitive to merely the presence of the fingertip on or near the key. The four keys provide four inputs into the microprocessor. The four inputs are illustrated as four discrete lines, but it will be understood that four data states can be represented by two data lines (with or without a “data active” line).
 The data entry device 10 may have further buttons (not shown) for other functions. For example, it preferably has a 12-key keypad for entry of digits 0-9 and for calling. Scrolling keys may be provided for menu control.
 The memory 15 is illustrated as having two tables 25 and 26. Tables are not essential, but it will be explained below that tables are a useful and convenient manner of translating inputs from the input device 12 into primitive handwriting elements and (if required by the ultimate character set of the language in question) for translating primitive handwriting elements into data characters.
 In operation, a user enters vector or movement strokes into the input device 12 by passing his or her finger across the keys 20 in two dimensions in a horizontal plane. The input device 12 generates a series or sequence of discrete inputs to the microprocessor 11, dependent on the keys activated. This is explained in greater detail below with reference to language examples. The microprocessor, under the control of program code stored in program memory 14, performs a look-up or search operation in the memory 15 to uniquely identify the intended character or to identify the most likely intended character.
 Where a character is uniquely identified by the microprocessor 11, it is displayed on display 13. Where several likely intended characters are identified, one or more is or are displayed on display 13. Where one character only is displayed, the character that is deemed to be the most likely intended character is displayed. Where several candidate characters are displayed, these are displayed in a selection list in decreasing order of likelihood of usage. Likelihood of intended usage is determined by frequency information in the memory 15, showing the frequency of usage of a character in the language in question (or the relative frequency of usage from among several candidates) or the frequency of usage of a character in the context of the character in the language (e.g. taking into account the previous N characters). A more detailed explanation is set out below.
 In one example, an apparatus is described for inputting Chinese strokes.
 In operation, if a user wishes to input a left-right horizontal stroke, the user moves his or her thumb horizontally across key1 and key2. The signal that key1 and key2 have been consecutively visited in a particular sequence is sent to the microprocessor 11 for interpretation as horizontal stroke. In the same manner, for a top-down vertical stroke, the user moves his or her thumb across key1 and key2.
 The method presented here defines 6 fundamental strokes, namely “horizontal”, “vertical”, “slash”, “back slash”, “clockwise” and “counter-clockwise”. There now follows, with reference to FIGS. 2a-2 f, a brief description of the six fundamental strokes and how they can be entered via sequence of keys. In the following table (Table 1), “No.” is a serial number for the fundamental strokes; “Type” is the name of the fundamental strokes; “Var” is the number of variations of thumb move paths; and “Path” is the thumb move paths defined by the sequence of keys being visited.
 Strokes 1-6 are illustrated in FIGS. 2a-2 f respectively. A stroke is classified as “clockwise turn” type if and only if the first turn of the stroke is of clockwise direction, no matter how many turns may follow thereafter and no matter what direction they are. The same applies to the “counter-clockwise turn” stroke type. A thumb move path is the sequence of keys which have been visited during a preset time period from the first key of the sequence being visited until a pre-defined time-off is detected. A time-off is the time elapse from the moment the previous key is visited to the moment the next key is visited.
 Upon entering strokes using discrete thumb, finger, stylus, trackball, mouse or other two-dimensional stroke inputs, i.e. discrete signals representative of discrete vectors, the microprocessor 11 translates the vector inputs into strokes using Table 1 and then performs a tabular look-up or other search in table 26 in memory 15 for Chinese characters corresponding to the stroke inputs. A two-stage translation is preferred, as this reduces the size of table 26, but it is not necessary. Chinese character identifiers can alternatively be stored in table 26 in a manner indexable by a raw vector input sequence.
 A method for storing and performing look-up of Chinese characters from primitive stroke data is described in co-pending patent application No. 08/806,504 of Chen, assigned to the assignee of the present invention and incorporated herein by reference, which describes a table of digit streams representative of characters in a desired language and character addresses corresponding to the digit streams. The digit streams are equivalent to sequences of strokes (e.g. strokes 1-6 above). Alternatively, in accordance with the teaching of the present invention, the digit streams can be key sequences representative of stroke vectors.
 Also described in the co-pending patent application, and illustrated in FIG. 3 herein, is a further table 27 of characters located at addresses corresponding to character addresses from table 26. A character is read from table 27 located at a character address obtained from table 26. Characters in the table 27 that have a common stroke sequence (or common digit stream) are stored in order of relatively decreasing frequency of use. This scheme allows for changing the relative addresses of characters in the table 27 to adjust for relative frequency of use of characters. A further table 28 can optionally be provided to perform bigram look-up operations. The output of table 27 (or table 28 if used) is standard hexGB coding of one or more Chinese characters. A further look-up is used to obtain and display the pictorial representation of the character.
 The above method has the following two notable and advantageous features. First, it makes use of only four keys. Therefore, the keypad can be relatively small in size. The four keys are distinctively positioned, the chance of getting confused with different keys and thus resulting misfires on wrong keys has been greatly reduced.
 Second, this method does not require a one-to-one match from an actual writing stroke to a designated key. Instead, it associates the thumb move path over the keys with the actual writing trajectory of the intended stroke. This builds an intuitive relationship between the stroke writing and key pressing processes. In addition, the definition of six fundamental strokes allows 17 different kinds of entry variations in total, which cover all N basic stroke types. It has an unprecedented feature that the majority of basic strokes can be drawn on the 4-key keypad. This scheme makes it possible that a user only needs to look at the set of candidate characters and make a selection among them without the need of watching closely the movement of the thumb over the keys on the keypad. It allows a user to input strokes in a natural and convenient manner.
 Some editing functions have also been defined by the thumb move path over the keys, as described below.
 A further embodiment of the invention is now described with reference to entry of Roman letters and Arabic numerals (alphanumeric data entry). The embodiment will again be described using a 4-key keypad.
 In operation, to enter a Roman letter, the user draws a stroke with his thumb on the four keys. Different schemes can be used to represent Roman letters using a single stroke (i.e. pen-down to pen-up strokes) or a small number of strokes. Graffiti (trademark) is an example of a scheme.
 A list of strokes is defined for each of the 26 uppercase Roman letters, as shown in Table 2. The first column is the corresponding uppercase Roman letter, the second column is the number of variations of strokes. The third column is the strokes defined as the sequence of keys to be pressed during the time period from the first key of the sequence is visited until a pre-defined time-off is detected. Here a time-off is defined as the maximum time elapse allowed from the moment a key is visited to the moment its subsequent key is visited.
 On a well-designed keypad, the strokes can be drawn on four keys by moving around the thumb or a finger. This is illustrated in FIG. 4 for the first four letters of the Roman alphabet. Thus, the letter “A” can be drawn in four (or more) ways and can be drawn as a single pen-down (or thumb-down) stroke (as in “Graffiti”) or using two pen-down strokes, each starting at the apex. A dotted line in FIG. 4 illustrates the thumb being lifted, generally to return the thumb to a key to begin a new stroke at another key. Note that the table does not distinguish between two keys being visited in a thumb-down sequence and two keys being visited in a thumb-up sequence. These distinctions are shown only for ease of understanding.
 Note also that other sequences can be added to the sequences shown, provided the time-out timer permits. Thus, for example, a horizontal cross-stroke can be added to the sequence of stroke vectors that represent the letter “A” (e.g. 232434 or 32434). In this particular example, such an additional stroke is redundant, because the extra stroke 3-4 adds nothing to disambiguate the input sequence, which is already uniquely identified by the first two strokes 2-3-2-4 or 3-2-4. Thus it can be seen that it is not necessary to include further strokes of a letter in Table 1.
 A timer is started for each letter entry. Preferably the timer begins with the first key-press of a new character. All strokes entered before the time-out are considered as a single entry (character or numeral). Any additional strokes not necessary for character recognition (e.g. the cross-stroke of the letter “A”) are simply disregarded. This has the advantage that a user does not need to learn a special style of input (e.g. the user does not need to drop the cross-stroke on the letter “A”). In addition, the timer is reset as soon as a character is uniquely decoded and the immediate following stroke entered is not a legitimate stroke, thus allowing a user to immediately begin entry of the next character. However, if the immediate following stroke entered before the time-out is a legitimate stroke although not necessary, it is still disregarded, but the timer will not be reset until either after the time-out or the immediate following stroke entered is not a legitimate stroke.
 Ultimately the exact choices for stroke sequences in Table 2 is a matter for compromise and is dependent on handwriting habits and preferences of typical users.
 Preferably no two letters of the Roman alphabet can be represented by the same sequence of keys, but this is not essential. If there are ambiguities (e.g 3143 for “A” and for “B”), these ambiguities will need to be resolved using disambiguation techniques such as through n-grams.
 The above method has the following advantageous features. First, it uses a small number of keys. Hence, the keypad can be designed relatively small to allow very compact implementations. Second, this method uses an intuitive relationship between the actual letter writing and key pressing. In addition, the multiple definitions of strokes provide reasonable variations to give users more freedom in writing than, for example rigid adherence to a single-stroke scheme (such as “Graffiti”). The scheme makes it possible to do blind typing of Roman letters on a small keypad.
 Based on the same concept, the ten Arabic digits 0-9 can also be input by stroke sequences, as presented below.
 It can be easily seen that there are some overlaps between the strokes defined for letters and those for digits. Therefore, a mode switch between these two is preferably introduced to avoid confusions.
 Referring to FIGS. 5, 6 and 7, a front view, rear view and elevation view respectively of an alternative input device 50 are shown. The alternative input device has a joystick element 100 (which term is to be understood as including other button or lever devices moveable in two dimensions in a horizontal plane, including mouse-buttons). The joystick element 100 is mounted on a spring-loaded mounting illustrated as a ball-and-socket mounting 101 by way of example. The mounting is biased such that the joystick element returns to a central resting position (not shown) when not under thumb or finger pressure. Springs 104-107 are shown as providing bias, but it will be understood that these need not be discrete helical springs and may be replaced by a single elastomeric member. Four discrete contacts 110-113 are shown at four equally spaced compass points around the center (north-west, north-east, south-east and south-west respectively).
 As shown in FIG. 6, there is a silvered circle 120 on the rear of the ball of the ball-and-socket mounting 101 and there is a ground contact 121 fixed relative to the ball-and socket mounting and positioned centrally behind the ball.
 In operation, a user moves the joystick element 100 with his or her thumb or finger and the ball rotates such that the silvered circle 120 makes contact between the ground contact 121 and one of the discrete compass-point contacts 110-113. In this way, the input device of FIG. 5 can generate a series of discrete inputs just like the four-key input device 12 of FIG. 1. A north-west movement of the joystick generates the same input as key 1 of input device 12, and so on.
 It will be understood by one of ordinary skill in the act that other joystick elements can achieve the same result. For example, a ball-and-socket arrangement with an asymmetric ball can be used that activates four or more microswitches similar to the buttons of input device 12 of FIG. 1. The joystick does not need to have a ball-and-socket at all.
 It will also be understood by one of ordinary skill in the art that more than four contacts can be used for the input device 12 of FIG. 1 or the input device 50 of FIG. 5. For example, six, eight, twelve or sixteen compass point contacts can be used. Alternatively, a matrix of 3×3 or 4×4 buttons or contacts could be used. Tables 1 and 2 would need to be reformulated accordingly, and there would be many more stroke variations permissible for each item in these tables. Alternatively, the joystick button input device of FIG. 5 does not have a ball-and-socket, but is fixed on its mounting and uses orthogonal strain gauge elements to provide a continuous (i.e. progressive, non-discrete) 2-dimensional output (e.g. two analog voltage outputs) which is divided into discrete values by the microprocessor 11 or by an interface into the microprocessor 11 (e.g. an analog-to-digital converter).
 Referring to FIG. 7, a microswitch 150 is shown mounted beneath the ball-and-socket mounting 101 of the input device 50 of FIG. 5. The microswitch 150 is a push-to-make switch and can be used for a number of purposes.
 In one embodiment, the microswitch 150 is used as a pen-down indicator. In this variation, a single input stroke is measured from pen-down to pen-up. This has the advantage of disambiguating between pen-down and pen-up segments. All contiguous pen-down segments can be captured and used for character recognition, regardless of whether they are captured within a time-out time or after expiry of a time-out timer. This allows for greater flexibility in user-variations of time duration when entering strokes or characters. A “data active” line on the input device 12 of FIG. 1 can perform the same function, such that all continuous thumb-down movements cause an activation of at least one button and cause activation of the “data active” line, whereas a thumb-up event gives no data active signal. Instead of a data active line, timing measurements by the processor 11 can be used to measure the time lapse between button presses (if any) and so determine if there has been a thumb-up event.
 If an input device 50 such as that of FIG. 7 is used that gives a continuous analog output, the preprocessing technique of U.S. Pat. No. 5,740,273 followed by the recognition technique of U.S. Pat. No. 5,742,705 can be used to interpret and recognize Roman letters or Chinese characters written as pen-down writing segments. This is illustrated in FIG. 8.
 In FIG. 8, a joystick element 200 is shown having strain gauges (or other analog elements) 201 and 202 that provide analog movement indications for movement of the joystick element 200 in orthogonal x and y dimensions in a horizontal plane. Integral with the joystick element 200 is a push switch 204, preferably a push-to-make switch.
 The analog elements 201 and 202 are connected to analog-to-digital (A/D) converters 210 and 211 (or to a single shared A/D converter), which are coupled to a processor 220. The switch 204 is also coupled to the processor 220.
 The processor 220 has a program stored in program memory that causes it to perform a scaling (normalizing) function 221 on the inputs from the A/D converters 210 and 211, for example as described in U.S. Pat. No. 5,740,273. Inputs from the A/D converters are accepted by the scaling function 221 when the switch 204 indicates a “push” condition (equivalent to a pen-down state). Following the scaling function, an optional smoothing function 222 is carried out and a segmentation function 223. The segmentation function segments the two-dimensional input into segments at natural bends in the input, thereby providing a sequence of raw stroke segments. A matching function 224 matches the segments against pre-stored templates from template store 230 in a manner known in the art, for example as described in U.S. Pat. No. 5,742,705.
 The arrangement of FIG. 8 is particularly useful for entry and recognition of ideographic characters (e.g. Chinese characters), but is not limited thereto, and is useful for Roman character entry or Grafitti (trade mark) type of stroke entry. The smoothing, segmenting and matching steps can be modified (or omitted where unnecessary) to suit the type of data entry.
 This method makes inputting of alphanumeric letter on a small input device in a convenient and efficient manner, and it also leads to very compact implementations.