« PreviousContinue »
ALPHANUMERIC INFORMATION INPUT
This invention relates generally to user interfaces and more particularly to user interfaces that convert user input into alphanumeric information.
An increasing number of user devices require an increasing amount of user input and often that user input represents alphanumeric information such as, for example, the text of a message or the address of a recipient. At the same time, 15 many user devices, and especially portable user devices, are becoming smaller. Smaller form factors unfortunately reduce the amount of physical space available for the user interface (this tends not only to impact the user input mechanism but also corresponding output mechanisms, 20 memory, and computational resources as also support the overall user interface). As a result, various existing user interfaces tend to be unsatisfactory for one reason or another.
This general dissatisfaction exists essentially regardless of the input modality itself (including keyboards and key- 25 pads, speech recognition, handwriting recognition, and more exotic input mechanisms such as those driven by alpha waves and other biological electromagnetic signals and fields). Such approaches tend to be too large and/or require too much in the way of device resources (such as memory 30 or computational capability), and/or are otherwise time consuming, tedious, error prone, and for many users tend to discourage rather than encourage usage. Generally put, present solutions for small user devices often tend to result in a poor user experience, one way or the other, ranging from 35 an inability to properly utilize the input mechanism to an uncomfortably high and fatiguing cognitive load to achieve what amounts to modestly acceptable performance for the input mechanism.
Recognition techniques (or, for some keypad-based input 40 mechanisms, so-called predictive techniques) attempt to address these issues by resolving ambiguous input data into words that presumably a user is likely to input into the device. Such techniques are usually based upon standard models that represent common word usage amongst a sam- 45 pling study, along with other statistical information (for example the distribution of character shapes for a so-called handwriting recognizer). As a result, these techniques can work well for some users and quite poorly for others. Further, such techniques often require considerable training 50 as the user interface is not sufficiently intuitive to the user. In addition, even users for whom the technique works relatively well often find that the technique does not work equally well for all of their needs (for example, the resultant selections may be relatively useful when supporting creation 55 of a business-related message but relatively unhelpful or even annoying when the user creates more casual correspondence).
As a partial remedy for this concern, it has been suggested that such standard models can be augmented by using an 60 additional dictionary of words that the user must create and customize. While this approach can improve recognition for a given user, the editing process itself can be time consuming, tedious, and otherwise typify some of the same problems that the predictive techniques were initially trying to 65 alleviate. In addition, such solutions tend to be highly consumptive of both memory and processing capacity.
These needs can render such techniques ill suited for many portable user devices where these resources are either limited or other corresponding limitations of concern exist (such as respecting a need to minimize current consumption in a portable device).
Simply put, present user input devices and techniques do not provide, especially for small user devices, a suitable and relatively intuitive user input interface that will reduce the time and associated cognitive load required to accurately input alphanumeric information, either for a given input modality or in compatible cooperation with a variety of input modalities.
BRIEF DESCRIPTION OF THE DRAWINGS
The above needs are at least partially met through provision of the alphanumeric information input method described in the following detailed description, particularly when studied in conjunction with the drawings, wherein:
FIG. 1 comprises a block diagram depiction of various user and system operating embodiments in accordance with the invention;
FIG. 2 comprises a flow diagram depicting various embodiments of a process in accordance with the invention;
FIG. 3 comprises a representative depiction of potential contents of a personal context model in accordance with various embodiments of the invention;
FIG. 4 comprises a block diagram depiction of a particular illustrative embodiment configured in accordance with the invention;
FIG. 5 comprises a flow diagram depicting various embodiments of a process as configured in accordance with the invention;
FIG. 6 comprises a detailed flow diagram depicting an alternative embodiment of a process as in accordance with the invention; and
FIG. 7 comprises a flow diagram depicting yet another embodiment of a process as configured in accordance with the invention.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. Also, some components may be shown in reduced number in order to render more clearly an understanding of various embodiments of the present invention.
A prediction engine that receives data meant to represent alphanumeric information from a user and predicts at least one additional item of alphanumeric information based upon that received data and upon a personal context model for that user is disclosed. The personal context model includes both a personal language model that represents a statistical analysis of prior alphanumeric content for this particular user and context markers that correlate to the personal language model in significant and dynamic ways.
Generally speaking, the various embodiments described below provide, support provision of, or utilize a prediction engine that receives data meant to represent alphanumeric information from a user (including all manner of fontrepresentable characters and symbols) and predicts at least one additional item of alphanumeric information based upon that received data and upon a personal context model for that
user. The personal context model includes both a personal language model that represents a statistical analysis of prior alphanumeric content for this particular user and context markers that correlate to the personal language model in significant and dynamic ways. The prediction is then typi- 5 cally presented to the user. Preferably, one word-only predictions are presented as part of an overall strategy to minimize cognitive load on the user (even when other candidate predictions exist) (as used herein, "word" refers to a single sequence of one or more alphanumeric characters 10 representing a semantic unit, and includes traditional words as one finds in a dictionary, so-called texting language (such as using "u" for "you" or "L8R" for "later"), so-called emoticons (such as ":-)" or the corresponding font character "©" to express happiness), font-based symbols (alone or in 15 combination with other symbols or alphanumeric characters, such as "I?U"), and other expressions of various kinds (such as all or part of Internet addresses, telephone numbers, passwords, credit card numbers, and so forth). In one embodiment, the personal context model is created by use of 20 existing alphanumeric material of the user, including email files. This greatly facilitates provision of a vocabulary that is relevant to the user and that also allows the contextual links to be noted as mentioned above.
Referring now to FIG. 1, a suitable working environment 25 for practice of various embodiments in accordance with the invention includes a user device 11 having an adaptive input recognition and prediction engine 12 and a personal context model 17 included therewith, a user input interface 13, and a display 14. The device 11 will typically constitute an 30 electronic device such as, but not limited to, a communication device (for example, cellular telephones, 2-way radios, pagers, and so forth), personal digital assistant, pre-recorded audio playback device (for example, MP3 and CD players), remote control, teletext interface as used in some countries 35 to facilitate interactive television services, computer (including both desktop, laptop, and other portable variations), or portable or desktop game module. It will also be seen that the device 11 can itself constitute a component in other products, including various consumer electronic devices 40 (clocks, radios, stereo components, and kitchen appliances to name a few illustrative examples) where the device would facilitate some type of alphanumeric information exchange between a user and the other product.
The user input interface 13 essentially serves to detect 45 bio-sourced physical indicia from a user to thereby transduce such indicia into corresponding alphanumeric information. Examples include keypads with one or more keys that can be physically asserted by a user, script (including both cursive and printed handwriting) input devices, audio- 50 responsive inputs (including voice recognition mechanisms), and a wide variety of electromagnetic biological signal transducers (including devices responsive to muscle flexure, brain waves, eye movement, and so forth). The display 14 will usually comprise a visual display of size and 55 nature as appropriate to the given application that presents alphanumeric characters (other options, including text-tospeech or other user feedback mechanism, can serve as well when appropriate to a given application).
The device 11 will typically include an adaptive input 60 recognition and prediction engine 12 as an integrated element thereof (this can be done through dedicated integrated hardware and/or software) though these elements could be made more severable as might be appropriate to a given application. Many suitable candidate devices 11 already 65 have a microprocessor and for such devices a substantially or fully software implemented embodiment would probably
often be a preferred approach (additional detail regarding the adaptive input recognition and prediction engine 12 will be presented below).
In addition, as may be appropriate to some embodiments, a remote platform 15 and/or one or more alternative user platforms 16 can interface to each other and/or to the user device 11. Examples will be provided below.
The adaptive input recognition and prediction engine 12 uses a personal context model 17 to both recognize ambiguous input and provide completion and prediction capability. In order for the adaptive input recognition and prediction engine 12 to function optimally, the personal context model 17 needs to represent a personal context model for the user in question. To some extent, such a model can include a certain amount of generic or at least categorical content. Pursuant to these embodiments, however, the model will preferably contain considerable content that is uniquely developed from and for the user in question, hence making the context model a personal context model. As depicted, the personal context model 17 comprises an integral part of the device 11 itself. The personal context model 17 can typically reside in the device 11, or, if desired, can reside in a separate portable device. Such a portable device (such as a small memory and associated interface mechanism) could hold a user's current personal context model and provide this model to whatever device 11 the user coupled the portable device to (using, for example, USB or other known data porting mechanisms or standards). In this way, a user could utilize a variety of devices during the course of a day without necessarily having each such device constantly maintain that user's personal context model in its own resident memory.
Pursuant to one embodiment, and referring now to FIG. 2, at least portions of the personal context model can be developed and installed in the user device 11 prior to significant use of the device 11 (thus allowing the user to begin use without any training or explicit user-directed customization of the model). In particular, pre-existing alphanumeric material from the user can be received 21 and statistically analyzed 22. The alphanumeric material itself can include email files for the user, including subject matter content, address book content, and even the unique language usage and patterns of the individual user. The subject matter content can be statistically analyzed to note, in general, individual words that tend to be used with notable frequency by this particular user as well as word pairs (or longer expressions or phrases) that similarly occur in this user's communications with frequent occurrence. Such analysis can be used in general to create a personal language model that includes words (and word sequences) that are relatively likely to be used by this particular user.
In addition, a considerable amount of context information will typically be included with such email files. For example, a given email has corresponding identified recipients, a time and day of transmission, a subject header and the like. Such information can be used to better inform the statistical analysis of the textual information. For example, the analysis can discern that one set of specific unique words and/or expressions tends to be used when the user is sending messages to a given individual. These correlations, when statistically significant, can be noted and used to create a personal context model. As another example, the analysis can discern that another set of specific unique words or expressions tends to be used when the user is sending messages at a particular time of day (for example, communications created between 11:30 AM and 12:15 PM may, for a given user, be statistically more likely to include the words "lunch," "eat," and "Fred's diner." As a result, the personal
context model includes not only words (and word sequences) that are likely to be used by a given user, but also correlations between such usage and discernable user contexts. For example, while it will be discerned that a given user might use the expression "John Doe" more frequently 5 than any other multiple word expression that contains the word "John," it will also be discerned that, on Wednesdays, "John" is more likely to be used in the multiple word expression "John White" (because, presumably, the user tends, on that day, to regularly source communications to the 10 latter individual). As noted below, combining the unique vocabulary of a user with correlations between specific instances of usage and various user contexts allows for rapid and relatively reliable prediction of intended subsequent alphanumeric information when a user is inputting informa- 15 tion to the user device 11.
With momentary reference to FIG. 3, the personal context model 17 so developed includes a personal language model 32 containing words and word sequences that are of greater likelihood to be used by the given individual for whom the 20 personal context model 17 is made as well as correlations between these words and their usage with a variety of user contexts 33. If desired, only one or two such contexts can be used, however, greater reliability can in most cases be achieved with more contextual information. Some useful 25 contexts include the nature of a recipient's device (for example, a pager or other short message service as distinguished from a standard computer email account), the content of a message to which the user is presently replying, the content of previous messages to this particular recipient, the 30 recipient's name, the present time of day, the present day (of the week, month, or as a calendar date), the subject line for the message, a given geographic location, heading, speed of travel, and so forth (when that information is known and available), and a given transactional context (contrasting, for 35 example, business communications from friendly personal communications), to name a few. Each of these contexts can potentially be monitored and/or sensed in a user device 11 and used to inform the adaptive input recognition and prediction engine 12 by exploiting the correlation informa- 40 tion in the personal context model 17 as between the personal language model 32 and the various contexts 33 used.
Referring again to FIG. 2, when making 22 the statistical analysis, the process can additionally consider other material 45 and or analysis 23 as may be available. For example, prior raw material for this individual could be processed anew in conjunction with the newly received 21 information. Or, previous analysis results for this user, including previously developed or modified personal context models, could be 50 accessed and used to facilitate and/or supplement the analysis of the new material. For example, in one embodiment of the method the personal context model will adjust itself with usage on the device automatically. In the above-suggested examples, the additional material relates to the user himself 55 (or herself). If desired, however, the additional material could constitute other information that, although not originating with the user, may still relate to the user in some appropriate way. For example, if standard language models are available for various professions, hobbies, cultures, and 60 so forth, such models could be used to supplement and enrich the statistical analysis (in this way, for example, a few occurrences of the word "baby" in the user's materials could be statistically weighted more heavily if it were known that the user was a pediatrician and the word "baby" had an 65 overall high frequency of use in a pediatrician-specific language model).
As noted, the result of this statistical analysis is to provide 24 a personal context model for a specific user. When this process is effected on the user device 11 itself, then the device 11 will simply begin to use the model. As will be shown below, however, this analysis and development of a personal context model can also be done at a remote location. In that event, the personal context model is then transmitted 25 to the relevant user device so that it may be used. Nonetheless, even in this case, the transmitted personal context model on the device can also then be modified directly on the device itself as well as use the remote location for updates to the model for more powerful versions or to transfer to or update a second device.
While it is possible for a personal context model to be developed for a given user and then used thereafter without change, it would be more beneficial to many users for the model to be updated to better ensure that changes in the user's textual content are tracked and incorporated. In this manner the model evolves naturally to the user's own changing language patterns. Such updating can be frequent, infrequent (though periodic), or driven by some non-temporal stimuli (such as by the user initiating the process when desired). The processing of updating the personal context model at the remote location can be effected both independently of the updates on the device as well as in addition to device updating. The purpose of updating at a remote location where appropriate resources (dedicated if desired) can be brought to bear are at least threefold: First, to facilitate initial use of the device 11 or corresponding product without any particular training or programming; second, to take advantage of more powerful processing and updating versions of the personal context model at the remote location; and third, to use the remote location as a medium for transferring the personal context model to other devices and also assuring that all models are in sync with the current model.
For example, with reference to FIG. 4, email files from a user's personal computer 41 could be communicated via a communication medium 44 (such as, for example, the Internet or an intranet or other data distribution mechanism) to a model server 43. The model server 43 can function to work the process as described above to statistically analyze the information available and to form the personal context model for the user. That personal context model can then be forwarded to a different user's device 42 (again through the communication medium 44 if available and/or through any other information delivery mechanism, such as wireless services, as available and appropriate). Through such a process, user-relevant information can be used to inform the development of a personal context model at a remote site, which model is then transferred to and used in one or more other user devices. Of course, other ways of communicating with other user devices could also be readily supported. Regardless of what services and/or connecting mediums are used, it might also be desirable to use encryption or other mechanisms to protect the confidentiality of the data during such exchanges.
As noted above, the user device 11 uses the personal context model 17 for alphanumeric information prediction purposes. More particularly, and referring now to FIG. 5, the user device 11 receives 51 alphanumeric input from the user via the user input interface 13 described above. The user device 11, using the adaptive input recognition and prediction engine 12, then uses shared or dedicated processing capability to predict 52 subsequent alphanumeric information based both upon the received user data and the personal context model 17. As one example, the user could begin a
message on a wireless short message device (such as a two-way pager, a two-way personal digital assistant, or a cellular telephone with short message service capability) by addressing it to "John Doe." This alphanumeric information constitutes not only user data, but also provides context 5 information (i.e., the identity of the recipient of the message). In addition, the user device 11 can be aware of the time of day and the day, which for purposes of this example will be 11:57 AM on a Wednesday. All of this context information can be used in conjunction with the personal 10 context model 17 to note correlalations with this user's personal language model. For example, taking the above context information into account, it may be evident that the user, under these conditions, tends to use the following words and word sequences with high frequency: "lunch," 15 "How's about," "Bob's cafe," "the cafeteria," and "r u available?" These high frequency words, in context, and high frequency word sequences, are used to form the desired predictions.
In most cases the present prediction will likely be pre- 20 sented 53 to the user on the display 14 of the user device 11. To minimize cognitive load on the user, pursuant to a preferred embodiment, only one-word predictions are presented at a time. For example, to continue the example above, the prediction may in fact be that the next alphanu- 25 meric information to be entered by the user will be, "How's about Bob's cafe for lunch?" Notwithstanding this prediction, the only prediction presented will be "How's." With each acceptance of a correct prediction, the user will be offered the next prediction. In this manner a user can 30 conceivably complete an entire sentence or message without entering additional specific text
Generally speaking, the prediction mechanism should be non-intrusive, i.e., no prediction would be inserted into the text without an explicit action from the user to accept it. In 35 the preferred embodiment, the predicted alphanumeric information will have a different visual appearance than the text already entered so far (for example in highlighted fashion) and positioned after the insertion cursor. The user may ignore it and continue entering alphanumeric data with a 40 single input 54 instance. Conversely, a specific input 54 event could be used to accept 55 the prediction as is (as described above).
When a prediction is accepted 55, a next predicted 52 alphanumeric information is then presented 53 without 45 requiring the user to effect entry of additional alphanumeric information. For example, upon accepting "How's" in the example above, the word "about" would be displayed. Again the word "about" would be highlighted or otherwise rendered immediately editable to allow a user to reject the 50 prediction and seek other alphanumeric expressions. If the word "about" were accepted, however, the word "Bob's" would then be displayed as the next prediction. In this way, it is possible for entire messages to be created with only very little current textual entry by the user. Furthermore, the 55 interface operates relatively intuitively and users can achieve success with little or no training.
If desired, and as presented in this embodiment, a timer can determine 56 whether a predetermined amount of time T has elapsed from the time a prediction is presented without 60 the user making a subsequent input Generally speaking, a time delay of this sort can indicate that the user is not accepting the presented prediction (but has not yet determined what alphanumeric information to provide by way of substitution). When the time T is met in this way, pursuant 65 to this embodiment, a new prediction is made and presented. Continuing the above example, "How's" and "about" may
have been accepted, but the timer expired following presentation of "Bob's." The process would remove "Bob's" from the display and substitute a new prediction. For purposes of this example, if we presume that the next most statistically likely word is "the" then the word "the" would be automatically presented without further action being necessary on the part of the user. If this word were accepted, then the next predicted word would be presented (which, for purposes of this example, would be "cafeteria") as before.
The particular time period used for this purpose can be arbitrarily set, or can be modifiable by a user, and/or can be automatically alterable by the device itself. For example, the time duration can be dynamically set to match observed experience (i.e., a time T that corresponds appropriately to how long it often takes the user to respond to an inaccurate prediction with substitute alphanumeric information). In general, it is expected that this time T would tend to be shorter rather than longer, or more on the order of less than two seconds in duration.
As described above, when the time T expires, a new prediction is substituted for the previous prediction. In fact, a wholly new prediction could be made as suggested, or, in the alternative, a previous candidate prediction as already made could simply be substituted at this time. Referring to FIG. 6, an alternative embodiment to that described with respect to FIG. 5 provides for an additional feature—the provision of a synonym that corresponds to a predicted alphanumeric expression. In particular, once the process has predicted 52 the next alphanumeric entry, the process can then use a digital thesaurus to identify 61 a synonym for the predicted entry and the process can then proceed as described above by presenting the synonym to the user. Such a feature should ordinarily be selectable by the user such that it can be active or inactive as desired by the user. It may also be useful in a given application to limit such a synonym function to specific parts of speech (such as nouns, verbs, adjectives, and so forth).
In the embodiments described above with respect to FIG. 5, new predictions can be displayed in the absence of receiving any inputs from the user. For some users this may be useful. In general, however, many users will probably find such a feature confusing. With reference to FIG. 7, an alternative embodiment can again receive user input 71, and then determine 72 whether that input constitutes an acceptance of a previously presented prediction. If true, then the process can insert 73 the prediction into a device 11 text buffer, reset a refused prediction list 74, and continue by predicting 52 additional subsequent alphanumeric information (again as described above and as informed by the personal context model 17), presenting 53 the new prediction, and in this embodiment, setting 80 a timer. Another alternative embodiment is depicted in FIG. 7, in that a refuse list 79 of previously unaccepted predictions for the present prediction can be maintained and used when forming a present prediction to avoid reusing an already-presented prediction that was refused by the user.
When the user input does not constitute an acceptance of a previously presented prediction, but instead provides additional information, the process determines 75 whether the information being input by the user remains consistent with the previous prediction 76, if any. If not true (either because the user input contains information that is inconsistent with the previous prediction 75 or because there is no outstanding previous prediction), then the process again makes a new prediction 52 and the process continues as described above. If, however, the information being input by the user remains consistent with the previous prediction, the process deter