A method and apparatus for recognizing a gesture input on a display screen of a computer system. A method of recognizing an ellipse gesture comprises the steps of receiving a stroke input by a user onto a computer screen, smoothing the stroke by reducing the number of points that represent the stroke, determining whether the smoothed stroke includes at least a predetermined minimum number of points, and, if so, calculating angles between adjacent segments of the smoothed stroke and calculating derivatives of the angles. It is determined if the stroke substantially represents an ellipse by utilizing the angles and the derivatives of the angles. The method preferably includes selecting an ink object on the screen if the ink object is substantially overlapped by the smoothed stroke. A different method of implementing fuzzy recognition includes smoothing the stroke, normalizing the smoothed stroke, determining whether the normalized stroke matches one of a plurality of gesture prototype... |
Citations|
| US4365235 | Dec 31, 1980 | Dec 21, 1982 | International Business Machines Corporation | Chinese/Kanji on-line recognition system | | US4680804 | Mar 28, 1985 | Jul 14, 1987 | Hitachi, Ltd. | Method for designating a recognition mode in a hand-written character/graphic recognizer | | US4739317 | Mar 30, 1984 | Apr 19, 1988 | International Business Machines Corporation | Draw graphics capabilities | | US4812832 | Jul 15, 1987 | Mar 14, 1989 | Brother Kogyo Kabushiki Kaisha | Input control device | | US4821030 | Dec 19, 1986 | Apr 11, 1989 | Tektronix, Inc. | Touchscreen feedback system | | US4992630 | Jul 11, 1989 | Feb 12, 1991 | Summagraphics Corporation | Digitizer tablet with user customization of stylus proximity and pressure | | US5034989 | Jan 17, 1989 | Jul 23, 1991 | | On-line handwritten character recognition apparatus with non-ambiguity algorithm | | US5038382 | Oct 13, 1989 | Aug 6, 1991 | International Business Machines Corporation | Multi-scale recognizer for hand drawn strokes | | US5063600 | May 14, 1990 | Nov 5, 1991 | | Hybrid information management system for handwriting and text | | US5121442 | Oct 2, 1991 | Jun 9, 1992 | Sharp Kabushiki Kaisha | Figure input system | | US5157737 | May 14, 1990 | Oct 20, 1992 | Grid Systems Corporation | Handwritten keyboardless entry computer system | | US5218645 | Mar 29, 1991 | Jun 8, 1993 | Cell Analysis Systems, Inc. | Method and apparatus for separating cell objects for analysis | | US5220649 | Apr 29, 1991 | Jun 15, 1993 | | Script/binary-encoded-character processing method and system with moving space insertion mode |
Referenced by|
| US5796866 | Dec 8, 1994 | Aug 18, 1998 | Matsushita Electric Industrial Co., Ltd. | Apparatus and method for editing handwritten stroke | | US5861886 | Jun 26, 1996 | Jan 19, 1999 | Xerox Corporation | Method and apparatus for grouping graphic objects on a computer based system having a graphical user interface | | US6144764 | Jan 20, 1998 | Nov 7, 2000 | Mitsui High-tec, Inc. | Method and apparatus for on-line handwritten input character recognition and recording medium for executing the method | | US6249606 | Feb 19, 1998 | Jun 19, 2001 | Mindmaker, Inc. | Method and system for gesture category recognition and training using a feature vector | | US6438523 | May 18, 1999 | Aug 20, 2002 | | Processing handwritten and hand-drawn input and speech input | | US6459442 | Dec 1, 1999 | Oct 1, 2002 | Xerox Corporation | System for applying application behaviors to freeform data | | US6476834 | May 28, 1999 | Nov 5, 2002 | International Business Machines Corporation | Dynamic creation of selectable items on surfaces | | US6647145 | Jul 23, 1999 | Nov 11, 2003 | Co-Operwrite Limited | Means for inputting characters or commands into a computer | | US6883145 | Jun 12, 2001 | Apr 19, 2005 | | Arrow logic system for creating and operating control systems | | US7017124 | Jan 18, 2002 | Mar 21, 2006 | | Method for controlling electronic devices using digital recall tool | | US7254787 | Aug 5, 2003 | Aug 7, 2007 | | Method for formatting text by hand drawn inputs | | US7339580 | Dec 17, 2004 | Mar 4, 2008 | Apple Inc. | Method and apparatus for integrating manual input | | US7487461 | May 4, 2005 | Feb 3, 2009 | International Business Machines Corporation | System and method for issuing commands based on pen motions on a graphical keyboard | | US7511702 | May 9, 2006 | Mar 31, 2009 | Apple Inc. | Force and location sensitive display | | US7515135 | Jun 15, 2004 | Apr 7, 2009 | Research In Motion Limited | Virtual keypad for touchscreen display | | US7538760 | Mar 30, 2006 | May 26, 2009 | Apple Inc. | Force imaging input device and system | | US7567238 | Nov 21, 2001 | Jul 28, 2009 | NEC Corporation | Method for supporting medical treatment system and medical treatment support system | | US7580829 | Feb 11, 2005 | Aug 25, 2009 | Tegic Communications, Inc. | Apparatus and method for reordering of multiple language databases for text disambiguation | | US7610194 | Jul 16, 2003 | Oct 27, 2009 | Tegic Communications, Inc. | Dynamic database reordering system | | US7614008 | Sep 16, 2005 | Nov 3, 2009 | Apple Inc. | Operation of a computer with touch screen interface | | US7619618 | Jul 3, 2006 | Nov 17, 2009 | Apple Inc. | Identifying contacts on a touch surface | | US7653883 | Sep 30, 2005 | Jan 26, 2010 | Apple Inc. | Proximity detector in handheld device | | US7656393 | Jun 23, 2006 | Feb 2, 2010 | Apple Inc. | Electronic device having display and surrounding touch sensitive bezel for user interface and control | | US7656394 | Jul 3, 2006 | Feb 2, 2010 | Apple Inc. | User interface gestures | | US7663607 | May 6, 2004 | Feb 16, 2010 | Apple Inc. | Multipoint touchscreen | | US7705830 | Feb 10, 2006 | Apr 27, 2010 | Apple Inc. | System and method for packing multitouch gestures onto a hand | | US7750891 | Jun 28, 2004 | Jul 6, 2010 | Tegic Communications, Inc. | Selective input system based on tracking of motion parameters of an input device | | US7764274 | Jul 3, 2006 | Jul 27, 2010 | Apple Inc. | Capacitive sensing arrangement | | US7765486 | Sep 15, 2004 | Jul 27, 2010 | NBOR Corporation | Arrow logic system for creating and operating control systems | | US7778818 | Sep 21, 2007 | Aug 17, 2010 | Tegic Communications, Inc. | Directional input system with automatic correction | | US7782307 | Nov 14, 2006 | Aug 24, 2010 | Apple Inc. | Maintaining activity after contact liftoff or touchdown | | US7812828 | Feb 22, 2007 | Oct 12, 2010 | Apple Inc. | Ellipse fitting for multi-touch surfaces | | US7821503 | May 11, 2009 | Oct 26, 2010 | Tegic Communications, Inc. | Touch screen and graphical user interface | | US7844914 | Sep 16, 2005 | Nov 30, 2010 | Apple Inc. | Activating virtual keys of a touch-screen virtual keyboard | | US7880730 | Sep 11, 2007 | Feb 1, 2011 | Tegic Communications, Inc. | Keyboard system with automatic correction | | US7920131 | Aug 28, 2009 | Apr 5, 2011 | Apple Inc. | Keystroke tactility arrangement on a smooth touch surface | | US7925088 | Feb 23, 2005 | Apr 12, 2011 | Hewlett-Packard Development Company, L.P. | System, method and apparatus for automatic segmentation and analysis of ink stream | | US7929765 | May 12, 2010 | Apr 19, 2011 | Fujitsu Limited | Video text processing apparatus | | US7932897 | Aug 15, 2005 | Apr 26, 2011 | Apple Inc. | Method of increasing the spatial resolution of touch sensitive devices | | US7978181 | Apr 25, 2006 | Jul 12, 2011 | Apple Inc. | Keystroke tactility arrangement on a smooth touch surface | | US8115745 | Dec 19, 2008 | Feb 14, 2012 | Tactile Displays, LLC | Apparatus and method for interactive display with tactile feedback | | US8125463 | Nov 7, 2008 | Feb 28, 2012 | Apple Inc. | Multipoint touchscreen | | US8130203 | May 31, 2007 | Mar 6, 2012 | Apple Inc. | Multi-touch input discrimination | | US8147248 | Mar 21, 2005 | Apr 3, 2012 | Microsoft Corporation | Gesture training | | US8159501 | Jul 12, 2008 | Apr 17, 2012 | International Business Machines Corporation | System and method for smooth pointing of objects during a presentation | | US8201087 | Jan 31, 2008 | Jun 12, 2012 | Tegic Communications, Inc. | Spell-check for a keyboard system with automatic correction | | US8217908 | Jun 19, 2008 | Jul 10, 2012 | Tactile Displays, LLC | Apparatus and method for interactive display with tactile feedback | | US8225203 | Nov 4, 2010 | Jul 17, 2012 | Nuance Communications, Inc. | Spell-check for a keyboard system with automatic correction | | US8237681 | Jul 2, 2010 | Aug 7, 2012 | Tegic Communications, Inc. | Selective input system and process based on tracking of motion parameters of an input object | | US8237682 | Oct 22, 2010 | Aug 7, 2012 | Tegic Communications, Inc. | System and process for selectable input with a touch screen | | US8238662 | Jul 17, 2007 | Aug 7, 2012 | SMART Technologies ULC | Method for manipulating regions of a digital image | | US8239784 | Jan 18, 2005 | Aug 7, 2012 | Apple Inc. | Mode-based graphical user interfaces for touch sensitive input devices | | US8243041 | Jan 18, 2012 | Aug 14, 2012 | Apple Inc. | Multi-touch input discrimination | | USRE40153 | May 27, 2005 | Mar 18, 2008 | Apple Inc. | Multi-touch system and method for emulating modifier keys via fingertip chords | | USRE40993 | Jan 13, 2006 | Nov 24, 2009 | Apple Inc. | System and method for recognizing touch typing under limited tactile feedback conditions | | USRE42738 | Oct 8, 1998 | Sep 27, 2011 | Apple Inc. | Portable computers |
Claims1. A method for recognizing a gesture input on a display screen for a computer system, the method comprising the steps of: - (a) receiving a stroke input by a user onto a computer screen, the stroke being represented as a string of points;
- (b) smoothing said stroke by reducing the number of points that represent the stroke such that the points of the smoothed stroke are located at least a threshold distance apart from one another, said smoothing step including the substeps of:
- (i) calculating a position for a new point that is positioned between two adjacent points in the stroke that are separated by less than the threshold distance; and
- (ii) replacing the two adjacent points that are separated by less than the threshold distance with the new point; and
- (iii) repeating steps (i) and (ii) until all of the remaining points in the stroke are separated by at least the threshold distance;
- (c) determining whether said smoothed stroke includes at least a predetermined minimum number of points;
- (d) calculating angles between adjacent segments of said smoothed stroke and calculating derivatives of said angles; and
- (e) determining whether said stroke substantially represents an ellipse, the ellipse determining step utilizing said angles and said derivatives of said angles and including the substeps of:
- (i) determining whether said stroke substantially curves in one direction, wherein said stroke is not considered to substantially represent an ellipse when said stroke does not curve in one direction;
- (ii) determining whether a point on said stroke that is within a predetermined portion including an end point of said stroke is within a predetermined distance of an opposite end point of said stroke, wherein said stroke is not considered to substantially represent an ellipse when said point is not within said predetermined distance of said opposite end point; and
- (iii) summing said derivatives of said angles and checking whether said sum is within a predetermined range of values, wherein said stroke is not considered to substantially represent an ellipse when said sum is outside said range of values;
- wherein steps (d) and (e) are only executed when said smoothed stroke includes at least said predetermined minimum number of points.
2. A method as recited in claim 1 further comprising the steps of: - determining whether an object displayed on a computer screen is substantially overlapped by said smoothed stroke; and
- selecting said object when said object is overlapped by said smoothed stroke and said smoothed stroke substantially represents an ellipse.
|