DISPLAY SYSTEM CAPABLE OF
ACCEPTING USER COMMANDS BY USE OF
VOICE AND GESTURE INPUTS
 Inventors: Haru Ando, Kokubunji; Yoshinori
Kitahara, Musashimurayama, both of
 Assignee: Hitachi, Ltd., Tokyo, Japan
 Appl. No.: 137,788
 Filed: Oct. 19, 1993
 Foreign Application Priority Data
Oct. 20, 1992 [JP] Japan 4-282057
 Int. CI.6 G06T 11/00
 U.S. CI 395/133; 395/326; 395/806
 Field of Search 395/161, 155-161,
395/133, 700, 153, 154, 118
 References Cited
U.S. PATENT DOCUMENTS
4,811,243 3/1989 Racine 395/161
5,201,034 4/1993 Matsuura et al 395/155
FOREIGN PATENT DOCUMENTS
4-239832 8/1992 Japan .
"Maclife", No. 25,9, Sep. 1990, Kawade Publishers, pp. 82-88, & 97-99.
"Put-That-There: Voice and Gesture at the Graphics Interface", R. Bolt, Computer Graphics, vol. 14, No. 3,1980, pp. 262-270.
"Fundamentals of Voice Information Processing", Saito et al, The Ohm-sha, Ltd., 1981, pp. 95-104. "Study on Colloquial Sentence Accepting Method in Information Retrieval System Based on Voice Inputs", Japan Institute of Acoustics, 3-5-7, Mar. 1991, Kitahara et al, pp. 101-102.
"Media Space for Systemization of Conceptual Knowledge", Computer Science, vol. 2, No. 1, 1991, Fujisawa, pp. 39-45.
"Personal Computing in the Year of 2010", Nikkei Byte, Jun. 1992, Takachiho et al, pp. 228-233.
"Simultaneous Language and Image Information Processing in a User-Interface Link Between Language and Image Taking Group Information into Consideration" 36th Information Processing Institute, Hataka et al, pp. 1371-1372.
"Study on Integration-Model F Information on Multi-Model Interface", Collection of Data for 8th Symposium on Human Interface, Oct. 1992, Ando et al, pp. 259-264.
W. Wahlster, "User and Discourse Models for Multimodal Communication", Intelligent User Interfaces, 1991, ACM Press, Chapter 3, pp. 45-67.
Primary Examiner—Almis R. Jankus
Attorney, Agent, or Firm—Fay, Sharpe, Beall, Fagan, Min
nich & McKee
A method of accepting multimedia operation commands wherein, while pointing to either of a display object or a display position on a display screen of a graphics display system through a pointing input device, a user commands the graphics display system to cause an event on a graphics display, through a voice input device; comprising a first step of allowing the user to perform the pointing gesture so as to enter a string of coordinate points which surround one area for either of the display object and any desired display position; a second step of allowing the user to give the voice command together with the pointing gesture; a third step of recognizing a command content of the voice command by a speech recognition process in response to the voice command; a fourth step of recognizing a command content of the pointing gesture in accordance with the recognized result of the third step; and a fifth step of executing the event on the graphics display in accordance with the command contents of the voice command and the pointing gesture. Thus, the method provides a man-machine interface which utilizes the plurality of media of the voice and the pointing gesture, which offers a high operability to the user, and with which an illustration etc. can be easily edited.