« PreviousContinue »
HAPTIC TRACKBALL DEVICE
CROSS REFERENCE TO RELATED
This application is a continuation of patent application Ser. No. 09/507,539 filed on Feb. 18, 2000, now U.S. Pat. No. 6,707,443, which is a continuation-in-part of parent patent applications:
Application Ser. No. 09/103,281, now U.S. Pat. No. 6,088, 10 019, filed Jun. 23, 1998 on behalf of Louis Rosenberg, entitled, "Low Cost Force Feedback Device with Actuator for Non-Primary Axis,"
Application Ser. No. 09/253,132, filed Feb. 18, 1999 now U.S. Pat. No. 6,243,078 on behalf of Louis Rosenberg, 15 entitled, "Low Cost Force Feedback Pointing Device,"
Application Ser. No. 09/456,887, now U.S. Pat. No. 6,211, 861, filed Dec. 7, 1999, on behalf of Louis Rosenberg, entitled, "Tactile Mouse Device," all assigned to the assignee of this present application, and all of which are incorporated 20 by reference herein in their entirety.
BACKGROUND OF THE INVENTION
The present invention relates generally to interface devices 25 for allowing humans to interface with computer systems, and more particularly to computer interface devices that allow the user to provide input to computer systems and allow computer systems to provide haptic feedback to the user.
A user can interact with an environment displayed by a 30 computer to perform functions and tasks on the computer, such as playing a game, experiencing a simulation or virtual reality environment, using a computer aided design system, operating a graphical user interface (GUI), etc. Common human-computer interface devices used for such interaction 35 include a mouse, joystick, trackball, steering wheel, stylus, tablet, pressure-sensitive sphere, or the like, that is connected to the computer system controlling the displayed environment. Typically, the computer updates the environment in response to the users manipulation of a physical manipulan- 40 dum such as a joystick handle or mouse, and provides visual and audio feedback to the user utilizing the display screen and audio speakers. The computer senses the user's manipulation of the user object through sensors provided on the interface device that send locative signals to the computer. For 45 example, the computer displays a cursor or other graphical object in a graphical environment, where the location of the cursor is responsive to the motion of the user object.
In some interface devices, force feedback or tactile feedback is also provided to the user, more generally known 50 herein as "haptic feedback." These types of interface devices can provide physical sensations which are felt by the user manipulating a user manipulandum of the interface device. One or more motors or other actuators are coupled to the joystick or mouse and are connected to the controlling com- 55 puter system. The computer system controls forces on the joystick or mouse in conjunction and coordinated with displayed events and interactions by sending control signals or commands to the actuators. The computer system can thus convey physical force sensations to the user in conjunction 60 with other supplied feedback as the user is grasping or contacting the interface device or manipulatable object of the interface device. For example, when the user moves the manipulatable object and causes a displayed cursor to interact with a different displayed graphical object, the computer can 65 issue a command that causes the actuator to output a force on the physical object, conveying a feel sensation to the user.
One problem with current force feedback controllers in the home consumer market is the high manufacturing cost of such devices, which makes the devices expensive for the consumer. A large part of this manufacturing expense is due to the inclusion of multiple actuators and corresponding control electronics in the force feedback device. In addition, high quality mechanical and force transmission components such as linkages and bearings must be provided to accurately transmit forces from the actuators to the user manipulandum and to allow accurate sensing of the motion of the user object. These components are complex and require greater precision in their manufacture than many of the other components in an interface device, and thus further add to the cost of the device. A need therefore exists for a haptic device that is lower in cost to manufacture yet offers the user haptic feedback to enhance the interaction with computer applications.
SUMMARY OF THE INVENTION
The present invention is directed to a low-cost haptic feedback trackball device connected to a computer system, the trackball device having a simple actuator for low cost force feedback for enhancing interactions and manipulations in a displayed graphical environment.
More specifically, the present invention relates to a haptic feedback trackball device that is coupled to a host computer which implements a host application program. The device includes a housing that is physically contacted by said user, the housing resting on a support surface. A sphere is positioned in the housing, the sphere being rotatable in two rotary degrees of freedom. A sensor device detects the movement of the sphere in the rotary degrees of freedom and outputs sensor signals representative of the movement. An actuator applies a force to the housing approximately along an axis that is substantially perpendicular to the support surface, where the force is transmitted to the user contacting the housing. The force is preferably correlated with a graphical representation displayed by the host computer, where a position of the sphere in the rotary degrees of freedom corresponds with a position of a cursor displayed in the graphical representation.
Preferably, at least one compliant element is provided between a portion of the housing contacted by the user and the support surface, where the compliant element amplifies the force output from the actuator by allowing the contacted portion of the housing to move with respect to the support surface. For example, the compliant element can be one or more feet provided on the underside of the housing and made of a compliant material such as rubber or foam. Or, the compliant element can be a compliant coupling provided between the contacted portion of the housing and a non-contacted portion of the housing.
In some embodiments, the force is an inertial force that is output approximately along the axis that is substantially perpendicular to the support surface, where the actuator outputs the inertial force to the housing by moving an inertial mass. The actuator can be coupled to a flexure that provides a centering spring bias to the inertial mass. The inertial force can be a pulse, vibration or texture correlated with the interaction of a user-controlled cursor with a graphical object displayed in a graphical user interface. For example, the pulse can be output when the cursor moves between menu items in a displayed graphical menu. In other embodiments, the force is a contact force that is provided by driving a moving element that contacts the user. The moving element can be a cover portion of the housing that is movably coupled to a base portion of the housing. Alternatively, the moving element can be a button that also provides button input to the host com3
puter. Some embodiments may include a second actuator, such as a passive brake, for outputting a force on the sphere in its degrees of freedom. A method for providing haptic feedback similarly includes detecting the motion of a sphere of the trackball device, receiving information from the host com- 5 puter indicating that a tactile sensation is to be output, and outputting a force on the housing of the trackball device approximately along an axis perpendicular to a support surface.
The present invention advantageously provides a haptic 10 feedback trackball device that is significantly lower in cost than other types of haptic feedback devices and is thus quite suitable for home consumer applications. A single actuator can be provided that applies a force in a particular degree of freedom, such as the Z-axis perpendicular to the support 15 surface, and compliance is provided between surface and user contact. This allows more compelling forces to be experienced by the user, and also enhances the user's experience of a third dimension relative to the surface plane. Furthermore, the actuator of the present invention can provide a variety of 20 different types of force sensations to enhance the user's interfacing and experience with a computer application.
These and other advantages of the present invention will become apparent to those skilled in the art upon a reading of the following specification of the invention and a study of the 25 several figures of the drawing.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a perspective view of system including a haptic 30 trackball device of the present invention connected to a host computer;
FIG. 2 is a side cross sectional view of the trackball device of FIG. 1 providing inertial forces;
FIG. 3 is a perspective view of one embodiment of an 35 actuator assembly suitable for use with the present invention;
FIG. 4 is a side cross sectional view of the trackball device of FIG. 1 providing contact forces;
FIG. 5 is a block diagram of the haptic device and host computer of the present invention; and 40
FIG. 6 is a diagrammatic view of a display screen showing graphical objects associated with force sensations output using the haptic device of the present invention.
DETAILED DESCRIPTION OF PREFERRED 45
FIG. 1 is a perspective view of a haptic feedback interface system 10 of the present invention capable of providing input to a host computer based on the user's manipulation of a 50 trackball and capable of providing haptic feedback to the user of the interface system based on events occurring in a program implemented by the host computer. System 10 includes a trackball device 12 and a host computer 14. It should be noted that the term "trackball" as used herein, indicates any 55 device in which a spherical object can be rotated by the user to provide input to the host computer.
Trackball device 12 includes a housing 13 and a sphere or ball 15. Sphere 15 is contacted by a user's finger and/or palm and is rotated in any direction, e.g. in two degrees of freedom. 60 In the embodiment shown, the user preferably rests his or her palm on the housing 13 when moving the sphere 15 to provide a hand rest. In other embodiments, the trackball device can provide a large sphere 15 and a small housing 13 with buttons, or can provide a sphere that can be contacted in multiple 65 areas, such as with a thumb and a forefinger on both sides of the sphere.
The sphere 15 can be rotated in the two degrees of freedom to provide input to the host computer 14. For example, a user can move sphere 15 to provide two-dimensional input to a computer system to correspondingly move a computer generated graphical object, such as a cursor or other image, in a graphical environment provided by computer 14 or to control a virtual character, vehicle, or other entity in a game or simulation. In addition, trackball device 12 preferably includes one or more buttons 16a and 16b to allow the user to provide additional commands to the computer system, and may also include additional buttons.
Trackball device 12 preferably includes an actuator 18 which is operative to produce forces on the trackball device
12. This operation is described in greater detail below with reference to FIG. 2.
Trackball device 12 rests on a ground surface 22 such as a tabletop or other reference surface. A user contacts the sphere 15 and housing 13 while using the device 12. Since the sphere 15 can move in both directions without moving the housing
13, the housing typically remains stationary with respect to the surface 22. Sensor mechanisms used to detect and measure sphere 15 rotation are described below with reference to FIG. 2.
Trackball device 12 is preferably a relative device, in which the device 12 reports a change in position to the host computer and the host controls a graphical object, adjusts a value, etc., based on the change in position. Thus, the sphere 15 can be rotated indefinitely in any direction. In other embodiments, the trackball device 12 may be implemented as an absolute device, in which the sphere 15 has an absolute position in a workspace and the absolute position is reported to the host computer. In one absolute embodiment, stops canbe placed in the workspace of the sphere 15 to prevent the sphere from moving outside the bounded workspace.
Trackball device 12 is coupled to the computer 14 by a bus 20, which communicates signals between device 12 and computer 14 and may also, in some preferred embodiments, provide power to the trackball device 12. Components such as an actuator (described below) require power that can be supplied from through the bus 20 if the bus is, for example, a USB or Firewire bus. In other embodiments, signals can be sent between trackball device 12 and computer 14 by wireless transmission/reception. In some embodiments, the power for the actuator can be supplemented or solely supplied by a power storage device provided on the device 12, such as a capacitor or one or more batteries. Some embodiments of such are disclosed in U.S. Pat. No. 5,691,898, incorporated herein by reference.
Host computer 14 is preferably a personal computer or workstation, such as a PC compatible computer or Macintosh personal computer, or a Sun or Silicon Graphics workstation. For example, the computer 14 can operate under the WindowsTM, MacOS, Unix, or MS-DOS operating system. Alternatively, host computer system 14 can be one of a variety of home video game console systems commonly connected to a television set or other display, such as systems available from Nintendo, Sega, or Sony. In other embodiments, host computer system 14 can be a "set top box" which can be used, for example, to provide interactive television functions to users, a "network-" or "internet-computer" which allows users to interact with a local or global network using standard connections and protocols such as used for the Internet and World Wide Web, or other appliance or device allowing the user to provide two-dimensional (or greater) input for selection or control. Host computer preferably includes a host microprocessor, random access memory (RAM), read only memory
(ROM), input/output (I/O) circuitry, and other components of computers well-known to those skilled in the art.
Host computer 14 preferably implements a host application program with which a user is interacting via trackball device 12 and other peripherals, if appropriate, and which 5 may include force feedback functionality. For example, the host application program can be a video game, word processor or spreadsheet, Web page or browser that implements HTML or VRML instructions, scientific analysis program, virtual reality training program or application, or other appli- 10 cation program that utilizes input of device 12 and outputs force feedback commands to the device 12. Herein, for simplicity, operating systems such as WindowsTM, MS-DOS, MacOS, Linux, Be, etc. are also referred to as "application programs." In one preferred embodiment, an application pro- 15 gram utilizes a graphical user interface (GUI) to present options to a user and receive input from the user. Herein, computer 14 may be referred as providing a "graphical environment", which can be a graphical user interface, game, simulation, or other visual environment. The computer dis- 20 plays "graphical objects" or "computer objects," which are not physical objects, but are logical software unit collections of data and/or procedures that may be displayed as images by computer 14 on display screen 26, as is well known to those skilled in the art. A displayed cursor or a simulated cockpit of 25 an aircraft might be considered a graphical object. The host application program checks for input signals received from the electronics and sensors of trackball device 12, and outputs force values and/or commands to be converted into forces output for trackball device 12. Suitable software drivers 30 which interface such simulation software with computer input/output (I/O) devices are available from Immersion Corporation of San Jose, Calif.
Display device 26 can be included in host computer 14 and can be a standard display screen (LCD, CRT, flat panel, etc.), 35 3-D goggles, or any other visual output device. Typically, the host application provides images to be displayed on display device 26 and/or other feedback, such as auditory signals. For example, display screen 26 can display images from a GUI.
As shown in FIG. 1, the host computer may have its own 40 "host frame" 28 which is displayed on the display screen 26. In contrast, the device 12 has its own workspace or "local frame" in which the sphere 15 is moved. In a position control paradigm, the position (or change in position) of a usercontrolled graphical object, such as a cursor, in host frame 28 45 corresponds to a position (or change in position) of the sphere 15 in the local frame. The offset between the object in the host frame and the object in the local frame can be changed by the user by indexing, i.e., moving the sphere 15 while no change in input is provided to the host computer. Indexing is typically 50 not needed for a trackball since the workspace of the sphere 15 is infinite.
In alternative embodiments, the device 12 can be a different interface or control device. For example, a hand-held remote control device used to select functions of a television, 55 video cassette recorder, sound stereo, internet or network computer (e.g., Web-TVTM), mouse device, or a gamepad controller for video games or computer games, can include a sphere 15 for input and can be used with the haptic feedback components described herein. Handheld devices can still ben- 60 efit from the directed inertial sensations described herein which, for example, can be output perpendicularly from the device's top surface. In yet other embodiments, the actuator 18 (and all the variations described herein) can be positioned within a handle of a large joystick, and so provide tactile 65 sensations such as pulses and vibrations to the user grasping the joystick handle. Similarly, the actuator embodiments
mentioned herein can be placed in the grasped steering wheel of a wheel controller. The actuator assembly can be scaled to the desired size to provide haptic sensations appropriate for the size and mass of the controller device.
FIG. 2 is a side cross-sectional view of the trackball device 12 of FIG. 1. Trackball device 12 includes one or more actuators 18 for imparting haptic feedback such as tactile sensations to the user of the device 12. The actuator outputs forces on the device 12 which the user is able to feel.
Trackball device 12 includes a housing 13, a sensing system 40, and an actuator 18. Housing 13 can be provided in a variety of shapes to allow the user to manipulate the sphere 15 and buttons 16. Sensing system 40 detects the position of the sphere in its two rotary degrees of freedom. In some embodiments, sensing system 40 can include cylindrical rollers 52 which are coupled to sensors 54, such as optical encoders, for detecting the motion of the sphere 15. Each roller 52 is frictionally coupled to the sphere 15 and rotates when the sphere 15 rotates, and the associated sensor 54 detects the rotation of the roller. A roller and sensor can be used for each of the degrees of freedom of the sphere 15, e.g. the x-direction and y-direction.
Other types of mechanisms and/or electronics for detecting motion of the sphere 15 can be used in other embodiments. For example, some trackball devices employ non-contact optical emitters and detectors to sense motion of the sphere 15. In some of these embodiments, the motion of a surface or pattern on the sphere, such as dots or bars, is detected. Such optical sensing methods can be used in the present invention. Other types of sensors can also be used, such as magnetic sensors, analog potentiometers, etc.
An actuator 18 is coupled to the housing 13 to provide haptic feedback to the user. The haptic feedback can generally be provided in two forms: inertial forces and contact forces. Inertial forces are provided by moving an inertial mass, which causes forces on the housing felt by the user. Contact forces are more direct forces applied to the user, such as by moving an element of the housing which contacts the user's hand.
A preferred embodiment creates inertial forces that are directed substantially in a particular degree of freedom, i.e. along a particular axis. The inertial forces can be created, for example, using a high bandwidth linear actuator; preferred actuators include a linear moving voice coil actuator and a linear moving-magnet actuator, which are suitable for high bandwidth actuation. A traditional servo motor used in a harmonic drive configuration can also be a suitable high bandwidth actuator. This embodiment allows for high fidelity control of force sensations in both the frequency and magnitude domains. This also allows the forces to be directed along a desired axis and allows for crisp tactile sensations that can be independently modulated in magnitude and frequency.
The preferred direction for the output forces is along the Z-axis. Since the tactile sensations are directed in a third degree of freedom relative to the two-dimensional planar surface and display screen, jolts or pulses output along the Z axis feel much more like three-dimensional bumps or divots to the user, increasing the realism of the tactile sensations and creating a more compelling interaction. For example, an upwardly-directed pulse that is output when the cursor is moved over a window border creates the illusion that the sphere or whole device 12 is moving "over" a bump at the window border.
In a first inertial force feedback embodiment, actuator 18 is preferably a linear actuator having a stationary portion coupled to the device housing 13 (and thus stationary only with respect to the portion of the housing to which it is coupled), and a moving portion that moves linearly approxi