WO2006103196A1

WO2006103196A1 - Input forecasting method and a user input forecasting interface

Info

Publication number: WO2006103196A1
Application number: PCT/EP2006/060993
Authority: WO
Inventors: Birgit Otto; Klaus-Peter Schneider
Original assignee: Siemens Aktiengesellschaft
Priority date: 2005-03-31
Filing date: 2006-03-23
Publication date: 2006-10-05
Also published as: DE102005014774A1

Abstract

The invention relates to an input forecasting method in which a sensor detects that a user extremity comes closer to an input element or is brought into contact therewith when said input element is not actuated. In addition, a data processing unit outputs, by means of an output device, information about at least one effect which can take place by the input element actuation.

Description

description

Procedure for input preview and user interface with input preview

The invention relates to a method for input preview. Conventional user interfaces assume that a user can extract the function of their input elements from their labeling, prior knowledge or operating instructions. For the input elements of a

The user interface may include buttons, knobs, switches, knobs, jog dials, swashplates, joysticks, pedals, rotary pushbuttons, barrel keys, rotary dials, footswitches, pedals, and similar simple or complex input elements. For example, the keys of conventional keyboards are labeled so that the user can predict the effect of a key press before pressing the key concerned. Input elements in a vehicle are also labeled by symbols or color coding to give the user an indication of their function.

However, with a variety of input elements, such as found in today's vehicles, or a user without previous knowledge, it must be assumed that the user can not extract the function of the input elements alone their caption. In this case, the user must consult a manual. The same applies if complex or context-dependent functions can be triggered via an input element. Thus, with rotary push buttons in vehicles, navigated in a complex menu and a variety of functions can be selected context-dependent, a simple label of the element to explain its function is no longer sufficient. The same applies to

Keyboard shortcuts when operating a computer whose function can not be deduced from the lettering on the keyboard. Here the user must either the function memorize the input items or read a manual. Alternatively, it is possible to simply operate the respective input element or the key combination and the function of the trial and error principle according to the trial and error principle

To learn input elements. However, such an approach can lead to operator errors and is not accepted by all users.

In particular, when driving a vehicle, it is not desirable for the user to have to look down to identify the correct input element from the label. Here, the driver's eye and attention should, if possible, remain on the road. This is problematic because, depending on the position and complexity of the input elements, it is difficult for the driver to locate, correct, and predict the system response to his input without looking. In conventional user interfaces, the input elements are therefore adapted so that, for example, embracing is not required. However, this creates considerable restrictions on the design of the input elements. Otherwise, it has to be accepted that the driver has to look at the input elements a few times to make his input.

It is therefore the object to provide a method and a user interface, which make it easier for a user to operate input elements whose function is unknown to him or he does not want to view at the time of operation.

This object is achieved by the input preview method and the input-preview user interface according to the independent claims. Further developments of the invention are defined in the dependent claims. In the input preview method, a sensor detects an approach of a user's extremity to an input element or a touch of the extremity to the input element without actuating the input element. Furthermore, a data processing unit inputs via

Output device information about at least one effect, which may have an actuation of the input element.

The user interface with input preview has a sensor that can detect an approach of a user's extremity to an input element or a touch of the extremity with the input element, without the input element being actuated. Furthermore, the user interface to a data processing unit, which can output via an output device information about at least one effect, which can have an actuation of the input element.

The extremity may be a user's hand, foot or fingertip, for example. Either an approach or a touch, but also both can be detected.

The method of input preview or the user interface with input preview have numerous advantages. This gives the user a preview of possible interactions he might trigger. For example, if the preview does not reflect his intention to act, he can simply reposition his hand differently. Elaborate corrections for incorrect entries are not necessary as long as the input element has not actually been actuated. This increases the efficiency of operation. Input errors are reduced and the distraction of about a driver in traffic is significantly reduced, thereby improving traffic safety.

Furthermore, numerous applications are simplified or made possible for the first time, including a blind operation of Require input elements. Here, too, the risk of operating errors decreases while at the same time increasing efficiency. A self-descriptiveness, expectation conformity and user-friendliness of the user interface are increased.

The input preview allows a user to locate the desired input element without looking at it. Furthermore, the output information can save a lookup in a manual.

In one development, an attitude of the extremity is classified. Furthermore, information is output about at least one effect that an actuation of the input element can have with this attitude.

This development comes in complex input elements to fruition, which can be pressed, rotated or moved depending on the hand position or hand position. The training now identifies the respective posture of the extremity of the user and provides information about which interactions can be made with the current posture. Thus, with a rotary selector, the information is only output when classifying a pose that actually allows rotation of the rotary selector. In this way, it is ensured that the information is output only when the user prepares for the operation, and that the information can be issued specifically for the form of the operation which the user intends.

According to one embodiment, the output device is a screen. An item is displayed on the screen representing the input item. After detection, the color or shape of the item is changed on the screen.

This embodiment allows the user to quickly and easily access the desired input element locate it by displaying a corresponding selection. It can be changed either the color or the shape of the element on the screen, but also both. Furthermore, the position, size or arrangement of the element can be varied on the screen. The element may be, for example, an icon.

In a further embodiment, after the detection on the screen, a state appears, which would occur by pressing the input element.

This embodiment enables a particularly intuitive preview of the effect of an actuation of the input element.

In another embodiment, the information is output on a screen as a textual description. Similar to the "Tool Tips" for unknown icons on toolbars in Windows applications, the user thus learns in advance the function of the respective input element.

Additionally or alternatively, the information can also be output acoustically. This is particularly advantageous in traffic or for blind users, since the view of a screen can be omitted entirely. The acoustic

Output can be made by a non-voice signal, thereby affecting less the driver's conversation.

However, the acoustic output can also be a voice output. This makes it possible to give the user an accurate indication of the function of the input element.

In a further development, the information is output only after the approach or contact has been detected over a predetermined period of time. Similar to the "tool tips", the information is thus output only if the user is uncertain about the function of the respective input element.

In a further embodiment, the actuation of the input element may have different effects. The information describes the different effects.

This is particularly advantageous for complex input elements that provide a variety of functions. Furthermore, a spectrum, a scale or a list of selectable values can be displayed in this way, for example when touching a rotary selector.

In a further development, the input element is a key on a keyboard. Through the preview, information about its function can also be output here without pressing the key. This is particularly advantageous with function keys.

In a particular embodiment, multiple approaches or touches are detected simultaneously. The information here describes at least one effect that would be caused by pressing the corresponding key combination. In this way it becomes possible, even one

Provide input preview for keyboard shortcuts.

According to one embodiment, the sensor is a touch sensor, a camera or an infrared sensor. The touch sensor allows a particularly accurate tracking of the extremity of the user, especially in a variety of small input elements. The camera is advantageous because it can replace a variety of touch sensors. Furthermore, it allows the use of one of the most mature techniques in the field of camera tracking. The infrared sensor is particularly advantageous in dark environments. In a development, the sensor is formed by a system with multiple sensors. For example, a large number of small switches require a corresponding number of touch sensors. Alternatively, different technologies can be combined to increase the accuracy of tracking or choose the most cost-effective tracking technology.

In the following, the invention will be explained in more detail with reference to exemplary embodiments, which are shown schematically in the figures. The same reference numerals in the individual figures indicate the same elements. In detail shows:

Figure 1 is a user interface

Figure 2 shows a PC workstation

Figure 3 is a vehicle cockpit

Figure 4 is a mobile phone

Figure 5 is a multimedia device.

FIG. 1 shows an arrangement 100 which enables an input preview. The arrangement 100 is a user interface with which a user can make inputs. For this purpose, an input element 102 is available to the user, which he can operate with a limb 101. As input element 102, for example, switches, dials, jog dial, swashplate, joystick, foot switch, pedal, rotary selector, rotary push button, barrel key etc. come into question. The extremity 101 of the user may be about a hand, a fingertip or a foot. In this case, by a sensor 103, an approach or touch the

Extremity 101 to the input element 102 detected. The sensor 103 may be a camera, a touch sensor, a temperature sensor, or the like act. It is also possible to use a sensor system comprising a plurality of sensors. The sensor 103 detects the approach or touch without actually actuating the input member 102. A later operation is not excluded. After the detection, there is one

Data processing unit 104 information about at least one effect, which may have an actuation of the input element 102. The output here takes place via an output device 105. The data processing unit 104 may be, for example, a personal computer, a processor in a mobile terminal (video cell phone, PDA, notebook,...) Or a multimedia device or an embedded system, for example in a vehicle or in a dashboard , As output device 105 can both visual output devices such as screens, projectors or LCD displays and acoustic

Output devices such as single speakers or surround systems are used.

The information indicates what the actuation of the input element 102 would do. The information can be as

Text, as a symbol or as a graphic to be displayed visually.

Acoustically, the information can be output as a speech signal or as a non-speech signal. Sorry the

For example, if the user's fingertip is on a pushbutton, which may increase the volume of music playback, e.g. temporarily increases the volume of music playback.

In this case, the information is thus output acoustically, but non-linguistically.

The information may also indicate how to operate the input element 102, such as that the input element 102 may be rotated to achieve a particular effect.

FIG. 2 shows a PC workstation 200, which enables an input preview. Here, a user with a hand 201 and a foot 211 operates a keyboard 210 and a foot switch 212. Further, a camera 203 detects an approach or touch of a fingertip of the finger User to a key 202 without the user pressing this key 202. Subsequently, information is output via a personal computer 204, which would result in the pressure of the key 202 in the current context. The information output here may depend on the current menu or state of an application on the personal computer 204. The output can be made either via a screen 205 or via speaker 206. Also with respect to the foot switch 212, corresponding information may be output depending on its current function. In this as well as in all other embodiments, it may be useful to make the output of the information after a predetermined period of time. For example, if the user lingers on button 202 with his fingertip for more than 1.5 seconds, it is considered that he needs help, and then the information is output.

If the actuation of an input element leads to a state change of the respective application, it can be superimposed on the screen 205 as a preview in the respective state. If, for example, pressing the key 202 causes the change to a submenu, then an approaching or touching of the key by the user can cause the respective submenu to be temporarily displayed. When the user actually presses the button 202, the respective application actually changes to the state shown above. In this and all other embodiments, a combination of the aforementioned types of output is possible.

If the user touches several keys at the same time without pressing them, a preview of information about the effect which the respective key combination would currently effect in the current application can be output. For example, in a word processor, touching several keys may temporarily cause formatting - italics, underlining, etc. The changed format will only be displayed temporarily, as long as the corresponding format is displayed Keyboard shortcut is touched, and only accepted when the user actually presses the keys.

FIG. 3 shows a cockpit 300. The cockpit 300 may be a pilot's cockpit in an airplane, a driver's cockpit in a vehicle or else a cockpit in a ship, submarine or simulator. The cockpit may also be a medical surgical system operator console or any other user interface. The cockpit is characterized by a variety of controls, sometimes with high complexity. For a user or driver, a blind operation of all these controls or memorizing their function is difficult. A blind operation is desirable because the user is about in the

Road traffic would like to observe the environment through a Windschutzschreibe 320, without directing the view of the controls. In this context, usually the controls are separated from a screen 315 so that it can be placed as close to the windshield 320 as possible. In order to avoid encompassing the user, several functions are often made accessible via a multifunction input element 301 here. Since such a multi-function input element 301 can be operated in various ways, such as by pressing, rotating or shifting, it makes sense to output information on all the effects that the different types of operation may have. Alternatively, an attitude of about a hand 201 of the user may be detected, which approaches or touches the multi-function input member 301. Since certain types of operation, such as rotation, are possible only with a certain attitude, the information can be specifically outputted for the effects that can be achieved by operating the multi-function input member 301 with the current posture. The same applies, for example, to a rotary control 304. Since this can only be rotated with a certain hand position, the information is output only when the corresponding posture is detected. In this way, superfluous expenses are avoided during fleeting movements of the user.

The cockpit 300 has a plurality of input elements 302. These can for example be designed as push buttons. If the user is looking for a particular input element 302 but does not want to look away from the screen 315, textual or graphical elements may be displayed on the latter. If an approach or contact of the user to one of the input elements 302 is detected, the shape or color of the associated element on the screen 315 is changed, for example.

A touch of other controls, such as a toggle switch 303, a pedal 305 or a steering wheel 310 may lead to an output corresponding information. Alternatively or in addition to a visual output via the screen 315, an acoustic output via loudspeaker 316 can also take place. Detection of the approach or contact is effected by a sensor system 313. For the sensor system 313, contact sensors, cameras, infrared sensors, temperature sensors, laser monitoring systems can be used alone or in combination. The detection is based here, for example, in a video surveillance system on conventional tracking methods. As a reference point for the tracking, for example, the fingertip of the user can be determined. Depending on the configuration of the sensor system 313, an approach or a touch can be detected. Thus, on the basis of a camera tracking approach can be detected as soon as the distance of the hand 201 to the respective input element falls below a predetermined threshold. The information can be output as text or as language: "By By activating this input element you can increase the room temperature. "

Figure 4 shows a mobile phone 400. This has keys 402 which can be pressed by a user with a fingertip 401. Furthermore, the mobile phone 400 includes a screen 405 and a speaker 406. In addition to the buttons 402, a swash plate 412 is available for input. In this embodiment, a touch of the fingertip 401 with one of the keys 402 or the swashplate 412 is preferably registered by touch sensors. For example, in the case of the swash plate 412, it can additionally be detected which side of the swash plate 412 is touched. Subsequently, information is output via the screen 405 or the speaker 406, which would cause the depression of the respective button 402 or the respective side of the swash plate 412.

FIG. 5 shows a multimedia device 500. This has a display 505. Furthermore, it contains loudspeaker 506 and input elements 502. Here, too, hints on the function of the input elements 502 can be output visually or acoustically already before their actuation via the display 505 or the loudspeakers 506. For example, if the multimedia device 500 includes a knob that controls the tuner of a built-in radio receiver, touching or approaching the dial on the display 505 may display a scale or list of possible frequencies or stations. Here, the input preview is not the blind operation, but a simplified operation.

Other conceivable embodiments may extend to any household appliances such as washing machines.

Also in the context of medical technology, user interfaces are conceivable which enable an input preview in accordance with the exemplary embodiments mentioned. This may allow a surgeon to perform an operation via a user interface, for example actual intervention is guided by robotic arms. Since the surgeon can not proceed according to the trial-end error principle here, he needs information on which functions the individual input elements are assigned. The information may also include medical knowledge and physiological data of the patient.

The embodiments may be implemented as hardware, such as electronic circuitry, or software. All mentioned embodiments can be used in any combination.

Claims

claims

1. Method for input preview, in which

a sensor (103) an approach of a limb (101) of a user to an input element (102) or a

Touching the extremity (101) with the input element (102) detected without the input element (102) being actuated,

- A data processing unit (104) via an output device (105) outputs information about at least one effect that an actuation of the input element (102) may have.

2. The method of claim 1, wherein a posture of the extremity (102) is classified and in which a

Information about at least one effect is output, which can have an actuation of the input element (102) with this attitude.

3. The method according to claim 1,

in which the output device (105) is a screen (205),

in which an element representing the input element (102) is displayed on the screen (205),

- Wherein after the detection color and / or shape of the element on the screen (205) are changed.

4. The method according to claim 1,

in which the output device (105) is a screen (205),

- In which after the detection on the screen (205), a state is displayed, which by pressing the

Input element (102) would occur.

5. The method according to claim 1,

- In which the output device (105) is a screen (205), - in which the information on the screen (205) is output as a textual description.

6. The method of claim 1, wherein the information via the output device (105) is output acoustically.

The method of claim 6, wherein the acoustic output is a linguistic output.

8. The method of claim 1, wherein the information is output only after the approach or touch has been detected over a predetermined period of time.

9. The method of claim 1, wherein the actuation of the input element (102) may have different effects and wherein the information describes the different effects.

The method of claim 1, wherein the input element (102) is a key (202) on a keyboard (210).

11. The method of claim 10, wherein a plurality of approaches or touches are detected simultaneously,

- where the information describes at least one effect that would be caused by pressing the appropriate key combination.

12. User interface with input preview,

- With a sensor (103), the approach of a limb (101) of a user to an input element

(102) or can detect a contact of the limb (101) with the input element (102), without the input element (102) being actuated,

- With a data processing unit (104), which can output via an output device (105) information about at least one effect that an actuation of the input element (102) may have.

13. User interface according to claim 12, wherein the sensor (103) a touch sensor, a camera (203) or a Infrared sensor is.

The user interface of claim 12, wherein the sensor (103) is formed by a multi-sensor system.

15. User interface according to claim 12, wherein the output device (105) is a screen (205) and / or a loudspeaker (206).