WO1993015476A1

WO1993015476A1 - Transducer arrangement for controlling machine(s) by perception of distance, especially for making choices from the computer's menu

Info

Publication number: WO1993015476A1
Application number: PCT/HU1992/000003
Authority: WO
Inventors: Dezso^' FODOR
Original assignee: Mie Modern Information Technology And Electronics Development, Services And Sales Co. Ltd.
Priority date: 1992-01-27
Filing date: 1992-01-27
Publication date: 1993-08-05
Also published as: AU1195392A

Abstract

The subject of the invention is a transducer arrangement for controlling machine(s) by perception of distance, especially for making choices from the computer's menu, in which transducer arrangement the output of displacement sensors being displaceable closer or farther from each other - as the ultrasonic sender (5) and the ultrasonic receiver (3) in the realization example shown - is connected to a signal processing unit. The essence of the invention is that the sensors can be fixed to the thumb and the fingers, favourably to the tips of those of the operator who controls the machine(s). The realization example shows that the sensors can be fixed onto the tip of the thumb (1) as well as onto the tip of at least one out of the fingers - forefinger (4), middle finger (7), ring finger (8) and little finger (9) -, preferably onto the tip of the forefinger (4) and they can be controlled by shortening or lengthening the distance between them.

Description

Transducer arrangement for controlling machine (s)

by perception of distance,

especially for making choices from the computer's menu.

The subject of the invention is a transducer arrangement for controlling machine((s) by perception of distance, especially for making choices from the computer's menu, where the output of the sensors, being displaceable closer or further from each other, is connected to a signal processing unit.

It's well known, that different solutions of controlling a machine by man are compareable to each other in different aspects. Such aspects are, among others that

- by which of their organs people make the machine work

- whether the directly controlled part of that produces and transmits control signals to other parts of the machine on mechanical, electrical or electronic principle,

- in which field of the human activity is the machine applied, (design, manufacturing, transport, service, etc.),

- how much work (force by displacement) is required to make it work for 1 intance,

- how complicated is the mental process in preceding and following 1 instance of activation,

- how speedy the process, consisting of a series of activations can be,

- at last, which are the strain characteristics of the activation process: energy consumption, psychic load.

According to the the above aspects a device can be activated mechanically, being operated by hand or foot. There is evolving the control, by speech. The directly controlled part of the machine can be mechanic handle, lever, lock, steering wheel, switch-stick, hand-wheel, steering-rod, etc.; electric contact, switch, push-button, potentiometer, sliding contact, seisin, etc.; electronic push-button, keyboard, capacity-sensor, optical sensor, microphone or adequately configurated sets of any of those. The mechanic work needed to activate them can be between the tiresome force and the application of the zero-force touch-sensor. The complexity of the mental process pertaining to one instance of actiavation can be of any degree from the simplest, which requires only the perception of one binary signal an the output of a control signal of the same type (this is e.g. the case of a prescribed action when noting an optic signal: the driver brakes when the tail light in front of him is on), to the most complicated mental processes, as the perception of many analog signals, decision depending on the complicated dependencies between them, and as a result to produce a selected value of a variable control signal - e.g. the value of a key in a full keyboard - as it is the case when typist read the text or dictated to. The possible rapidity of the process consisting of the sequence of activation instances depends on the device and the person applying it: on the device to the extent, that how rapid it makes it possible to percept the result of one activation instance in the sequence (the possible rapidity of signalling back through sensing) to decide the next activation (the possible rapidity of processing a stimulus and of the decision - that can be increased significantly by exercising), and to pass the activity (that is the time between the start of the activation and the start of the control signal resulting from it). At last, the strain characteristics of the activation processes are of the broadest variety: they can be compared to each other on the base of the two qualities, that how long can the machine be operated uninteruptedly by the same person without catastrophic decrease in quality, and that how intensive can one perform another activity during the work. From this point of view, ideal strain characteristics are shown perhaps by the controlling system of the automobile: it can be operated long time without getting tired in parallel with rather significantly more complicated other activities. In contrast, there is a big strain caused by handling the computer, despite the law physical work, if the program communicates intensively with the operator. The greater strain caused by the computer can be accepted undoubtedly by comparing it to the control of other machines - e.g. that of the motor car - based on the two preceding criteria: when operating an intensively communicating program, it's almost impossible to conversate or to give attention to the surroundings. That can be a reason for the popularity of the computer games: the great strain gives opportunity to the self-evaluation in playing form for the operator.

The effort to reduce the strain which is caused by working with a computer led to the recognition of the importance to find some method of reducing the - primarily mental - strain. The two most important from the point of view of the invention:

- The number of operating keys of a program which is intensively communicating with the operator has been restricted to a few keys rather than the whole keyboard by programming. There have been made, even, special devices to choose from some producible control signals (arrow keys, joystick).

- There is always shown the operator the whole menu of the reasonable control actions at the given point of time and state, or in case of a small number of control actions they are known ahead. This menu is always changing, just as the consequence of the performed operations. In the case of a large number of allowed operations, they are grouped into logically coherent groups. Each selection separates first a group, subgroup of operations, while achieving the selected aim perhaps after several steps only.

The application of the two above methods is called menu-selection, and the device for that is the menu-selector. There is an old trend to apply the two methods - operating keys, control signals have to be only few, and the full menu of the operation has to be shown to the operator or known in advance. Consequently, the menu-selector is just the application of traditional principles to handle computers. To realise these principles in practice is rather dificult for any machine. Mostly the acceptability of the construction of the machine depends on it. Never is it fully developed when epoche-making new machines are emerging, as today in the computer technology. The menu selector devices of the computer are now in an era of being renewed again and again. Common characteristic of the broadly known hand-operated menu selectors is, that the hand has always to manipulate on an object situated independently from the body, of the operator: on keyboard, joystick, mouse, touch-screen, etc.

Consequently, the situation of the operator's body is depending on the situation of the device, which he or she has to manipulate. We say, therefore, that those menu-selectors are not "situation-free". However, the demand and striving exists to improve this, as shown by the success of some new products: cordless link between the keyboard and the controlled machine; the smaller and smaller costruction of computers; which are first of all portable; and are or can be equipped with menu selectors, such as joystick, trackball, mouse-emulating arrow-button; "infra-mouse", which also has cordless link to the computer; and the general strive to build any device intended for personal use in a portable, "laptop", "pocket" version as it is shown just by the development of the computer or calculator. The example of achieving perfect freedom from the situation of the body when using a computer-controlling device is found in the device developed by the Californian VPL Research Inc., called "data-glove", described e.g. by James D. Foley in the "Scientific American" in 1987. This man-machine communication device intends to map the human hand's precision, sensitivity and mobility in full extent for the computer. There are sewn a number of different transducers into the glove, so that the computer can reconstruct the hand's mechanical model and movements through a multi-wire connection from the measured data in its memory. That device selects the control operation as intended by the operator with the recognition algorithms of the computer's program-system from a rather large mass of measured data.

The "data-glove" is of pioneering significance from another point of view also: it achieves freedom from the situation of the whole body. It is enough for the operator to use only one hand to work. Other position-free devices: e.g., the pocket-calculator, TV-set infra-red telecontrol ler are situation-free only to the extent that both hands of the operator are occupied. The designers of the data-glove had got the progressive idea to achieve freedom from situation of the body by making the control action with a sensor which is directly connected to the hand's movement. It's mounted on to the hand in such a way, that it transmits the data of change coused by the hand's movements to the machine, e.g. to a robot or computer. So, it's possible even to plan controlling movements along all the three dimensions of the space.

The data glove has the disadvantage of utilising those geometric features of the hand, which are possible to sense and to map only in an extremly complicated way, therefore it's application is uneconomical to transmit simpler control actions.

The most general disadvatage of the known menu-selector devices is, that they do not provide in parallel both freedom from the situation of the body and useability with one hand only. An exception is the data-glove with the disadvantage, that it "overshoots" the aim. The device itself hasn't the above characteristics of menu-selecting: choosing between a small number of possible operations and being the menu known in advance or displayed.

Although it can be used also as a menu-selector if applied as part of a system, but to transmit the extremly large quantity of measured data from the hand and to process them with a recognizing algorithm for the menu-selection, is a rather sub-optimal technical solution. In the known applications of the data-glove arn't defined the criteria of the one-handed situation-free menu-selector, as we call this invention, as an aim or possibility of usage.

It is quite justified, therefore, to make a device for controlling the machine in the man-machine communication which intends to achieve fully the above principles, i.e.:

- it serves for selection from a small number of possible operations (that is from a restricted, comprehensible number of possibilities),

- the menu is known in advance or can be displayed,

- the situation of the operator's body is independent from the situation of any other object(s), needed to the control operation.

- the usage of the device occupies one hand only

The correct and for menu selection possibly optimal fulfillment of these criteria didn't exist until now, as it is proven by the above said circumstaces at the present state of the technical progress.

The invention is based on the realization that the variation of a very simple feature of the hand, the distance of the fingertips from each other, is sufficient for control actions. The action is done along one single dimension of space, that it is economical to use in some simpler control operations, as e.g., the menu-selection, or other simple movements of the hand. That distance should be onlyconverted into electrical signal by distance transducers.

The principle of the measurement is unimportant from the inventions point of view. Many things are worth mentioning from the simplest traditional means to the application of the most modern products of the electronic component industry. Also there are such principles which deserve attention, although there are not of any value of an invention. To illustrate this, we list some known principles of the distance transducers:

- potentiometer with sliding contact

- capacitive distance-transducer

- optical distance-transducer

- inductive distance-transducer

- magnetic (magneto-resistive) distance-transducer

The essence of the invention is that the sensors can be fixed to the thumb and the fingers, favourably to the tips of those of the operator who controls the machine (s), and the tranducers can be controlled by causing the distance between them to get shorter or longer.

We discuss the invention in more details using the realization example shown by the figures:

- Figure 1 illustrates a realization example of the transducer arrangement, as in the invention, when using ultrasonic sensors fixed on to fingertips. - Figure 2 shows the structure of an ultrasonic sensor of the same example in cross-section.

- Figure 3 shows the connection diagram of the electronic signal-processing unit

- Figure 4 shows the output signals of the comparator and the contents of the counter in the signal processing unit of Figure 2.

As shown on Figure 1 the sensor, which is the ultrasonic receiver 3 in the example, is fixed to thimble 2 which can be pulled onto the thumb 1; as well as the other sensor, which is the ultrasonic sender 5 in the example, is fixed to that thimble 2 which can be pulled on to the forefinger 4. The sensed data will be sent on the leads 6 from the ultrasonic receiver 3 and the ultrasonic sender 5 to the electronic signal-processing unit illustrated on Figure 3 but not being shown on Figure 1. It's possible to fix a thimble 2 onto all other fingers as needed, i.e. onto the middle finger 7, the ring finger 8 and the little finger 9, the ultasonic sender 5 being possible to fix to that thimble 2. It's convenient to sense the distance between the thumb 1 and some of the other fingers, because it's so the easiest to change the distance. The suitable position of the sensors on the fingers is the tip of the finger or the touching side of the finger just at the last joint, so that they can "see" each other, but they mustn't cover the touching pad of the fingers and so hinder the operator in making use of its hand. The sensors fixed on the different fingers can serve for different functions too. E.g., the sensors fixed on the thumb 1 and the middle finger 7 can serve as an on/off switch of some function of the machine(s). If we fix a sensor onto the little finger as well, the change of the distance between the thumb 1 and the little finger 9 can be used as an on/off switch.

Figure 2 shows the structural solution of the thimble (the same which is shown on Figure 1) pulled onto the forefinger 4 and the ultrasonic sender 5 fixed on it. This sulution is the same for any of the thimbles pulled onto the thumb 1, forefinger 4, middle finger 7, ring finger 8 and little finger 9. The difference between them is only that the inner diameter has to be fitted in any case to the diameter of the respective finger.

To fix the ultrasonic receiver 3 to the thimble 2 may require the same structural solution, because the ultrasonic sender 5 and the ultrasonic receiver 3 may happen to be manufactured in a case of the same shape. Those are cylindric metal cases, with two connecting pins on the insulator bottom of the cylinder and a surface on the top serving for capturing or radiation of the sound respectively.

In the construction of the given realization example, the thimble is of some unusual shape (it can be manufactured by, e.g., pressure casting) with a cylindric boring, which can hold the ultrasonic sender 5 or the ultrasonic receiver 3, at an angle of about 80 degrees to the thumble's longitudinal axis. The ultrasonic sender 5 or the ultrasonic receiver 3 is soldered to the disk 21 by its connecting pins. The disk 21 can be made of printed circuit material. Its diameter is somewhat larger than that of the ultrasonic sender 5 and ultrasonic receiver 3, consequently it leans against the step made in the boring where the ultrasonic sender 5 and ultrasonic receiver 3 are to be fixed. It's held in this supported position by the spring 24. Figure 3 shows the electronic signal processing unit in which the integrated circuit oscillator's output is connected firstly through a signal-forming resistor and choking coil to the piezoceramic sender 11 - that is identical with the ultrasonic sender 5 - , secondly to one of the inputs of a phase comparator, i.e. to the input driving the integrated circuit monostable multivibrator of the comparator.

The phase comparator consists, as the connection diagram shows on the figure, of the integrated circuit monostable multivibrators 25,26 and 28, the integrated circuit NAND gate 27, the integrated circuit D-flip-flop 12, the integrated circuit Schmitt-trigger 15 and delaying RC circuits as well.

The condenser microphone 13, being identical with the ultrasonic receiver 3, is connected to the other input of the phase comparator through amplifier transistors and the integrated circuit Schmitt trigger 14. The phase comparator has two outputs. One of them is connected to the enabling input of the up-down counter 16 through the delay circuit consisting of the integrated circuit Schmitt trigger 15 and the RC circuit in front of him, while the other is connected firstly to the up-down control input of the up-down counter 16 and secondly to the input of a differentiating circuit.

The differentiating circuit consists, as the connection diagram shows on the figure, of the integrated circuit Schmitt trigger 17, the integrated circuit inverters 29 and 30, limiting Zener diodes, RC circuits making negative impuls from the change of the logic signal from high to low, the integrated circuit NAND gate 27 which derives the OR function of those impulses and of the impulse resetting the up-down counter 16.

The output of the differentiating circuit, being identical with the output of the integrated circuit Schmitt trigger 17, is connected to the impulse counting input of the up-down counter 16, while the outputs of the up-down counter 16 are the outputs of the electronic signal processing unit. The outputs of the signal processing unit may be connected to a transmitter such as working without wire connection (e.g. wireless or infrared connection) as well. The resetting inputs of the up-down counter 16 are linked with the resetting contacts 18 and 19 positioned beside the ultrasonic receiver 3 and the ultrasonic sender 5. The measurement display unit 20 is connected to the outputs of of the up-down counter 16.

The above discussed example of the transducer arrangement can be operated as follows:

The oscillator 10 provides an ultrasonic frequency signal of about 40 Khz for the piezoceramic sender 11. Figure 1 shows the space between the thumb 1 and the forefinger 4, being a region of a resolution of about 8 mm. Namely, the 40 kHz frequency corresponds with a sound-wavelength of about 8 mm. The condenser microphone 13 converts the received soundwaves into a sinusoidal electronic signal of the same frequency but of a delayed phase in contrast to that signal driving the sender. The phase comparator is essentially an electronic circuit sensing the identity of the phase of two different sine waves. Figure 4 illustrates its output signals as the function of the distance between the piezoceramic sender 11 and the condenser microphone 13.

The phase delay, being identical with the time delay between the input signals of the phase comparator, results from the time being required to the propagation of the sound through the air from the ultrasonic sender 5 to the ultrasonic receiver 3, which is proportional to the length of the way done in the air and is overwhelming in contrast to the delay caused by other parts of the arrangement. The delay, the time lag makes a phase difference which changes by a whole period when the distance changes by the wavelength, i.e. by 8 mm. The phase comparator recognizing that the phase of the two signals are identical means that the phase difference between them is zero or the multiple of the cicle time. This means at the same time that the distance between the ultrasonic sender 5 and the ultrasonic receiver 3 is the multiple of the wavelength of the ultrasound, which is in our case 8 mm.

The phase comparator signals with one of its outputs, being the output of the integrated circuit monostable multivibrator 28, that the phase difference between the two signals is within certain small + and - limits, while the other output, being the output of the integrated circuit D flip-flop 12. means that if it is, which signal is ahead (is the pase difference + or -). This later output retains its value also when the phase difference isn't within the limits, i.e. the value of the mesured distance is inside an 8 mm long sector and not in any boundary region of the sectors. Consequently, this later signal shows whether the value of the distance entered the 8 mm long sector by increasing or decreasing. Using an expression customary in electronics the signal showing the phase difference between the input signals of the phase comparator changes its value by a "hysteresis character" when the value of the measured distance alters by increasing and decreasing between the opposite boundary regions of an 8 mm long sector.

The output signals of the phase comparator shown by Figure 4 are suitable for driving and controlling the up-down counter 16 in such a way that its content shows always the number referring to that 8 mm long sector of distance which corresponds to the measured distance.

The signal from the phase comparator representing the phase identity, being the output of the integrated circuit monostable multivibrator 28, controls the enabling input of the up-down counter 16 in such a way that when the compared inputs become of equal phase, it enables triggering the up-down counter 16 after a certain delay produced by a delay circuit consisting of the integrated circuit Schmitt trigger 15 and the RC circuit in front of him. That delay is necessary because the counter triggering signal, the value of which shows whether the received signal is delayed or the other, inverts its value at varying distance when the phase just becomes equal within the given limits. This change, however, mustn't be allowed to clock the counter. This same signal inverts its value also later, when the phase continues to alter within the given limits and the situation that the received signal is delayed or the other is inverted about in the middle of the field of tolerance. At that time the output of the integrated circuit monostable multivibrator 28 enables already to trigger the up-down counter 16 because the delay which began at the point of time the phase was just becoming equal within the given limits ended long ago. The triggering takes place in such a way that the signal, the value of which shows whether the received signal is delayed or the other, being the output of the integrated circuit D flip-flop 12, prepares the up-down control input of the up-down counter 16 for the respective count direction. From the inversion of this same signal, the differentiating circuit makes the impuls which triggers the up-down counter 16.

By setting the distance between the thumb and a finger we can input 5 - 6 different values to a computer in a fully free situation of the operator's body. The input devices being used till now (e.g. keyboard, joystick, mouse) doesn't allow the situation of the body to be free. To make suitable this device for the broadest variety of the input processes is only a matter of programming. So, as a first step, the operation of either a joystick or a keyboard or even a mouse can be simulated (eventually with a certain limit in speed) by an approprate program. I think, a bigger adventage can be achieved, however, by the development of menu selecting programsystems based especially on this device.

An experimental transducer arrangement displays on a LED display the distance between the thumb and a finger. The display and its driver circuit, as well as the circuits connecting the sensors to him, are the stable parts of the equipment. Two leads, being shielded separately, and a third one connects the sensors being on the thumb and a finger to them. The leads can be moved freely and are 1 - 2 m long. The non-shielded lead beeing grounded resets the display to zero in the nearest position of the thumb and finger to each other. Removing them remoter from each other increments the display by one per 8 mm, while moving them backwards closer to each other decrements that.

There would not be difficult to develop a cordless data connection between the equipment and the computer. That makes the body much more free. Another possibility for the development to fix sensors onto more than one finger. That enables to input more values than above.

The thimbles holding the sensors pulled onto the fingertips don't particularly hinder the user of the device in other action. Figure 2 illustrates the fact, that an adequately constructed thimble doesn't hinder the operator to work even with the traditional keyboard: The thimble 2 pulled onto the forefinger 4 allows to push the key 22 of the keyboard in such a way that the key 23 and the other ones remain untouched.

Of course, the foreseeable adventages of the invention will, moreover, make it so no need for the use of a keyboard.

The expirements made with the experimental transducer arrangement show that the operator of the device can find the wanted function definitely and quickly. The expected adventages can be observed. The situation of the body, freer from the machine, enables to use large-screen monitor easily at the computer which, as per the ergonomics, is more favourable. You can feel that applying the device in an appropriate system makes easier to operate the computer also when reading a lecture. There is no bound in manipulating the machine, in contrast to the necessity to be well accustomed to the key-positions when using a keyboard. There is no motivation to look during the action at the operating device - the hand - , because the feedback of the result of the action takes place by percepting the funcion caused by that on the screen. This is a great advantage in contrast to the keyboard, where the position of each key gets conditioned only after a long training, or even so not, to the extent that one hasn't to look at the keyboard.

In contrast to the use of the mouse, another advantage is that the menu selecting action isn't controlled by the movement of the hand and arm on a table but by the more precise monipulation of the fingers.

An aspect of interest can be in the future, verifyed only by longer experience, to what extent the speed of this input device is less than the speed of a well trained typing. It seems, however, also to be an adventage if the speed of an untrained, every-times-looking-at-the-keyboard, typing process can be reached.

To sum up, the invention discussed here, as the means of controlling the "MACHINE" by man, regarding the former given aspects has the benefits as follows:

- it is operated by hand,

- it can produce the electronic control signal on any principle and that signal can be transmitted to other parts of the machine with the many known and used methods of the remote control,

- it is suitable for control of machines used in any region of man's working activity, where digital electonics can be used. - the work required to activate it once is very little,

- the mental process preceding and following each activation is simple,

- the sequence of activations can be fast,

- the mental and physical strain on the operator is low.

The invention makes it more economical the mental and physical energy consumption of the human activity. We mean here by the human activity the intentional intervention with defined purpose in the natural or artificial surroundings. That is, in essence, the human work.

One of the important domains of economizing on the human energy spend on labour is the interaction between the man and the means of his activity, the machine. The invention can be used in this sense by applying it in that field of the ergonomics dealing with the man-machine interaction: it is a new means to control the generally taken "MACHINE" by man. This is the most general definition of the sphere of the application.

The claim isn't limited to the sulutions descripted in the realization examples, but it extends to all solutions which are protected by the claims, especially by the main claim. Legend

1 thumb

2 thimble

3 ultrasonic receiver

4 forefinger

5 ultrasonic sender

6 lead

7 middle finger

8 ring finger

9 little finger

10 oscillator

11 piesoceramic sender

12 integrated circuit D flip-flop

13 condenser microphone

14, 15, 17 integrated circuit Schmitt trigger

16 up-down counter

18, 19 resetting contacts

20 measurement display unit

21 disk

22, 23 key

24 spring

25, 26, 28 integrated circuit monostable multivibrator 27, 31 integrated circuit NAND gate

29, 30 integrated circuit inverter

Claims

CLA IMS :

1. Transducer arrangement for controlling machine(s) by perception of distance, especially for making choices from the computer's menu, in which transducer arrangement the output signals of distance sensors being displaceable closer or farther from each other are connected to a signal processing unit,

characterised in that

the sensors are fixed onto the thumb and finger(s), preferably onto the tips of them, of the operator of the machine(s) and that the sensors are controllable by altering the distance from each other.

2. The transducer arrangement as in claim 1,

further characterised in that

the sensors are fixed onto the tips of the thumb and finger(s) with holding elements, preferably thimbles (2).

3. The transducer arrangement as in claim 1 or 2,

further characterised in that

a signal processing unit, producing output signals of a resolution between 3 and 20 mm, is connected to the outputs of the sensors.

4. The transducer arrangement as in any of the claims from 1 to 3,

further characterised in that

the sensors are fixed onto the thumb (1) and one of the fingers, preferably onto the forefinger (4).

5. The transducer arrangement as in any of the claims from 1 to 3,

further characterised in that

the sensors are fixed onto the thumb 1 and two of the fingers, preferably onto the forefinger (4) and the middle finger (7).

6. The transducer arrangement as in claim 5,

further characterised in that

sensors are fixed onto the ring finger (8) or the little finger (9) as well.

7. The transducer arrangement as in any of the claims from 1 to 6,

further characterised in that

the signal processing unit is electronic logic signal processing unit, the outputs of which are connected to a transmitter working on any principle of transmission without wire connection.