CA2263151C - Picture-transmitting remote object inspection system - Google Patents
Picture-transmitting remote object inspection system Download PDFInfo
- Publication number
- CA2263151C CA2263151C CA002263151A CA2263151A CA2263151C CA 2263151 C CA2263151 C CA 2263151C CA 002263151 A CA002263151 A CA 002263151A CA 2263151 A CA2263151 A CA 2263151A CA 2263151 C CA2263151 C CA 2263151C
- Authority
- CA
- Canada
- Prior art keywords
- image
- observation station
- recording system
- image data
- image recording
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
Abstract
An image transmitting remote object examination device includes (a) an image recording system at the location of the object to be examined, (b) an observation station located at a distance from the image-recording system for the evaluation of the image and for the remote control of the image recording system, and (c) an intervening, narrow band telecommunications link, by way of which the images taken are transferred to the observation station using image data compression and/or reduction together with the remote control commands to the image system. A system status-determining unit continuously records the overall system status relevant to image transmission, and dependent upon the overall system status, variably controls the image data compression and/or reduction and/or the number of individual telecommunication channels used in a transmission packet. The image quality required for a particular situation, together with the optimal on-line image impression, is thereby obtained.
Description
S U M M A R Y
1. Image-transmitting remote object examination device 2.1. The invention relates to a remote object examination de-vice with an image-recording system provided at the loca-tion of the object to be examined (43), an observation station spaced apart from it for evaluating the image and for the remote control of the image-recording system as well as an intervening narrow-band telecommunications link (7) .
2.2. A device (42) for determining the system status is pro-vided in accordance with the invention, which continuously records the overall system status relevant to image trans-fer and as a function thereof automatically controls the image data compression and/or reduction and/or the number of telecommunication-channels used in a transmission packet so that both the desired image quality for a given situation and the best possible on-line-image impression are achieved in each case.
2.3. Application for example to telemicroscopy, teleendoscopy and telesonography.
1. Image-transmitting remote object examination device 2.1. The invention relates to a remote object examination de-vice with an image-recording system provided at the loca-tion of the object to be examined (43), an observation station spaced apart from it for evaluating the image and for the remote control of the image-recording system as well as an intervening narrow-band telecommunications link (7) .
2.2. A device (42) for determining the system status is pro-vided in accordance with the invention, which continuously records the overall system status relevant to image trans-fer and as a function thereof automatically controls the image data compression and/or reduction and/or the number of telecommunication-channels used in a transmission packet so that both the desired image quality for a given situation and the best possible on-line-image impression are achieved in each case.
2.3. Application for example to telemicroscopy, teleendoscopy and telesonography.
3. Figure 1.
' CA 02263151 2004-03-22 Image-transmitting remote object examination device Field of the Invention The invention refers to a remote object examination device pro-vided with an image-recording system at the location of the ob-ject to be examined,,a remote observation station for evaluating the image and for remote control of the image-recording system, as well as with an intervening nar-row-band telecommunications link, over which the image-data generated by the image-recording system is transferred to the ob-servation -station using image data compression and/or reduc-tion, as well as ' control command signals generated from the observation station for y remote controlling the image-recording system.
Background of the Invention Such remote object examination devices are known in form of telemicroscopy systems for example, particularly as applied to telepathology. The object to be examined in this case i.s a tis-sue section prepared, for example, during an operation. The im-age-recording system comprises a microscope remote-controllable in all functions, an image-recording device for the microscope field, a transmitter for the image signal a.nd a receiver for the control commands at the microscope. Transmitter and re-ceiver as well as an additional speech channel are connected over an interface with a telecommunications link. The observa-tion-station comprises a receiver, a monitor for the images to be returned, an appliance for the production of the control P 11696. PCT
commands for the microscope and a transmitter for these control commands. The observer or expert, for example, a suitable com-puter or a pathologist, can appraise the transmitted microscope image. Transmitter and receiver as well as a speech channel are in turn connected by way of a suitable interface with the same telecommunications link as the image-recording system.
If the telecommunications link has a sufficient width for the transfer of a television signal, the observer at the observa-tion station can use all functions of the spatially removed mi-croscope just as if this were at his workplace. Such a telepathology system with wide band data transfer is described in U.S. Patent 5,216,596. However, wide band telecommunication channels are comparatively costly for television signals and of only limited availability. It has already been proposed there-fore, to operate telemicroscopy over the widely available and inexpensive, narrow-band channels of the telephone network. An installation with such narrow band technology employing the ISDN-Network is described by M. Oberholzer et al. in the paper "Telepathology: frozen section diagnosis at a distance" (1995), 426, page 3. However, the limited channel capacity forms a bot-tleneck for the transfer of the image data so that the on-line-impression for the user can be lost and instead of a moving im-age he will only receive a slow consecutive string of individ-ual images. The result will be that the user will also lose the immediate impression of the effect of his microscope control commands, since for him their effect will be seen only after a substantial delay and in an altered context. The application of the technologies of video-telephony only allows the transmis-sion with a quality of image which is inadequate for most of the applications considered here, such as telemicroscopy, teleendoscopy and telesonography.
Some measures have also already been proposed for the economic application of ISDN channels for narrow band telemicroscopy systems, such as, for example, the use of a particular auto-P 11696!PC'I' " CA 02263151 2004-03-22 focus system and the application of a fixed, pre-determined, compression and reduction adapted to the particular system of the image data to be transferred, during their encoding, (see the papers by P. Schwarzmann, Telemikroskopie, Zentralbl.
Pathol, 138, (1992) 6, page 383 and P. Schwarzmann et al., Telemicroscopy Stations for Telepathology Based on Broadband and ISDN Connections, 1995. 43, No. 4, page 209.
In an information bulletin published jointly by the Institute for Physical Electronics of the University of Stuttgart and German Telekom AG in~1995 on the scenario for the use of telepathology in rapid section diagnosis it is proposed to use the limited capacity of the existing ISDN-Network with the nar-row-band telemicroscopy, but in an optimised form, where the system, dependent on the behaviour of the user, i.e. the ob-server at the observation station, employs different strategies for image data-compression and reduction, for example, selec-tion of a greater data-reduction in respect of colour, local resolution and quantisation of the image data when a large num-ber of image transfers are required per unit of time by reason of more rapid translation of the object under the microscope.
Summary of the Invention The technical problem which,the invention seeks to solve is the preparation of an image transmitting remote object examination device of the type mentioned in the preamble, which is capable of automatically adapting itself in its transfer behaviour very flexibly and variably to the appropriate situation, in such a way as to make optimal use of the existing narrow band data-carrying capacity.
The invention solves this problem by providing an image-transferring remote object examination device comprising a system status-determining unit which continuously records the entire image-transfer-relevant system status including the system resources available in each case and as a function thereof automatically ~ CA 02263151 2004-03-22 -controls the image data compression and/or image data reduction process and/or the number of channels of a transmission path to be used in a transmission packet so that as well as the desired image-quality for a particular situation the best possible on-line image impression for the corresponding situation is also obtained in each case. In this way the device can be matched flexibly and variably in respect of image-transfer to the ex-isting system status in each case.
This system status relevant to image-transfer and monitored by the system status-determining unit includes, depending on the system design, not only the user-behaviour, i.e. the control commands for the image-recording system entered by the ob-server, but also in particular the nominal condition in respect of the number individual channels of the telecommunication channel packet used, the type of telecommuni-cation network to be used, the status of the coding unit,carry-ing out.image data compression and/or reduction, the type of image-recording equipment used and the current content of the image.
In this way, despite narrow-band telecommunication connection between the image-recording system and the. observation posi-tion, the equipment in accordance with the invention is capable of maintaining for the user the impression of an on-line or real-time operation, with high image-quality. It is suitable for different types of remote object examinations, for example, remote material testing, telemicroscopy, teleendoscopy and telesonography, depending on the object examined and image-recording system employed.
With application of the. equipment as a telemicroscopy system in telepathology, the pathologist is given a tool with which he can carry out telepathology with a high degree of telepresence.
The parameters for the image data compression and/or reduction are among other things set as a function of the image content and the chosen image enlargement. Thus, for example, little structured, highly magnified images can, if necessary, be transferred at higher image transfer rates_and lower resolutions than highly structured images of lower magnification.
The transmission path uses a telecommunication channel packet, where the num-ber of channels currently used is set in accordance with the recorded system status and where the channel run times are automatically compensated for. The latter is of significance therefore, because the user himself in general has no control over the ways in which the individual communication channels, for example telephone channels, are switched. The ISDN Network technology has proved to be particularly suitable for such a packet solution.
The image-recording system has auto-focus equipment available to it, the function of which is controlled as a function of the system status through a plurality of control parameters. By this means it is possible to avoid the transfer of the usually consider-able quantity of image data required,for image focusing alone.
The control of the auto-focus equipment can be effected inter-actively or alternatively automatically from the system.
An automatic brightness control on the side of the image-recording-system is optionally interactively, or alternatively, auto-matically operable. By means of such a brightness control also the transfer of image data which is redundant for image evalua-tion can be avoided so that the on-line impression of the im-ages transmitted can be increased.
P 1169fi~PCT
-Inasmuch as the individual images which are transferred in course steps still partially overlap, an impression of movement is created by generating subsequent interpolated intervening images in the observation station which are successively brought to the display.
An impression of movement of transmitted, moving images is also conveyed, by not transferring the entire individual image in consecutive steps in each case during such an image movement but only the image areas which are new when compared with the preceding in-dividual image. These image areas are then joined in the obser-vation station to the previous individual image to produce the new individual image.
The activity state of the same, i.e. of the transmission path utilisation and-system activity, is shown optically, acoustically and/or mechanically by means of power reaction on an associated oper-ating control.
As an operating control for the observer at the o bservation station, a phantom, i.e. a copy of the imam-recording-equipmentwsed in the image-recording system, is used so that activities of the ob-server on the phantom are recorded and are converted into con-trol commands for the corresponding adjustment of the real im-age-recording equipment.
A prophylactic transfer of image scenes, is also possible for images, which are forecast to be required by the system at a later-time, at peri-ods with unused transfer channel capacity. For this, the equip-ment contains a predictor for the likely continuation of the remote examination of the object, which, for example, closes up the locations of object areas transmitted up to now to the places of the next image transfers, for example, through corre--sponding continuation of the.path of the last transmitted ar-eas.
A representation tool may be provided for reproducing the image at the observation station, with which an overall image can be repre sented on a separate screen or in the form of a window inserted on the screen used for the detail images, where a marking sur-face of variable size defines the position of the current vis-ual field of the image-recording equipment. If necessary, posi-tions of the marking surface already used could remain marked on the overall image, in order to avoid duplicate examinations.
Archiving selected images is also possible.
Fining a certain desired image quickly and displaying it on the basis of the previously stored archived data by corresponding controls of the image acquisition system is provided. This is useful, for example, if a preliminary examination first takes place and only subsequently. the final evaluation is to be carried out on the basis of selected images. Here, it can be envisaged that the observer moves with the marking-surface to a certain area in the overall,image, that is then represented at greater magnification, so that the equipment fulfils for the user a convenient magnifying glass function.
It is possible to transfer small, selectable image area of an overall image in real time at maximum image quality, for which the capacity of the narrow band transmission path is generally adequate.
A preferred embodiment of the invention is shown in the draw-ings and is described below.
P 11696-'PCT
Brief Description of the Drawings Fig. 1 a block-diagram of a telemicroscopy installation with narrow band telecommunication channel transmission path, Fig. 2 a sequence of images to illustrate a first procedure for conveying an impression of movement with the telemicroscopy installation of Fig. l, Fig. 3 a sequence of images to demonstrate a second proce-dure for conveying an impression of movement with the telemicroscopy installation of Fig. l, Fig. 4 a schematic representation of a mechanical operating element power feedback for the telemicroscopy instal-lation of Fig. 1, Fig. 5 a flow diagram of a background image transfer.~roce-dure for the telemicroscopy installation of Fig. 1 and Fig. 6 a schematic screen representation to demonstrate a procedure for investigating area marking with the telemicroscopy installation-of Fig. 1.
Detailed Description of the Preferred Embodiment Fig. 1 shows by means of a block diagram the construction of an image-transferring remote object examination device in form of a telemicroscopy installation which may be used, for example, for telepathology. One half of the installation comprises an image-recording system in form of a microscope station and the other half an observation station remote from the location of the microscope station, where the observer, for example a pa-thologist andlor an image-evaluating computer, is located.
The microscope station contains a conventional microscope la, all functions of which are remote-controllable, with associated microscope-control lb, an image-recording device photographing the microscope view field 2, an image data coding device 14, which encodes the image-data supplied by the image-recording device 2 by application of conventional procedures for image-data compression and image-data reduction, a transmitter 3 and a following interface 36, which sends out the encoded image-data from the microscope-station supplied by the transmitter 3.
A speech channel S in form of a microscope-station telephone channel is also connected to this interface 36. An auto-focus device 19 serves for the automatic focusing of the microscope la by issuing the corresponding focusing commands to the micro-scope control lb. A brightness-control unit 20 provides for an automatic brightness-control, for which it is supplied with the image data from the image-recording equipment 2 and issues cor-responding brightness control commands to the microscope con-trol 1b. Data supplied from the microscope station by way of the interface 36 are recorded by a transmit/receive unit 4, which passes them to the relevant target components. It simul-taneously serves as transmitter for the sending of status in-formation from the microscope control lb and the coding device 14 to the observation station, for which it is connected to these two units 14, lb by way of duplex signal lines.
The observation station contains an image data receiver 8 with a following monitor 9 to reproduce the image, where the re-ceiver 8 is connected through an image transmission control unit 30 to an interface 37, through which the observation sta-tion sends and receives data. Like the microscope station, the observation station also has a telephone channel 13 connected to its interface 37. In addition, the observation station has an operating unit 10, which functions as a user interface. As known from conventional telemicroscopy installatio:~s, the user can control the spatially distant microscope la in all its functions by inputting the corresponding control commands to P ~ ~6~~rc~r the operating unit 10 for the examination of an object 43 or preparations placed under the microscope la, for example, in respect of the movement of the microscope table and the selec-tion of the respective objective to provide different magnifi-cations. The control commands are sent on the one hand by way of a transmitter/receiver unit 11 to the interface 37 and on the other to a requirement-determining unit 15, which evaluates the control command issued, to analyse user behaviour as a pa-rameter of the system status of the installation. Like the im-age transmission control unit 30, the requirement-determining unit 15 also passes signals to a control display unit at the observation station, comprising a control image screen 28 and a loudspeaker 29 from one and which is in addition connected di-rectly with the operating unit 10. The control image screen 28 may moreover be supplied with image-data received from the re-ceiver 8.
As a further, essential component, the installation comprises a system status-determining unit 42 arranged by way of example in the observation station, which receives data signals from the requirement-determining unit 15 on the behaviour of the user to the operating unit 10 and by way of the receiver function of the transmitter/receiver unit 11 the system status-information from the microscope station which is supplied there specifi-cally from the microscope-control lb and the coding unit 14 over the local transmitter/receiver unit 4.
The microscope station and observation station com,~~unicate with one another by way of a virtual telescopy-microscopy device, which is shown in Fig. 1 by the area between the t:ao broken lines 34, 35 plus the two interfaces 36, 37 of the microscope station on the one hand and observation station on the other hand and which has a universal command set for con~rol com-mands, a suitable image-transfer protocol and a co-responding interface 6, 12 to an intervening, telecommunication channel packet 7 functioning as transmission path, which is constructed from a number, variable with time, of narrow band individual telecommunication channels. The virtual telemicroscopy device may be used to connect together any of several possible micros-copy stations with any of several possible observation stations for the purpose of data exchange without significant adaptive measures. In order to be able to set up in each case the de-sired number of individual channels for the channel packet 7, the two terminal communication channel interfaces 6, 12 com-prise a corresponding channel selection unit 18, 17, where the channel selection unit situated at the microscope station end 18 is controlled by the transmitter/receiver unit 4, while the other channel selection unit 17 is controlled by the system status-determining unit 42. For the recording of the system status, the system status-determining unit 42 communication channel interface 12 at the observation station end obtains data on the number of switched individual channels in each case.
The characteristics of the telemicroscopy installation of Fig.
1 are to be seen particularly in that the system status includ-ing the availability of the system resources is continuously determined by the system status-determining unit 42 and as a function thereof, on the one hand the appropriate number of in-dividual channels condensed to a transfer channel packet of the transmission path 7 between microscope-station and observation-station is determined, and on the other hand the parameters for the image data compression and reduction in the coding unit 14 are suitably adjusted. The system status influencing parameters are especially those of the user behaviour determined from the activities carried out at the operating unit 10 determined by the requirement determining unit 15, the nominal status of the number of individual channels condensed to the transmission package, the type of transmission path 7 used, the nominal status of the coding unit 14, the type of microscope la and im-age recording equipment 2 used as well as the relevant image content. All these data are interrogated from the relevant com-P I I G9fi PC~r ponents of the installation by the system status determining unit 42. The number of transmission channels and the type of the image data compression and reduction are then adjusted by the system status determining unit 42 in each case, so that the best possible on-line-impression is maintained despite the use of narrow band transmission channels for the image of an object lying under the microscope 43 reproduced at the monitor 9 with an image quality sufficient for any situation. For the telemi-croscopy installation of Fig 1 a number of special procedures, details of which are provided below, are envisaged for doing this.
An important point is the system status dependent image data compression and reduction in the coding unit 14. In order to make optimal use for image transfer of the transmission channel capacity, which is limited because of the narrowness of the band, the image-data from the coding unit 14 is compressed in each case and as far as allowable for the relevant application also reduced. Since the user during his work at the telemicro-scopy installation places different demands on resolution, the image transfer rate and image size of the microscope image, a fixed image compression and image reduction procedure would be inconvenient. The telemicroscopy installation therefore uses a system status dependent variable image data compression and re-duction. Dependent on the system status, as determined continu-ously by the system status determining unit 42, particularly dependent on the requirements of the user on the system estab-lished by the requirement determining unit, the best compromise in each case between image transfer rate and image resolution is determined and set on the coding unit 14 automa~ically by the system. For this the system status-determining unit 42 sends corresponding coding signals over the transr;.it-ter/receiver unit 11 and the transmission path 7 to the micro-scope station, which there receives them from the ~ransmit-ter/receiver unit 4 and relays the:,1 to the coding anit 14.
P 1169~~'PCT
If the user shows through frequent and swift, remote-controlled movement of the preparation 43 under the microscope la for ex-ample or through acoustic input signals, that he wants to scan areas of the preparation quickly for a general view, then the system will set the high image rate required for this, while the image resolution can be reduced. On the other hand, the system interprets a longer pause at the same area of the prepa-ration as indicating, that this area should be examined more precisely with a high local resolution and lower image rate, for which purpose the type of encoding will again be set suita-bly. The requirements derived in such a way from the user be-haviour as well as from the remaining system status parameters for the image encoding serves for the~control of the operating properties of the coding unit 14 with regard to lateral resolu-tion, colour resolution and temporal resolution in the form of image transfer rate. All statuses from highest resolution with minimal image rate to very low resolution or small image field at television image rate can be automatically set in any de-sired steps. The coding settings are effected completely auto-matically and necessitate no actions from the user, which could distract him from his actual examination activity. As coding methods, conventional,.procedures can be used, provided they al-low an intervention into their mode of operation with the aim of ensuring the above-mentioned setting of a compromise between image transfer rate and resolution, as is the case, for exam-ple, with the coding procedures "Joined Photographic Experts Group" and "Motion Picture Expert Group" commonly known under the abbreviations JPEG and MPEG respectively. This type of variable and flexible image data compression and reduction leads to a very economical usage of the transmission channel capacity and to a situation-dependent best possible on-line impression of the transmitted image sequences.
Another measure for the optimal utilisation of the transmission capacity of the narrow band transmission path 7 with as goad as possible on-line-impression of the transmitted image-sequences is the automatic packet formation of .an appropriate, variable P 11696~PCT
number of narrow band individual channels. This is realisable simply and advantageously by employing the public telephone network as a transmission path, for example, since conceivably at branch exchanges several individual channels are already available there or alternatively additional connections can easily be made. Dependent on the system status, the system status-determining unit 42 decides in each case the optimal number of individual channels condensed to form a transmission packet and issues corresponding control commands for the chan-nel selection units 17, 18, where it controls the channel se-lection unit 17 at the observation station side directly through a control line 16. The control commands for the other channel selection unit 18 are issued by way of the transmit-ter/receiver unit 11 on the observation station side and the subsequent interface 37 to the transmission path 7 and are transferred by the latter to the microscope station, where they are taken from the local transmitter/receiver unit 4 and used for the control of the relevant channel selection unit 18. Al-ternatively the interface 6 can be designed so that it takes the channel occupancy 12 set at the counter-interface from the data stream, so that the control by way of the transmit-ter/receiver unit 4 can be dropped.
In this way, it is possible to realise an automatic voting in or voting out of appropriate individual channels, where the signal to be transferred is suitably apportioned and distrib-uted over the various channels. Such signal splitting is known in other areas of data transfer, so that it is not necessary to detail it here. With this technology care must be taken that at the receiver end the signals from the individual channels are combined again in the correct time sequence. This is an impor-tant control process, since the user has no control over the ways in which the individual telephone channels are switched through. This can happen in the one case over a ground link and in another case by way of a satellite link, for example. The ISDN Network technology has proved itself to be particularly P l 1696 'PCT
suitable in the case of the present application for the reali-sation of such a channel packet solution.
By using the autofocus equipment 19 the image transmission ca-pacity, which was otherwise required for focusing purposes, can be saved, because the transfer of very many close focus and also very high resolution images with the corresponding signal production is necessary for interactive focusing of a micro-scope image. This transfer of image data not relevant for the actual evaluation of the microscope image scene is avoided by the auto-focus-equipment 19, which can be activated automati-cally by a request for a new image extract or a different en-largement or the parameters of which can be adjusted according to the status of the system by the system status-determining unit 42. As autofocus equipment 19, equipment such as that dis-closed in German Offenlegungsschrift DE 42 26 523 Al, to which reference should be made, can be used.
In an analogous manner to this automatic focusing, the telemi-croscopy station has available a brightness-control 20, which can be controlled either automatically or, dependent on the status of the system, by the system status-determining unit 42, where the latter conveys control commands to the brightness-control unit 20. This automatic brightness control also avoids the transfer of image information which is redundant for the evaluation and again allows improved utilisation of the trans-fer capacities to achieve an as good as possible on-line im-pression of the image sequences returned at the monitor 9.
The telemicroscopy installation in addition allows the impres-sion of movement of a preparation 43 moved under the microscope 1a to be conveyed to the image sequences represented at the monitor 9, in that an on-line impression is created for such moving image sequences, for which two different procedures can be considered. It must be said in advance, that the observation of the movement of the preparation under the r.~icroscope la fre-quently represents an important bearings-aid for the user, ex-actly as in telepathology, and with narrow band transmission lines such a moving image impression of the microscope field of view in real-time by permanent on-line transfer is impossible because of the limited transfer channel capacities.
A first procedure for conveying an impression of movement is shown in Fig. 2. With this procedure, after transfer of an im-age 21, the image data transmitter 3 waits after a request from the user for a preparation movement before the transfer of a next image 22 in each case, until a sufficiently large prepara-tion movement has taken place, which is laid down by corre-sponding translation threshold values. These are chosen so that the next transmitted image 22 at least partially overlaps with the previously transmitted image 21, as shown in Fig. 2 above by a hatched overlap region LT. On the observation station side, i.e. in a computer assigned to the image reproducing monitor 9, a pre-determinable number of intermediate images 23a, 23b, ..., 23n with uniform mutual displacement are generated by uniform interpolation and presented to the observer continuously one after the other between the two transmitted individual images 21, 22. The observer thus receives the impression of an even movement of the preparation that is only somewhat delayed com-pared with the request for movement. For this, only the two terminal individual images 21, 22 of the entire image sequence in accordance with Fig. 2, need to be transferred, while the intermediate images 23a,..., 23n shown in Fig. 2 below are only generated at the observation station and therefore demand no transmission channel capacity.
A second possible procedure for providing the impression of movement despite a narrow band transmission path 7 is illus-trated in Fig. 3. With this procedure with a preparation move-ment, as illustrated in Fig. 3 by a movement Grrow 24, in sev-eral steps in each case only the fresh image areas entering into the field of view 25, which represent those areas of the P 1 1696 PC'1.
current visual field 45a, which do not overlap with the visual field 45b of the preceding steps are transferred. Since the surface newly arriving is clearly smaller than the total sur-face of the microscope field of view image areas 25 again in each case, a corresponding reduction of the quantity of image data to be transferred results. At the observation station, the transmitted image areas 25 are then joined at the existing im-age, which was first correspondingly shifted, as shown by the displacement arrow 26 in Fig. 3. With this procedure, the tem-poral delay between movement request and reaction on the moni-tor 9 of the observation station is smaller in comparison to the first named process, consequently the speed of movement of the preparation is somewhat more limited.
The parameters in both processes can be set entirely automati-cally by the system in either case, in that the latter records the preparation movement ordered by the user by way of the re-quirement determining unit 15. The image montage can take place so that the observer is not aware of any disturbances in the join area 27 of two consecutive images, as shown hatched in Fig. 3, which results from the fact that a small overlap area of consecutive images is used to carry out a correlation analy-sis and from it to decide the exact projection. Since size and direction of the requested movement are known to the system from the requirement determining unit 15, the approximate situation of the consecutive images is also known, so that the correlation analysis has only to make minor corrections. This consequently represents a mixture of prior knowledge and evaluation of image data. The system can select in each case a suitable matching image-sequencing strategy, which may extend from the movement by single columns or lines ~o whole images, depending on the recorded user behaviour. Witr it, the time-consuming and unnecessary transfer of redundant image informa-tion is eliminated.
F 11G9G~PCT
The coding unit 14 is designed in such a way that it decides the image quality according to pre-determined measures and re-lays it for use at the transmitter 3. For this, quality meas-ures, which involve the local resolution, the colour fidelity and the contrast are determined and serve to enable the extent of any image deterioration by data reduction to be appraised quantitatively and hence on the one hand the information on the image content and on the other the visual loss in picture qual-ity be judged by a human observer. Such a judgement can also be made, if a computer aided image evaluation is intended instead of the human observer. The quantifying measures possible with these image quality measurements allow systematic optimisation of the system parameters and the investigation of optimal com-promises between user demands and system limitations.
The design of the operating unit 10 serving as user interface is chosen from the point of view of being able to use the sys-tem effectively interactively, taking into account the fact that the image data throughput and with it the effective possi-ble working speed is determined by the bottleneck of the trans-mission path 7. For such a design, there are different alterna-tives. Preferably the operating unit 10 shows the observer in ergonomically suitable manner the system loading status and limiting factors, so that the latter can organise its behaviour accordingly.
One possible design consists of indicating after a request for a microscope control action by way of the operating unit 10 through the user to show the latter on the associated control screen 28 and/or the associated loudspeakers 29 at the end of each image transfer or microscope action which has taken place on this request, so that he can then start a new action or con-tinue one which is already begun. The optical signalling con-sists of a message, such as, for example, "system carrying out objective changes" or "autofocus process running" or of a sym-bol or a colour, in order to show the degree of availability of the system resources, such as channel occupancy. Acoustically, the system activity status can be signalled by different pitches or sound pulse sequences, for example. With this proce-dure, the image transmission control unit 30 controls the con-trol screen 28 and the loudspeaker 29 accordingly.
As further possible embodiment, a mechanical feedback can be provided, as shown in Fig. 4 in the form of a power feedback on an operating element 31 of the operating unit 10. The operating element 31 can be a rotating knob on the operating unit 10 for the remote control of the microscope table, for example. The degree of system performance or, in the borderline case, system overload is shown to the user mechanically by way of power feedback to the operating element 31. For this, the initial data on system status, as determined by the system status de-termining unit 42, is received through a control unit 44 of the operating unit 10, sets a defined friction by means of a fric-tional force servo component 32 and / or an active reaction force by means of a force feedback motor 33 as a function of the input data. With obstruction of the installation, the oper-ating element 31 is also blocked. If present, the strength of the force sent back signals not only the force on the micro-scope but also in particular the occupancy status of the in-stallation or its busiest component, which is generally the transmission path 7. The user senses immediately, whether he can continue his action or start a new action.
The ergonomic adaptation of the operating unit 10 ~.o the field of application is an important point for the Gccep~ance of such an installation, for example, as telemicroscopy installation in medicine. One embodiment may, for example, take the form of an operating mouse, with which all microscope actions are con-trolled by a device similar to a computer mouse. I~. addition to the current mouse technology, one possible solutio-: is a close adaptation to a conventional microscope work station, which consists of forming the operating unit 10 as a pha~:tom micro-P I f69G~PCT
scope, which takes the form of a microscope reproducing the real microscope of the image recording system without optics, but otherwise with all operating elements. These operating ele-ments are connected with transmitters for recording the user requests and if necessary with the afore-mentioned force feed-backs in accordance with Fig. 4 and permit the control of the actual microscope 1a of the microscope station through manifest operating actions at the phantom microscope o= the observation station. The phantom-microscope provides the user with equip-ment which he is accustomed to working with, so that he does not need to readapt or to train. The only dif~erence from his conventional job is the observation of the microscope view field on the monitor 9 at the observation station instead of through the microscope eyepiece.
As further functionality, the telemicroscopy _nstallation shown has the ability to transfer images prophylactically. Proceeding from the fact that in a detailed examination of images in gen-eral, time intervals appear, in which the existing transmission channel capacity is not used, these time rote=vals are used by the installation to transfer further images ir_ the background from areas of the microscope view field, on tie assumption that they will subsequently be required by the use_. For this, the installation contains a predictor, not shown ~n detail, for the likely continuation of an examination. This p_edictor infers the locations for the next image transfer frog. the locations of the previously transmitted areas of the prepa_ation. The pro-gram course for such image transfers is shown as a flow diagram in Fig. 5.
After a start step 50 there next folloGas a query s'ep 51 to de-termine whether a new image request is avaiiaJle. .f this is the case, a check is made in a subsequent ret_ieva_ step 52, as to whether the image now requested has alread.r bee-: recorded as previously transmitted background image in an associated stor-age. If this is the case, this stored image can be immediately P f 1G9G PCT
be called up from the storage in the next step 53 and shown on the monitor. A following retrieval step 54 then judges, whether the program run has been interrupted so that a stop 55 is reached or whether there should be a jump back to behind the start step 50 to carry out a new program cycle.
If it emerges in the associated retrieval step 52, that the newly ordered image is not yet available in memory, the co-ordinates of the image field are first stored (step 56), after which the new image ordered is then transferred (step 57), so that it can be shown on the monitor. For the storage of the im-age co-ordinates, a corresponding address memory 58 is envis-aged, which is symbolically connected in Fig. 5 by the broken lines with those points in the program relevant to it.
If it is found in the relevant retrieval step 51 that there is no new image request, the predictor in the next step 59 deter-mines the most likely next image address, for example, by ex-tending the track of the previous transmitted preparation areas by a previously determined incremental displacement. This most likely next image is then transferred to the background during a period of unused transmission channel capacity and its ad-dress is stored (step 60). The transmitted image field is then stored in the associated image memory (step 61), where it is then ready for retrieval. Finally, the disconnect enquiry step 54 is continued. This prophylactic image transfer in the back-ground effects an improvement in the image display speed, since the most likely next image is frequently already available at the observation station in the image-storage at the moment when it is required and only needs to be called from there.
Another advantageous measure, found with the telemicroscopy in-stallation of Fig. 1, relates to the permanent orientation help for the observer, as shown in Fig. 6. This guidance comprises the use of a separate monitor screen 39, on which is repre-sented an overall image 38 of the object accumulated since the P l 1696 PCT
start of the examination. Alternatively this overall image rep-resentation can also be presented as an inserted window on the image monitor 9 or on the display screen 28. In the overall im-age is a marking rectangle 40, variable in size, which indi-cates the position of the current microscope viewing field in the overall image, where the size of the marking rectangle 40 represents the area of the current visual field in respect of the overall image. With a selectable operating mode every view-ing field which has once been visited and examined in the over-all image 38 is shown in colour, so that the total colour marked area 41 shows the preparation area already examined in each case, which gives the user a rapid overview of the situa-tion and the surface of the areas of the preparation already examined. This avoids duplicate examinations of preparation ar-eas, which again reduces the data flow.
The telemicroscopy installation shown possesses the further functionality of being able to transfer small image areas in real time. This is based on the fact that the transmission ca-pacity of the transmission path 7 is normally enough to trans-fer a small segment of image from the entire -_mage scene in real-time. This function can be used in telep~thology, for ex-ample, to focus on single cell nuclei in several focus levels.
The user chooses the appropriate detail-image with a small, part image framework (not shown) in the overall image 38 in ac-cordance with Fig. 6 and can then request an _mage-transfer in real-time with maximum image quality for the =ramed image -section.
In addition, the telemicroscopy installation shown allows the archiving of selected image scenes. With this func~ion, the user is given the possibility of storing selected _mage scenes in a data base or electronic patient file. A comme::t is stored with the image, indicating the place of the i~.age '_n the over-all preparation examined together with all re~epticn condi-tions, especially the microscope settings employed. Simultane-r u6» rc-r ously, these images are held ready in thumbnail format on the image-rendering monitor 9 during the examination of a case.
As counterpart to this archiving function moreover, the telemi-croscopy installation shown allows the possibility of calling up again selected preparation locations. By selecting a thumb-nail image or by calling from the data base or the patient card the desired image-scene of the preparation, by means of the data stored with it relating to the image location and the mi-croscope parameters, is called up and displayed. If required, the examination of the preparation can be continued subse-quently in the neighbourhood of this image location. This func-tionality is especially useful, if first the preparation is pre-examined and subsequently assessed by an expert on the ba-sis of the image-scenes selected in advance.
In the telemicroscopy installation shown moreover, a magnifier function is implemented for the inspection of pre-selected preparation locations, with which the observer in the overall image 38 in accordance with Fig. 6 can jump to a selectable po-sition and examine the same with a higher enlargement. This magnifying glass function is executed automatically by the sys-tem. The previous state of enlargement and the previous exami-nation location are automatically restored again at the end of a magnifying glass function process. This functionality offers the user the possibility of detailed examinations with rela-tively few system inputs required from him.
In order to offer the users on the two sides cf the telemicro-scopy installation, for example the pathologist on the observa-tion station side and the operating personnel on the microscope station side, an optical discussion-aid, the telemicroscopy in-stallation shown possesses a pointer-symbol for each partner in the discussion. The two pointer symbols are different in colour and/or shape and are steered and moved by the respective sta-tion and also shown on any other station, which mares the dis-P 11696 PC'I
cussion of image details between the separated remote partners to the discussion easier and unambiguous.
It is to be understood, that the different functional compo-nents of the telemicroscopy installation shown in Fig. 1, can in each case take the form of separate prefabricated parts or of integrated units for example of a computer, in so far as they can be realised in software instead of hardware, as is well-known to the expert. It is further to be understood, that the expert, in addition to that shown, can realise other tele-microscopy installations in accordance with the invention and moreover that the invention includes other types of remote ob-ject examination devices, especially teleendoscopy and tele-sonography installations. With teleendoscopy installations, the microscope in the telemicroscopy installation shown is replaced with a remote-controlled endoscope or with an endoscope con-trolled by an operator. In telesonography installations, the microscope of the telemicroscopy installation shown is replaced with a remote-controlled or operator-controlled sonography sys-tem. In addition, the same advantageous procedures can also be achieved with these types of installation, as detailed above for the telemicroscopy installation.
P 11G9GlPCT
' CA 02263151 2004-03-22 Image-transmitting remote object examination device Field of the Invention The invention refers to a remote object examination device pro-vided with an image-recording system at the location of the ob-ject to be examined,,a remote observation station for evaluating the image and for remote control of the image-recording system, as well as with an intervening nar-row-band telecommunications link, over which the image-data generated by the image-recording system is transferred to the ob-servation -station using image data compression and/or reduc-tion, as well as ' control command signals generated from the observation station for y remote controlling the image-recording system.
Background of the Invention Such remote object examination devices are known in form of telemicroscopy systems for example, particularly as applied to telepathology. The object to be examined in this case i.s a tis-sue section prepared, for example, during an operation. The im-age-recording system comprises a microscope remote-controllable in all functions, an image-recording device for the microscope field, a transmitter for the image signal a.nd a receiver for the control commands at the microscope. Transmitter and re-ceiver as well as an additional speech channel are connected over an interface with a telecommunications link. The observa-tion-station comprises a receiver, a monitor for the images to be returned, an appliance for the production of the control P 11696. PCT
commands for the microscope and a transmitter for these control commands. The observer or expert, for example, a suitable com-puter or a pathologist, can appraise the transmitted microscope image. Transmitter and receiver as well as a speech channel are in turn connected by way of a suitable interface with the same telecommunications link as the image-recording system.
If the telecommunications link has a sufficient width for the transfer of a television signal, the observer at the observa-tion station can use all functions of the spatially removed mi-croscope just as if this were at his workplace. Such a telepathology system with wide band data transfer is described in U.S. Patent 5,216,596. However, wide band telecommunication channels are comparatively costly for television signals and of only limited availability. It has already been proposed there-fore, to operate telemicroscopy over the widely available and inexpensive, narrow-band channels of the telephone network. An installation with such narrow band technology employing the ISDN-Network is described by M. Oberholzer et al. in the paper "Telepathology: frozen section diagnosis at a distance" (1995), 426, page 3. However, the limited channel capacity forms a bot-tleneck for the transfer of the image data so that the on-line-impression for the user can be lost and instead of a moving im-age he will only receive a slow consecutive string of individ-ual images. The result will be that the user will also lose the immediate impression of the effect of his microscope control commands, since for him their effect will be seen only after a substantial delay and in an altered context. The application of the technologies of video-telephony only allows the transmis-sion with a quality of image which is inadequate for most of the applications considered here, such as telemicroscopy, teleendoscopy and telesonography.
Some measures have also already been proposed for the economic application of ISDN channels for narrow band telemicroscopy systems, such as, for example, the use of a particular auto-P 11696!PC'I' " CA 02263151 2004-03-22 focus system and the application of a fixed, pre-determined, compression and reduction adapted to the particular system of the image data to be transferred, during their encoding, (see the papers by P. Schwarzmann, Telemikroskopie, Zentralbl.
Pathol, 138, (1992) 6, page 383 and P. Schwarzmann et al., Telemicroscopy Stations for Telepathology Based on Broadband and ISDN Connections, 1995. 43, No. 4, page 209.
In an information bulletin published jointly by the Institute for Physical Electronics of the University of Stuttgart and German Telekom AG in~1995 on the scenario for the use of telepathology in rapid section diagnosis it is proposed to use the limited capacity of the existing ISDN-Network with the nar-row-band telemicroscopy, but in an optimised form, where the system, dependent on the behaviour of the user, i.e. the ob-server at the observation station, employs different strategies for image data-compression and reduction, for example, selec-tion of a greater data-reduction in respect of colour, local resolution and quantisation of the image data when a large num-ber of image transfers are required per unit of time by reason of more rapid translation of the object under the microscope.
Summary of the Invention The technical problem which,the invention seeks to solve is the preparation of an image transmitting remote object examination device of the type mentioned in the preamble, which is capable of automatically adapting itself in its transfer behaviour very flexibly and variably to the appropriate situation, in such a way as to make optimal use of the existing narrow band data-carrying capacity.
The invention solves this problem by providing an image-transferring remote object examination device comprising a system status-determining unit which continuously records the entire image-transfer-relevant system status including the system resources available in each case and as a function thereof automatically ~ CA 02263151 2004-03-22 -controls the image data compression and/or image data reduction process and/or the number of channels of a transmission path to be used in a transmission packet so that as well as the desired image-quality for a particular situation the best possible on-line image impression for the corresponding situation is also obtained in each case. In this way the device can be matched flexibly and variably in respect of image-transfer to the ex-isting system status in each case.
This system status relevant to image-transfer and monitored by the system status-determining unit includes, depending on the system design, not only the user-behaviour, i.e. the control commands for the image-recording system entered by the ob-server, but also in particular the nominal condition in respect of the number individual channels of the telecommunication channel packet used, the type of telecommuni-cation network to be used, the status of the coding unit,carry-ing out.image data compression and/or reduction, the type of image-recording equipment used and the current content of the image.
In this way, despite narrow-band telecommunication connection between the image-recording system and the. observation posi-tion, the equipment in accordance with the invention is capable of maintaining for the user the impression of an on-line or real-time operation, with high image-quality. It is suitable for different types of remote object examinations, for example, remote material testing, telemicroscopy, teleendoscopy and telesonography, depending on the object examined and image-recording system employed.
With application of the. equipment as a telemicroscopy system in telepathology, the pathologist is given a tool with which he can carry out telepathology with a high degree of telepresence.
The parameters for the image data compression and/or reduction are among other things set as a function of the image content and the chosen image enlargement. Thus, for example, little structured, highly magnified images can, if necessary, be transferred at higher image transfer rates_and lower resolutions than highly structured images of lower magnification.
The transmission path uses a telecommunication channel packet, where the num-ber of channels currently used is set in accordance with the recorded system status and where the channel run times are automatically compensated for. The latter is of significance therefore, because the user himself in general has no control over the ways in which the individual communication channels, for example telephone channels, are switched. The ISDN Network technology has proved to be particularly suitable for such a packet solution.
The image-recording system has auto-focus equipment available to it, the function of which is controlled as a function of the system status through a plurality of control parameters. By this means it is possible to avoid the transfer of the usually consider-able quantity of image data required,for image focusing alone.
The control of the auto-focus equipment can be effected inter-actively or alternatively automatically from the system.
An automatic brightness control on the side of the image-recording-system is optionally interactively, or alternatively, auto-matically operable. By means of such a brightness control also the transfer of image data which is redundant for image evalua-tion can be avoided so that the on-line impression of the im-ages transmitted can be increased.
P 1169fi~PCT
-Inasmuch as the individual images which are transferred in course steps still partially overlap, an impression of movement is created by generating subsequent interpolated intervening images in the observation station which are successively brought to the display.
An impression of movement of transmitted, moving images is also conveyed, by not transferring the entire individual image in consecutive steps in each case during such an image movement but only the image areas which are new when compared with the preceding in-dividual image. These image areas are then joined in the obser-vation station to the previous individual image to produce the new individual image.
The activity state of the same, i.e. of the transmission path utilisation and-system activity, is shown optically, acoustically and/or mechanically by means of power reaction on an associated oper-ating control.
As an operating control for the observer at the o bservation station, a phantom, i.e. a copy of the imam-recording-equipmentwsed in the image-recording system, is used so that activities of the ob-server on the phantom are recorded and are converted into con-trol commands for the corresponding adjustment of the real im-age-recording equipment.
A prophylactic transfer of image scenes, is also possible for images, which are forecast to be required by the system at a later-time, at peri-ods with unused transfer channel capacity. For this, the equip-ment contains a predictor for the likely continuation of the remote examination of the object, which, for example, closes up the locations of object areas transmitted up to now to the places of the next image transfers, for example, through corre--sponding continuation of the.path of the last transmitted ar-eas.
A representation tool may be provided for reproducing the image at the observation station, with which an overall image can be repre sented on a separate screen or in the form of a window inserted on the screen used for the detail images, where a marking sur-face of variable size defines the position of the current vis-ual field of the image-recording equipment. If necessary, posi-tions of the marking surface already used could remain marked on the overall image, in order to avoid duplicate examinations.
Archiving selected images is also possible.
Fining a certain desired image quickly and displaying it on the basis of the previously stored archived data by corresponding controls of the image acquisition system is provided. This is useful, for example, if a preliminary examination first takes place and only subsequently. the final evaluation is to be carried out on the basis of selected images. Here, it can be envisaged that the observer moves with the marking-surface to a certain area in the overall,image, that is then represented at greater magnification, so that the equipment fulfils for the user a convenient magnifying glass function.
It is possible to transfer small, selectable image area of an overall image in real time at maximum image quality, for which the capacity of the narrow band transmission path is generally adequate.
A preferred embodiment of the invention is shown in the draw-ings and is described below.
P 11696-'PCT
Brief Description of the Drawings Fig. 1 a block-diagram of a telemicroscopy installation with narrow band telecommunication channel transmission path, Fig. 2 a sequence of images to illustrate a first procedure for conveying an impression of movement with the telemicroscopy installation of Fig. l, Fig. 3 a sequence of images to demonstrate a second proce-dure for conveying an impression of movement with the telemicroscopy installation of Fig. l, Fig. 4 a schematic representation of a mechanical operating element power feedback for the telemicroscopy instal-lation of Fig. 1, Fig. 5 a flow diagram of a background image transfer.~roce-dure for the telemicroscopy installation of Fig. 1 and Fig. 6 a schematic screen representation to demonstrate a procedure for investigating area marking with the telemicroscopy installation-of Fig. 1.
Detailed Description of the Preferred Embodiment Fig. 1 shows by means of a block diagram the construction of an image-transferring remote object examination device in form of a telemicroscopy installation which may be used, for example, for telepathology. One half of the installation comprises an image-recording system in form of a microscope station and the other half an observation station remote from the location of the microscope station, where the observer, for example a pa-thologist andlor an image-evaluating computer, is located.
The microscope station contains a conventional microscope la, all functions of which are remote-controllable, with associated microscope-control lb, an image-recording device photographing the microscope view field 2, an image data coding device 14, which encodes the image-data supplied by the image-recording device 2 by application of conventional procedures for image-data compression and image-data reduction, a transmitter 3 and a following interface 36, which sends out the encoded image-data from the microscope-station supplied by the transmitter 3.
A speech channel S in form of a microscope-station telephone channel is also connected to this interface 36. An auto-focus device 19 serves for the automatic focusing of the microscope la by issuing the corresponding focusing commands to the micro-scope control lb. A brightness-control unit 20 provides for an automatic brightness-control, for which it is supplied with the image data from the image-recording equipment 2 and issues cor-responding brightness control commands to the microscope con-trol 1b. Data supplied from the microscope station by way of the interface 36 are recorded by a transmit/receive unit 4, which passes them to the relevant target components. It simul-taneously serves as transmitter for the sending of status in-formation from the microscope control lb and the coding device 14 to the observation station, for which it is connected to these two units 14, lb by way of duplex signal lines.
The observation station contains an image data receiver 8 with a following monitor 9 to reproduce the image, where the re-ceiver 8 is connected through an image transmission control unit 30 to an interface 37, through which the observation sta-tion sends and receives data. Like the microscope station, the observation station also has a telephone channel 13 connected to its interface 37. In addition, the observation station has an operating unit 10, which functions as a user interface. As known from conventional telemicroscopy installatio:~s, the user can control the spatially distant microscope la in all its functions by inputting the corresponding control commands to P ~ ~6~~rc~r the operating unit 10 for the examination of an object 43 or preparations placed under the microscope la, for example, in respect of the movement of the microscope table and the selec-tion of the respective objective to provide different magnifi-cations. The control commands are sent on the one hand by way of a transmitter/receiver unit 11 to the interface 37 and on the other to a requirement-determining unit 15, which evaluates the control command issued, to analyse user behaviour as a pa-rameter of the system status of the installation. Like the im-age transmission control unit 30, the requirement-determining unit 15 also passes signals to a control display unit at the observation station, comprising a control image screen 28 and a loudspeaker 29 from one and which is in addition connected di-rectly with the operating unit 10. The control image screen 28 may moreover be supplied with image-data received from the re-ceiver 8.
As a further, essential component, the installation comprises a system status-determining unit 42 arranged by way of example in the observation station, which receives data signals from the requirement-determining unit 15 on the behaviour of the user to the operating unit 10 and by way of the receiver function of the transmitter/receiver unit 11 the system status-information from the microscope station which is supplied there specifi-cally from the microscope-control lb and the coding unit 14 over the local transmitter/receiver unit 4.
The microscope station and observation station com,~~unicate with one another by way of a virtual telescopy-microscopy device, which is shown in Fig. 1 by the area between the t:ao broken lines 34, 35 plus the two interfaces 36, 37 of the microscope station on the one hand and observation station on the other hand and which has a universal command set for con~rol com-mands, a suitable image-transfer protocol and a co-responding interface 6, 12 to an intervening, telecommunication channel packet 7 functioning as transmission path, which is constructed from a number, variable with time, of narrow band individual telecommunication channels. The virtual telemicroscopy device may be used to connect together any of several possible micros-copy stations with any of several possible observation stations for the purpose of data exchange without significant adaptive measures. In order to be able to set up in each case the de-sired number of individual channels for the channel packet 7, the two terminal communication channel interfaces 6, 12 com-prise a corresponding channel selection unit 18, 17, where the channel selection unit situated at the microscope station end 18 is controlled by the transmitter/receiver unit 4, while the other channel selection unit 17 is controlled by the system status-determining unit 42. For the recording of the system status, the system status-determining unit 42 communication channel interface 12 at the observation station end obtains data on the number of switched individual channels in each case.
The characteristics of the telemicroscopy installation of Fig.
1 are to be seen particularly in that the system status includ-ing the availability of the system resources is continuously determined by the system status-determining unit 42 and as a function thereof, on the one hand the appropriate number of in-dividual channels condensed to a transfer channel packet of the transmission path 7 between microscope-station and observation-station is determined, and on the other hand the parameters for the image data compression and reduction in the coding unit 14 are suitably adjusted. The system status influencing parameters are especially those of the user behaviour determined from the activities carried out at the operating unit 10 determined by the requirement determining unit 15, the nominal status of the number of individual channels condensed to the transmission package, the type of transmission path 7 used, the nominal status of the coding unit 14, the type of microscope la and im-age recording equipment 2 used as well as the relevant image content. All these data are interrogated from the relevant com-P I I G9fi PC~r ponents of the installation by the system status determining unit 42. The number of transmission channels and the type of the image data compression and reduction are then adjusted by the system status determining unit 42 in each case, so that the best possible on-line-impression is maintained despite the use of narrow band transmission channels for the image of an object lying under the microscope 43 reproduced at the monitor 9 with an image quality sufficient for any situation. For the telemi-croscopy installation of Fig 1 a number of special procedures, details of which are provided below, are envisaged for doing this.
An important point is the system status dependent image data compression and reduction in the coding unit 14. In order to make optimal use for image transfer of the transmission channel capacity, which is limited because of the narrowness of the band, the image-data from the coding unit 14 is compressed in each case and as far as allowable for the relevant application also reduced. Since the user during his work at the telemicro-scopy installation places different demands on resolution, the image transfer rate and image size of the microscope image, a fixed image compression and image reduction procedure would be inconvenient. The telemicroscopy installation therefore uses a system status dependent variable image data compression and re-duction. Dependent on the system status, as determined continu-ously by the system status determining unit 42, particularly dependent on the requirements of the user on the system estab-lished by the requirement determining unit, the best compromise in each case between image transfer rate and image resolution is determined and set on the coding unit 14 automa~ically by the system. For this the system status-determining unit 42 sends corresponding coding signals over the transr;.it-ter/receiver unit 11 and the transmission path 7 to the micro-scope station, which there receives them from the ~ransmit-ter/receiver unit 4 and relays the:,1 to the coding anit 14.
P 1169~~'PCT
If the user shows through frequent and swift, remote-controlled movement of the preparation 43 under the microscope la for ex-ample or through acoustic input signals, that he wants to scan areas of the preparation quickly for a general view, then the system will set the high image rate required for this, while the image resolution can be reduced. On the other hand, the system interprets a longer pause at the same area of the prepa-ration as indicating, that this area should be examined more precisely with a high local resolution and lower image rate, for which purpose the type of encoding will again be set suita-bly. The requirements derived in such a way from the user be-haviour as well as from the remaining system status parameters for the image encoding serves for the~control of the operating properties of the coding unit 14 with regard to lateral resolu-tion, colour resolution and temporal resolution in the form of image transfer rate. All statuses from highest resolution with minimal image rate to very low resolution or small image field at television image rate can be automatically set in any de-sired steps. The coding settings are effected completely auto-matically and necessitate no actions from the user, which could distract him from his actual examination activity. As coding methods, conventional,.procedures can be used, provided they al-low an intervention into their mode of operation with the aim of ensuring the above-mentioned setting of a compromise between image transfer rate and resolution, as is the case, for exam-ple, with the coding procedures "Joined Photographic Experts Group" and "Motion Picture Expert Group" commonly known under the abbreviations JPEG and MPEG respectively. This type of variable and flexible image data compression and reduction leads to a very economical usage of the transmission channel capacity and to a situation-dependent best possible on-line impression of the transmitted image sequences.
Another measure for the optimal utilisation of the transmission capacity of the narrow band transmission path 7 with as goad as possible on-line-impression of the transmitted image-sequences is the automatic packet formation of .an appropriate, variable P 11696~PCT
number of narrow band individual channels. This is realisable simply and advantageously by employing the public telephone network as a transmission path, for example, since conceivably at branch exchanges several individual channels are already available there or alternatively additional connections can easily be made. Dependent on the system status, the system status-determining unit 42 decides in each case the optimal number of individual channels condensed to form a transmission packet and issues corresponding control commands for the chan-nel selection units 17, 18, where it controls the channel se-lection unit 17 at the observation station side directly through a control line 16. The control commands for the other channel selection unit 18 are issued by way of the transmit-ter/receiver unit 11 on the observation station side and the subsequent interface 37 to the transmission path 7 and are transferred by the latter to the microscope station, where they are taken from the local transmitter/receiver unit 4 and used for the control of the relevant channel selection unit 18. Al-ternatively the interface 6 can be designed so that it takes the channel occupancy 12 set at the counter-interface from the data stream, so that the control by way of the transmit-ter/receiver unit 4 can be dropped.
In this way, it is possible to realise an automatic voting in or voting out of appropriate individual channels, where the signal to be transferred is suitably apportioned and distrib-uted over the various channels. Such signal splitting is known in other areas of data transfer, so that it is not necessary to detail it here. With this technology care must be taken that at the receiver end the signals from the individual channels are combined again in the correct time sequence. This is an impor-tant control process, since the user has no control over the ways in which the individual telephone channels are switched through. This can happen in the one case over a ground link and in another case by way of a satellite link, for example. The ISDN Network technology has proved itself to be particularly P l 1696 'PCT
suitable in the case of the present application for the reali-sation of such a channel packet solution.
By using the autofocus equipment 19 the image transmission ca-pacity, which was otherwise required for focusing purposes, can be saved, because the transfer of very many close focus and also very high resolution images with the corresponding signal production is necessary for interactive focusing of a micro-scope image. This transfer of image data not relevant for the actual evaluation of the microscope image scene is avoided by the auto-focus-equipment 19, which can be activated automati-cally by a request for a new image extract or a different en-largement or the parameters of which can be adjusted according to the status of the system by the system status-determining unit 42. As autofocus equipment 19, equipment such as that dis-closed in German Offenlegungsschrift DE 42 26 523 Al, to which reference should be made, can be used.
In an analogous manner to this automatic focusing, the telemi-croscopy station has available a brightness-control 20, which can be controlled either automatically or, dependent on the status of the system, by the system status-determining unit 42, where the latter conveys control commands to the brightness-control unit 20. This automatic brightness control also avoids the transfer of image information which is redundant for the evaluation and again allows improved utilisation of the trans-fer capacities to achieve an as good as possible on-line im-pression of the image sequences returned at the monitor 9.
The telemicroscopy installation in addition allows the impres-sion of movement of a preparation 43 moved under the microscope 1a to be conveyed to the image sequences represented at the monitor 9, in that an on-line impression is created for such moving image sequences, for which two different procedures can be considered. It must be said in advance, that the observation of the movement of the preparation under the r.~icroscope la fre-quently represents an important bearings-aid for the user, ex-actly as in telepathology, and with narrow band transmission lines such a moving image impression of the microscope field of view in real-time by permanent on-line transfer is impossible because of the limited transfer channel capacities.
A first procedure for conveying an impression of movement is shown in Fig. 2. With this procedure, after transfer of an im-age 21, the image data transmitter 3 waits after a request from the user for a preparation movement before the transfer of a next image 22 in each case, until a sufficiently large prepara-tion movement has taken place, which is laid down by corre-sponding translation threshold values. These are chosen so that the next transmitted image 22 at least partially overlaps with the previously transmitted image 21, as shown in Fig. 2 above by a hatched overlap region LT. On the observation station side, i.e. in a computer assigned to the image reproducing monitor 9, a pre-determinable number of intermediate images 23a, 23b, ..., 23n with uniform mutual displacement are generated by uniform interpolation and presented to the observer continuously one after the other between the two transmitted individual images 21, 22. The observer thus receives the impression of an even movement of the preparation that is only somewhat delayed com-pared with the request for movement. For this, only the two terminal individual images 21, 22 of the entire image sequence in accordance with Fig. 2, need to be transferred, while the intermediate images 23a,..., 23n shown in Fig. 2 below are only generated at the observation station and therefore demand no transmission channel capacity.
A second possible procedure for providing the impression of movement despite a narrow band transmission path 7 is illus-trated in Fig. 3. With this procedure with a preparation move-ment, as illustrated in Fig. 3 by a movement Grrow 24, in sev-eral steps in each case only the fresh image areas entering into the field of view 25, which represent those areas of the P 1 1696 PC'1.
current visual field 45a, which do not overlap with the visual field 45b of the preceding steps are transferred. Since the surface newly arriving is clearly smaller than the total sur-face of the microscope field of view image areas 25 again in each case, a corresponding reduction of the quantity of image data to be transferred results. At the observation station, the transmitted image areas 25 are then joined at the existing im-age, which was first correspondingly shifted, as shown by the displacement arrow 26 in Fig. 3. With this procedure, the tem-poral delay between movement request and reaction on the moni-tor 9 of the observation station is smaller in comparison to the first named process, consequently the speed of movement of the preparation is somewhat more limited.
The parameters in both processes can be set entirely automati-cally by the system in either case, in that the latter records the preparation movement ordered by the user by way of the re-quirement determining unit 15. The image montage can take place so that the observer is not aware of any disturbances in the join area 27 of two consecutive images, as shown hatched in Fig. 3, which results from the fact that a small overlap area of consecutive images is used to carry out a correlation analy-sis and from it to decide the exact projection. Since size and direction of the requested movement are known to the system from the requirement determining unit 15, the approximate situation of the consecutive images is also known, so that the correlation analysis has only to make minor corrections. This consequently represents a mixture of prior knowledge and evaluation of image data. The system can select in each case a suitable matching image-sequencing strategy, which may extend from the movement by single columns or lines ~o whole images, depending on the recorded user behaviour. Witr it, the time-consuming and unnecessary transfer of redundant image informa-tion is eliminated.
F 11G9G~PCT
The coding unit 14 is designed in such a way that it decides the image quality according to pre-determined measures and re-lays it for use at the transmitter 3. For this, quality meas-ures, which involve the local resolution, the colour fidelity and the contrast are determined and serve to enable the extent of any image deterioration by data reduction to be appraised quantitatively and hence on the one hand the information on the image content and on the other the visual loss in picture qual-ity be judged by a human observer. Such a judgement can also be made, if a computer aided image evaluation is intended instead of the human observer. The quantifying measures possible with these image quality measurements allow systematic optimisation of the system parameters and the investigation of optimal com-promises between user demands and system limitations.
The design of the operating unit 10 serving as user interface is chosen from the point of view of being able to use the sys-tem effectively interactively, taking into account the fact that the image data throughput and with it the effective possi-ble working speed is determined by the bottleneck of the trans-mission path 7. For such a design, there are different alterna-tives. Preferably the operating unit 10 shows the observer in ergonomically suitable manner the system loading status and limiting factors, so that the latter can organise its behaviour accordingly.
One possible design consists of indicating after a request for a microscope control action by way of the operating unit 10 through the user to show the latter on the associated control screen 28 and/or the associated loudspeakers 29 at the end of each image transfer or microscope action which has taken place on this request, so that he can then start a new action or con-tinue one which is already begun. The optical signalling con-sists of a message, such as, for example, "system carrying out objective changes" or "autofocus process running" or of a sym-bol or a colour, in order to show the degree of availability of the system resources, such as channel occupancy. Acoustically, the system activity status can be signalled by different pitches or sound pulse sequences, for example. With this proce-dure, the image transmission control unit 30 controls the con-trol screen 28 and the loudspeaker 29 accordingly.
As further possible embodiment, a mechanical feedback can be provided, as shown in Fig. 4 in the form of a power feedback on an operating element 31 of the operating unit 10. The operating element 31 can be a rotating knob on the operating unit 10 for the remote control of the microscope table, for example. The degree of system performance or, in the borderline case, system overload is shown to the user mechanically by way of power feedback to the operating element 31. For this, the initial data on system status, as determined by the system status de-termining unit 42, is received through a control unit 44 of the operating unit 10, sets a defined friction by means of a fric-tional force servo component 32 and / or an active reaction force by means of a force feedback motor 33 as a function of the input data. With obstruction of the installation, the oper-ating element 31 is also blocked. If present, the strength of the force sent back signals not only the force on the micro-scope but also in particular the occupancy status of the in-stallation or its busiest component, which is generally the transmission path 7. The user senses immediately, whether he can continue his action or start a new action.
The ergonomic adaptation of the operating unit 10 ~.o the field of application is an important point for the Gccep~ance of such an installation, for example, as telemicroscopy installation in medicine. One embodiment may, for example, take the form of an operating mouse, with which all microscope actions are con-trolled by a device similar to a computer mouse. I~. addition to the current mouse technology, one possible solutio-: is a close adaptation to a conventional microscope work station, which consists of forming the operating unit 10 as a pha~:tom micro-P I f69G~PCT
scope, which takes the form of a microscope reproducing the real microscope of the image recording system without optics, but otherwise with all operating elements. These operating ele-ments are connected with transmitters for recording the user requests and if necessary with the afore-mentioned force feed-backs in accordance with Fig. 4 and permit the control of the actual microscope 1a of the microscope station through manifest operating actions at the phantom microscope o= the observation station. The phantom-microscope provides the user with equip-ment which he is accustomed to working with, so that he does not need to readapt or to train. The only dif~erence from his conventional job is the observation of the microscope view field on the monitor 9 at the observation station instead of through the microscope eyepiece.
As further functionality, the telemicroscopy _nstallation shown has the ability to transfer images prophylactically. Proceeding from the fact that in a detailed examination of images in gen-eral, time intervals appear, in which the existing transmission channel capacity is not used, these time rote=vals are used by the installation to transfer further images ir_ the background from areas of the microscope view field, on tie assumption that they will subsequently be required by the use_. For this, the installation contains a predictor, not shown ~n detail, for the likely continuation of an examination. This p_edictor infers the locations for the next image transfer frog. the locations of the previously transmitted areas of the prepa_ation. The pro-gram course for such image transfers is shown as a flow diagram in Fig. 5.
After a start step 50 there next folloGas a query s'ep 51 to de-termine whether a new image request is avaiiaJle. .f this is the case, a check is made in a subsequent ret_ieva_ step 52, as to whether the image now requested has alread.r bee-: recorded as previously transmitted background image in an associated stor-age. If this is the case, this stored image can be immediately P f 1G9G PCT
be called up from the storage in the next step 53 and shown on the monitor. A following retrieval step 54 then judges, whether the program run has been interrupted so that a stop 55 is reached or whether there should be a jump back to behind the start step 50 to carry out a new program cycle.
If it emerges in the associated retrieval step 52, that the newly ordered image is not yet available in memory, the co-ordinates of the image field are first stored (step 56), after which the new image ordered is then transferred (step 57), so that it can be shown on the monitor. For the storage of the im-age co-ordinates, a corresponding address memory 58 is envis-aged, which is symbolically connected in Fig. 5 by the broken lines with those points in the program relevant to it.
If it is found in the relevant retrieval step 51 that there is no new image request, the predictor in the next step 59 deter-mines the most likely next image address, for example, by ex-tending the track of the previous transmitted preparation areas by a previously determined incremental displacement. This most likely next image is then transferred to the background during a period of unused transmission channel capacity and its ad-dress is stored (step 60). The transmitted image field is then stored in the associated image memory (step 61), where it is then ready for retrieval. Finally, the disconnect enquiry step 54 is continued. This prophylactic image transfer in the back-ground effects an improvement in the image display speed, since the most likely next image is frequently already available at the observation station in the image-storage at the moment when it is required and only needs to be called from there.
Another advantageous measure, found with the telemicroscopy in-stallation of Fig. 1, relates to the permanent orientation help for the observer, as shown in Fig. 6. This guidance comprises the use of a separate monitor screen 39, on which is repre-sented an overall image 38 of the object accumulated since the P l 1696 PCT
start of the examination. Alternatively this overall image rep-resentation can also be presented as an inserted window on the image monitor 9 or on the display screen 28. In the overall im-age is a marking rectangle 40, variable in size, which indi-cates the position of the current microscope viewing field in the overall image, where the size of the marking rectangle 40 represents the area of the current visual field in respect of the overall image. With a selectable operating mode every view-ing field which has once been visited and examined in the over-all image 38 is shown in colour, so that the total colour marked area 41 shows the preparation area already examined in each case, which gives the user a rapid overview of the situa-tion and the surface of the areas of the preparation already examined. This avoids duplicate examinations of preparation ar-eas, which again reduces the data flow.
The telemicroscopy installation shown possesses the further functionality of being able to transfer small image areas in real time. This is based on the fact that the transmission ca-pacity of the transmission path 7 is normally enough to trans-fer a small segment of image from the entire -_mage scene in real-time. This function can be used in telep~thology, for ex-ample, to focus on single cell nuclei in several focus levels.
The user chooses the appropriate detail-image with a small, part image framework (not shown) in the overall image 38 in ac-cordance with Fig. 6 and can then request an _mage-transfer in real-time with maximum image quality for the =ramed image -section.
In addition, the telemicroscopy installation shown allows the archiving of selected image scenes. With this func~ion, the user is given the possibility of storing selected _mage scenes in a data base or electronic patient file. A comme::t is stored with the image, indicating the place of the i~.age '_n the over-all preparation examined together with all re~epticn condi-tions, especially the microscope settings employed. Simultane-r u6» rc-r ously, these images are held ready in thumbnail format on the image-rendering monitor 9 during the examination of a case.
As counterpart to this archiving function moreover, the telemi-croscopy installation shown allows the possibility of calling up again selected preparation locations. By selecting a thumb-nail image or by calling from the data base or the patient card the desired image-scene of the preparation, by means of the data stored with it relating to the image location and the mi-croscope parameters, is called up and displayed. If required, the examination of the preparation can be continued subse-quently in the neighbourhood of this image location. This func-tionality is especially useful, if first the preparation is pre-examined and subsequently assessed by an expert on the ba-sis of the image-scenes selected in advance.
In the telemicroscopy installation shown moreover, a magnifier function is implemented for the inspection of pre-selected preparation locations, with which the observer in the overall image 38 in accordance with Fig. 6 can jump to a selectable po-sition and examine the same with a higher enlargement. This magnifying glass function is executed automatically by the sys-tem. The previous state of enlargement and the previous exami-nation location are automatically restored again at the end of a magnifying glass function process. This functionality offers the user the possibility of detailed examinations with rela-tively few system inputs required from him.
In order to offer the users on the two sides cf the telemicro-scopy installation, for example the pathologist on the observa-tion station side and the operating personnel on the microscope station side, an optical discussion-aid, the telemicroscopy in-stallation shown possesses a pointer-symbol for each partner in the discussion. The two pointer symbols are different in colour and/or shape and are steered and moved by the respective sta-tion and also shown on any other station, which mares the dis-P 11696 PC'I
cussion of image details between the separated remote partners to the discussion easier and unambiguous.
It is to be understood, that the different functional compo-nents of the telemicroscopy installation shown in Fig. 1, can in each case take the form of separate prefabricated parts or of integrated units for example of a computer, in so far as they can be realised in software instead of hardware, as is well-known to the expert. It is further to be understood, that the expert, in addition to that shown, can realise other tele-microscopy installations in accordance with the invention and moreover that the invention includes other types of remote ob-ject examination devices, especially teleendoscopy and tele-sonography installations. With teleendoscopy installations, the microscope in the telemicroscopy installation shown is replaced with a remote-controlled endoscope or with an endoscope con-trolled by an operator. In telesonography installations, the microscope of the telemicroscopy installation shown is replaced with a remote-controlled or operator-controlled sonography sys-tem. In addition, the same advantageous procedures can also be achieved with these types of installation, as detailed above for the telemicroscopy installation.
P 11G9GlPCT
Claims (14)
1. In an image transmitting remote object examination device comprising:
(a) an image recording system disposed at a location of an object to be examined;
(b) an observation station located at a distance from the image recording system, the observation station for evaluating the image and remotely controlling the image recording system;
and (c) a narrow-band telecommunications link interposed between the image recording system and the observation station, whereby images recorded at the image recording system are transmitted to the observation station by the narrow band telecommunications link using at least one of image data compression and image data reduction, the narrow band telecommunications link also transmitting remote control signals to the image recording system;
the improvement comprising:
an automated system status determining unit for continuously supervising the automated management of image transmission, and as a function of determined system status, including availability of system resources, variably and automatically controlling at least one of the image data compression, the image data reduction and the number of individual telecommunication channels employed in the transmission of the image data.
(a) an image recording system disposed at a location of an object to be examined;
(b) an observation station located at a distance from the image recording system, the observation station for evaluating the image and remotely controlling the image recording system;
and (c) a narrow-band telecommunications link interposed between the image recording system and the observation station, whereby images recorded at the image recording system are transmitted to the observation station by the narrow band telecommunications link using at least one of image data compression and image data reduction, the narrow band telecommunications link also transmitting remote control signals to the image recording system;
the improvement comprising:
an automated system status determining unit for continuously supervising the automated management of image transmission, and as a function of determined system status, including availability of system resources, variably and automatically controlling at least one of the image data compression, the image data reduction and the number of individual telecommunication channels employed in the transmission of the image data.
2. The improved image transmitting remote object examination device of claim 1, wherein parameters for the at least one of image data compression and image data reduction are set as a function of image content and image enlargement.
3. The improved image transmitting remote object examination device of claim 1, wherein a variable number of telecommunication channels serves as a transmission path, the number of channels being determined by the system status determining unit as a function of the determined system status, the system status determining unit also automatically compensating for channel duration.
4. The improved image transmitting remote object examination device of claim 1, wherein the image recording system comprises an automatic image focusing mechanism.
5. The improved image transmitting remote object examination device of claim 1, wherein the image recording system comprises an automatic brightness control mechanism.
6. The improved image transmitting remote object examination device of claim 1, wherein the observation station is for matching a previously recorded image with only the area in a subsequent image that is not contained in the previously recorded image.
7. The improved image transmitting remote object examination device of claim 1, wherein the observation station further comprises at least one of an optical display, an acoustic display and a mechanical force feedback display of system status information.
8. The improved image transmitting remote object examination device of claim 1, wherein the observation station further comprises a phantom image recording mechanism for reproducing a recorded image as a system operating parameter.
9. The improved image transmitting remote object examination device of claim 1, wherein images are periodically transmitted to and stored at the observation station, the images being selected based upon the probability that the images will be subsequently required by the system.
10. The improved image-transmitting remote object examination device of claim 1, wherein the observation station has a storage mechanism for archiving data for transmitted images.
11. The improved image-transmitting remote object examination device of claim 1, the improvement further comprising a mechanism for viewing image field areas in real time.
12. An image transmitting remote object examination device comprising:
(a) an image recording system disposed at a location of an object to be examined;
(b) an observation station located at a distance from the image recording system, the observation station for evaluating the image and remotely controlling the image recording system, the observation station comprising a mechanism for generating and displaying intermediate images between successively transmitted, partially overlapping individual images;
(c) a narrow-band telecommunications link interposed between the image recording system and the observation station, whereby images recorded at the image recording system are transmitted to the observation station by the narrow band telecommunications link using at least one of image data compression and image data reduction, the narrow band telecommunications link also transmitting remote control signals to the image recording system; and (d) an automated system status determining unit for continuously supervising the automated management of image transmission, and as a function of determined system status, including availability of system resources, variably and automatically controlling at least one of the image data compression, the image data reduction and the number of individual telecommunication channels employed in the transmission of the image data.
(a) an image recording system disposed at a location of an object to be examined;
(b) an observation station located at a distance from the image recording system, the observation station for evaluating the image and remotely controlling the image recording system, the observation station comprising a mechanism for generating and displaying intermediate images between successively transmitted, partially overlapping individual images;
(c) a narrow-band telecommunications link interposed between the image recording system and the observation station, whereby images recorded at the image recording system are transmitted to the observation station by the narrow band telecommunications link using at least one of image data compression and image data reduction, the narrow band telecommunications link also transmitting remote control signals to the image recording system; and (d) an automated system status determining unit for continuously supervising the automated management of image transmission, and as a function of determined system status, including availability of system resources, variably and automatically controlling at least one of the image data compression, the image data reduction and the number of individual telecommunication channels employed in the transmission of the image data.
13. An image transmitting remote object examination device comprising:
(a) an image recording system disposed at a location of an object to be examined;
(b) an observation station located at a distance from the image recording system, the observation station for evaluating the image and remotely controlling the image recording system;
(c) a narrow-band telecommunications link interposed between the image recording system and the observation station, whereby images recorded at the image recording system are transmitted to the observation station by the narrow band telecommunications link using at least one of image data compression and image data reduction, the narrow band telecommunications link also transmitting remote control signals to the image recording system;
(d) an automated system status determining unit for continuously supervising the automated management of image transmission, and as a function of determined system status, including availability of system resources, variably and automatically controlling at least one of the image data compression, the image data reduction and the number of individual telecommunication channels employed in the transmission of the image data; and (e) a display capable of representing an overall image of an object in which the current viewing field and selected areas of the object already examined are represented as respective marker areas.
(a) an image recording system disposed at a location of an object to be examined;
(b) an observation station located at a distance from the image recording system, the observation station for evaluating the image and remotely controlling the image recording system;
(c) a narrow-band telecommunications link interposed between the image recording system and the observation station, whereby images recorded at the image recording system are transmitted to the observation station by the narrow band telecommunications link using at least one of image data compression and image data reduction, the narrow band telecommunications link also transmitting remote control signals to the image recording system;
(d) an automated system status determining unit for continuously supervising the automated management of image transmission, and as a function of determined system status, including availability of system resources, variably and automatically controlling at least one of the image data compression, the image data reduction and the number of individual telecommunication channels employed in the transmission of the image data; and (e) a display capable of representing an overall image of an object in which the current viewing field and selected areas of the object already examined are represented as respective marker areas.
14. An image transmitting remote object examination device comprising;
(a) an image recording system disposed at a location of an object to be examined;
(b) an observation station located at a distance from the image recording system, the observation station for evaluating the image and remotely controlling the image recording system, the observation station having a storage mechanism for archiving data for transmitted images;
(c) a narrow-band telecommunications link interposed between the image recording system and the observation station, whereby images recorded at the image recording system are transmitted to the observation station by the narrow band telecommunications link using at least one of image data compression and image data reduction, the narrow band telecommunications link also transmitting remote control signals to the image recording system;
(d) an automated system status determining unit for continuously supervising the automated management of image transmission, and as a function of determined system status, including availability of system resources, variably and automatically controlling at least one of the image data compression, the image data reduction and the number of individual telecommunication channels employed in the transmission of the image data;
wherein the system is for automatically adjusting current images on the basis of archived data.
(a) an image recording system disposed at a location of an object to be examined;
(b) an observation station located at a distance from the image recording system, the observation station for evaluating the image and remotely controlling the image recording system, the observation station having a storage mechanism for archiving data for transmitted images;
(c) a narrow-band telecommunications link interposed between the image recording system and the observation station, whereby images recorded at the image recording system are transmitted to the observation station by the narrow band telecommunications link using at least one of image data compression and image data reduction, the narrow band telecommunications link also transmitting remote control signals to the image recording system;
(d) an automated system status determining unit for continuously supervising the automated management of image transmission, and as a function of determined system status, including availability of system resources, variably and automatically controlling at least one of the image data compression, the image data reduction and the number of individual telecommunication channels employed in the transmission of the image data;
wherein the system is for automatically adjusting current images on the basis of archived data.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE19633997.9 | 1996-08-23 | ||
| DE19633997A DE19633997C1 (en) | 1996-08-23 | 1996-08-23 | Remote image inspection facility for image transmission |
| PCT/EP1997/004473 WO1998008342A1 (en) | 1996-08-23 | 1997-08-16 | Picture-transmitting remote object inspection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2263151A1 CA2263151A1 (en) | 1998-02-26 |
| CA2263151C true CA2263151C (en) | 2004-12-07 |
Family
ID=7803435
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002263151A Expired - Fee Related CA2263151C (en) | 1996-08-23 | 1997-08-16 | Picture-transmitting remote object inspection system |
Country Status (8)
| Country | Link |
|---|---|
| US (1) | US6650703B1 (en) |
| EP (1) | EP0920780B1 (en) |
| JP (1) | JP2000516784A (en) |
| AT (1) | ATE290748T1 (en) |
| AU (1) | AU718452B2 (en) |
| CA (1) | CA2263151C (en) |
| DE (2) | DE19633997C1 (en) |
| WO (1) | WO1998008342A1 (en) |
Families Citing this family (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| ES2288760T3 (en) | 1996-04-25 | 2008-01-16 | Bioarray Solutions Ltd. | ELECTROCINETIC ASSEMBLY CONTROLLED BY LIGHT OF PARTICLES NEXT TO SURFACES. |
| AU9476798A (en) * | 1997-09-10 | 1999-03-29 | Bellsouth Corporation | Digital telepathology imaging system with bandwidth optimization and virtual focus control functions |
| US6606413B1 (en) | 1998-06-01 | 2003-08-12 | Trestle Acquisition Corp. | Compression packaged image transmission for telemicroscopy |
| US20040083085A1 (en) | 1998-06-01 | 2004-04-29 | Zeineh Jack A. | Integrated virtual slide and live microscope system |
| DE69925084T3 (en) * | 1998-10-14 | 2017-12-28 | Hamamatsu Photonics K.K. | MICROSCOPY |
| US6847729B1 (en) | 1999-04-21 | 2005-01-25 | Fairfield Imaging Limited | Microscopy |
| DE10026392A1 (en) * | 2000-05-27 | 2001-11-29 | Leica Microsystems | Method and arrangement for coding live images in microscopy |
| US9709559B2 (en) | 2000-06-21 | 2017-07-18 | Bioarray Solutions, Ltd. | Multianalyte molecular analysis using application-specific random particle arrays |
| ES2259666T3 (en) | 2000-06-21 | 2006-10-16 | Bioarray Solutions Ltd | MOLECULAR ANALYSIS OF MULTIPLE ANALYTICS USING SERIES OF RANDOM PARTICLES WITH APPLICATION SPECIFICITY. |
| DE10031746A1 (en) | 2000-06-29 | 2002-01-10 | Leica Microsystems | Method and arrangement for adjusting the lateral and temporal resolution of a microscope image |
| DE10056438A1 (en) * | 2000-11-14 | 2002-05-23 | Forschungszentrum Fuer Medizin | Endoscopic device system for medical applications, has supply unit integrated into probe, and has transmission unit for wireless transmission of image data to reception unit |
| JP2002202463A (en) * | 2000-12-27 | 2002-07-19 | Nikon Corp | Image display device, microscope system provided with the image display device and recording medium |
| DE10128532A1 (en) * | 2001-06-13 | 2003-01-02 | Siemens Ag | Data compression determination procedure e.g. for medical equipment, involves selecting a data compression procedure giving the highest compression factor |
| US7262063B2 (en) | 2001-06-21 | 2007-08-28 | Bio Array Solutions, Ltd. | Directed assembly of functional heterostructures |
| US20040233284A1 (en) * | 2001-10-09 | 2004-11-25 | Vehicle Enhancement Systems, Inc. | Apparatus and methods for providing surveillance data concerning a scene of interest to a user located at a remote location |
| EP2722395B1 (en) | 2001-10-15 | 2018-12-19 | Bioarray Solutions Ltd | Multiplexed analysis of polymorphic loci by concurrent interrogation and enzyme-mediated detection |
| WO2004047007A1 (en) | 2002-11-15 | 2004-06-03 | Bioarray Solutions, Ltd. | Analysis, secure access to, and transmission of array images |
| DE10357496A1 (en) | 2002-12-09 | 2004-07-01 | Carl Zeiss | Surgical operation microscope for use in operating theatre has binocular optical system with single large objective lens, peripheral lighting system and radio transmission of data |
| GB2398196B (en) | 2003-02-05 | 2005-06-01 | Fairfield Imaging Ltd | Microscope system and method |
| WO2005029705A2 (en) | 2003-09-18 | 2005-03-31 | Bioarray Solutions, Ltd. | Number coding for identification of subtypes of coded types of solid phase carriers |
| ES2375962T3 (en) | 2003-09-22 | 2012-03-07 | Bioarray Solutions Ltd | IMMOBILIZED SURFACE POLYELECTROLYTE WITH MULTIPLE FUNCTIONAL GROUPS ABLE TO JOIN COVALENTLY TO BIOMOLECULES. |
| US7136936B2 (en) | 2003-10-03 | 2006-11-14 | Asoka Usa Corporation | Method and system for virtual powerline local area networks |
| EP1692298A4 (en) | 2003-10-28 | 2008-08-13 | Bioarray Solutions Ltd | Optimization of gene expression analysis using immobilized capture probes |
| ES2533876T3 (en) | 2003-10-29 | 2015-04-15 | Bioarray Solutions Ltd | Multiplexed nucleic acid analysis by double stranded DNA fragmentation |
| US7363170B2 (en) * | 2004-07-09 | 2008-04-22 | Bio Array Solutions Ltd. | Transfusion registry network providing real-time interaction between users and providers of genetically characterized blood products |
| US7848889B2 (en) | 2004-08-02 | 2010-12-07 | Bioarray Solutions, Ltd. | Automated analysis of multiplexed probe-target interaction patterns: pattern matching and allele identification |
| US8486629B2 (en) | 2005-06-01 | 2013-07-16 | Bioarray Solutions, Ltd. | Creation of functionalized microparticle libraries by oligonucleotide ligation or elongation |
| CN101461026B (en) * | 2006-06-07 | 2012-01-18 | Fei公司 | Slider bearing for use with an apparatus comprising a vacuum chamber |
| JP4861364B2 (en) * | 2008-04-14 | 2012-01-25 | 株式会社インテック | Remote diagnostic device |
| DE102011087454A1 (en) * | 2011-11-30 | 2013-06-06 | Leica Microsystems (Schweiz) Ag | Optical device e.g. surgical microscope for used in medical technology, has electric line that is provided to transmit electric signals, and monitoring unit that is provided for monitoring signal transfer over electric line |
| JP6106967B2 (en) * | 2012-06-04 | 2017-04-05 | ソニー株式会社 | Information processing apparatus, information processing method, program, and image display apparatus |
| DE102015110305B3 (en) * | 2015-06-26 | 2016-12-15 | Leica Microsystems Cms Gmbh | Microscope with remote-controlled camera |
Family Cites Families (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0293083A3 (en) * | 1987-04-30 | 1990-07-18 | Corabi International Telemetrics, Inc. | Remote transmission diagnostic system |
| US5216596A (en) * | 1987-04-30 | 1993-06-01 | Corabi International Telemetrics, Inc. | Telepathology diagnostic network |
| US5331551A (en) * | 1989-10-02 | 1994-07-19 | Olympus Optical Co., Ltd. | Endoscope image recording system for compressing and recording endoscope image data |
| US5543939A (en) * | 1989-12-28 | 1996-08-06 | Massachusetts Institute Of Technology | Video telephone systems |
| US5619995A (en) * | 1991-11-12 | 1997-04-15 | Lobodzinski; Suave M. | Motion video transformation system and method |
| US5544649A (en) * | 1992-03-25 | 1996-08-13 | Cardiomedix, Inc. | Ambulatory patient health monitoring techniques utilizing interactive visual communication |
| WO1994003014A1 (en) * | 1992-07-24 | 1994-02-03 | Koz Mark C | Low power video security monitoring system |
| US5793969A (en) * | 1993-07-09 | 1998-08-11 | Neopath, Inc. | Network review and analysis of computer encoded slides |
| US5432871A (en) * | 1993-08-04 | 1995-07-11 | Universal Systems & Technology, Inc. | Systems and methods for interactive image data acquisition and compression |
| US5469353A (en) * | 1993-11-26 | 1995-11-21 | Access Radiology Corp. | Radiological image interpretation apparatus and method |
| GB9407548D0 (en) * | 1994-04-15 | 1994-06-08 | Harris David | Diagnostic method and apparatus |
| CA2125300C (en) * | 1994-05-11 | 1999-10-12 | Douglas J. Ballantyne | Method and apparatus for the electronic distribution of medical information and patient services |
| US5715326A (en) * | 1994-09-08 | 1998-02-03 | Neopath, Inc. | Cytological system illumination integrity checking apparatus and method |
| US5926210A (en) * | 1995-07-28 | 1999-07-20 | Kalatel, Inc. | Mobile, ground-based platform security system which transmits images that were taken prior to the generation of an input signal |
| US5704364A (en) * | 1995-11-08 | 1998-01-06 | Instromedix, Inc. | Concurrent medical patient data and voice communication method and apparatus |
| US6031930A (en) * | 1996-08-23 | 2000-02-29 | Bacus Research Laboratories, Inc. | Method and apparatus for testing a progression of neoplasia including cancer chemoprevention testing |
| US5987519A (en) * | 1996-09-20 | 1999-11-16 | Georgia Tech Research Corporation | Telemedicine system using voice video and data encapsulation and de-encapsulation for communicating medical information between central monitoring stations and remote patient monitoring stations |
| US6108027A (en) * | 1996-12-17 | 2000-08-22 | Netergy Networks, Inc. | Progressive still frame mode |
| US5993001A (en) * | 1997-06-05 | 1999-11-30 | Joslin Diabetes Center, Inc. | Stereoscopic imaging system for retinal examination with remote examination unit |
| US6014451A (en) * | 1997-10-17 | 2000-01-11 | Pioneer Hi-Bred International, Inc. | Remote imaging system for plant diagnosis |
-
1996
- 1996-08-23 DE DE19633997A patent/DE19633997C1/en not_active Expired - Fee Related
-
1997
- 1997-08-16 WO PCT/EP1997/004473 patent/WO1998008342A1/en active IP Right Grant
- 1997-08-16 EP EP97938912A patent/EP0920780B1/en not_active Expired - Lifetime
- 1997-08-16 CA CA002263151A patent/CA2263151C/en not_active Expired - Fee Related
- 1997-08-16 US US09/147,896 patent/US6650703B1/en not_active Expired - Fee Related
- 1997-08-16 AT AT97938912T patent/ATE290748T1/en not_active IP Right Cessation
- 1997-08-16 DE DE59712223T patent/DE59712223D1/en not_active Expired - Lifetime
- 1997-08-16 AU AU41188/97A patent/AU718452B2/en not_active Ceased
- 1997-08-16 JP JP10510389A patent/JP2000516784A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| AU4118897A (en) | 1998-03-06 |
| ATE290748T1 (en) | 2005-03-15 |
| JP2000516784A (en) | 2000-12-12 |
| EP0920780A1 (en) | 1999-06-09 |
| EP0920780B1 (en) | 2005-03-09 |
| DE59712223D1 (en) | 2005-04-14 |
| CA2263151A1 (en) | 1998-02-26 |
| WO1998008342A1 (en) | 1998-02-26 |
| US6650703B1 (en) | 2003-11-18 |
| DE19633997C1 (en) | 1998-03-26 |
| AU718452B2 (en) | 2000-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2263151C (en) | Picture-transmitting remote object inspection system | |
| US6646677B2 (en) | Image sensing control method and apparatus, image transmission control method, apparatus, and system, and storage means storing program that implements the method | |
| JP5030261B2 (en) | VIDEO DISTRIBUTION DEVICE, VIEWER DEVICE, VIDEO DISTRIBUTION DEVICE CONTROL METHOD, VIEWER DEVICE CONTROL METHOD, AND COMPUTER PROGRAM | |
| US6985178B1 (en) | Camera control system, image pick-up server, client, control method and storage medium therefor | |
| EP0738076B1 (en) | Camera control system and method of controlling same | |
| US8004561B2 (en) | Methods and systems for networked camera control | |
| US6954224B1 (en) | Camera control apparatus and method | |
| US20010026318A1 (en) | Communication apparatus, image processing apparatus, communication method, and image processing method | |
| US20020180878A1 (en) | Image capturing system, image capturing apparatus, and manual operating apparatus | |
| EP0893919B1 (en) | Camera control system | |
| US6707947B1 (en) | Frame switcher and method of switching, digital camera, and monitoring system | |
| US20020168177A1 (en) | Program recording system and program recording method | |
| US7562380B2 (en) | Internet camera system | |
| US20030225845A1 (en) | Method of providing multicast service and server employing the method | |
| US20140111871A1 (en) | Lens system and image pickup system including the same | |
| JPWO2003084225A1 (en) | Video relay apparatus, video distribution system, and video relay method | |
| JP2009200697A (en) | Image transmitter, field angle control method, image receiver, image display system, and image display method | |
| JPH08228340A (en) | Image selection display system | |
| KR20140111324A (en) | A method for video surveillance, a related system, a related surveillance server, and a related surveillance camera | |
| US6907137B2 (en) | Method and arrangement for adapting the lateral and temporal resolution of a microscope image | |
| JPH10322581A (en) | Image pickup device controller and image pickup system | |
| JP3441776B2 (en) | Image transmission system | |
| JPH10164562A (en) | Device and method for processing information, storage medium and information communication system | |
| JPH0638087A (en) | Remote camera apparatus | |
| JP2008236785A (en) | Image providing device, image providing method, and image providing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |