|Publication number||US6899675 B2|
|Application number||US 10/050,601|
|Publication date||May 31, 2005|
|Filing date||Jan 15, 2002|
|Priority date||Jan 15, 2002|
|Also published as||DE60324559D1, EP1476071A2, EP1477103A1, EP1520508A1, EP1520508B1, US20030135092, US20050143627, WO2003059159A2, WO2003059159A8|
|Publication number||050601, 10050601, US 6899675 B2, US 6899675B2, US-B2-6899675, US6899675 B2, US6899675B2|
|Inventors||Richard W. Cline, John J. P. Fengler|
|Original Assignee||Xillix Technologies Corp.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (102), Non-Patent Citations (3), Referenced by (79), Classifications (21), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present invention relates to medical imaging systems in general, and in particular to fluorescence endoscopy video systems.
Fluorescence endoscopy utilizes differences in the fluorescence response of normal tissue and tissue suspicious for early cancer as a tool in the detection and localization of such cancer. The fluorescing compounds or fluorophores that are excited during fluorescence endoscopy may be exogenously applied photo-active drugs that accumulate preferentially in suspicious tissues, or they may be the endogenous fluorophores that are present in all tissue. In the latter case, the fluorescence from the tissue is typically referred to as autofluorescence or native fluorescence. Tissue autofluorescence is typically due to fluorophores with absorption bands in the UV and blue portion of the visible spectrum and emission bands in the green to red portions of the visible spectrum. In tissue suspicious for early cancer, the green portion of the autofluorescence spectrum is significantly suppressed. Fluorescence endoscopy that is based on tissue autofluorescence utilizes this spectral difference to distinguish normal from suspicious tissue.
Since the concentration and/or quantum efficiency of the endogenous fluorophores in tissue is relatively low, the fluorescence emitted by these fluorophores is not typically visible to the naked eye. Fluorescence endoscopy is consequently performed by employing low light image sensors to acquire images of the fluorescing tissue through the endoscope. The images acquired by these sensors are most often encoded as video signals and displayed on a color video monitor. Representative fluorescence endoscopy video systems that image tissue autofluorescence are disclosed in U.S. Pat. No. 5,507,287, issued to Palcic et al.; U.S. Pat. No. 5,590,660, issued to MacAulay et al.; U.S. Pat. No. 5,827,190, issued to Palcic et al., U.S. patent application Ser. Nos. 09/615,965, and 09/905,642, all of which are herein incorporated by reference. Each of these is assigned to Xillix Technologies Corp. of Richmond, British Columbia, Canada, the assignee of the present application.
While the systems disclosed in the above-referenced patents are significant advances in the field of early cancer detection, improvements can be made. In particular, it is desirable to reduce the size, cost, weight, and complexity of the camera described for these systems by eliminating moving parts.
A fluorescence endoscopy video system in accordance with the present invention includes an endoscopic light source that is capable of operating in multiple modes to produce either white light, reflectance light, fluorescence excitation light, or fluorescence excitation light with reference reflectance light. An endoscope incorporates a light guide for transmitting light to the tissue under observation and includes either an imaging guide or a camera disposed in the insertion portion of the endoscope for receiving light from the tissue under observation. A compact camera with at least one low light imaging sensor that receives light from the tissue and is capable of operating in multiple imaging modes to acquire color or multi-channel fluorescence and reflectance images. The system further includes an image processor and system controller that digitizes, processes and encodes the image signals produced by the image sensor(s) as a color video signal and a color video monitor that displays the processed video images.
The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
As shown in
A processor/controller 64 controls the multi-mode camera 100 and the light source 52, and produces video signals that are displayed on a video monitor 66. The processor/controller 64 communicates with the multi-mode camera 100 with wires or other signal carrying devices that are routed within the endoscope. Alternatively, communication between the processor/controller 64 and the camera 100 can be conducted over a wireless link.
The light from the arc lamp 70 is coupled to a light guide 54 of the endoscope 60 through appropriate optics 74, 76, and 78 for light collection, spectral filtering and focusing respectively. The light from the arc lamp is spectrally filtered by one of a number of optical filters 76A, 76B, 76C . . . that operate to pass or reject desired wavelengths of light in accordance with the operating mode of the system. As used herein, “wavelength” is to be interpreted broadly to include not only a single wavelength, but a range of wavelengths as well.
An intensity control 80 that adjusts the amount of light transmitted along the light path is positioned at an appropriate location between the arc lamp 70 and the endoscope light guide 54. The intensity control 80 adjusts the amount of light that is coupled to the light guide 54. In addition, a shutter mechanism 82 may be positioned in the same optical path in order to block any of the light from the lamp from reaching the light guide. A controller 86 operates an actuator 77 that moves the filters 76A, 76B or 76C into and out of the light path. The controller 86 also controls the position of the intensity control 80 and the operation of the shutter mechanism 82.
The transmission characteristics of filters 76A, 76B, 76C, . . . , the characteristics of the actuator 77 mechanism, and the time available for motion of the filters 76A, 76B, 76C, . . . , into and out of the light path, depend on the mode of operation required for use with the various camera embodiments. The requirements fall into two classes. If the light source shown in
A light source 52A of a second class is illustrated in
The transmission characteristics of light source filters, the characteristics of the filter actuator mechanism, and the time available for motion of the filters into and out of the light path, for the two different classes of light sources are described in more detail below in the context of the various camera embodiments.
Because fluorescence endoscopy is generally used in conjunction with white light endoscopy, each of the various embodiments of the multi-mode camera 100 described below may be used both for color and fluorescence/reflectance and/or fluorescence/fluorescence imaging. These camera embodiments particularly lend themselves to incorporation within a fluorescence video endoscope due to their compactness and their ability to be implemented with no moving parts.
In a first embodiment, shown in
The low light image sensor 104 preferably comprises a charge coupled device with charge carrier multiplication (of the same type as the Texas Instruments TC253 or the Marconi Technologies CCD65), electron beam charge coupled device (EBCCD), intensified charge coupled device (ICCD), charge injection device (CID), charge modulation device (CMD), complementary metal oxide semiconductor image sensor (CMOS) or charge coupled device (CCD) type sensor. The monochrome image sensor 102 is preferably a CCD or a CMOS image sensor.
An alternative configuration of the camera 100B is shown in FIG. 4C. All aspects of this embodiment of this camera 110C are similar to the camera 100B shown in
The processor/controller 64 as shown in
Based on operator input, the processor/controller 64 also provides control functions for the fluorescence endoscopy video system. These control functions include providing control signals that control the camera gain in all imaging modes, coordinating the imaging modes of the camera and light source, and providing a light level control signal for the light source.
The reason that two separate images in different wavelength bands are acquired in fluorescence imaging modes of the fluorescence endoscopy video systems described herein, and the nature of the fluorescence/reflectance and fluorescence/fluorescence imaging, will now be explained. It is known that the intensity of the autofluorescence at certain wavelengths changes as tissues become increasingly abnormal (i.e. as they progress from normal to frank cancer). When visualizing images formed from such a band of wavelengths of autofluorescence, however, it is not easy to distinguish between those changes in the signal strength that are due to pathology and those that are due to imaging geometry and shadows. A second fluorescence image acquired in a band of wavelengths in which the image signal is not significantly affected by tissue pathology, utilized for fluorescence/fluorescence imaging, or a reflected light image acquired in a band of wavelengths in which the image signal is not significantly affected by tissue pathology consisting of light that has undergone scattering within the tissue (known as diffuse reflectance), utilized for fluorescence/reflectance imaging, may be used as a reference signal with which the signal strength of the first fluorescence image can be “normalized”. Such normalization is described in two patents previously incorporated herein by reference: U.S. Pat. No. 5,507,287, issued to Palcic et al. describes fluorescence/fluorescence imaging and U.S. Pat. No. 5,590,660, issued to MacAulay et al. describes fluorescence/reflectance imaging.
One technique for performing the normalization is to assign each of the two image signals a different display color, e.g., by supplying the image signals to different color inputs of a color video monitor. When displayed on a color video monitor, the two images are effectively combined to form a single image, the combined color of which represents the relative strengths of the signals from the two images. Since light originating from fluorescence within tissue and diffuse reflectance light which has undergone scattering within the tissue are both emitted from the tissue with a similar spatial distribution of intensities, the color of a combined image is independent of the absolute strength of the separate image signals, and will not change as a result of changes in the distance or angle of the endoscope 60 to the tissue sample 58, or changes in other imaging geometry factors. If, however, there is a change in the shape of the autofluorescence spectrum of the observed tissue that gives rise to a change in the relative strength of the two image signals, such a change will be represented as a change in the color of the displayed image. Another technique for performing the normalization is to calculate the ratio of the pixel intensities at each location in the two images. A new image can then be created wherein each pixel has an intensity and color related to the ratio computed. The new image can then be displayed by supplying it to a color video monitor.
The mixture of colors with which normal tissue and tissue suspicious for early cancer are displayed depends on the gain applied to each of the two separate image signals. There is an optimal gain ratio for which tissue suspicious for early cancer in a fluorescence image will appear as a distinctly different color than normal tissue. This gain ratio is said to provide the operator with the best combination of sensitivity (ability to detect suspect tissue) and specificity (ability to discriminate correctly). If the gain applied to the reference image signal is too high compared to the gain applied to the fluorescence image signal, the number of tissue areas that appear suspicious, but whose pathology turns out to be normal, increases. Conversely, if the relative gain applied to the reference image signal is too low, sensitivity decreases and suspect tissue will appear like normal tissue. For optimal system performance, therefore, the ratio of the gains applied to the image signals must be maintained at all times. The control of the gain ratio is described in two patent applications previously incorporated herein by reference: U.S. patent application Ser. Nos. 09/615,965, and 09/905,642.
In vivo spectroscopy has been used to determine which differences in tissue autofluorescence and reflectance spectra have a pathological basis. The properties of these spectra determine the particular wavelength bands of autofluorescence and reflected light required for the fluorescence/reflectance imaging mode, or the particular two wavelength bands of autofluorescence required for fluorescence/fluorescence imaging mode. Since the properties of the spectra depend on the tissue type, the wavelengths of the important autofluorescence band(s) may depend on the type of tissue being imaged. The specifications of the optical filters described below are a consequence of these spectral characteristics, and are chosen to be optimal for the tissues to be imaged.
As indicated above, the filters in the light source and camera should be optimized for the imaging mode of the camera, the type of tissue to be examined and/or the type of pre-cancerous tissue to be detected. Although all of the filters described below can be made to order using standard, commercially available components, the appropriate wavelength range of transmission and degree of blocking outside of the desired transmission range for the described fluorescence endoscopy images are important to the proper operation of the system. The importance of other issues in the specification of such filters, such as the fluorescence properties of the filter materials and the proper use of anti-reflection coatings, are taken to be understood.
The operation of the preferred embodiment of the fluorescence endoscopy video system will now be described. The cameras 100A as shown in
The processor/controller 64 also protects the sensitive low light image sensor 104 during color imaging by decreasing the gain of the amplification stage of the sensor. The light reflected by the tissue 58 is collected and transmitted by the endoscope image guide 56 to the camera where it is projected through beamsplitter 106 onto the monochrome image sensor 102, or the light is directly projected through the camera beamsplitter 106 onto the monochrome image sensor 102 if the sensor is located within the insertion portion of the endoscope. The image projected during each of red, green, and blue illuminations is transduced by the monochrome image sensor 102 and the resulting image signals are transmitted to the processor/controller 64.
Based on the brightness of the images captured, the processor/controller 64 provides a control signal to the multi-mode light source 52 to adjust the intensity control 80 and thereby adjust the level of light output by the endoscope light guide 54. The processor/controller 64 may also send a control signal to the camera 100A, 100B or 100C to adjust the gain of the monochrome image sensor 102.
The processor/controller 64 interpolates the images acquired during sequential periods of red, green, and blue illumination to create a complete color image during all time periods, and encodes that color image as video signals. The video signals are connected to color video monitor 66 for display of the color image. All of the imaging operations occur at analog video display rates (30 frames per second for NTSC format and 25 frames per second for PAL format).
When switching to the fluorescence/reflectance imaging mode, the processor/controller 64 provides a control signal to the multi-mode light source 52 to indicate that it should be operating in fluorescence/reflectance mode. In response to this signal, the light source filter wheel 79 stops rotating and the light source 52 selects and positions the appropriate blue optical filter 79A continuously into the optical path between the arc lamp 70 and the endoscope light guide 54. This change from sequentially changing filters to a static filter occurs in a period of approximately one second. Filter 79A transmits only those wavelengths of light that will induce the tissue 58 under examination to fluoresce. All other wavelengths of light are substantially blocked as described above. The filtered light is then projected into the endoscope light guide 54 and transmitted to the tip of the endoscope 60 to illuminate the tissue 58.
As part of setting the system in the fluorescence/reflectance mode, the processor/controller 64 also increases the gain of the amplification stage of the low light image sensor 104. The fluorescence emitted and excitation light reflected by the tissue 58 are either collected by the endoscope image guide 56 and projected through the camera beamsplitter 106 onto the low light image sensor 104 and the image sensor 102, or are collected and directly projected through the camera beamsplitter 106 onto the low light image sensor 104 and the image sensor 102 at the insertion tip of the endoscope 60. Spectral filter 118 limits the light transmitted to the low light image sensor 104 to either green or red autofluorescence light only and substantially blocks the light in the excitation wavelength band. The autofluorescence image is transduced by the low light image sensor 104. The reference reflected excitation light image is transduced by the monochrome image sensor 102 and the resulting image signals are transmitted to the processor/controller 64.
Based on the brightness of the transduced images, the processor/controller 64 may provide a control signal to the multi-mode light source 52 to adjust the intensity control 80 and thereby adjust the level of light delivered to the endoscope 60. The processor/controller 64 may also send control signals to the cameras 100A, 100B or 100C to adjust the gains of the low light image sensor 104 and the monochrome image sensor 102, in order to maintain constant image brightness while keeping the relative gain constant.
After being processed, the images from the two sensors are encoded as video signals by processor/controller 64. The fluorescence/reflectance image is displayed by applying the video signals to different color inputs on the color video monitor 66.
In order for the combined image to have optimal clinical meaning, for a given proportion of fluorescence to reference light signals emitted by the tissue and received by the system, a consistent proportion must also exist between the processed image signals that are displayed on the video monitor. This implies that the (light) signal response of the fluorescence endoscopy video system is calibrated. The calibration technique is described in two patent applications previously incorporated herein by reference: U.S. patent application Ser. Nos. 09/615,965, and 09/905,642.
The cameras 100A, 100B, 100C can be operated in a variation of the fluorescence/reflectance mode to simultaneously obtain fluorescence images and reflectance images with red, green, and blue illumination. The operation of the system is similar to that described previously for color imaging, so only the points of difference from the color imaging mode will be described.
In this variation of the fluorescence/reflectance mode, instead of changing from sequential red, green, and blue illumination to static blue illumination when switching from color imaging to fluorescence/reflectance imaging, the multi-mode light source 52 provides the same sequential illumination utilized in the color imaging mode, for all imaging modes. Capture and display of the light reflected by the tissue is similar to that described previously for the color imaging mode. However, in addition to the reflectance images captured in that mode, the gain of the amplification stage of the low light image sensor 104 is adjusted to a value that makes it possible to capture autofluorescence images during blue illumination. During red and green illumination, the gain of amplification stage of the low light sensor is decreased to protect the sensor while the image sensor 102 captures reflectance images.
In this modified fluorescence/reflectance mode, the camera captures both reflectance and fluorescence images during the blue illumination period, in addition to reflected light images during the red and green illumination periods. As for the color imaging mode, the reflectance images are interpolated and displayed on the corresponding red, green and blue channels of a color video monitor to produce a color image. Like the previously described fluorescence/reflectance mode, a fluorescence/reflectance image is produced by overlaying the fluorescence image and one or more of the reflectance images displayed in different colors on a color video monitor.
Since individual reflectance and fluorescence images are concurrently captured, both a color image and a fluorescence/reflectance image can be displayed simultaneously on the color video monitor. In this case, there is no need to utilize a separate color imaging mode. Alternatively, as described for the previous version of fluorescence/reflectance operation, only the fluorescence/reflectance image may be displayed during fluorescence/reflectance imaging and a color image displayed solely in the color imaging mode.
Yet another embodiment of this invention will now be described. All points of similarity with the first embodiment will be assumed understood and only points that differ will be described.
In this second embodiment, all aspects of the fluorescence endoscopy video system are similar to those of the first embodiment except for the camera and the light source. A camera 100D for this embodiment of a system is as shown in FIG. 7A. It differs from the cameras 100A, 100B or 100C as described above in that all imaging modes utilize a single, low light color image sensor 103 (preferably a color CCD with charge carrier multiplication such as the Texas Instruments TC252) and that no beamsplitter is required. Alternatively, the color image sensor 103 may be a three-CCD with charge carrier multiplication color image sensor assembly, a color CCD, a three-CCD color image sensor assembly, a color CMOS image sensor, or a three-CMOS color image sensor assembly.
Each of the pixel elements on the low light color sensor 103 is covered by an integrated filter, typically red, green or blue. These filters define the wavelength bands of fluorescence and reflectance light that reach the individual pixel elements. Such mosaic filters typically have considerable overlap between the red, green, and blue passbands, which can lead to considerable crosstalk when imaging dim autofluorescence light in the presence of intense reflected excitation light. Therefore, a separate filter 118 is provided to reduce the intensity of reflected excitation light to the same level as that of the autofluorescence light and, at the same time, pass autofluorescence light.
In this embodiment, the primary fluorescence and reference images are projected onto the same image sensor 103, but, because of the individual filters placed over each pixel, these different images are detected by separate sensor pixels. As a result, individual primary fluorescence and reference image signals can be produced by processor/controller 64 from the single CCD image signal.
The operation of a system based on camera 100D of
In the color imaging mode, the processor/controller 64 provides a control signal to the multimode light source 52 that it should be in white light mode. The light source selects and positions the appropriate optical filter 76A into the optical path between the arc lamp 70 and endoscope light guide 54. Given the presence of filter 118 in cameras 100D, 100E which have reduced transmission for excitation light at blue wavelengths, the light source filter 76A should incorporate reduced transmission at red and green wavelengths to obtain a balanced color image at image sensor 103 with the proper proportions of red, green, and blue components.
Image signals from the color low light sensor 103 are processed by processor/controller 64. Standard techniques are utilized to produce a color image from a single color sensor: the image signals from pixels having the same filter characteristics are interpolated by processor/controller 64 to produce an image signal, related to the pass band of each element of the mosaic filter (e.g. red, green, and blue), at every pixel location. The resulting multiple images, which when combined produce a color image, are encoded by processor/controller 64 as video signals. The color image is displayed by connecting the video signals to the appropriate inputs of color video monitor 66.
Processor/controller 64 also maintains the overall image brightness at a set level by monitoring the brightness of the image signal at each pixel and adjusting the intensity of the light source output and camera amplifier gains according to a programmed algorithm.
When switching to the fluorescence/fluorescence imaging mode, processor/controller 64 provides a control signal to the multi-mode light source 52 to indicate that it should be in fluorescence/fluorescence mode. The light source 52 moves light source filter 76B into position in the light beam. Filter 76B transmits excitation light and blocks the transmission of light at the green and red fluorescence detection wavelengths, as described below. The characteristics of light source fluorescence excitation filter 76B and excitation filter 118, along with the mosaic filter elements on the color sensor 103, are such that the intensity of blue light at the color sensor is less than the intensities of red and green autofluorescence at the sensor, and are such that the ratio of the intensity of red autofluorescence to the intensity of green autofluorescence at the color sensor 103 has the appropriate value for optimal differentiation between normal and abnormal tissue. The fluorescence images are processed, as previously described for color imaging, by processor/controller 64 to produce separate images corresponding to each of the pass bands of the mosaic filter (e.g. red, green, and blue). These separate images are encoded as video signals by processor/controller 64. A composite fluorescence/fluorescence image is displayed on the color video monitor 66 by applying the video signals from red and green pass bands of the mosaic filter to different color inputs of the monitor.
When switching to the fluorescence/reflectance imaging mode, processor/controller 64 provides a control signal to the multi-mode light source 52 to indicate that it should be in fluorescence/reflectance mode. The light source 52 moves light source filter 76C into position in the light beam. Filter 76C transmits both excitation light and reference light and blocks the transmission of light at fluorescence detection wavelengths, as described below. The characteristics of the light source filter 76C for fluorescence excitation and the reflectance illumination and the camera filter 118, along with the mosaic filter on the color sensor 103, as detailed below, are such that the intensity of reflected excitation light at the color sensor is comparable to the intensity of autofluorescence at the sensor, and should be such that the ratio of the intensity of autofluorescence to the intensity of reflected reference light at the color sensor 103 has the appropriate value. The fluorescence and reflectance images are processed, as previously described for color imaging, by processor/controller 64 to produce separate images corresponding to each of the pass bands of the mosaic filter (e.g. red, green, and blue). These separate images are encoded as video signals by processor/controller 64. A composite fluorescence/reflectance image is displayed on color video monitor 66 by applying the video signals from the appropriate mosaic filter pass bands (as discussed below) to different color inputs of the monitor.
As indicated above, the filters in the light source and camera should be optimized for the imaging mode of the camera, the type of tissue to be examined and/or the type of pre-cancerous tissue to be detected. Although all of the filters described below can be made to order using standard, commercially available components, the appropriate wavelength range of transmission and degree of blocking outside of the desired transmission range for the described fluorescence endoscopy images modes are important to the proper operation of the system. The importance of other issues in the specification of such filters such as the fluorescence properties of the filter materials and the proper use of anti-reflection coatings are taken to be understood.
As discussed above, the filters in the light source and camera should be optimized for the imaging mode of the camera, the type of tissue to be examined and/or the type of pre-cancerous tissue to be detected, based on in vivo spectroscopy measurements. The preferred filter characteristics for use in the fluorescence endoscopy video systems with a camera of the type shown in
The fluorescence endoscopy video systems described in the above embodiments have been optimized for imaging endogenous tissue fluorescence. They are not limited to this application, however, and may also be used for photo-dynamic diagnosis (PDD) applications. As mentioned above, PDD applications utilize photo-active drugs that preferentially accumulate in tissues suspicious for early cancer. Since effective versions of such drugs are currently in development stages, this invention does not specify the filter characteristics that are optimized for such drugs. With the appropriate light source and camera filter combinations, however, a fluorescence endoscopy video system operating in either fluorescence/fluorescence or fluorescence/reflectance imaging mode as described herein may be used to image the fluorescence from such drugs.
As will be appreciated, each of the embodiments of a camera for a fluorescence endoscopy video system described above, due to their simplicity, naturally lend themselves to miniaturization and implementation in a fluorescence video endoscope, with the camera being incorporated into the insertion portion of the endoscope. The cameras can be utilized for both color imaging and fluorescence imaging, and in their most compact form contain no moving parts.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US3971068||Aug 22, 1975||Jul 20, 1976||The United States Of America As Represented By The Secretary Of The Navy||Image processing system|
|US4115812||Aug 15, 1977||Sep 19, 1978||Hitachi, Ltd.||Automatic gain control circuit|
|US4149190||Oct 17, 1977||Apr 10, 1979||Xerox Corporation||Automatic gain control for video amplifier|
|US4200801||Mar 28, 1979||Apr 29, 1980||The United States Of America As Represented By The United States Department Of Energy||Portable spotter for fluorescent contaminants on surfaces|
|US4378571||Jul 6, 1981||Mar 29, 1983||Xerox Corporation||Serial analog video processor for charge coupled device imagers|
|US4449535 *||Mar 25, 1982||May 22, 1984||Compagnie Industrielle Des Lasers Cilas Alcatel||Apparatus for measuring in situ the state of oxidation-reduction of a living organ|
|US4532918||Oct 7, 1983||Aug 6, 1985||Welch Allyn Inc.||Endoscope signal level control|
|US4556057||Mar 11, 1983||Dec 3, 1985||Hamamatsu Tv Co., Ltd.||Cancer diagnosis device utilizing laser beam pulses|
|US4638365||Jan 22, 1985||Jan 20, 1987||Canon Kabushiki Kaisha||Image sensing device|
|US4768513||Mar 30, 1987||Sep 6, 1988||Agency Of Industrial Science And Technology||Method and device for measuring and processing light|
|US4786813||Oct 22, 1985||Nov 22, 1988||Hightech Network Sci Ab||Fluorescence imaging system|
|US4821117||Nov 10, 1987||Apr 11, 1989||Kabushiki Kaisha Toshiba||Endoscopic system for producing fluorescent and visible images|
|US4837625||Feb 12, 1988||Jun 6, 1989||Sgs-Thomson Microelectronics S.A.||Automatic gain control device for video signals|
|US4930516||Apr 25, 1988||Jun 5, 1990||Alfano Robert R||Method for detecting cancerous tissue using visible native luminescence|
|US4951135||Dec 28, 1988||Aug 21, 1990||Olympus Optical Co., Ltd.||Electronic-type endoscope system having capability of setting AGC variation region|
|US4954897||Sep 13, 1989||Sep 4, 1990||Nikon Corporation||Electronic still camera system with automatic gain control of image signal amplifier before image signal recording|
|US5007408||Nov 7, 1989||Apr 16, 1991||Olympus Optical Co., Ltd.||Endoscope light source apparatus|
|US5034888||Oct 11, 1990||Jul 23, 1991||Olympus Optical Co., Ltd.||Electronic endoscope apparatus having different image processing characteristics for a moving image and a still image|
|US5134662||Mar 4, 1991||Jul 28, 1992||Cell Analysis Systems, Inc.||Dual color camera microscope and methodology for cell staining and analysis|
|US5165079||Jan 24, 1990||Nov 17, 1992||Linotype-Hell Ag||Optical color-splitter arrangement|
|US5214503||Jan 31, 1992||May 25, 1993||The United States Of America As Represented By The Secretary Of The Army||Color night vision camera system|
|US5225883||Jun 5, 1991||Jul 6, 1993||The Babcock & Wilcox Company||Video temperature monitor|
|US5255087||Dec 27, 1991||Oct 19, 1993||Olympus Optical Co., Ltd.||Imaging apparatus and endoscope apparatus using the same|
|US5365057||Jul 2, 1993||Nov 15, 1994||Litton Systems, Inc.||Light-weight night vision device|
|US5371355||Jul 30, 1993||Dec 6, 1994||Litton Systems, Inc.||Night vision device with separable modular image intensifier assembly|
|US5377686||Oct 11, 1991||Jan 3, 1995||The University Of Connecticut||Apparatus for detecting leakage from vascular tissue|
|US5408263||Jun 15, 1993||Apr 18, 1995||Olympus Optical Co., Ltd.||Electronic endoscope apparatus|
|US5410363||Dec 8, 1992||Apr 25, 1995||Lightwave Communications, Inc.||Automatic gain control device for transmitting video signals between two locations by use of a known reference pulse during vertical blanking period so as to control the gain of the video signals at the second location|
|US5419323||Nov 17, 1989||May 30, 1995||Massachusetts Institute Of Technology||Method for laser induced fluorescence of tissue|
|US5420628||Jun 3, 1994||May 30, 1995||Research Development Foundation||Video densitometer with determination of color composition|
|US5421337||Mar 29, 1994||Jun 6, 1995||Massachusetts Institute Of Technology||Spectral diagnosis of diseased tissue|
|US5424841||May 28, 1993||Jun 13, 1995||Molecular Dynamics||Apparatus for measuring spatial distribution of fluorescence on a substrate|
|US5430476||Jun 11, 1993||Jul 4, 1995||Richard Wolf Gmbh||Device for supplying light to endoscopes|
|US5485203||Aug 5, 1992||Jan 16, 1996||Olympus Optical Co., Ltd.||Color misregistration easing system which corrects on a pixel or block basis only when necessary|
|US5507287||Apr 27, 1995||Apr 16, 1996||Xillix Technologies Corporation||Endoscopic imaging system for diseased tissue|
|US5585846||Jun 27, 1995||Dec 17, 1996||Samsung Electronics Co., Ltd.||Image signal processing circuit in a digital camera having gain and gamma control|
|US5590660||Mar 28, 1994||Jan 7, 1997||Xillix Technologies Corp.||Apparatus and method for imaging diseased tissue using integrated autofluorescence|
|US5596654||Feb 16, 1994||Jan 21, 1997||Fuji Photo Film Co., Ltd.||Method of determining desired image signal range based on histogram data|
|US5646680||Sep 12, 1995||Jul 8, 1997||Olympus Optical Co., Ltd.||Endoscope system having a switch forcibly set to display video signals not passed through outer peripheral apparatus|
|US5647368||Feb 28, 1996||Jul 15, 1997||Xillix Technologies Corp.||Imaging system for detecting diseased tissue using native fluorsecence in the gastrointestinal and respiratory tract|
|US5749830||Oct 27, 1994||May 12, 1998||Olympus Optical Co., Ltd.||Fluorescent endoscope apparatus|
|US5772580||Feb 27, 1996||Jun 30, 1998||Asahi Kogaku Kogyo Kabushiki Kaisha||Biological fluorescence diagnostic apparatus with distinct pickup cameras|
|US5827190||Aug 19, 1996||Oct 27, 1998||Xillix Technologies Corp.||Endoscope having an integrated CCD sensor|
|US5891016||Oct 30, 1996||Apr 6, 1999||Asahi Kogaku Kogyo Kabushiki Kaisha||Fluorescence endoscope having an exciting light filter and a fluorescence filter|
|US5986271||Jul 3, 1997||Nov 16, 1999||Lazarev; Victor||Fluorescence imaging system|
|US6002137||Feb 13, 1998||Dec 14, 1999||Fuji Photo Film Co., Ltd.||Fluorescence detecting system|
|US6008889||Apr 16, 1997||Dec 28, 1999||Zeng; Haishan||Spectrometer system for diagnosis of skin disease|
|US6021344||Dec 3, 1997||Feb 1, 2000||Derma Technologies, Inc.||Fluorescence scope system for dermatologic diagnosis|
|US6028622 *||Apr 23, 1998||Feb 22, 2000||Olympus Optical Co., Ltd.||Observation apparatus for endoscopes|
|US6059720||Mar 5, 1998||May 9, 2000||Asahi Kogaku Kogyo Kabushiki Kaisha||Endoscope system with amplification of fluorescent image|
|US6069689||Apr 14, 1998||May 30, 2000||Derma Technologies, Inc.||Apparatus and methods relating to optical systems for diagnosis of skin diseases|
|US6070096 *||Feb 19, 1998||May 30, 2000||Fuji Photo Film Co., Ltd.||Fluorescence detecting apparatus|
|US6099466||Nov 10, 1997||Aug 8, 2000||Asahi Kogaku Kogyo Kabushiki Kaisha||Fluorescence diagnosis endoscope system|
|US6120435||May 28, 1998||Sep 19, 2000||Olympus Optical Co., Ltd.||Endoscope system in which operation switch sets designed to function and be handled same way are included in endoscope and image processing apparatus respectively|
|US6148227||Jun 16, 1998||Nov 14, 2000||Richard Wolf Gmbh||Diagnosis apparatus for the picture providing recording of fluorescing biological tissue regions|
|US6161035||Apr 29, 1998||Dec 12, 2000||Asahi Kogaku Kogyo Kabushiki Kaisha||Fluorescence diagnostic apparatus|
|US6192267||Mar 21, 1995||Feb 20, 2001||Scherninski Francois||Endoscopic or fiberscopic imaging device using infrared fluorescence|
|US6212425||Sep 26, 1996||Apr 3, 2001||Karl Storz Gmbh & Co., Kg||Apparatus for photodynamic diagnosis|
|US6280378||May 28, 1999||Aug 28, 2001||Fuji Photo Film Co., Ltd.||Fluorescence endoscope|
|US6293911||Nov 19, 1997||Sep 25, 2001||Olympus Optical Co., Ltd.||Fluorescent endoscope system enabling simultaneous normal light observation and fluorescence observation in infrared spectrum|
|US6364829||Jul 28, 1999||Apr 2, 2002||Newton Laboratories, Inc.||Autofluorescence imaging system for endoscopy|
|US6422994||Sep 15, 1998||Jul 23, 2002||Olympus Optical Co., Ltd.||Fluorescent diagnostic system and method providing color discrimination enhancement|
|US6603552||Dec 22, 1999||Aug 5, 2003||Xillix Technologies Corp.||Portable system for detecting skin abnormalities based on characteristic autofluorescence|
|DE19535114A1||Sep 21, 1995||Mar 28, 1996||Asahi Optical Co Ltd||Endoscope with illumination source of multiple wavelengths|
|DE19608027A1||Mar 1, 1996||Sep 5, 1996||Asahi Optical Co Ltd||Biological fluorescence diagnosis device for medical use|
|EP0512965A1||May 5, 1992||Nov 11, 1992||Xillix Technologies Corporation||Endoscopic imaging system for diseased tissue|
|EP0774685A2||Nov 11, 1996||May 21, 1997||Kodak Limited||Photographic materials for use in redox amplification and process|
|EP0792618A1||Jan 31, 1997||Sep 3, 1997||Xillix Technologies Corporation||Imaging system for detecting diseased tissue using native fluorescence in the gastrointestinal and respiratory tract|
|FR2671405A1||Title not available|
|JPH1189789A||Title not available|
|JPH07155285A||Title not available|
|JPH07155286A||Title not available|
|JPH07155290A||Title not available|
|JPH07155291A||Title not available|
|JPH07155292A||Title not available|
|JPH07204156A||Title not available|
|JPH07222712A||Title not available|
|JPH07250804A||Title not available|
|JPH07250812A||Title not available|
|JPH08224208A||Title not available|
|JPH08224209A||Title not available|
|JPH08224210A||Title not available|
|JPH08224240A||Title not available|
|JPH08252218A||Title not available|
|JPH10127563A||Title not available|
|JPH10151104A||Title not available|
|JPH10201700A||Title not available|
|JPH10225426A||Title not available|
|JPH10243915A||Title not available|
|JPH10243920A||Title not available|
|JPH10308114A||Title not available|
|JPH10309281A||Title not available|
|JPH10309282A||Title not available|
|JPH10328129A||Title not available|
|JPH11104059A||Title not available|
|JPH11104060A||Title not available|
|JPH11104061A||Title not available|
|JPH11104070A||Title not available|
|JPH11113839A||Title not available|
|JPH11155812A||Title not available|
|JPS60246733A||Title not available|
|JPS61159936A||Title not available|
|1||Alfano, R.R., et al., "Fluorescence Spectra From Cancerous and Normal Human Breast and Lung Tissues," IEEE Journal of Quantum Electronics. QE-23(10):1806-1811, 1987.|
|2||Andersson-Engles, S. et al., "Tissue Diagnostics Using Laser Induced Fluorescence," Ber. Bunsenges Physical Chemistry (93):335-342, 1989.|
|3||Hung, J., et al., "Autofluorescence of Normal and Malignant Bronchial Tissue," Lasers in Surgery and Medicine (11):99-105, 1991.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7713265||Dec 22, 2006||May 11, 2010||Ethicon Endo-Surgery, Inc.||Apparatus and method for medically treating a tattoo|
|US7925333||Aug 28, 2007||Apr 12, 2011||Ethicon Endo-Surgery, Inc.||Medical device including scanned beam unit with operational control features|
|US7929151 *||Jan 11, 2008||Apr 19, 2011||Carestream Health, Inc.||Intra-oral camera for diagnostic and cosmetic imaging|
|US7982776||Jul 13, 2007||Jul 19, 2011||Ethicon Endo-Surgery, Inc.||SBI motion artifact removal apparatus and method|
|US7983739||Aug 27, 2007||Jul 19, 2011||Ethicon Endo-Surgery, Inc.||Position tracking and control for a scanning assembly|
|US7995045||Aug 9, 2011||Ethicon Endo-Surgery, Inc.||Combined SBI and conventional image processor|
|US8050520||Nov 1, 2011||Ethicon Endo-Surgery, Inc.||Method for creating a pixel image from sampled data of a scanned beam imager|
|US8075308||Dec 13, 2011||Carestream Health, Inc.||Intra-oral camera for diagnostic and cosmetic imaging|
|US8078265||Jul 11, 2006||Dec 13, 2011||The General Hospital Corporation||Systems and methods for generating fluorescent light images|
|US8144410||Nov 3, 2008||Mar 27, 2012||General Electric Company||Multimodal optical imaging|
|US8144411||Nov 3, 2008||Mar 27, 2012||General Electric Company||Multimodal optical imaging|
|US8160678||Jun 18, 2007||Apr 17, 2012||Ethicon Endo-Surgery, Inc.||Methods and devices for repairing damaged or diseased tissue using a scanning beam assembly|
|US8185176||Apr 26, 2006||May 22, 2012||Novadaq Technologies, Inc.||Method and apparatus for vasculature visualization with applications in neurosurgery and neurology|
|US8187189||May 29, 2012||The Invention Science Fund I, Llc||Imaging via blood vessels|
|US8216214||Mar 12, 2007||Jul 10, 2012||Ethicon Endo-Surgery, Inc.||Power modulation of a scanning beam for imaging, therapy, and/or diagnosis|
|US8273015||May 16, 2007||Sep 25, 2012||Ethicon Endo-Surgery, Inc.||Methods for imaging the anatomy with an anatomically secured scanner assembly|
|US8306600||Jul 20, 2009||Nov 6, 2012||Carl Zeiss Meditec Ag||Filter set for observing fluorescence radiation in biological tissue|
|US8332014||Apr 25, 2008||Dec 11, 2012||Ethicon Endo-Surgery, Inc.||Scanned beam device and method using same which measures the reflectance of patient tissue|
|US8423127||Apr 16, 2013||The General Hospital Corporation||Systems and methods for generating fluorescent light images|
|US8498695 *||Dec 26, 2007||Jul 30, 2013||Novadaq Technologies Inc.||Imaging system with a single color image sensor for simultaneous fluorescence and color video endoscopy|
|US8626271||Apr 13, 2007||Jan 7, 2014||Ethicon Endo-Surgery, Inc.||System and method using fluorescence to examine within a patient's anatomy|
|US8630698 *||Apr 26, 2006||Jan 14, 2014||Novadaq Technologies, Inc.||Filter for use with imaging endoscopes|
|US8647605||Apr 26, 2006||Feb 11, 2014||Novadaq Technologies, Inc.||Real time imaging during solid organ transplant|
|US8659651||Dec 16, 2009||Feb 25, 2014||Carl Zeiss Meditec Ag||Surgical microscope for observing an infrared fluorescence and corresponding method|
|US8730601||Aug 9, 2011||May 20, 2014||Carl Zeiss Meditec Ag||Fluorescence observation system and set of filters|
|US8801606||Jan 9, 2007||Aug 12, 2014||Ethicon Endo-Surgery, Inc.||Method of in vivo monitoring using an imaging system including scanned beam imaging unit|
|US8825140||Feb 15, 2012||Sep 2, 2014||Xenogen Corporation||Imaging system|
|US8952312||May 14, 2012||Feb 10, 2015||Olive Medical Corporation||Image sensor for endoscopic use|
|US9072445||Jan 24, 2008||Jul 7, 2015||Lifeguard Surgical Systems Inc.||Common bile duct surgical imaging system|
|US9079762||Sep 22, 2006||Jul 14, 2015||Ethicon Endo-Surgery, Inc.||Micro-electromechanical device|
|US9123602||May 14, 2012||Sep 1, 2015||Olive Medical Corporation||Pixel array area optimization using stacking scheme for hybrid image sensor with minimal vertical interconnects|
|US9125552||Jul 31, 2007||Sep 8, 2015||Ethicon Endo-Surgery, Inc.||Optical scanning module and means for attaching the module to medical instruments for introducing the module into the anatomy|
|US9143746 *||Jun 28, 2013||Sep 22, 2015||Novadaq Technologies, Inc.||Imaging system with a single color image sensor for simultaneous fluorescence and color video endoscopy|
|US9153609||May 14, 2012||Oct 6, 2015||Olive Medical Corporation||Image sensor with tolerance optimizing interconnects|
|US9226645||Dec 20, 2010||Jan 5, 2016||The General Hospital Corporation||System and method for normalized diffuse emission epi-illumination imaging and normalized diffuse emission transillumination imaging|
|US9307893||Dec 27, 2012||Apr 12, 2016||Cook Medical Technologies Llc||Space-optimized visualization catheter with camera train holder in a catheter with off-centered lumens|
|US9343489||Feb 8, 2015||May 17, 2016||DePuy Synthes Products, Inc.||Image sensor for endoscopic use|
|US9386909||Mar 29, 2013||Jul 12, 2016||Novadaq Technologies Inc.||System and method for deposition and removal of an optical element on an endoscope objective|
|US9421280||Feb 10, 2014||Aug 23, 2016||Novadaq Technologies Inc.||Real time imaging during solid organ transplant|
|US9462234||Jul 26, 2013||Oct 4, 2016||DePuy Synthes Products, Inc.||Camera system with minimal area monolithic CMOS image sensor|
|US20050154319 *||Dec 10, 2004||Jul 14, 2005||Xillix Technologies Corporation||Fluorescence endoscopy video systems with no moving parts in the camera|
|US20060226336 *||Mar 23, 2006||Oct 12, 2006||Tir Systems Ltd.||Apparatus and method for collecting and detecting light emitted by a lighting apparatus|
|US20060239921 *||Apr 26, 2005||Oct 26, 2006||Novadaq Technologies Inc.||Real time vascular imaging during solid organ transplant|
|US20070015963 *||Apr 26, 2006||Jan 18, 2007||Xillix Technologies Corp.||Filter for use with imaging endoscopes|
|US20070122345 *||Sep 1, 2006||May 31, 2007||University Of Rochester Medical Center||Intraoperative determination of nerve location|
|US20070255097 *||Apr 28, 2006||Nov 1, 2007||Searete Llc, A Limited Liability Corporation Of The State Of Delaware||Imaging via blood vessels|
|US20080015446 *||Jul 11, 2006||Jan 17, 2008||Umar Mahmood||Systems and methods for generating fluorescent light images|
|US20080161744 *||Sep 6, 2007||Jul 3, 2008||University Of Rochester Medical Center||Pre-And Intra-Operative Localization of Penile Sentinel Nodes|
|US20080239070 *||Dec 26, 2007||Oct 2, 2008||Novadaq Technologies Inc.||Imaging system with a single color image sensor for simultaneous fluorescence and color video endoscopy|
|US20080312540 *||Dec 8, 2005||Dec 18, 2008||Vasilis Ntziachristos||System and Method for Normalized Flourescence or Bioluminescence Imaging|
|US20090181339 *||Jan 11, 2008||Jul 16, 2009||Rongguang Liang||Intra-oral camera for diagnostic and cosmetic imaging|
|US20090192349 *||Jul 30, 2009||Lifeguard Surgical Systems||Common bile duct surgical imaging system|
|US20090203993 *||Apr 26, 2006||Aug 13, 2009||Novadaq Technologies Inc.||Real time imagining during solid organ transplant|
|US20090203994 *||Apr 26, 2006||Aug 13, 2009||Novadaq Technologies Inc.||Method and apparatus for vasculature visualization with applications in neurosurgery and neurology|
|US20100044583 *||Jul 20, 2009||Feb 25, 2010||Carl Zeiss Surgical Gmbh||Filter set for observing fluorescence radiation in biological tissue|
|US20100106013 *||Jan 30, 2008||Apr 29, 2010||Olympus Corporation||Fluorescence observation device for organism tissue|
|US20100110572 *||Nov 3, 2008||May 6, 2010||General Electric Company||Multimodal optical imaging|
|US20100111353 *||Nov 3, 2008||May 6, 2010||General Electric Company||Multimodal optical imaging|
|US20100182418 *||Jul 22, 2010||Carl Zeiss Surgical Gmbh||Surgical microscope for observing an infrared fluorescence and corresponding method|
|US20100222673 *||Aug 10, 2006||Sep 2, 2010||Novadaq Technologies Inc.||Intra-operative head and neck nerve mapping|
|US20100261965 *||Apr 6, 2010||Oct 14, 2010||Gerd Beck||Apparatus For Fluorescence Diagnosis|
|US20110087111 *||Apr 14, 2011||The General Hospital Corporation||System and Method for Normalized Diffuse Emission Epi-illumination Imaging and Normalized Diffuse Emission Transillumination Imaging|
|US20110221880 *||Sep 15, 2011||Rongguang Liang||Intra-oral camera for diagnostic and cosmetic imaging|
|US20130286176 *||Jun 28, 2013||Oct 31, 2013||Novadaq Technologies Inc.|
|US20140155717 *||Feb 4, 2014||Jun 5, 2014||Fujifilm Corporation||Endoscope system, processor device, and method for operating endoscope system|
|DE102008034008A1 *||Jul 21, 2008||Jan 28, 2010||Carl Zeiss Surgical Gmbh||Filtersatz zur Beobachtung von Fluoreszenzstrahlung in biologischem Gewebe|
|DE102008034008B4 *||Jul 21, 2008||Jul 1, 2010||Carl Zeiss Surgical Gmbh||Filtersatz zur Beobachtung von Fluoreszenzstrahlung in biologischem Gewebe|
|DE102008062650A1 *||Dec 17, 2008||Jul 15, 2010||Carl Zeiss Surgical Gmbh||Operationsmikroskop zur Beobachtung einer Infrarot-Fluoreszenz und Verfahren hierzu|
|DE102009018141A1||Apr 8, 2009||Oct 21, 2010||Karl Storz Gmbh & Co. Kg||Vorrichtung zur Fluoreszenzdiagnose|
|DE102010033825A1 *||Aug 9, 2010||Feb 9, 2012||Carl Zeiss Meditec Ag||Filter set for use in fluorescence tracking system to carry out fluorescence observation of object, has illuminating light filter whose transmission characteristic is sum of two partial characteristics|
|DE102011016138A1 *||Mar 30, 2011||Oct 4, 2012||Karl Storz Gmbh & Co. Kg||Vorrichtung zur Fluoreszenzdiagnose|
|DE102011101934A1 *||May 18, 2011||Nov 22, 2012||Christian-Albrechts-Universität Zu Kiel||Optical biofilm sensor for determining biological properties of biofilm on settlement substrate, has set of photoconductive glass fibers arranged under substrate and around light source, where light source is arranged in sensor head|
|DE102013108189A1 *||Jul 31, 2013||Feb 5, 2015||Endress + Hauser Conducta Gesellschaft für Mess- und Regeltechnik mbH + Co. KG||Anordnung zur optischen Messung einer Prozessgröße und Messgerät umfassend eine solche|
|DE102013111368A1 *||Oct 15, 2013||Apr 16, 2015||Karl Storz Gmbh & Co. Kg||Endoskopische, exoskopische oder mikroskopische Vorrichtung zur Fluoreszenzdiagnose|
|DE102014008243A1 *||May 27, 2014||Dec 3, 2015||Carl Zeiss Meditec Ag||Fluoreszenbeobachtungssystem und optisches Filtersystem hierfür|
|DE102014016850A1 *||Nov 13, 2014||May 19, 2016||Carl Zeiss Meditec Ag||Optisches System zur Fluoreszenzbeobachtung|
|EP2238886A1||Apr 7, 2010||Oct 13, 2010||Karl Storz GmbH & Co. KG||Device for fluorescence diagnosis|
|EP2888989A1||Dec 22, 2014||Jul 1, 2015||Karl Storz Imaging, Inc.||Switching between white light imaging and excitation light imaging leaving last video frame displayed|
|WO2007016790A1 *||Aug 10, 2006||Feb 15, 2007||Novadaq Technologies, Inc.||Intra-operative head & neck nerve mapping|
|U.S. Classification||600/160, 600/109|
|International Classification||A61B1/00, A61B5/00, A61B1/04, A61B6/00, A61B1/06|
|Cooperative Classification||A61B1/0638, A61B5/0084, A61B1/00186, A61B1/0646, A61B5/0071, A61B1/043, A61B1/00009|
|European Classification||A61B1/04F, A61B1/06J, A61B1/00C1D, A61B5/00P12B, A61B1/00S5, A61B1/06M, A61B5/00P5|
|Apr 30, 2002||AS||Assignment|
Owner name: XILLIX TECHNOLOGIES CORPORATION, CANADA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CLINE, RICHARD W.;FENGLER, JOHN J.P.;REEL/FRAME:012847/0354
Effective date: 20020422
|Oct 11, 2005||CC||Certificate of correction|
|May 16, 2007||AS||Assignment|
Owner name: NOVADAQ TECHNOLOGIES INC., CANADA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:XILLIX TECHNOLOGIES CORP.;REEL/FRAME:019297/0691
Effective date: 20070502
|Dec 1, 2008||FPAY||Fee payment|
Year of fee payment: 4
|Dec 21, 2012||FPAY||Fee payment|
Year of fee payment: 8
|Dec 21, 2012||SULP||Surcharge for late payment|
Year of fee payment: 7