|Publication number||US8040373 B2|
|Application number||US 11/377,272|
|Publication date||Oct 18, 2011|
|Priority date||Mar 18, 2005|
|Also published as||EP1702556A1, EP1702556B1, US20060252988|
|Publication number||11377272, 377272, US 8040373 B2, US 8040373B2, US-B2-8040373, US8040373 B2, US8040373B2|
|Inventors||Daisuke Ayame, Kazunori Abe, Shinji Takeuchi|
|Original Assignee||Fujinon Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (29), Non-Patent Citations (2), Referenced by (2), Classifications (25), Legal Events (2)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present invention relates to an endoscope system apparatus, particularly relates to a constitution used in a medical field for forming and displaying a spectral image (image) comprising image information of an arbitrary selected wavelength region.
In recent years, in an electronic endoscope apparatus using a solid state image sensor, attention is attracted to spectroscopic imaging combined with a narrow band pass filter, that is, a narrow band filter incorporated electronic endoscope apparatus (Narrow Band Imaging-NBI) based on a prediction of a spectroscopic reflectance in the digesting organ (stomach mucosa or the like). According to the apparatus, three band pass filters of narrow (wavelength) bands are provided in place of a rotating filter of R (red), G (green), B (blue) of a face sequential type, and a spectral image is formed by successively outputting illuminating light by way of the narrow band path filters and processing three signals provided by the illuminating light while changing respective weights thereof similar to a case of R, G, B (RGB) signals. According to the spectral image, in the digesting organ of the stomach, the large intestine or the like, a fine structure which cannot be provided in a background art is extracted.
Meanwhile, it has been proposed to form a spectral image by operation processing based on the image signal provided by white light not by the face sequential type using the narrow band pass filters but by a simultaneous type for arranging a color filter of a small mosaic to a solid state image sensor as shown by JP-A-2003-93336 and Tokyo University Printing Association Foundation ‘Analysis and Evaluation of Digital Color Image’ by MIYAKE, yoichi (P148 through P153). According thereto, a relationship between respective color sensitivity characteristics of RGB which are formed into numerical value data and a spectroscopic characteristic of a specific narrow band pass filter which is formed into numerical value data is calculated as matrix data (coefficient set) and by operation of the matrix data and RGB signals, a spectral image signal provided by way of the narrow band pass filter is pseudonically provided. When the spectral image is formed by such an operation, it is not necessary to prepare a plurality of filters in correspondence with a desired band pass region, an interchanging arrangement thereof is dispensed with and therefore, large-sized formation of the apparatus is avoided and low cost formation thereof can be achieved.
Meanwhile, according to a spectral image extracting a specific fine structure or the like of an object, it is necessary to select a preferable wavelength region or adjusting the selected wavelength region and there is a case in which a necessary and sufficient spectral image cannot be provided in inspection by an endoscope in a limited time period. Further, it is necessary to observe and diagnose the spectral image in details by comparing with a normal color image and when a spectral image having an arbitrary wavelength region can be formed and displayed after inspection by the endoscope, an apparatus having an excellent way of use can be provided.
Further, in the operation processing of the spectral image by the endoscope apparatus, for example, color image signals of RGB constituting a basis thereof differs by a spectroscopic sensitivity characteristic including a kind of a color filter of an imaging element (solid state image sensor or the like), a kind of a light source, a spectroscopic sensitivity characteristic of an optical system member of the endoscope of a light guide or the like to pose a problem that such differences in the spectroscopic characteristics of the endoscope of the light source effect an influence on reproducibility on the same wavelength region. That is, there are CCDs constituting solid state image sensors of a complementally color type having color filters of Mg, Ye, Cy, G and a primary color type having color filters of RGB, further, even in the CCD of the same kind, the spectroscopic sensitivity characteristic differs by an individual difference.
Further, a spectroscopic characteristic of illuminating light differs by an aperture amount of a diaphragm blade in the light source apparatus owing to chromatic aberration of lenses, there is constituted a characteristic in which the more the light amount is reduced, the more the red color component is gradually cut from a long wavelength side to pose a problem that the reproducibility of the spectral image is deteriorated even by the spectroscopic characteristic of the illuminating light.
An object of an illustrative, non-limiting embodiment of the present invention is to provide an endoscope spectral image system apparatus capable of forming and displaying a spectral image of an arbitrary wavelength region after inspection by an endoscope, capable of forming a spectral image having excellent reproducibility in the same wavelength region even when a spectroscopic characteristic of an imaging element or an endoscope or a spectroscopic characteristic of a light source or illuminating light differs and having an excellent way of use.
The above object are accomplished with the following constitutions:
According to the above-described constitution, the spectroscopic characteristic information is supplied from the signal processor to the image recording apparatus along with the normal color image data, at the image recording apparatus, the matrix data (coefficient set) in correspondence with the spectroscopic characteristic information is read from the plurality of matrix data stored to the storing portion, and the spectral image is formed by the matrix operation based on the data. That is, the matrix data includes coefficients for calculating λ1, λ2, λ3 signals of wavelength narrow bands (components) by the matrix operation from, for example, RGB signals (may be other signals), or 61 of wavelength region parameters (coefficient sets p1 through p61) constituted by dividing a wavelength region from 400 nm to 700 nm by an interval of 5 nm, and a plurality of table data comprising 61 of the coefficient sets are prepared in accordance with the spectroscopic characteristic. Further, when the operator selects three wavelength regions λ1, λ2, λ3 (may be one wavelength region), λ1, λ2, λ3 signals are formed from matrix data (coefficient set) in correspondence with the three wavelength regions and the RGB signals output from DVP, DSP ort the like, the spectral image having excellent reproducibility is formed by the λ1, λ2, λ3 signals and displayed on a monitor or the like. That is, according to the image recording apparatus, not only the recorded normal image (stationary picture and dynamic picture) is reproduced and displayed, but also, based on the normal image, the spectral image (stationary picture and dynamic picture) in consideration of the spectroscopic characteristic of the endoscope (CCD) can be generated and displayed.
According to the constitution of the above (2), the matrix data in accordance with a difference of the spectroscopic characteristic of a Xenon lamp or a halogen lamp is read, according to the constitution of claim 3, the matrix data divided by, for example, 6 stages, in accordance with the diaphragm position (state) and in accordance with the spectroscopic characteristics of the 6 stages is read, the spectral image in accordance with the spectroscopic characteristics is formed and therefore, the reproducibility is further improved.
According to an exemplary embodiment of the endoscope apparatus of the invention, by holding the spectroscopic characteristic information along with the normal color image, after inspection by the endoscope, the spectral image of the arbitrary wavelength region can be formed and displayed and object image information useful for diagnosis or the like can be provided. Further, even when the spectroscopic characteristic of the imaging element or the endoscope taking the image of the normal color image, or the spectroscopic characteristic of the light source or the illuminating light differs, the spectral image having the excellent reproducibility which is not influenced by the differences of the spectroscopic characteristics can be formed and the apparatus having an excellent way of use can be provided.
The scope 10 is provided with CCD 18 constituting a solid state image sensor at a front end portion thereof, as the CCD 18, for example, a complementally color type having color filters of Mg (magenta), Ye (yellow), Cy (cyan), G (green) or a primary color type having color filters of RGB is used at an image taking face thereof. The CCD 18 is provided with a CCD driving circuit 19 for forming a drive pulse based on a synchronizing signal output from a timing generator (TG) 20 and is provided with a CDS/AGC (correlated double sampling/automatic gain control) circuit 21 for sampling and amplifying a picture image (image) signal input from the CCD 18, an A/D converter 22. Further, a microcomputer 24 for controlling various circuits in the scope 10 and communicating with a second microcomputer (42) at inside of the processor apparatus 12, and a memory (ROM or the like) 25 for storing a spectroscopic characteristic (spectroscopic characteristic in primary color type, complementary color type) of CCD 18, spectroscopic characteristic information of the scope 10 including spectroscopic characteristics or the like of an abject optical system, optical system members including a light guide and other identifying information are arranged. Further, the scope 10 is provided with an illuminating window 26 at a front end thereof, and the illuminating window 26 is connected to the light source apparatus 14 by a light guide 27.
On the other hand, the processor apparatus 12 is provided with a DVP (digital video processor) 30 for subjecting an image signal converted into digital to various image processing, and at the DVP 30, a Y/C signal constituted by a brightness (Y) signal and a chrominance [C (R-Y, B-Y)] signal is formed from an output signal of the CCD 18 and output. According to the embodiment, a normal image (dynamic picture and stationary picture) and a spectral image (dynamic picture and stationary picture) can selectively be formed and displayed, the DVP 30 is connected with a signal processing circuit 32 for forming a normal image by way of a switch 31 (at one terminal) for switching whether the normal image is formed or the spectral image is formed, the signal processing circuit 32 carries out a signal processing of character mix or the like for adding an image taking condition, patient information or the like to the image signal data. Other terminal of the switch 31 is arranged with a spectral image-forming circuit 34A for forming the spectral image at inside of the processor apparatus 12, a D/A converter 35 for inputting both outputs of the circuit 34A and the signal processing circuit 32, and an output of the D/A converter 35 is supplied to the monitor 15.
Further, the signal processing circuit 32 is arranged with an image memory 38 for temporarily holding a stationary picture, a packet generating circuit 39 for correlating the stationary picture and the spectroscopic characteristic information, and a network I/F (interface) 40 as a constitution for outputting the stationary picture to the image recording apparatus 16. Further, the processor apparatus 12 is provided with the second microcomputer 42 for controlling an inner circuit thereof and communicating with the first microcomputer 24, a third microcomputer 43 for carrying out similar processing, a memory 44 (ROM or the like) for storing operation information at inside of the processor apparatus 12, matrix data (Table) for forming the spectral image based on RGB signals, a serial I/F (interface) 45 for outputting a dynamic picture, the dynamic picture and the spectroscopic characteristic information are output from the serial I/F 45 and a packet for the stationary picture is output from network I/F 40. That is, scope side spectroscopic characteristic information (data) stored to the memory 25 of the scope 10 is transmitted from the first microcomputer 24 to the third microcomputer 43 by way of the second microcomputer 42, the stationary picture is added to image data at the packet generating circuit 39 and the dynamic picture is transmitted by the serial I/F 45. Therefore, the first microcomputer 24 through the third microcomputer 43 (and fourth microcomputer 50), the packet generating circuit 39 and the interfaces 40, 45 constitute information-outputting circuits. Further, the matrix data stored to the memory 44 is read by the second microcomputer 42 and is provided to the spectral image-forming circuit 34A.
Further, the light source apparatus 14 is provided with a light converging lens 48 for outputting illuminating light to the light guide 27, a diaphragm (blade) 49, a light source lamp (Xenon lamp or halogen lamp) 50, a lamp driving circuit 51 and a diaphragm position sensor 52 for detecting a drive diaphragm position of the diaphragm 49, and arranged with a memory (ROM or the like) 55 for storing information with regard to the fourth microcomputer 54, a kind of the light source lamp 50 or the like. Further, the fourth microcomputer 54 supplies the diaphragm position information (or spectroscopic characteristic information in correspondence with the diaphragm position), information of whether the light source lamp 50 is a Xenon lamp or a halogen lamp (or spectroscopic characteristic information in correspondence with the kind of lamp) to the second microcomputer 42, and the information is transmitted to the image recording apparatus 16 along with other spectroscopic characteristic information by being supplied to the third microcomputer 43.
Further, the image recording apparatus 16 is provided with a hard disk 61 for storing the image, a hard disk controller 62, CPU (or microcomputer) 63 for governing to control respective circuits, ROM (Read Only Memory) 64 for storing the matrix data for forming the spectral image from the RGB signals and a plurality of matrix data (table data) in correspondence with the spectroscopic characteristic information output from the processor apparatus 12, RAM (Readable Writable Memory) 65 for inputting to process data or the like, a spectral image-forming circuit 34B for forming the spectral image by using the read matrix data, a frame memory 67 for a monitor display processing and a D/A converter 68, and an output to the D/A converter 68 is supplied to the monitor 15.
The matrix data (one table) used in the matrix operation of the color space conversion processing circuit 71 and stored to the memory 44, the ROM 64 is as shown by Table 1, shown below.
The matrix data of Table 1 comprises 61 of wavelength region parameters (coefficient sets) p1 through p61 constituted by dividing, for example, a wavelength region from 400 nm through 700 nm by an interval of 5 nm, and the parameters p1 through p61 are constituted by coefficients kpr, kpg, kpb (p corresponds to p1 through p61) for matrix operation.
Further, at the color space conversion processing circuit 71, matrix operation of Equation 1, shown below, is carried out by the coefficient kpr, kpg, kpb, and the RGB signal output from the first color converting circuit 70.
That is, when, as λ1, λ2, λ3, for example, parameters p21 (center wavelength 50 nm), p45 (center wavelength 620 nm), p51 (center wavelength 650 nm) of Table 1 are selected, as coefficients (kpr, kpg, kpb), (−0.00119, 0.002346, 0.0016) of p21, (0.004022, 0.000068, −0.00097) of p45, (0.005152, −0.00192, 0.000088) of p51 may be substituted therefor.
Further, the color space conversion processing circuit 71 is provided with a mode selector 72 for selecting either of a spectral image (single color mode) of one wavelength region (narrow band region) and a spectral image (3 colors mode) comprising three wavelength regions, and an amplifying circuit 73 is connected to a post stage of the mode selector 72. The amplifying circuit 73 amplifies λ1, λ2, λ3 signals for forming the spectral image by respective gain values e1, e2, e3, and outputs amplified signals of e1×λ1, e2×λ2, e3×λ3. The amplifying circuit 73 is provided with a second color converting circuit 74 for inputting the signals of λ1, λ2, λ3 as amplified as Rs, Gs, Bs signals for carrying out a processing in correspondence with the RGB signals of the background art and converting the Rs, Gs, Bs signals into the Y/C signal.
The embodiment is constructed by the above-described constitution, first, according to the light source apparatus 14 of
At the DVP 30, various processing are carried out, and the Y/C signal comprising the brightness (Y) signal and the chrominance (R-Y, B-Y) signal is formed. An output of the DVP 30 is normally supplied to the signal processing circuit 32 by way of the switch 31, here, subjected to a predetermined processing, thereafter, supplied to the monitor 15 by way of the D/A converter 35 and the monitor is displayed with a color image of the normal object. Further, according to the embodiment, a spectral image signal can be formed by operating the spectral image-forming circuit 34A by the switch 31 and also the spectral image signal in this case is displayed on the monitor 15 by way of the D/A converter 35.
Next, operation in a case in which the stationary picture and the dynamic picture are recorded to the image recording apparatus 16 by recording operation of the scope 10 in reference to
Next, an explanation will be given of forming the spectral image at the spectral image-forming circuit 34B shown in
Further, when 3 colors mode is selected by the mode selector 72, the signals of λ1, λ2, λ3 are supplied to the amplifying circuit 73, further, when the single color mode is selected, any signal of λ1, λ2, λ3 is supplied thereto, amplified by the respective gains e1, e2, e3 to provide e1×λ1, e2×λ2, e3×λ3. The amplified signals output from the amplifying circuit 73 are supplied to the second color converting circuit 74 as the signals of Rs (=e1·λ1), Gs (=e2·λ2), Bs (=e3·λ3), further, when the single color mode is selected, any signal of λ1, λ2, λ3 (for example, when λ2 is selected, e2·λ2) is supplied to the second color converting circuit 74 as signals of Rs, Gs, Bs. At the second color converting circuit 74, the signals of λ1, λ2, λ3 as the Rs, Gs, Bs signals are converted into the Y/C signal (Y, Rs-Y, Bs-Y), and by supplying the Y/C signal to the monitor 15 by way of the D/A converter 68 (35), the spectral image is displayed on the monitor 15.
In this way, the spectral image displayed on the monitor 15 is constituted by color components of the wavelength regions shown in
Further, the spectral image provided by the image recording apparatus 16 can maintain the reproducibility in the same wavelength regions even when the color image provided by the scope 10 and the light source 14 kinds of which differ is held. That is, the excellent spectral image in which the reproducibility is not dispersed is provided when in the scope 10, the spectroscopic characteristic of CCD 18, the spectroscopic characteristic in consideration of the object optical system, the light guide or the like differ and even when at the light source apparatus 14, the spectroscopic characteristic differs by the difference of whether the light source lamp 50 is the Xenon lamp or the halogen lamp, or by the difference of the diaphragm position of the diaphragm 49, as mentioned later.
According to the example, the spectral image-forming circuit 34A is provided even at the processor apparatus 12, by selecting the wavelength regions of λ1, λ2, λ3 signals by operating the operation panel or the like from the processor apparatus 12, in carrying out observation, treatment by the scope 10, the spectral image can also be formed to be displayed on the monitor 15.
It will be apparent to those skilled in the art that various modifications and variations can be made to the described embodiments of the invention without departing from the spirit or scope of the invention. Thus, it is intended that the invention cover all modifications and variations of this invention consistent with the scope of the appended claims and their equivalents.
The present application claims foreign priority based on Japanese Patent Application No. JP2005-80426 filed Mar. 18 of 2005, the contents of which is incorporated herein by reference.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4885634||Aug 1, 1988||Dec 5, 1989||Olympus Optical Co., Ltd.||Endoscope apparatus capable of monochrome display with respect to specific wavelength regions in the visible region|
|US4926247||Oct 14, 1987||May 15, 1990||Olympus Optical Co., Ltd.||Color imaging apparatus including a means for electronically non-linearly expanding and compressing dynamic range of an image signal|
|US5331551||Jul 27, 1990||Jul 19, 1994||Olympus Optical Co., Ltd.||Endoscope image recording system for compressing and recording endoscope image data|
|US5408263||Jun 15, 1993||Apr 18, 1995||Olympus Optical Co., Ltd.||Electronic endoscope apparatus|
|US7393321 *||Jun 18, 2004||Jul 1, 2008||Olympus Corporation||Endoscope apparatus with solid-state pick-up|
|US7573499 *||Jun 22, 2004||Aug 11, 2009||Olympus Corporation||Endoscope apparatus for obtaining properly dimmed observation images|
|US7800656 *||Sep 21, 2010||Fujinon Corporation||Endoscope apparatus|
|US7850599 *||Dec 14, 2010||Fujinon Corporation||Endoscope apparatus|
|US7944466 *||May 17, 2011||Fujinon Corporation||Endoscope apparatus|
|US20030001952||Jun 11, 2002||Jan 2, 2003||Asahi Kogaku Kogyo Kabushiki Kaisha||Electronic endoscope with color adjustment function|
|US20040141054||Dec 22, 2003||Jul 22, 2004||Olympus Corporation||Image processing unit whose ability to process endoscopic image signal can be expanded, and endoscopic imaging system|
|US20040225222 *||May 8, 2003||Nov 11, 2004||Haishan Zeng||Real-time contemporaneous multimodal imaging and spectroscopy uses thereof|
|US20040257438 *||Jun 22, 2004||Dec 23, 2004||Olympus Corporation||Endoscope apparatus for obtaining properly dimmed observation images|
|US20050010081 *||Jun 18, 2004||Jan 13, 2005||Olympus Corporation||Endoscope apparatus|
|US20060197830 *||Feb 17, 2006||Sep 7, 2006||Fujinon Corporation||Endoscope apparatus|
|US20060197831 *||Feb 17, 2006||Sep 7, 2006||Fujinon Corporation||Endoscope apparatus|
|US20060198551 *||Mar 2, 2006||Sep 7, 2006||Fujinon Corporation||Endoscope apparatus|
|US20060211915 *||Mar 2, 2006||Sep 21, 2006||Fujinon Corporation||Endoscope apparatus|
|US20060253036 *||Mar 17, 2006||Nov 9, 2006||Fujinon Corporation||Endoscope system apparatus|
|JP2000014629A||Title not available|
|JP2000221417A||Title not available|
|JP2000350231A||Title not available|
|JP2001112712A||Title not available|
|JP2003093336A||Title not available|
|JP2004194993A||Title not available|
|JPH0690900A||Title not available|
|JPH06335449A||Title not available|
|JPS6417621A||Title not available|
|JPS6443227A||Title not available|
|1||Japanese Notification of Reasons for Refusal, Jul. 1, 2010.|
|2||Miyake, Tokyo University Printing Association Foundation, pp. 148-153.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8698884 *||Feb 6, 2013||Apr 15, 2014||Olympus Medical Systems Corp.||Light source apparatus|
|US9066676||Sep 28, 2011||Jun 30, 2015||Fujifilm Corporation||Endoscopic image display apparatus|
|U.S. Classification||348/71, 600/160, 600/476, 600/181|
|International Classification||A61B6/00, A61B1/06, A61B1/04|
|Cooperative Classification||A61B1/05, A61B1/00009, A61B1/045, A61B5/0084, H04N9/07, H04N5/2354, H04N2005/2255, H04N1/646, A61B5/0075, H04N9/67|
|European Classification||A61B1/05, H04N9/07, A61B1/045, H04N1/64D, H04N5/235L, H04N9/67, A61B1/00C1D, A61B5/00P7|
|Mar 17, 2006||AS||Assignment|
Owner name: FUJINON CORPORATION, JAPAN
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AYAME, DAISUKE;ABE, KAZUNORI;TAKEUCHI, SHINJI;REEL/FRAME:017697/0912
Effective date: 20060314
|Apr 1, 2015||FPAY||Fee payment|
Year of fee payment: 4