WO1997013135A2 - Method and system for multiple wavelength microscopy image analysis - Google Patents
Method and system for multiple wavelength microscopy image analysis Download PDFInfo
- Publication number
- WO1997013135A2 WO1997013135A2 PCT/US1996/014995 US9614995W WO9713135A2 WO 1997013135 A2 WO1997013135 A2 WO 1997013135A2 US 9614995 W US9614995 W US 9614995W WO 9713135 A2 WO9713135 A2 WO 9713135A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- stain
- wavelengths
- stained
- absorbance
- stains
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000004458 analytical method Methods 0.000 title claims abstract description 18
- 238000001000 micrograph Methods 0.000 title claims description 8
- 238000013500 data storage Methods 0.000 claims abstract description 17
- 238000002835 absorbance Methods 0.000 claims description 43
- 238000003384 imaging method Methods 0.000 claims description 8
- 230000004075 alteration Effects 0.000 claims description 7
- 230000003287 optical effect Effects 0.000 claims description 7
- 230000001419 dependent effect Effects 0.000 claims description 4
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 claims 3
- 210000001519 tissue Anatomy 0.000 description 46
- WZUVPPKBWHMQCE-UHFFFAOYSA-N Haematoxylin Chemical compound C12=CC(O)=C(O)C=C2CC2(O)C1C1=CC=C(O)C(O)=C1OC2 WZUVPPKBWHMQCE-UHFFFAOYSA-N 0.000 description 26
- 238000000354 decomposition reaction Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 230000011218 segmentation Effects 0.000 description 9
- WLDHEUZGFKACJH-UHFFFAOYSA-K amaranth Chemical compound [Na+].[Na+].[Na+].C12=CC=C(S([O-])(=O)=O)C=C2C=C(S([O-])(=O)=O)C(O)=C1N=NC1=CC=C(S([O-])(=O)=O)C2=CC=CC=C12 WLDHEUZGFKACJH-UHFFFAOYSA-K 0.000 description 8
- 238000000862 absorption spectrum Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000010191 image analysis Methods 0.000 description 6
- 238000010186 staining Methods 0.000 description 6
- 238000003860 storage Methods 0.000 description 6
- 238000013459 approach Methods 0.000 description 5
- 230000009977 dual effect Effects 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 210000004027 cell Anatomy 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000003595 spectral effect Effects 0.000 description 4
- 238000010521 absorption reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 239000012620 biological material Substances 0.000 description 2
- 210000000805 cytoplasm Anatomy 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- GZCWLCBFPRFLKL-UHFFFAOYSA-N 1-prop-2-ynoxypropan-2-ol Chemical compound CC(O)COCC#C GZCWLCBFPRFLKL-UHFFFAOYSA-N 0.000 description 1
- 102000013563 Acid Phosphatase Human genes 0.000 description 1
- 108010051457 Acid Phosphatase Proteins 0.000 description 1
- CIWBSHSKHKDKBQ-JLAZNSOCSA-N Ascorbic acid Chemical compound OC[C@H](O)[C@H]1OC(=O)C(O)=C1O CIWBSHSKHKDKBQ-JLAZNSOCSA-N 0.000 description 1
- RZSYLLSAWYUBPE-UHFFFAOYSA-L Fast green FCF Chemical compound [Na+].[Na+].C=1C=C(C(=C2C=CC(C=C2)=[N+](CC)CC=2C=C(C=CC=2)S([O-])(=O)=O)C=2C(=CC(O)=CC=2)S([O-])(=O)=O)C=CC=1N(CC)CC1=CC=CC(S([O-])(=O)=O)=C1 RZSYLLSAWYUBPE-UHFFFAOYSA-L 0.000 description 1
- 206010060862 Prostate cancer Diseases 0.000 description 1
- 208000000236 Prostatic Neoplasms Diseases 0.000 description 1
- 238000010420 art technique Methods 0.000 description 1
- 235000013405 beer Nutrition 0.000 description 1
- 210000003855 cell nucleus Anatomy 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000002380 cytological effect Effects 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000012760 immunocytochemical staining Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 210000004940 nucleus Anatomy 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 210000002307 prostate Anatomy 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N1/00—Sampling; Preparing specimens for investigation
- G01N1/28—Preparing specimens for investigation including physical details of (bio-)chemical methods covered elsewhere, e.g. G01N33/50, C12Q
- G01N1/30—Staining; Impregnating ; Fixation; Dehydration; Multistep processes for preparing samples of tissue, cell or nucleic acid material and the like for analysis
- G01N1/31—Apparatus therefor
- G01N1/312—Apparatus therefor for samples mounted on planar substrates
Abstract
A method and system for segmenting images of samples stained with two, three, or more stains accurately and reliably into multiple single-color images which represent the mass density of the stain at each position in the sample. The system includes a computer-controlled microscope and CCD camera, and at least one computer which includes data storage media for storing image data and which runs software for controlling image acquisition and analysis. To analyze tissue stained with N stains, images must be obtained at N different wavelengths, from multiply stained and singly stained samples. Relative stain mass densities are calculated with the use of the ratio of absorptivities measured at different wavelengths. The theoretical and practical basis for implementing the method with two, three, or more stains is presented. A method for selecting the wavelengths which provide optimal image accuracy and resolution is also provided.
Description
METHOD AND SYSTEM FOR
MULTIPLE WAVELENGTH MICROSCOPY IMAGE ANALYSIS MICROFICHE APPENDICES.
A copy of the source code used in an example of the preferred embodiment of the invention is attached hereto as microfiche Appendix A which has 1 page and 31 frames and microfiche Appendix B which has 1 page and 44 frames. The code in Appendix A can be run on a PC-486DX running at 33MHz. The code in Appendix B can be run on a SUN workstation. This source code represents one element of one preferred
embodiment of the invention. It should be understood that the inventive concepts could be implemented in ways other than those shown in the microfiche appendix without departing from the inventive concept.
BACKGROUND OF THE INVENTION
FIELD OF THE INVENTION
This invention relates to the field of analysis of microscopy images. In particular, it relates to the analysis of microscopy images which are stained with several stains of different colors. Different colored stains are used to mark elements in the microscopically imaged material which have distinct structures and/or functions. Imaging of several stains simultaneously permits investigation of the
interrelationship between different structural and functional elements. Moreover, automated imaging makes possible the quantification of the density of each stain, and hence of the corresponding stained element. The inventive method was developed particularly for the evaluation of histological and cytological specimens, but could also be used to evaluate structures in samples of non-biological materials. THE BACKGROUND ART
The conventional approach for analyzing stained tissue is inspection of a section of the tissue under a
microscope by a trained pathologist. However, it is
difficult to obtain quantitative results with this method, and variability may exist between different observers, or between the same observer on different occasions. Therefore, methods for automated analysis of stained tissue sections is particularly desirable. Image analysis can provide
quantitative information about the geometry (e.g., size and shape) of cells or other structures, density of staining, distribution of features within an image, etc.
Biochemical or immunocytochemical staining can be used to enhance the contrast of a structure or substance to which stain is attached. Measurement of stain density is relatively simple in a monochromic image. However, if several different stains are used, this determination is more difficult.
If the absorption bands of different stains do not overlap, the stains can be imaged independently by
appropriate choice of narrow-band filters. This is known as the Exclusive Filtering Technique, and has been used by
Bacus, et al. Am. J. Clin. Pathol. Vol. 90, pp. 223-239, 1988. This method, however, is dependent on the use of stains with non-overlapping absorption bands, and only a limited number of suitable stain combinations are available. This method can seldom be used with more than two stains simultaneously due to overlap of absorption bands.
Another approach is to use a color-space transformation model to separate stains, in images acquired with a three-channel color video camera. [e.g. Umbaugh et al., IEEE Engr. Med. Biol. Magazine, Dec. 1989, pp. 43-52; MacAuley et al., Anal, and Quant. Cytol. and Histol. Vol. II, No. 1, pp. 53-58, 1989.] Three images are acquired (one red, one green, and one blue) after filtering of the image with broad-band filters. User-defined thresholds in color space are used to separate the different colored stains. However, this method has the drawbacks that spatial resolution is low, no more than three stains can be used, and the use of broad-
band filters in the red, green and blue ranges may not be optimal for separating certain combinations of stains.
Principal component analysis has been suggested as a method for determining component spectral curves from the spectrum of a mixture of unknown components [Kawata et al., J. Opt. Soc. Am. A, Vol. 4, No. 11, pp. 2101-2106, 1987.] With this method, it is not necessary that spatial and spectral information about the components be known, but it is necessary to capture 13 images with 13 different narrow bandpass filters. This is the case even if only two
components are being analyzed. The component spectrum is estimated with uncertainty, so quantitative analysis of a single component's spatial or spectral properties is not possible.
Prior art techniques may be satisfactory for two or possibly three stains, with certain restrictions on the absorption spectra of the stains. However, there is no method which reliably separates two or more stains with good spatial resolution and without limitations on the absorption spectra of the stains to be used. The present invention overcomes these limitations.
BRIEF SUMMARY AND OBJECTS OF THE INVENTION
The present invention is a method and system for performing analysis of multiply stained microscopy images. The preferred embodiment of the system includes a computer-controlled microscope and camera, software for controlling digitization and storage of images, and for decomposing multiply stained images to generate multiple single-stain images, and at least one computer on which said software is run. Several stains are used on the same sample so that the relationship between differently stained components can be seen. Color segmentation is performed to quantify the areas labelled with each stain. The color segmentation method used in the invention is based on differences of absorptivity spectra of stains (as measured at several different
wavelengths) and can be used to decompose images including two, three or more stains.
The primary objectives of the present invention are:
1) to provide efficient separation of two or more different stains in a single sample.
2) to provide good spatial resolution of multiple single-stain images in multiply stained samples.
3) to provide a method for determining the optimum
wavelengths for decomposing multiply stained images.
4) to provide a method and system for quantitative
analysis of multiply stained images.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram of equipment used in acquisition and processing of images of stained tissue sections.
FIG. 2 is a flow chart diagram outlining the major steps in data acquisition and analysis.
FIG. 3 is a flow chart diagram of the image
acquisition procedure.
FIG. 4 is a flow chart diagram of the batch
processing procedure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
Theory of Operation
The inventive method and system are based on the following theory. Let Gk(x,y) be the light intensity at location x,y in the image formed by passing light of
wavelength λk through a stained sample. Let GOk(x,y) be the light intensity at location x,y in the image formed by passing light through a region without stain. According to Beer's Law,
where there are N stain components, mik is the absorptivity of the ith stain component at wavelength λk, ci(x,y) is the concentration of the ith stain component at location (x,y), and ti(x,y) is the thickness of the ith stain component at location (x,y).
If Ak is defined to be the absorbance of the sample at wavelength λk, and δi, the mass density of stain component i, is defined as ci (x,y) · ti (x, y) then
If pairs of images (one with stain and one without stain) are acquired at N different wavelengths λk (k =
1...N), N linear equations of the following form are
obtained:
If matrices A, D, and M are defined as:
A = M · D
It should be noted that indices for array elements are given in the form (column, row). That is, mik is in row k and column i.
If |M | ≠ 0, then
for i = 1 N, where
and so forth.
The densities of the N different stain components can be determined from EQN. 5 if the N absorbance images Ak and NxN absorptivity values mik (i= 1...N, k= 1..N) are available. According to Equation 2, the N absorbance images Ak can be obtained from N image pairs Gk and G0k measured at the N different wavelengths. The absorptivity values, mik, are more difficult to determine because the absorption spectrum of a stain in a tissue sample is usually different from the absorption spectrum of the stain in solution.
Furthermore, the amount of stain in a given tissue sample is unknown. Thus it is not feasible to use an equation which requires knowledge of absorptivity values to determine mass densities of stains in tissue.
In the present invention, the problem of unknown absorptivities is solved by selecting a wavelength λN at which the absorptivity mlN is non-zero for each stain component and defining the following ratio matrices:
respectively. δi is constant for a particular stain at a particular location, so it cancels out. The absorptivity value miN used in equation 9 simply indicates the relative absorbances of the different stains and acts as a constant scaling factor for the intensity of pixels within a
particular image. Therefore, it is not necessary to have an exact measurement of miN in order to perform color image segmentation.
Based on the theory outlined above, in the inventive method and system, a mass density value is
calculated for each stain at each field of the slide. EQN. 9 is used, substituting in absorptivity ratios calculated from single-stain absorbance values (EQN. 10) and absorbance values measured in multiple-stained samples at each
wavelength (EQN. 2). It is then possible to generate a separate mass-density image for each stain. The procedure for determining mass density is described more specifically in connection with the examples presented subsequently.
Optimization of Wavelengths for Image Segmentation
Although the inventive method can be carried out with any wavelengths at which the absorptivity ratios of the stain components are different, it is preferable to select wavelengths that minimize error in the image measurement and segmentation. Optimization of wavelengths increases the accuracy and resolution of color image segmentation.
Two wavelength-dependent parameters should be minimized. One is the variance of the mass density estimate. The other is the chromatic aberration in the optical path of the imaging system. The chromatic aberration in the optical path of the imaging system can be determined from the shift in the images obtained at different wavelengths. For
example, image shift can be measured conveniently from images of a cross hair stage micrometer.
In order to choose the wavelengths at which
variance of the mass density estimate is minimal, the
spectral properties of the stain components must be known. The absorption spectra of stains can be estimated from singly-labelled slides with the use of a microphotometer. A microphotometer with intervals of 10 nm or less is suitable for measuring absorption spectra.
As shown in EQN.5, mass density is a function of absorbance and absorptivity. Since the absorptivity mik is constant for a specific stain i at a specific wavelength λk, we can assume that δ. is simply a function of absorbance, i.e. fi (Ak). fi(Ak) can be represented as a Taylor series:
where Ak0 is the mean absorbance at λk.
Assuming that higher order terms approach zero, using only the first terms of the series gives a good approximation:
The variance of δi can be represented by the expected value of the square of the difference between fi (Ak) and the mean value of the function, fi(Ak0):
By substituting EQN. 12 into EQN. 13, and assuming the Aks are independent, the following equation is obtained:
Since the partial derivative of fi (Ak0) with respect to Ak does not vary as a function of λk, the variance of δ. can be obtained from the variances of the Aks:
If the absorption spectra of the stain components are known, the variance of δi can be obtained for each stain component i by using equation 15. The optimum wavelengths for multiple wavelength segmentation of a particular combination of stains is determined by minimizing the sum of the variances.
Practice of the Invention
The practice of the invention requires that samples of interest be prepared, and that images be acquired and analyzed. Image acquisition requires the use of suitable hardware and software; image analysis is performed by
software which implements the equations which have been described previously to determine mass density images for each stain component. Sample preparation, equipment, and software configuration will now be described. Finally, two examples showing the use of the invention for performing segmentation of dual- and triple-stained images will be provided.
Preparation of Samples
In the application of the invention to the analysis of biological tissue, the preferred approach is to prepare slides from thin (e.g., 4 μm) sections of tissue. Sections can be of any thickness which allows the microscope to be focussed clearly on the tissue. The tissue is fixed and dehydrated, stained, and cover-slipped. Certain non-biological materials may not need to be fixed and dehydrated, and if components of interest are naturally colored, staining may not be necessary. Moreover, materials which are
sufficiently rigid do not need to be mounted on slides.
Various stains known in the prior art can be used in the practice of the invention. The choice of stain will depend on the structures of interest in the tissue to be stained. Many suitable combinations of stains can be found.
Examples of some stains which may be used in the practice of the invention are:
1) hematoxylin - a blue stain which stains nuclei of cells
2) FastRed - a pink stain which stains the
cytoplasm of cells
3) PAP-DAB - a brown stain which can be used to immunocytochemically label various structures against which antibodies have been produced.
4) Feulgen - stains DNA in cell
5) FastGreen and NEC chromagen - used for
immunohistological staining
The above stains are intended to serve only as examples, and the practice of the invention is not limited to the use of these stains.
In the practice of the invention, in order to decompose images labelled with several different stains it is necessary to prepare a number of control slides along with the multiply-stained sample slides. One singly-stained control slide must be prepared with each different stain. In addition, at least one unstained control slide must be prepared (which will be used to determine the amount of attenuation caused by the tissue section alone). Sample and control slides should be taken from serial sections of the tissue of interest and treated identically except for the differences in staining procedure.
Equipment
The equipment used in the preferred embodiment of the invention is depicted in FIG. 1. Slide 1, which contains a sample to be imaged, is secured to stage 2 of microscope 3. Microscope 3 preferably has a repeatable focus level and a chromatic aberration-free objective. Stage 2 is an XY translation stage driven by an XY stepper motor 4 (max speed 30,000 μm/sec; resolution 0.1 μm; reproducibility 4 μm;
accuracy 3 μm). The position of the stage in the Z-direction is adjusted with focus knob 5. Z-axis stepper motor 6 is used to control focus knob 5 in the preferred embodiment of the invention (stepper motor resolution is 0.01 μm, maximum speed is 27,000 μm/sec). Light from light source 7 passes through field iris 8 and condenser 9. After passing through slide 1, said light passes through microscope observation tube 10 and through a filter 11 carried in filter wheel 12. Filter wheel 12 contains a number of filters which are preferably chromatic aberration-free narrow bandpass filters. Light Source 7 is preferably a conventional light source. In an alternate embodiment of the invention, a fluorescent light source may be placed above the sample and the light emitted from slide 1 will travel through microscope observation tube
10 and filter 11. The system must also include sensors for the incident and the emitted light. In the embodiment of the invention depicted here, microscope observation tube 10 is a conventional microscope observation tube which has been modified so that filter wheel 12 can be attached between photo eyepiece 13 and camera 14. Camera 14 is a high-resolution digital CCD camera (in the preferred embodiment of the invention, a XILLIX 1400 is used), which is attached to the top of microscope observation tube 10. Camera 14 is controlled by signals sent from camera control card 15.
Camera 14 is used to acquire and digitize microscopic images. These images are subsequently displayed on color monitor 21 by a pseudo color image display board (UNIVISION 2600) 16. Both camera control card 15 and image display board 16 are installed in computer 17. In the presently preferred
embodiment of the invention, computer 17 is a PC-486DX running at 33 MHz or comparable computer. Computer 17 is preferably equipped with a mouse 18, a keyboard 19, and an SVGA monitor 20. Text generated by the software
(instructions, data, etc.) is displayed on monitor 20.
Images from image display board 16 are displayed on color image monitor 21, which is preferably a 1280 X 1024 high resolution monitor. Light source 7 is powered by power supply 22. Light source 7 is modified so that it will operate when power supply 22 is either a conventional AC power supply or a DC stabilized power supply. In the
preferred embodiment of the invention, power supply 22 is a DC power supply which may be computer controlled. Power supply 22, Z-axis stepper motor 6, stage 2, shutter 30 of camera 14, and filter wheel 12 are computer controlled in the preferred embodiment of the invention. Stage 2, power supply 22, and filter wheel 12 are controlled by modular automation controller 23 (for example, a MAC-2000 Modular Automation Controller), which is in turn controlled by signals sent over a communication line from computer 17. In the preferred embodiment, said communication line is an RS232 line, but depending on the particular computer and controller used, it
may be equally or more preferable to use another communication line. In an alternate, less preferred
embodiment of the invention, power supply 22 is controlled manually, and XYZ motion of stage 2 is controlled by a manually operated joystick.
In the presently preferred embodiment of the invention, computer 17 is connected to a second computer 24, which is preferably a SUN workstation, via an ETHERNET network 25 or comparable network. A network card 26 is installed in computer 17 (an EtherCard Plusl6 in the
preferred embodiment of the invention) to control the
transfer of data over the network. Alternatively, the invention could be carried out on a single computer,
providing said computer was capable of running all required image acquisition, storage, and analysis software. The invention is not limited to the use of a particular computer system or systems. Image data acquired with computer 17 is transferred to second computer 24 for storage and analysis. Computer 24 is preferably attached to a printer 27 on which data and images can be printed out, data storage device 28, which stores data prior to and during processing and which is preferably a hard disk, and a data backup device 29, which is a tape driver in the preferred embodiment of the invention. Various devices may be used for short- and long-term data storage and for generation of hard copies of data and images, and the devices described herein are intended only to serve as examples.
Software
In the preferred embodiment of the invention, computer 17 is a 486DX PC running DOS 5.0 and Microsoft
Windows 3.1. The software running on computer 17 controls operation of the microscope, digitization of microscope images, and archival and retrieval of images. All of the above software was developed using the basic image processing function package OPTIMAS 4.10. This function package allows C subroutines to be linked in with OPTIMAS functions.
Computer 24 is a SUN workstation running the UNIX operating system. The software running on computer 24 comprises batch mode image processing software written in the C programming language.
The practice of the invention is not limited to a particular computer system, software package or programming language. It would be possible for one skilled in the art of software development and computer control to implement the invention on various types of computers and in various programming languages. Moreover, some or all of the
functions carried out by software could be controlled by hardware, instead (e.g., matrix operations could be carried out by a digital signal processing chip). It would also be possible to use programmable chips in place of software.
The inventive method is largely computer controlled. In the preferred embodiment of the invention, image acquisition is performed in an automated manner.
However, in an alternative embodiment of the invention, image acquisition can be performed manually. Both approaches are described below.
Manual Image Acquisition
When the "manual" image acquisition procedure is used, adjustment of the microscope (i.e. imaged field, focus, light intensity, etc.) is made by hand, by a user. The digitization and subsequent storage and analysis of digitized images is performed under software control. When image acquisition is done manually, a reference image must be acquired before each test image is acquired, since manually set light intensities may vary. Accordingly, if two
wavelengths are used, 4M images must be obtained if each slide has M fields which must be imaged (i.e., at each field, one reference image and one test image must be obtained at each wavelength) . If three wavelengths are used, 6M images must be obtained (i.e., at each field, one reference image and one test image at each wavelength). The steps for manual image acquisition in the two-wavelength case are shown in
Automated Image Acquisition
In "automated" image acquisition, control of the
microscope (imaged field, focus, light intensity, etc.) is carried out with software. Digitization, storage and
analysis of digitized images is also under software control. When image acquisition is automated, only one reference image must be obtained before all M fields are imaged. A
rectangular area containing the fields of interest is defined by the user before image acquisition is begun. The light intensities used at each wavelength are set beforehand.
Light intensities are set automatically during image
acquisition and therefore are less variable. If two
wavelengths are used, 2M + 2 images must be obtained if each slide has M fields (i.e., one reference image at each
wavelength, and one sample image at each field and each wavelength). Steps performed in dual-wavelength automated image acquisition are listed in Table 2.
If three wavelengths are used, 3M + 3 images must be obtained (i.e., one reference image at each wavelength and at each field, one sample image at each wavelength). Steps performed in triple-wavelength automated image acquisition are listed in Table 3.
The automated image acquisition procedure can readily be expanded for use with more than three wavelengths. In general, reference images are acquired at each wavelength from a clear region of the slide, and then tissue images are acquired at each wavelength, from each field within the area of interest of the slide. If N wavelengths are used, NxM + N or N(M + 1) images, must be obtained. It is clear that as N increases, the image acquisition process becomes increasingly time consuming and would be virtually impossible to perform manually.
Image Data Acquisition and Processing
The overall procedure used for image data acquisition and processing is shown in FIG. 2. For each slide (indexing is performed at step 201), a tissue image is acquired from each stained field of the tissue sample, at each wavelength of interest (step 202). In step 203, images are acquired from a clear region of the slide (i.e. a region where no tissue is present), at each wavelength. The absorbance is then calculated at each location and at each wavelength (step 204). Steps 202 through 204 are repeated for each slide, as indicated at step 201. After all images have been acquired, absorptivity ratios are calculated from the absorbances of slides stained with single stains, at each wavelength (step 205), and multiply-stained images are decomposed making use of the previously calculated absorbance and absorptivity ratios (step 206).
In the example of the preferred embodiment of the invention depicted in FIG. 1, the procedure shown in FIG. 2 is carried out on two different computers. The software which carries out the image acquisition steps is run on computer 17, which is a PC in the presently preferred
embodiment of the invention, while the image decomposition steps are carried out on computer 24. The software run on computer 17 in this example of the presently preferred embodiment is listed in microfiche Appendix A. The image acquisition steps are shown in FIG. 3. In step 302, images are acquired from stained field of the tissue sample, at each wavelength of interest, and saved. In step 303, images are acquired from a clear region of the slide (i.e. a region where no tissue is present), at each wavelength and saved.
Either the manual procedure or automated procedure described above may be used to acquire images. The image data is stored directly on the data storage device 28 of computer 24. Image data is preferably stored immediately after it is acquired; it would also be possible to store a large block of image data in one step after image acquisition is completed. In the presently preferred embodiment of the invention,
computer 24 is a SUN workstation, and data storage device 28 on the SUN system is transparent to the computer 17 (the PC) Data can thus be written to data storage device 28 using the same commands which can be used to write to disks of the PC. Images from which staining thresholds can be calculated are determined manually (stept 305). Absorptivity ratios are calculated (step 306), and in step 307, absorptivity ratios and the case number are stored in a "flag file" on data storage device 28 of computer 24.
Batch Mode Image Analysis
Because image analysis is time-consuming, it is
desirable that if many images are to be analyzed, this analysis be carried out at a time when the use of the
computer for other tasks is not needed, e.g. in the middle of the night. It is also preferable that multiple images be processed without the need for user intervention. The batch mode image analysis procedure makes it possible to process a large amount of image data without user intervention. This procedure is carried out by a computer program running on computer 24. The software run on computer 24 in this example of the presently preferred embodiment is listed in microfiche Appendix B. The procedure for batch mode image analysis is shown in FIG. 4. The batch mode procedure is started at step 401. In step 402, the program checks for the presence of a flag file on data storage device 28. The flag file is created and stored on data storage device 28 subsequent to the storage of image files on data storage device 28, as shown in FIG. 3. Accordingly, the presence of a flag file indicates that image files are also present on the disk. In step 402, the system time is also evaluated; the program waits until the desired starting time is reached. The starting time shown in FIG. 4. is 00:00 a.m. (midnight), but any time which is convenient can be used in the practice of the invention. When the starting time is reached, the first image file listed in the flag file is opened (step 403). For each image, the stain components are decomposed (step 404)
and the threshold stain density is determined for each component image (step 405). The threshold stain density has a value which is higher than the density of the background stain, but below the stain density of positively stained tissue. Finally, a data file containing the results of the stain component measurements is created (step 406). If additional files are listed in the flag file, as determined in step 407, steps 403 through 406 are repeated for each file, until all files listed in the flag file have been processed. A data file containing the parameters related to each stain component (e.g. stain intensity and extent) is then created (step 408). The flag file is deleted from data storage device 28 (step 409), and the batch procedure is ended (step 410).
The following examples illustrate the use of the
invention in the segmentation of dual- and triple-stained images. In each example, the method was used to analyze images of prostate cancer tissue specimens. Example 1: Dual-wavelength segmentation of dual-stained image. In this example, sample slides labelled with two stains were analyzed. PAP-DAB was used to stain Prostate Specific Acid Phosphatase (Stain 1) in the cytoplasm, and hematoxylin (Stain 2) was used to stain cell nuclei. Slides were prepared with 1) both PAP-DAB + hematoxylin, 2) PAP-DAB alone, 3) hematoxylin alone, or 4) no stain. Only one slide each was prepared with the single stains or no stain, while a set of slides were prepared with both stains. All slides were processed together to insure identical staining with each particular stain. The absorbances measured at
wavelengths λa and λb were:
\
where α and β are absorptivity rations, with a = (A-∇a)/(B- ∇b) measured in an image stained only with PAP-DAB, and averaged over the entire image, and β = (A-∇a) / (B-∇b)
measured in an image stained only with hematoxylin and averaged over the entire image. α and β were found to be
1.71 ± 0.039 and 0.51 ± 0.007, respectively. ∇a and ∇b were the average absorbances of unstained tissue at wavelengths λa and λb, respectively. In doubly labelled tissue sections, attenuation is caused by the tissue itself as well as by the two stains. Assuming that the tissue section itself is translucent and the attenuations at the two wavelengths are ∇a and ∇b, A and B in EQN. 17 were replaced by A-∇a and B-∇b, respectively. The tissue absorbancies are very small and appear uniform over the wavelengths used; thus, any errors in the estimation caused by neglecting the tissue absorbancies contribute only small variations to the other measurements. Each of mlb and m2b (the relative absorptivities of the two stains at wavelength λb) were assumed to be equal to 1.0.
In order to determine the optimum wavelengths to use in the segmentation, equation 14 was rewritten in the form:
Since m1b and m2b are unknown, rather than calculating the absolute variances, a variation index which expressed the variance relative to the variances at two standard
wavelengths λ1 and λ2 (in this case 480 nm and 570 nm) was calculated. The variation indices were:
The optimum wavelengths were determined by minimizing the summation of the variation indices. The optimum wavelengths thus determined were 400 nm and 600 nm. The chromatic aberration in the optical path of the imaging system was minimum at wavelengths between 480 nm and 630 nm. Chromatic
aberration was determined from the shift in the image at different wavelengths. In order to obtain a low variance while minimizing chromatic aberration, near-optimum
wavelengths of 480 nm and 570 nm were used.
The accuracy of the decomposition method was determined by comparing stain density estimates obtained by decomposing dual-stained tissue samples with estimates obtained from single-stained tissue samples. For PAP-DAB, it was found that coefficient of correlation between stain density
estimates obtained from single-stained tissue samples and from decomposition of dual-stained tissue samples was 0.994. The difference between the estimates obtained with the two methods was not significant. For hematoxylin, the
coefficient of correlation between stain density estimates obtained from single-stained tissue samples and from
decomposition of dual-stained tissue samples was 0.998.
Again, the difference was not significant. Accordingly, it was concluded that decomposition of dual-stained images produced accurate estimates of the densities of the
individual stains.
Example 2: Triple-wavelength segmentation of triple-stained image.
In this example, sample slides labelled with three stains were analyzed. The stains used were PAP-DAB, FastRed, and hematoxylin. Slides were prepared with 1) the
combination of PAP-DAB, FastRed and hematoxylin, 2) PAP-DAB alone, 3) FastRed alone, 4) hematoxylin alone, and 5) no stain.
The absorbances measured at wavelengths λa, λb and λc were:
, and
, the relative absorptivities of the three stain components at wavelength λc, were assumed to be 1.0.
The absorptivity ratios of PAP-DAB, FastRed, and hematoxylin, were calculated as follows:
α1 = (A-∇a) / (C-∇c) measured in the image stained only with PAP-DAB and averaged over all pixels, a2 = (B-∇b) / (C-∇c) measured in the image stained only with PAP-DAB and averaged over all pixels, βx = (A-∇a) / (C-∇c) measured in the image stained only with FastRed and averaged over all pixels, β2 = (B-∇b) / (C-∇c) measured in the image stained only with FastRed
and averaged over all pixels, γ1 = (A-∇a) / (C-∇c) measured in the image stained only with hematoxylin and averaged over all pixels, γ2 = (B-∇b) / (C-∇c) measured in the image stained only with hematoxylin and averaged over all pixels. The values obtained for these parameters are shown in Table 4.
∇a, ∇b, and ∇c, the effective absorbencies of unstained tissue sections at wavelength λc, were calculated from the absorbance of unstained tissue In [GOa (x, y) / Ga (x, y) ] , averaged over the entire image. Assuming that the tissue section itself is translucent and the attenuations at the three wavelengths are ∇a, ∇b, and ∇c, A,B, and C in EQN. 20 were replaced by A-∇a, B-∇b and C-∇c, respectively.
The wavelengths which minimized the summation of the variance values were determined. In the three component case, equation 14 was rewritten in the following form to determine the variances:
variation indices were thus:
where vδ1 2 , vδ2 2 , and vδ3 2 were def ined as
The optimum wavelengths were determined by minimizing the summation of the variation indices. The summation of the variation indices was minimized at 400, 560 and 610 nm.
Since the chromatic aberrations in the optical path of the imaging system were minimal between 480 and 630 nm,
wavelengths of 480, 570 nm, and 630 nm were used to minimize the overall error.
The accuracy of the triple-wavelength decomposition method was determined by comparing stain density estimates obtained by decomposing triple-stained tissue samples with estimates obtained from single-stained tissue samples. For PAP-DAB, it was found that the coefficient of correlation between stain density estimates obtained from single-stained tissue samples and from decomposition of triple-stained tissue samples was 0.996. For hematoxylin, the coefficient
of correlation between stain density estimates obtained from single-stained tissue samples and from decomposition of triple-stained tissue samples was 0.992. For FastRed, the coefficient of correlation between stain density estimates obtained from single-stained tissue samples and from
decomposition of triple-stained tissue samples was 0.973. Again, the differences between the estimates obtained from single-stained and triple-stained samples were not
significant, and it was concluded that decomposition of triple-stained images produced accurate estimates of the densities of the individual stains.
The practice of the invention is illustrated by the examples given above, which describe the use of invention for analyzing images in which two or three stains were used, and demonstrate the accuracy of the method. The invention may also be used to analyze images in which a larger number of stains are used. The theoretical basis for analysis of images stained with two, three, or more stains has been described, and the method may be used for images stained with more than three stains according to the procedures for image acquisition and analysis described herein.
The described embodiment is to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
Claims
What is claimed and desired to be secured by Letters Patent is: 1. A method for analyzing microscope images of multiply stained samples, comprising the steps of:
a) selecting N stains;
b) preparing at least one unstained sample;
c) preparing at least N singly-stained samples, each of said at least N singly-stained samples being stained with a different one of said N stains;
d) preparing at least one multiply-stained sample stained with all of said N stains;
e) selecting N wavelengths of light;
f) acquiring images formed by shining light of each of said N wavelengths through each of said at least one
unstained sample, said at least N singly-stained samples, and said at least one multiply-stained sample;
g) determining the absorbance of said at least one unstained sample for each of said N wavelengths of light; h) determining the absorbance of each of said at least N singly-stained samples for each of said N wavelengths of light;
i) determining the absorbance of said at least one multiply-stained sample for each of said N wavelengths of light;
j) determining the absorptivity ratio of each of said at least N singly-stained samples at each of said N wavelengths of light by dividing the absorbance measured at each of said N wavelengths by the absorbance measured at a selected one of said N wavelengths, wherein the absorptivity is non-zero for each stain component at said selected one wavelength;
k) determining the mass density of each of said N stains in said at least one multiply-stained sample from said absorbances of said at least one multiply-stained sample at each of said N wavelengths and from said absorptivity ratios of said at least N singly stained samples at each of said N wavelengths.
2. A method in accordance with claim 1, wherein the density of a stain i at a location (x,y) of a sample is calculated as:
wherein rik is the absorptivity ratio for stain i and
wavelength λk; Ak(x,y) is the absorbance measured at location (x,y) and wavelength λk, and miN is a constant.
3. A method in accordance with claim 1, wherein said N wavelengths selected in step e) are selected such that the summation of the wavelength-dependent variances of said mass densities of said N stains is minimized.
4. A method in accordance with claim 1, wherein said N wavelengths selected in step e) are selected such that the chromatic abberation in the optical path of the imaging system used to acquire said images is minimized.
5. A method in accordance with claim 1, wherein said N wavelengths selected in step e) are selected such that the summation of the wavelength-dependent variances of said mass densities of said N stains and the chromatic aberration in the optical path of the imaging system used to acquire said images are minimized.
6. A method in accordance with claim 1, wherein step e) comprises the further steps of:
i) estimating the variance σδ i 2 of the mass density δi for each stain i, from the variance of the absorbances Ak over all wavelengths λk, according to the following equation: 
wherein Ak0 is the mean value of Ak;
ii) calculating the sum of said variances for each stain i; and
iii) selecting N wavelengths λk which minimize said sum of said variances.
7. A method in accordance with claim 6, wherein N equals 2 and wherein said variances are determined as follows:
8. A method in accordance with claim 6, wherein N equals 2 and
wherein said variances are determined according to the following:
wherein λ1 and λ2 are predetermined constant wavelengths, and wherein the two wavelengths used are λa and λb; wherein A is the absorbance at λa and B is the absorbance at λb; wherein α is the absorptivity ratio for the first stain and β is the absorptivity ratio for the second stain; and wherein m1b and m2b are the absorptivities of said first stain and said second stain at λb.
9. A method in accordance with claim 6, wherein N equals 3 wherein said variances are calculated accorded to the
following equation:
wherein the three wavelengths used are λa, λb, and λc; wherein A is the absorbance at λa, B is the absorbance at λb and C is the absorbance at λc; wherein α1, β1 , and γ1 are the
absorptivity ratios for the first, second and third stains, respectively, calculated at λa and λc and α2 , β2 , and γ2 are the absorptivity ratios for the first, second and third stains, respectively, calculated at λb and λc, and wherein mic is the absorptivity of stain i at λc.
10. A method in accordance with claim 6, wherein N equals 3 and
wherein said variances are calculated according to the following: 
wherein vδ1 2 , vδ2 2 , and vδ3 2 are def ined as :
wherein λ1, λ2, and λ3 are predetermined constant wavelengths, wherein the three wavelengths used are λa, λb, and λc; wherein A is the absorbance at λa, B is the absorbance at λb, and C is the absorbance at λc; wherein α1 , β1 , and γ1 are the absorptivity ratios for the first, second and third stains, respectively, calculated at λa and λc, and α2 , β2 , and γ2 are the absorptivity ratios for the first, second and third stains, respectively, calculated at λb and λc, and wherein mic is the absorptivity of stain i at λc.
11. A method in accordance with claim 1, wherein said images acquired in step f) are saved on a data storage device;
wherein a flag file containing a list of said saved images is also stored on said data storage device; wherein steps g) through k) are initiated and carried out under the control of software; wherein steps g) through k) are not initiated until said software detects that the time on the system clock has reached a user-set time and that said flag file is present; wherein steps g) through k) are carried out for each image listed in said flag file; and wherein said flag file is deleted from said data storage device after steps g) through k) have been carried out.
12. A method in accordance with claim 1, wherein N = 2 and said mass density is determined in step k) according to the following equations:
wherein δ1(x,y) is the density of the first stain and δ2(x,y) is the density of the second stain at location (x,y) of said multiply stained sample; wherein A(x,y) is the absorbance at the first wavelength and B(x,y) is the absorbance at the second wavelength at location (x,y) of said multiply stained sample; wherein α and β are the absorptivity ratios of said first stain and said second stain; and wherein 
and 
are each equal to one.
13. A method in accordance with claim 1, wherein N = 3 and said mass density is determined in step k) according to the following equations:
wherein δ1(x,y) is the density of the first stain, δ2(x,y) is the density of the second stain, and δ3(x,y) is the density of the third stain at location (x,y) of said multiply stained sample; wherein A(x,y) is the absorbance at the first
wavelength, B(x,y) is the absorbance at the second
wavelength, and C(x,y) is the absorbance at the third
wavelength at location (x,y) of said multiply stained sample; wherein α1, β1 and γ1 are the absorptivity ratios of said first, second and third stains calculated at said first and third wavelengths and α2, β2 and γ2 are the absorptivity ratios of said first, second and third stains calculated at said second and third wavelengths; and wherein 
, 
, and are each equal to one.
14. A method for analyzing microscope images of multiply stained samples in accordance with claim 1, wherein each of said samples is on a microscope slide, wherein N = 2 and wherein step f) comprises the further steps of:
i) manually moving the microscope stage to a region of the slide without tissue or stain;
ii) manually switching to a filter selective for a first wavelength and adjusting the intensity of said light shining through said sample to an appropriate level;
iii) capturing and saving a reference image of said region without tissue or stain at said first wavelength; iv) manually moving said microscope stage to a tissue field of interest on said slide;
v) capturing and saving an image of said tissue field at said first wavelength;
vi) manually switching to a filter selective for a second wavelength and adjusting the intensity of said light shining through said sample to an appropriate level;
vii) capturing and saving an image of said tissue field at said second wavelength;
viii) manually moving said microscope stage to said region of said slide without tissue or stain; ix) capturing and saving a reference image of said region without tissue or stain at said second
wavelength; and
x) repeating steps i) through ix) for each tissue field of interest on said slide.
15. A method in accordance with claim 1, wherein each of said samples is on a microscope slide, and wherein step f) comprises the further steps of:
i) manually moving the stage of the microscope and focussing said microscope on a region of a slide without tissue or stain so that position and focus is recorded by software;
ii) manually moving said stage and focussing said microscope on a tissue structure of interest so that a first position and focus is recorded by software, wherein said first position is the position of said first tissue field to be imaged;
iii) moving said stage and focussing said microscope on said region of slide without tissue or stain;
iv) for wavelengths λk, for k = 1 to N:
(1) changing to filter selective for
wavelength λk and setting the intensity of said light shining through said sample appropriately;
(2) capturing reference image;
v) for M different tissue fields of said slide, performing the steps of:
(1) moving said stage and focussing said microscope on each tissue field, wherein each said tissue field has a position which is measured in relation to said first position;
(2) for wavelengths λk, for k = 1 to N:
(a) changing to filter selective for wavelength λk and setting illuminating
intensity appropriately; and (b) capturing tissue image;
wherein steps iii), iv) and v) are carried out automatically under computer control.
16. A method in accordance with claim 15, wherein N = 2.
17. A method in accordance with claim 15, wherein N = 3.
18. A system for analyzing multiply stained samples,
comprising:
a) a microscope having a stage, a stage x,y position adjustment and a focus knob for adjusting the z-axis position of said stage;
b) a filter wheel having a multiplicity of optical filters of selected wavelengths;
c) a light source;
d) a camera;
e) a first computer,
f) a second computer;
g) a data storage device, accessible by said first computer and said second computer;
h) software running on said first computer, said
software providing for the control of:
i) acquisition of images from said camera; and ii) archiving of said images in said data storage device; and
i) software running on said second computer, providing for the retrieval and analysis of said images archived in said data storage device;
wherein each of said multiply stained samples is stained with N different stains, and wherein analysis of said images includes the step of determining for each of said N different stains the ratios of the absorbance of each stain at each of N different wavelengths of light to the absorbance at a selected one of said N wavelengths of light.
19. A system in accordance with claim 18, further comprising a) a motorized filter wheel
b) an x-y-axis stepper motor which provides x-y motion to said stage;
c) a z-axis stepper motor which controls said focus knob; and
d) a microscope controller which controls said light source, said filter wheel, said x-y-stepper motor and said z-axis stepper motor;
wherein said motorized filter wheel, said x-y-axis stepper motor, said z-axis stepper motor, and said microscope
controller are controlled by software running on said first computer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| AU73643/96A AU7364396A (en) | 1995-09-19 | 1996-09-18 | Method and system for multiple wavelength microscopy image analysis |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US08/530,435 | 1995-09-19 | ||
| US08/530,435 US5732150A (en) | 1995-09-19 | 1995-09-19 | Method and system for multiple wavelength microscopy image analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO1997013135A2 true WO1997013135A2 (en) | 1997-04-10 |
Family
ID=24113634
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US1996/014995 WO1997013135A2 (en) | 1995-09-19 | 1996-09-18 | Method and system for multiple wavelength microscopy image analysis |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US5732150A (en) |
| AU (1) | AU7364396A (en) |
| WO (1) | WO1997013135A2 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1998052016A1 (en) * | 1997-05-09 | 1998-11-19 | Morphometrix Technologies Inc. | Multi-spectral imaging system and method for cytology |
| EP1042727A1 (en) * | 1997-12-04 | 2000-10-11 | Applied Spectral Imaging Ltd. | Method of cancer cell detection |
| DE10124340A1 (en) * | 2001-05-18 | 2002-12-05 | Fraunhofer Ges Forschung | Method of analyzing a biological sample |
| US20110223632A1 (en) * | 2010-03-11 | 2011-09-15 | Sysmex Corporation | Smear staining apparatus, smear preparing apparatus, smear processing system, and method for determining staining condition |
Families Citing this family (85)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6690817B1 (en) | 1993-08-18 | 2004-02-10 | Applied Spectral Imaging Ltd. | Spectral bio-imaging data for cell classification using internal reference |
| JPH09116852A (en) * | 1995-10-16 | 1997-05-02 | Nikon Corp | Electronic camera |
| US6165734A (en) * | 1995-12-12 | 2000-12-26 | Applied Spectral Imaging Ltd. | In-situ method of analyzing cells |
| US5989835A (en) * | 1997-02-27 | 1999-11-23 | Cellomics, Inc. | System for cell-based screening |
| US6008010A (en) * | 1996-11-01 | 1999-12-28 | University Of Pittsburgh | Method and apparatus for holding cells |
| US7853411B2 (en) * | 1997-02-27 | 2010-12-14 | Cellomics, Inc. | System for cell-based screening |
| US6727071B1 (en) | 1997-02-27 | 2004-04-27 | Cellomics, Inc. | System for cell-based screening |
| CA2282658C (en) | 1997-02-27 | 2003-02-25 | Cellomics, Inc. | A system for cell-based screening |
| US6759206B1 (en) | 1997-02-27 | 2004-07-06 | Cellomics, Inc. | System for cell-based screening |
| US7117098B1 (en) | 1997-02-27 | 2006-10-03 | Cellomics, Inc. | Machine-readable storage medium for analyzing distribution of macromolecules between the cell membrane and the cell cytoplasm |
| US6753161B2 (en) * | 1997-03-27 | 2004-06-22 | Oncosis Llc | Optoinjection methods |
| WO1998055026A1 (en) * | 1997-06-05 | 1998-12-10 | Kairos Scientific Inc. | Calibration of fluorescence resonance energy transfer in microscopy |
| US6221612B1 (en) * | 1997-08-01 | 2001-04-24 | Aurora Biosciences Corporation | Photon reducing agents for use in fluorescence assays |
| ATE292822T1 (en) | 1998-11-13 | 2005-04-15 | Cellomics Inc | METHOD AND SYSTEM FOR EFFICIENTLY OBTAINING AND STORING EXPERIMENTAL DATA |
| US6453060B1 (en) * | 1999-06-29 | 2002-09-17 | Tri Path Imaging, Inc. | Method and apparatus for deriving separate images from multiple chromogens in a branched image analysis system |
| US6403332B1 (en) | 1999-07-30 | 2002-06-11 | California Institute Of Technology | System and method for monitoring cellular activity |
| JP4441831B2 (en) * | 1999-09-16 | 2010-03-31 | 株式会社ニコン | Microscope equipment |
| US7045049B1 (en) * | 1999-10-01 | 2006-05-16 | Nanoplex Technologies, Inc. | Method of manufacture of colloidal rod particles as nanobar codes |
| US6919009B2 (en) * | 1999-10-01 | 2005-07-19 | Nanoplex Technologies, Inc. | Method of manufacture of colloidal rod particles as nanobarcodes |
| US20040209376A1 (en) * | 1999-10-01 | 2004-10-21 | Surromed, Inc. | Assemblies of differentiable segmented particles |
| US7225082B1 (en) | 1999-10-01 | 2007-05-29 | Oxonica, Inc. | Colloidal rod particles as nanobar codes |
| US20040178076A1 (en) * | 1999-10-01 | 2004-09-16 | Stonas Walter J. | Method of manufacture of colloidal rod particles as nanobarcodes |
| US20050032226A1 (en) * | 1999-10-01 | 2005-02-10 | Natan Michael J. | Encoded nanoparticles in paper manufacture |
| JP5087192B2 (en) | 1999-11-30 | 2012-11-28 | インテレクソン コーポレイション | Method and apparatus for selectively aiming a specific cell in a cell group |
| WO2001055951A2 (en) | 2000-01-25 | 2001-08-02 | Cellomics, Inc. | Method and system for automated inference of physico-chemical interaction knowl edge |
| US6917377B2 (en) * | 2000-02-04 | 2005-07-12 | Olympus Optical Co., Ltd. | Microscope system |
| US6768982B1 (en) | 2000-09-06 | 2004-07-27 | Cellomics, Inc. | Method and system for creating and using knowledge patterns |
| AU2001296885A1 (en) * | 2000-09-27 | 2002-04-08 | Hyperion, Inc. | Quantitative digital fluorography and related products and methods |
| CA2426871A1 (en) * | 2000-10-24 | 2002-05-16 | Oncosis Llc | Method and device for selectively targeting cells within a three -dimensional specimen |
| JP2004531048A (en) * | 2000-12-15 | 2004-10-07 | サイバーオプティクス コーポレーション | Substrate alignment image capture device with improved interface |
| US20020146745A1 (en) * | 2001-04-03 | 2002-10-10 | Surromed, Inc. | Methods and reagents for multiplexed analyte capture, surface array self-assembly, and analysis of complex biological samples |
| WO2002097716A1 (en) * | 2001-05-29 | 2002-12-05 | Tissueinformatics, Inc. | Robust stain detection and quantification for histological specimens based on a physical model for stain absorption |
| US6930773B2 (en) * | 2001-08-23 | 2005-08-16 | Cambridge Research And Instrumentation, Inc. | Spectral imaging |
| US7065236B2 (en) * | 2001-09-19 | 2006-06-20 | Tripath Imaging, Inc. | Method for quantitative video-microscopy and associated system and computer software program product |
| EP1451304A4 (en) * | 2001-11-07 | 2006-09-20 | Bioview Ltd | Kits and methods for preparing cell samples optimized for dual staining |
| US7133547B2 (en) * | 2002-01-24 | 2006-11-07 | Tripath Imaging, Inc. | Method for quantitative video-microscopy and associated system and computer software program product |
| AU2003211104B2 (en) * | 2002-02-13 | 2009-01-29 | Reify Corporation | Method and apparatus for acquisition, compression, and characterization of spatiotemporal signals |
| US8614768B2 (en) * | 2002-03-18 | 2013-12-24 | Raytheon Company | Miniaturized imaging device including GRIN lens optically coupled to SSID |
| US20040002154A1 (en) * | 2002-04-19 | 2004-01-01 | Palsson Bernhard O. | Methods for preparing libraries of unique tags and related screening methods |
| US7272252B2 (en) * | 2002-06-12 | 2007-09-18 | Clarient, Inc. | Automated system for combining bright field and fluorescent microscopy |
| US20050037406A1 (en) * | 2002-06-12 | 2005-02-17 | De La Torre-Bueno Jose | Methods and apparatus for analysis of a biological specimen |
| US8712118B2 (en) * | 2003-04-10 | 2014-04-29 | Carl Zeiss Microimaging Gmbh | Automated measurement of concentration and/or amount in a biological sample |
| WO2004113869A2 (en) * | 2003-06-17 | 2004-12-29 | Surromed, Inc. | Labeling and authentication of metal objects |
| DE10327839A1 (en) * | 2003-06-20 | 2005-01-05 | Arvinmeritor Gmbh | Vehicle roof module |
| JP2007531496A (en) * | 2003-07-11 | 2007-11-08 | サロメッド，インコーポレイテッド | Multiplex molecular beacon assay for detecting pathogens |
| US20050095578A1 (en) * | 2003-10-31 | 2005-05-05 | Koller Manfred R. | Method and apparatus for cell permeabilization |
| US7425426B2 (en) | 2004-03-15 | 2008-09-16 | Cyntellect, Inc. | Methods for purification of cells based on product secretion |
| US8953866B2 (en) * | 2004-03-16 | 2015-02-10 | Amnis Corporation | Method for imaging and differential analysis of cells |
| US7899624B2 (en) * | 2005-07-25 | 2011-03-01 | Hernani Del Mundo Cualing | Virtual flow cytometry on immunostained tissue-tissue cytometer |
| US20070091109A1 (en) * | 2005-09-13 | 2007-04-26 | Roscoe Atkinson | Image quality |
| JP4915071B2 (en) * | 2005-09-22 | 2012-04-11 | 株式会社ニコン | Microscope and virtual slide creation system |
| IL177550A0 (en) | 2006-08-17 | 2006-12-31 | Sialo Technology Israel Ltd | All-in-one optical microscopic handle |
| CN100427990C (en) * | 2006-09-13 | 2008-10-22 | 成都医学院 | Digital interactive microscope and method for controlling same |
| US7738094B2 (en) | 2007-01-26 | 2010-06-15 | Becton, Dickinson And Company | Method, system, and compositions for cell counting and analysis |
| US7835074B2 (en) | 2007-06-05 | 2010-11-16 | Sterling Lc | Mini-scope for multi-directional imaging |
| US8326014B2 (en) * | 2007-09-28 | 2012-12-04 | Cytyc Corporation | Methods and systems for processing biological specimens utilizing multiple wavelengths |
| US7812318B1 (en) | 2008-03-14 | 2010-10-12 | Advanced Technology Applications, Llc | Electromagnetic biosensor |
| US8735846B1 (en) | 2008-03-14 | 2014-05-27 | Advanced Technology Applications, Llc | Electromagnetic biosensor |
| US20090253163A1 (en) * | 2008-04-02 | 2009-10-08 | General Electric Company | Iterative staining of biological samples |
| US20090287048A1 (en) * | 2008-05-16 | 2009-11-19 | Sterling Lc | Method and apparatus for imaging within a living body |
| JP5596027B2 (en) | 2008-06-18 | 2014-09-24 | レイセオン カンパニー | catheter |
| WO2009155432A2 (en) * | 2008-06-18 | 2009-12-23 | Sterling Lc | Miniaturized imaging device multiple grin lenses optically coupled to multiple ssids |
| US8486735B2 (en) * | 2008-07-30 | 2013-07-16 | Raytheon Company | Method and device for incremental wavelength variation to analyze tissue |
| WO2010053916A2 (en) | 2008-11-04 | 2010-05-14 | Sterling Lc | Method and device for wavelength shifted imaging |
| US20100179310A1 (en) * | 2009-01-09 | 2010-07-15 | Cyntellect, Inc. | Genetic analysis of cells |
| KR20110108390A (en) | 2009-01-12 | 2011-10-05 | 신텔렉트 인코포레이티드 | Laser mediated sectioning and transfer of cell colonies |
| WO2011041730A2 (en) | 2009-10-01 | 2011-04-07 | Jacobsen Stephen C | Light diffusion apparatus |
| US9144664B2 (en) | 2009-10-01 | 2015-09-29 | Sarcos Lc | Method and apparatus for manipulating movement of a micro-catheter |
| US9661996B2 (en) | 2009-10-01 | 2017-05-30 | Sarcos Lc | Needle delivered imaging device |
| US8828028B2 (en) | 2009-11-03 | 2014-09-09 | Raytheon Company | Suture device and method for closing a planar opening |
| JP5631682B2 (en) * | 2010-09-30 | 2014-11-26 | オリンパス株式会社 | Microscope system and distribution system |
| US9523682B2 (en) | 2011-11-16 | 2016-12-20 | Becton, Dickinson And Company | Methods and systems for detecting an analyte in a sample |
| BR112015010695B1 (en) | 2013-01-11 | 2023-03-07 | Becton, Dickinson And Company | MICROFLUID DEVICE AND METHOD FOR PERFORMING A LIQUID SAMPLE TEST, METHOD FOR FORMING A MICROFLUID DEVICE, SYSTEM AND KIT |
| CN113477149B (en) | 2013-11-06 | 2023-09-12 | 贝克顿·迪金森公司 | Microfluidic devices and methods of making and using the same |
| US10018640B2 (en) | 2013-11-13 | 2018-07-10 | Becton, Dickinson And Company | Optical imaging system and methods for using the same |
| ES2962033T3 (en) | 2014-10-14 | 2024-03-14 | Becton Dickinson Co | Blood sample management using open cell foam |
| US9693723B2 (en) | 2014-10-14 | 2017-07-04 | Becton, Dickinson And Company | Blood sample management using open cell foam |
| US9927630B2 (en) * | 2014-10-31 | 2018-03-27 | Konica Minolta Laboratory U.S.A., Inc. | Color calibration for pathology slides |
| AU2016229837B2 (en) | 2015-03-10 | 2018-05-24 | Becton, Dickinson And Company | Biological fluid micro-sample management device |
| US10183290B2 (en) | 2015-04-16 | 2019-01-22 | Advanced Technology Applications, Llc | Method for analyte examination |
| EP3330694A4 (en) * | 2015-07-29 | 2019-03-27 | Olympus Corporation | Specimen staining system |
| US10578606B2 (en) | 2015-09-01 | 2020-03-03 | Becton, Dickinson And Company | Depth filtration device for separating specimen phases |
| CN105466733B (en) * | 2015-12-16 | 2017-12-05 | 中国船舶重工集团公司第七一九研究所 | A kind of step-by-step movement high flow rate aerosol sampler |
| US20180279862A1 (en) * | 2016-10-10 | 2018-10-04 | Jack Wade | Systems and methods for closed-loop medical imaging |
| KR102537664B1 (en) * | 2017-02-06 | 2023-05-30 | 이에프에이-엔지니어링 포 올 엘티디. | handheld digital diagnostic device |
Family Cites Families (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3919530A (en) * | 1974-04-10 | 1975-11-11 | George Chiwo Cheng | Color information leukocytes analysis system |
| US4191940A (en) * | 1978-01-09 | 1980-03-04 | Environmental Research Institute Of Michigan | Method and apparatus for analyzing microscopic specimens and the like |
| US4601537A (en) * | 1984-01-06 | 1986-07-22 | Ohio State University Research Foundation | Apparatus and methods for forming images and for optical demultiplexing |
| US4700298A (en) * | 1984-09-14 | 1987-10-13 | Branko Palcic | Dynamic microscope image processing scanner |
| US4724543A (en) * | 1985-09-10 | 1988-02-09 | Beckman Research Institute, City Of Hope | Method and apparatus for automatic digital image analysis |
| US4998284A (en) * | 1987-11-17 | 1991-03-05 | Cell Analysis Systems, Inc. | Dual color camera microscope and methodology for cell staining and analysis |
| US5109429A (en) * | 1985-11-04 | 1992-04-28 | Cell Analysis Systems,Inc. | Apparatus and method for analyses of biological specimens |
| US5134662A (en) * | 1985-11-04 | 1992-07-28 | Cell Analysis Systems, Inc. | Dual color camera microscope and methodology for cell staining and analysis |
| US5008185A (en) * | 1985-11-04 | 1991-04-16 | Cell Analysis Systems, Inc. | Methods and apparatus for the quantitation of nuclear proteins |
| US4741043B1 (en) * | 1985-11-04 | 1994-08-09 | Cell Analysis Systems Inc | Method of and apparatus for image analyses of biological specimens |
| US5121436A (en) * | 1987-08-14 | 1992-06-09 | International Remote Imaging Systems, Inc. | Method and apparatus for generating a plurality of parameters of an object in a field of view |
| US5124932A (en) * | 1988-03-10 | 1992-06-23 | Indiana University Foundation | Method for analyzing asymmetric clusters in spectral analysis |
| US4965725B1 (en) * | 1988-04-08 | 1996-05-07 | Neuromedical Systems Inc | Neural network based automated cytological specimen classification system and method |
| US5016173A (en) * | 1989-04-13 | 1991-05-14 | Vanguard Imaging Ltd. | Apparatus and method for monitoring visually accessible surfaces of the body |
| US5107422A (en) * | 1989-10-02 | 1992-04-21 | Kamentsky Louis A | Method and apparatus for measuring multiple optical properties of biological specimens |
| US5321771A (en) * | 1989-10-03 | 1994-06-14 | Thomson-Csf | Method and device for automatic image segmentation by textural analysis |
| US5162990A (en) * | 1990-06-15 | 1992-11-10 | The United States Of America As Represented By The United States Navy | System and method for quantifying macrophage phagocytosis by computer image analysis |
| US5235522A (en) * | 1990-10-10 | 1993-08-10 | Cell Analysis Systems, Inc. | Method and apparatus for automated analysis of biological specimens |
| US5218645A (en) * | 1991-03-29 | 1993-06-08 | Cell Analysis Systems, Inc. | Method and apparatus for separating cell objects for analysis |
| JP2741973B2 (en) * | 1991-06-24 | 1998-04-22 | 大日本スクリーン製造株式会社 | Image processing system |
| US5428690A (en) * | 1991-09-23 | 1995-06-27 | Becton Dickinson And Company | Method and apparatus for automated assay of biological specimens |
| US5586304A (en) * | 1994-09-08 | 1996-12-17 | Compaq Computer Corporation | Automatic computer upgrading |
-
1995
- 1995-09-19 US US08/530,435 patent/US5732150A/en not_active Expired - Lifetime
-
1996
- 1996-09-18 WO PCT/US1996/014995 patent/WO1997013135A2/en active Application Filing
- 1996-09-18 AU AU73643/96A patent/AU7364396A/en not_active Abandoned
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1998052016A1 (en) * | 1997-05-09 | 1998-11-19 | Morphometrix Technologies Inc. | Multi-spectral imaging system and method for cytology |
| EP1042727A1 (en) * | 1997-12-04 | 2000-10-11 | Applied Spectral Imaging Ltd. | Method of cancer cell detection |
| EP1042727A4 (en) * | 1997-12-04 | 2005-11-30 | Applied Spectral Imaging Ltd | Method of cancer cell detection |
| DE10124340A1 (en) * | 2001-05-18 | 2002-12-05 | Fraunhofer Ges Forschung | Method of analyzing a biological sample |
| US7257243B2 (en) | 2001-05-18 | 2007-08-14 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Method for analyzing a biological sample |
| US20110223632A1 (en) * | 2010-03-11 | 2011-09-15 | Sysmex Corporation | Smear staining apparatus, smear preparing apparatus, smear processing system, and method for determining staining condition |
| US9250165B2 (en) * | 2010-03-11 | 2016-02-02 | Sysmex Corporation | Smear staining apparatus, smear preparing apparatus, smear processing system, and method for determining staining condition |
Also Published As
| Publication number | Publication date |
|---|---|
| US5732150A (en) | 1998-03-24 |
| AU7364396A (en) | 1997-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5732150A (en) | Method and system for multiple wavelength microscopy image analysis | |
| US8285024B2 (en) | Quantitative, multispectral image analysis of tissue specimens stained with quantum dots | |
| US5548661A (en) | Operator independent image cytometer | |
| US7065236B2 (en) | Method for quantitative video-microscopy and associated system and computer software program product | |
| US6337472B1 (en) | Light imaging microscope having spatially resolved images | |
| EP0534948B1 (en) | Dual color camera microscope and methodology for cell staining and analysis | |
| US5134662A (en) | Dual color camera microscope and methodology for cell staining and analysis | |
| US6750964B2 (en) | Spectral imaging methods and systems | |
| DE60316113T2 (en) | METHOD FOR QUANTITATIVE VIDEO MICROSCOPY AND DEVICE AND COMPUTER PROGRAM FOR IMPLEMENTING THE PROCESS | |
| AU2002334590A1 (en) | Method quantitative video-microscopy and associated system and computer software program product | |
| BRPI0610115A2 (en) | methods for image analysis based on chromogen separation | |
| US7582872B2 (en) | Volumetric spectral imaging | |
| Rogers | Photometrie measurements of grain density in autoradiographs | |
| US20220228968A1 (en) | Hyperspectral quantitative imaging cytometry system | |
| Holmquist et al. | A microspectrophotometric study of Papanicolaou-stained cervical cells as an aid in computerized image processing. | |
| Caspersson | Techniques for cytochemical studies of the nucleus and its substructures | |
| Dhale | Advanced Techniques in Plant Sciences | |
| AU2007249081A1 (en) | Method quantitative video-microscopy and associated system and computer software program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AK | Designated states |
Kind code of ref document: A2 Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE HU IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK TJ TM TR TT UA UG UZ VN |
|
| AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): KE LS MW SD SZ UG AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN ML MR NE SN TD TG |
|
| DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
| REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
| NENP | Non-entry into the national phase |
Ref country code: CA |
|
| 122 | Ep: pct application non-entry in european phase |