CA1290060C - Interlaboratory quality assurance program - Google Patents
Interlaboratory quality assurance programInfo
- Publication number
- CA1290060C CA1290060C CA000533719A CA533719A CA1290060C CA 1290060 C CA1290060 C CA 1290060C CA 000533719 A CA000533719 A CA 000533719A CA 533719 A CA533719 A CA 533719A CA 1290060 C CA1290060 C CA 1290060C
- Authority
- CA
- Canada
- Prior art keywords
- performance
- pool
- apparatuses
- precision
- accuracy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N35/00—Automatic analysis not limited to methods or materials provided for in any single one of groups G01N1/00 - G01N33/00; Handling materials therefor
- G01N35/00584—Control arrangements for automatic analysers
- G01N35/00722—Communications; Identification
- G01N35/00871—Communications between instruments or with remote terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N35/00—Automatic analysis not limited to methods or materials provided for in any single one of groups G01N1/00 - G01N33/00; Handling materials therefor
- G01N35/00584—Control arrangements for automatic analysers
- G01N35/00594—Quality control, including calibration or testing of components of the analyser
- G01N35/00613—Quality control
- G01N35/00623—Quality control of instruments
- G01N2035/00653—Quality control of instruments statistical methods comparing labs or apparatuses
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N35/00—Automatic analysis not limited to methods or materials provided for in any single one of groups G01N1/00 - G01N33/00; Handling materials therefor
- G01N35/00584—Control arrangements for automatic analysers
- G01N35/00594—Quality control, including calibration or testing of components of the analyser
- G01N35/00712—Automatic status testing, e.g. at start-up or periodic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
ABSTRACT
An improved procedure for statistically evaluating the performance, precision and/or accuracy, of at least one analytical instrument in a pool of like or substantially like instruments by comparing the performance of the one instrument located in one laboratory with the performance of like instruments in other laboratories. The procedure includes the step of quantitatively analyzing a reference or control specimen of known constituents by each instrument in the pool and collecting the resultant data from the one instrument in the pool and of the pool instruments. The mean data from the one instrument and of the pool instruments are mathematically correlated in a series of statistical equations for computing a novel, sensitive index known as the "Instrument Performance Index" or IPI.
This IPI can be tabulated and/or graphically displayed, preferably as a single point in a Cartesian coordinate system, whereby an operator can tell at a glance the performance level with respect to both precision and accuracy of the one instrument in the pool. The ultimate IPI is composed of a novel Coefficient of Variation Index (CVI), which is a valuable measure of instrument precision, and the Standard Deviation Index (SDI), which is a measure of performance accuracy. Both the CVI and SDI are derived values according to the invention and are capable of tabulation and display.
An improved procedure for statistically evaluating the performance, precision and/or accuracy, of at least one analytical instrument in a pool of like or substantially like instruments by comparing the performance of the one instrument located in one laboratory with the performance of like instruments in other laboratories. The procedure includes the step of quantitatively analyzing a reference or control specimen of known constituents by each instrument in the pool and collecting the resultant data from the one instrument in the pool and of the pool instruments. The mean data from the one instrument and of the pool instruments are mathematically correlated in a series of statistical equations for computing a novel, sensitive index known as the "Instrument Performance Index" or IPI.
This IPI can be tabulated and/or graphically displayed, preferably as a single point in a Cartesian coordinate system, whereby an operator can tell at a glance the performance level with respect to both precision and accuracy of the one instrument in the pool. The ultimate IPI is composed of a novel Coefficient of Variation Index (CVI), which is a valuable measure of instrument precision, and the Standard Deviation Index (SDI), which is a measure of performance accuracy. Both the CVI and SDI are derived values according to the invention and are capable of tabulation and display.
Description
~J9~160 INTERLABORATORY QUALITY ASSURANCE PROGRAM
This invention relates to an improved procedure for evaluating the performance of at least one apparatus in a pool of like apparatuses. Such performance evaluation lies within the category of quality assurance or QA and quality control or qc. In the field of biomedical and laboratory equipment and apparatus, the QA can be accomplished by comparing the performance of a specific type of equipment or apparatus, located in one laboratory, with the performance of like apparatuses in other laboratories, often in other cities and states. Such an evaluation program is called an Interlaboratory Quality Assurance Program -- IQAP.
The field of this invention is directed to statistical comparisons of apparatus performance. Accordingly, to set forth the invention in a meaningful environment and best mode, a specific type of laboratory apparatus needs to be identified. One such apparatus is a blood analyzer sold by Coulter Electronics, Inc. under the trademark COULTER COUNTER~ and is exemplified by U.S. Patent 3,549,994. Such an apparatus was called the Model S and has been modified and improved over the past seventeen years to become two full series of semi-automatic hematology analyzers, well known throughout the world.
The need for a highly informative and readily usable IQAP for laboratory equipment, especially hematology analyzers, is self-evident. These apparatuses are complex and provide quantitative data on numerous blood parameters, for example: the red blood cell count (RBC), hematocrit (Hct), hemoglobin (Hgb), mean corpuscular volume (MCV), mean corpuscular hemoglobin (MCH), the mean corpuscular hemoglobin concentration (MCHC), and the white blood cell count (WBC).
Definitions of each of these parameters are set forth in U.S. Patent 3,549,994. Quantitative data also can be obtained from other components of the blood, for example, platelets, neutrophils, eosinophils, lymphocytes, and monocytes, by use of current generation COULTER COUNTER analyzers.
Quality control for hematology equipment is a relatively complex issue for two essential reasons, the first being the lack of standards, i.e. materials with a known number of analytical variables;
-and the second being the number of analytical variables that can affect test result performance. Other significant factors affecting the relative accuracy of test results can be (1) changes in calibration, (2) specimen atability or shelf life, (3) reagent condition, and (4) instrument performance.
Many hematology apparatus quality control techniques have been utiliæed in the laboratory and are in use today. They are:
calibration, commercial controls, XB analysis (called "X bar B"), interlaboratory comparison, and technologist review.
- Calibration is a method for achieving or "setting" instrument accuracy, namely by adjusting the instrument to duplicate a single assay value of a calibrator, such as fresh whole blood, values of which were assigned by a reference method.
Commercial control, while satisfactory, presents a problem of formulation which can make a substantial difference in control stability and performance. The impact of this problem, to a large degree, can be lessened by matching the controls to the instruments and utilizing compatible reagents.
XB analysis is a statistical technique utilizing patient results for continually monitoring COULTER COUNTER analyzers and other apparatus. As conceived by Dr. Brian Bull of Loma Linda University, the XB algorithm is based on the demonstrated stability of the erythrocyte (red cell) indices, mean corpuscular volume (~CV), mean corpuscular hemoglobin (MCH), and mean corpuscular hemoglobin concentration (MCHC) of select patient populations.
Daily instruments checks monitor calibration stability, reproducibility, background counts for confirming the proper functioning of reagent level, diluting and dispensing systems, and general instrument performance.
Interlaboratory comparison programs objectively establish the quality of a laboratory's performance referenced to peer performance for maintaining high performance levels in accordance with the various accreditation criteria of many state, regulatory, and professional agencies.
While each of the above techniques are satisfactory and have permitted achieving a hematologically acceptable degree of control 1?~
over the accuracy of analytical apparatus, thereby obtaining accurate information about a blood sample, each does suffer from some inherent disadvantages as follows: Whole blood calibration is time consuming.
It requires a large quantity of fresh whole blood. As pointed out by Gilmer and Williams in "The Status of Methods of Calibration in Hematology", Am. J. Clin. Path. 74(4):600, "there is always the real possibility that the mean value obtained may not be accurate".
Commercial controls can lead to serious analytical errors through improper use, in storage and handling. Furthermore, there are many circumstances where controls are not practical. XB analysis is not applicable when patient populations are non-representative so as to generate outlying indices, i.e. neonates and oncology patients.
Furthermore, XB only monitors ability of original calibration, which could be in error. Also, other generally measured parameters, such as with white cells or platelets, are not monitored by the XB analysis.
Professional interlaboratory comparison programs often lead to erroneous interpretation because of specimen/instrument incompatability. The use of a technologist review is at best arbitrary and depends on the skill level of the technologist.
Statistical quality control systems, which aim at determining the statistical distribution of certain quality characteristics of a sample, are disclosed in U.S. Patent 3,151,237. Other types of statistical systems are disclosed in U.S. Patent 4,320,463, in "Medical Research" by John M. England in Chapter l thereof, and in "Statistics in Medicine" by Theodore Colton, Sc.D. published by Little, Brown and Company, Boston, Eight Printing ~QA-276-Co-1974).
This invention is an improved procedure for evaluating the accuracy and/or precision of the performance of at least one apparatus in a pool of like apparatuses with respect to the arithmetic mean accuracy and/or precision of the performan~e of all the apparatuses in the pool. The apparatuses discussed in the preferred embodiment are hematology analyzing apparatuses, although other medical and biological apparatus can well benefit from the invention.
The novel procedure of this invention involves quantitatively analy~ing a reference or cell control specimen of known constituents by each apparatus in the pool and collecting the resultant data from ,:
the one apparatus in the pool and the a~ithmetic mean data of the pool apparatus. The data of the one apparatus and of the pool apparatuses are correlated mathematically in a series of statistical equations for computing a novel, sensitive index herein named the "Instrument Performance Index" or IPI. This IPI constitutes a major feature of this invention. This IPI preferably is displayed as a single point in a Cartesian graph, although other graphical and displaying forms of indicia are within the scope of this invention.
This IPI permits an operator of the one apparatus to determine readily the degree of accuracy and/or precision, or conversely the inaccuracy and/or imprecision, of the one apparatus as compared to the accuracy and/or precision of the pool apparatuses. If the IPI
indicates an inaccuracy and/or imprecision, the laboratory can consider directly making necessary adjustments or corrections to their apparatus, its environmental conditions, the laboratory routine, the handling and condition of the blood cell control material, etc., for again achieving optimum accuracy and/or precision of the one apparatus. Or, the laboratory can call upon the skill of the customer service department of the manufacturer who, guided by the IQAP and the IPI for each measured parameter, can direct and/or implement the necessary corrections and adjustments.
The reference control or cell sample analyzed by the apparatuses in performance of this invention is preferably ~hole blood. Other control sample materials can be employed, for example those which contain modified blood components, or components for simulating fresh whole blood, as disclosed in U.S. Patents 4,299,726 and 4,405,719 assigned to Coulter Electronics, Inc. Commercially available hematology instrument controls useful in practicing this IQAP
invention are 4C~ and 4C~ Plus cell controls produced by the Diagnostics Division of Coulter Electronics, Inc.
This invention is not limited to hematology analyzers of the COIJLTER COUNTER type, but can be successfully implemented for use with a wide range of apparatuses that would benefit from interlaboratory quality assurance programming.
.
~l;?J~
By way of example, illustrative embodiments of the invention now will be described with references to the accompanying drawings in which:
Figures lA-lC illustrate conventional characteristic3 of accuracy and precision;
Figures 2A-2D illustrate conventional characteristic distribution patterns that differentiate shifts from trends;
Figure 3 illustrates a conventional curve of normal distribution showing a comparison of the standard deviation values of a normal distribution; and Figure 4 illustrates a portion of a novel multiparameter hematology IQAP report form, displaying the novel Instrument Performance Index (IPI) results as one point on each plot.
This invention is an improved procedure for interlaboratory quality assurance (IQAP) for evaluating the accuracy and/or precision of one apparatus in a pool of like apparatuses. Accuracy and precision are terms describing two performance characteristics that ` determine analytical quality. Accuracy describes the closeness of a measured value to the true or accepted value. Precision describes the closeness of successive measurement of the same value to each other, i.e. reproducibility. The targets illustrated in Figures lA-lC
illustrate the characteristics of accuracy and precision. The different arrow locations in the target of Figure lA represent poor precision, although one arrow is in the bullseye. The arrows in Figure lB are in exactly the same location, thereby representing high precision, but also exemplifying inaccurate performance. Figure lC
shows a precision and accuracy indicative of especially good performance, since all the arrows are in the bullseye. It is therefore readily apparent that accuracy and precision are separate and distinct functions; however, this invention correlates both accuracy and precision and presents such as a single indicia (IPI); in contrast to prior art IQAP techniques and resulting data formats.
Shifts and trends are known terms describing successive mean-value distribution patterns that indicate developing changes. A
shift describes the distribution of six or more successive values, either above or belo~ the target mean. A trend describes the distribution of six or more successive values in one direction, either up or down. A trend can begin above or below the target mean and cross the mean as it develops. Figures 2A and 2B show, between dashed lines, an upward and a downward shift, respectively; these shifts having relatively flat distribution patterns compared with the upward and downward trend patterns of Figures 2C and 2D. Shifts can be caused by changes in calibration, the concentration of reagents, or the control or assay sample. Trends can be attributed to gradual changes in the reagent or the instrument. This invention can indicate both shifts and trends types of imprecision with a single data indicia, e.g. the Instrument Performance Index point (IPI).
Inasmuch as our invention is a novel procedure which employs statistical formulas from which the quantitative analytical data from one instrument is compared with like data from a pool of like instruments, to obtain our novel Coefficient of Variation Index (CVI) and our novel Instrument Performance Index (IPI), the statistical formulas should be set forth herein, even though most of them are well known.
I~ATHEMATICAL FORMULAS FOR CALCULATING INDICES
1. Mean Value (a) The calculation for each laboratory instrument mean value (a) iS:
Ex a = n wherein (a) is the mean of the analytical data values derived from the tests (x) from each laboratory instrument, using the same control sample; (E) (representative of the Çreek letter Sigma) means the sum of; and (n) is the number of tests. The symbol ~a) sometimes is referred to as "X barl'.
Mean values are used to statistically measure accuracy against the "known" measurements, i.e. control values. Mean values should be calculated using not less than 16 replicate samples for assuring statistical significance. An n of 16 reduces the error of the mean to o less than one standard deviation (SD). The formula of the standard error of the mean (SEM) is:
SEM = a 2. Pool Mean (b) The calculation for the pool mean (b) is:
b =
np wherein the pool mean (b) is the average of the arithmetic mean values (a) generated by all of the laboratories, including yours; and (np) is the number of reporting laboratories in the pool. The term "peer group" often is employed as meaning pool.
.
This invention relates to an improved procedure for evaluating the performance of at least one apparatus in a pool of like apparatuses. Such performance evaluation lies within the category of quality assurance or QA and quality control or qc. In the field of biomedical and laboratory equipment and apparatus, the QA can be accomplished by comparing the performance of a specific type of equipment or apparatus, located in one laboratory, with the performance of like apparatuses in other laboratories, often in other cities and states. Such an evaluation program is called an Interlaboratory Quality Assurance Program -- IQAP.
The field of this invention is directed to statistical comparisons of apparatus performance. Accordingly, to set forth the invention in a meaningful environment and best mode, a specific type of laboratory apparatus needs to be identified. One such apparatus is a blood analyzer sold by Coulter Electronics, Inc. under the trademark COULTER COUNTER~ and is exemplified by U.S. Patent 3,549,994. Such an apparatus was called the Model S and has been modified and improved over the past seventeen years to become two full series of semi-automatic hematology analyzers, well known throughout the world.
The need for a highly informative and readily usable IQAP for laboratory equipment, especially hematology analyzers, is self-evident. These apparatuses are complex and provide quantitative data on numerous blood parameters, for example: the red blood cell count (RBC), hematocrit (Hct), hemoglobin (Hgb), mean corpuscular volume (MCV), mean corpuscular hemoglobin (MCH), the mean corpuscular hemoglobin concentration (MCHC), and the white blood cell count (WBC).
Definitions of each of these parameters are set forth in U.S. Patent 3,549,994. Quantitative data also can be obtained from other components of the blood, for example, platelets, neutrophils, eosinophils, lymphocytes, and monocytes, by use of current generation COULTER COUNTER analyzers.
Quality control for hematology equipment is a relatively complex issue for two essential reasons, the first being the lack of standards, i.e. materials with a known number of analytical variables;
-and the second being the number of analytical variables that can affect test result performance. Other significant factors affecting the relative accuracy of test results can be (1) changes in calibration, (2) specimen atability or shelf life, (3) reagent condition, and (4) instrument performance.
Many hematology apparatus quality control techniques have been utiliæed in the laboratory and are in use today. They are:
calibration, commercial controls, XB analysis (called "X bar B"), interlaboratory comparison, and technologist review.
- Calibration is a method for achieving or "setting" instrument accuracy, namely by adjusting the instrument to duplicate a single assay value of a calibrator, such as fresh whole blood, values of which were assigned by a reference method.
Commercial control, while satisfactory, presents a problem of formulation which can make a substantial difference in control stability and performance. The impact of this problem, to a large degree, can be lessened by matching the controls to the instruments and utilizing compatible reagents.
XB analysis is a statistical technique utilizing patient results for continually monitoring COULTER COUNTER analyzers and other apparatus. As conceived by Dr. Brian Bull of Loma Linda University, the XB algorithm is based on the demonstrated stability of the erythrocyte (red cell) indices, mean corpuscular volume (~CV), mean corpuscular hemoglobin (MCH), and mean corpuscular hemoglobin concentration (MCHC) of select patient populations.
Daily instruments checks monitor calibration stability, reproducibility, background counts for confirming the proper functioning of reagent level, diluting and dispensing systems, and general instrument performance.
Interlaboratory comparison programs objectively establish the quality of a laboratory's performance referenced to peer performance for maintaining high performance levels in accordance with the various accreditation criteria of many state, regulatory, and professional agencies.
While each of the above techniques are satisfactory and have permitted achieving a hematologically acceptable degree of control 1?~
over the accuracy of analytical apparatus, thereby obtaining accurate information about a blood sample, each does suffer from some inherent disadvantages as follows: Whole blood calibration is time consuming.
It requires a large quantity of fresh whole blood. As pointed out by Gilmer and Williams in "The Status of Methods of Calibration in Hematology", Am. J. Clin. Path. 74(4):600, "there is always the real possibility that the mean value obtained may not be accurate".
Commercial controls can lead to serious analytical errors through improper use, in storage and handling. Furthermore, there are many circumstances where controls are not practical. XB analysis is not applicable when patient populations are non-representative so as to generate outlying indices, i.e. neonates and oncology patients.
Furthermore, XB only monitors ability of original calibration, which could be in error. Also, other generally measured parameters, such as with white cells or platelets, are not monitored by the XB analysis.
Professional interlaboratory comparison programs often lead to erroneous interpretation because of specimen/instrument incompatability. The use of a technologist review is at best arbitrary and depends on the skill level of the technologist.
Statistical quality control systems, which aim at determining the statistical distribution of certain quality characteristics of a sample, are disclosed in U.S. Patent 3,151,237. Other types of statistical systems are disclosed in U.S. Patent 4,320,463, in "Medical Research" by John M. England in Chapter l thereof, and in "Statistics in Medicine" by Theodore Colton, Sc.D. published by Little, Brown and Company, Boston, Eight Printing ~QA-276-Co-1974).
This invention is an improved procedure for evaluating the accuracy and/or precision of the performance of at least one apparatus in a pool of like apparatuses with respect to the arithmetic mean accuracy and/or precision of the performan~e of all the apparatuses in the pool. The apparatuses discussed in the preferred embodiment are hematology analyzing apparatuses, although other medical and biological apparatus can well benefit from the invention.
The novel procedure of this invention involves quantitatively analy~ing a reference or cell control specimen of known constituents by each apparatus in the pool and collecting the resultant data from ,:
the one apparatus in the pool and the a~ithmetic mean data of the pool apparatus. The data of the one apparatus and of the pool apparatuses are correlated mathematically in a series of statistical equations for computing a novel, sensitive index herein named the "Instrument Performance Index" or IPI. This IPI constitutes a major feature of this invention. This IPI preferably is displayed as a single point in a Cartesian graph, although other graphical and displaying forms of indicia are within the scope of this invention.
This IPI permits an operator of the one apparatus to determine readily the degree of accuracy and/or precision, or conversely the inaccuracy and/or imprecision, of the one apparatus as compared to the accuracy and/or precision of the pool apparatuses. If the IPI
indicates an inaccuracy and/or imprecision, the laboratory can consider directly making necessary adjustments or corrections to their apparatus, its environmental conditions, the laboratory routine, the handling and condition of the blood cell control material, etc., for again achieving optimum accuracy and/or precision of the one apparatus. Or, the laboratory can call upon the skill of the customer service department of the manufacturer who, guided by the IQAP and the IPI for each measured parameter, can direct and/or implement the necessary corrections and adjustments.
The reference control or cell sample analyzed by the apparatuses in performance of this invention is preferably ~hole blood. Other control sample materials can be employed, for example those which contain modified blood components, or components for simulating fresh whole blood, as disclosed in U.S. Patents 4,299,726 and 4,405,719 assigned to Coulter Electronics, Inc. Commercially available hematology instrument controls useful in practicing this IQAP
invention are 4C~ and 4C~ Plus cell controls produced by the Diagnostics Division of Coulter Electronics, Inc.
This invention is not limited to hematology analyzers of the COIJLTER COUNTER type, but can be successfully implemented for use with a wide range of apparatuses that would benefit from interlaboratory quality assurance programming.
.
~l;?J~
By way of example, illustrative embodiments of the invention now will be described with references to the accompanying drawings in which:
Figures lA-lC illustrate conventional characteristic3 of accuracy and precision;
Figures 2A-2D illustrate conventional characteristic distribution patterns that differentiate shifts from trends;
Figure 3 illustrates a conventional curve of normal distribution showing a comparison of the standard deviation values of a normal distribution; and Figure 4 illustrates a portion of a novel multiparameter hematology IQAP report form, displaying the novel Instrument Performance Index (IPI) results as one point on each plot.
This invention is an improved procedure for interlaboratory quality assurance (IQAP) for evaluating the accuracy and/or precision of one apparatus in a pool of like apparatuses. Accuracy and precision are terms describing two performance characteristics that ` determine analytical quality. Accuracy describes the closeness of a measured value to the true or accepted value. Precision describes the closeness of successive measurement of the same value to each other, i.e. reproducibility. The targets illustrated in Figures lA-lC
illustrate the characteristics of accuracy and precision. The different arrow locations in the target of Figure lA represent poor precision, although one arrow is in the bullseye. The arrows in Figure lB are in exactly the same location, thereby representing high precision, but also exemplifying inaccurate performance. Figure lC
shows a precision and accuracy indicative of especially good performance, since all the arrows are in the bullseye. It is therefore readily apparent that accuracy and precision are separate and distinct functions; however, this invention correlates both accuracy and precision and presents such as a single indicia (IPI); in contrast to prior art IQAP techniques and resulting data formats.
Shifts and trends are known terms describing successive mean-value distribution patterns that indicate developing changes. A
shift describes the distribution of six or more successive values, either above or belo~ the target mean. A trend describes the distribution of six or more successive values in one direction, either up or down. A trend can begin above or below the target mean and cross the mean as it develops. Figures 2A and 2B show, between dashed lines, an upward and a downward shift, respectively; these shifts having relatively flat distribution patterns compared with the upward and downward trend patterns of Figures 2C and 2D. Shifts can be caused by changes in calibration, the concentration of reagents, or the control or assay sample. Trends can be attributed to gradual changes in the reagent or the instrument. This invention can indicate both shifts and trends types of imprecision with a single data indicia, e.g. the Instrument Performance Index point (IPI).
Inasmuch as our invention is a novel procedure which employs statistical formulas from which the quantitative analytical data from one instrument is compared with like data from a pool of like instruments, to obtain our novel Coefficient of Variation Index (CVI) and our novel Instrument Performance Index (IPI), the statistical formulas should be set forth herein, even though most of them are well known.
I~ATHEMATICAL FORMULAS FOR CALCULATING INDICES
1. Mean Value (a) The calculation for each laboratory instrument mean value (a) iS:
Ex a = n wherein (a) is the mean of the analytical data values derived from the tests (x) from each laboratory instrument, using the same control sample; (E) (representative of the Çreek letter Sigma) means the sum of; and (n) is the number of tests. The symbol ~a) sometimes is referred to as "X barl'.
Mean values are used to statistically measure accuracy against the "known" measurements, i.e. control values. Mean values should be calculated using not less than 16 replicate samples for assuring statistical significance. An n of 16 reduces the error of the mean to o less than one standard deviation (SD). The formula of the standard error of the mean (SEM) is:
SEM = a 2. Pool Mean (b) The calculation for the pool mean (b) is:
b =
np wherein the pool mean (b) is the average of the arithmetic mean values (a) generated by all of the laboratories, including yours; and (np) is the number of reporting laboratories in the pool. The term "peer group" often is employed as meaning pool.
.
3. Standard Deviation (SD) The calculation for the standard deviation (SD) is:
SD =
n - 1 wherein (E), (x), (a) and (n) are defined herein above.
Standard deviation (SD) is a measure of the spread of values about a mean value. In quality control (QC), the standard deviation is used to measure precision and generally is expressed in terms of the units used to measure the mean. Precision of your system is evaluated by comparing its standard deviation to the standard deviation values of a pool, as represented by the typical normal distribution curve Figure 3, where 68% of all the values will fall within _1.0 measurement unit (1 SD) of the mean, and 95% of all values -~ will fall within +2 measurement units of the mean (2 SD). In QC, standard deviation values within +2 SD range denote an acceptable result and standard deviation values within +l SD range indicate an even higher degree of accuracy.
~L~9~)~60 4. SD of the Mean (MSD) The formula for calculating the MSD is:
Ea2 _ (Ea)2/L
MSD =
~ L ~ 1 wherein the symbols (E), and (a~ are defined above and (L) represents the number of laboratory instruments.
SD of the pool mean is the SD calculated from the average of all laboratories' mean values (MSD) and represents the spread of individual means about the pool mean. The MSD is used as the SD value in the calculation of the Standard Deviation Index (SDI) in calculation No. 8 below.
SD =
n - 1 wherein (E), (x), (a) and (n) are defined herein above.
Standard deviation (SD) is a measure of the spread of values about a mean value. In quality control (QC), the standard deviation is used to measure precision and generally is expressed in terms of the units used to measure the mean. Precision of your system is evaluated by comparing its standard deviation to the standard deviation values of a pool, as represented by the typical normal distribution curve Figure 3, where 68% of all the values will fall within _1.0 measurement unit (1 SD) of the mean, and 95% of all values -~ will fall within +2 measurement units of the mean (2 SD). In QC, standard deviation values within +2 SD range denote an acceptable result and standard deviation values within +l SD range indicate an even higher degree of accuracy.
~L~9~)~60 4. SD of the Mean (MSD) The formula for calculating the MSD is:
Ea2 _ (Ea)2/L
MSD =
~ L ~ 1 wherein the symbols (E), and (a~ are defined above and (L) represents the number of laboratory instruments.
SD of the pool mean is the SD calculated from the average of all laboratories' mean values (MSD) and represents the spread of individual means about the pool mean. The MSD is used as the SD value in the calculation of the Standard Deviation Index (SDI) in calculation No. 8 below.
5. Pool SD ~PSD) The formula for calculating the PSD is:
S ¦ E [E(x - a)2]
n - L
wherein (E), (L), (x), (a) and (n) are as defined above. The PSD
is the weighted average of all laboratories' SD values. It represents the average precision of all laboratories and is used in the calculation of Pool Coefficient of Variation (PCV) in calculation No.
7 below.
S ¦ E [E(x - a)2]
n - L
wherein (E), (L), (x), (a) and (n) are as defined above. The PSD
is the weighted average of all laboratories' SD values. It represents the average precision of all laboratories and is used in the calculation of Pool Coefficient of Variation (PCV) in calculation No.
7 below.
6. Coefficient of Variation (CV) The formula for calculating the CV is:
SD x 100 CV = a wherein (a) and (SD) are as defined above. The coefficient of variation is the standard deviation value expressed as a percentage of the mean value. The system CV, as calculated above, is an integral part o f this invention. The comparison of an individual system CV to ~o~
the pool CV is an index of the system imprecision as defined in calculation No. 9 below.
SD x 100 CV = a wherein (a) and (SD) are as defined above. The coefficient of variation is the standard deviation value expressed as a percentage of the mean value. The system CV, as calculated above, is an integral part o f this invention. The comparison of an individual system CV to ~o~
the pool CV is an index of the system imprecision as defined in calculation No. 9 below.
7. Pool Coefficient of Variation (PCV) The formula for calculating the PCV is:
PSD x 100 PCV = b The PCV is the PSD expressed as a percentage of the pool mean (b). It represents pool precision and is used in the calculation of system imprecision in calculation 9 below. In general terms, the PCV states the average imprecision of the pool against which your system imprecision is measured.
PSD x 100 PCV = b The PCV is the PSD expressed as a percentage of the pool mean (b). It represents pool precision and is used in the calculation of system imprecision in calculation 9 below. In general terms, the PCV states the average imprecision of the pool against which your system imprecision is measured.
8. Standard Deviation Index (SDI) The formula for calculating the SDI is:
a - b SDI =
MSD
The Standard Deviation Index is a comparative measure of the closeness of your mean value to the pool mean. This invention utilizes the SDI
value as an index of system accuracy relative to the pool mean. An SDI of greater than ~2 identifies individual system inaccuracy.
a - b SDI =
MSD
The Standard Deviation Index is a comparative measure of the closeness of your mean value to the pool mean. This invention utilizes the SDI
value as an index of system accuracy relative to the pool mean. An SDI of greater than ~2 identifies individual system inaccuracy.
9. Coefficient of Variation Index (CVI) The formula for calculating the CVI is:
CV
CVI =
PCV
The Coefficient of Variation Index is a novel comparative measure of your system imprecision relative to that of the pool imprecision.
This invention utilizes this CVI measurement to identify systems with poor precision when referenced to the pool precision.
~Q(~6~
The thus obtained Standard Deviation Index (SDI) and new Coefficient of Variation Index (CVI) are combined by our invention to constitute the set of coordinates in the Cartesian coordinate system of a graph. This set is plotted as a dot on a graph as illustrated in Figure 4. This focal point constitutes the novel Instrument Performance Index (IPI) of our invention. The location of the IPI
visually indicates the precision and accuracy of the instrument or apparatus resulting from the performance of one apparatus as compared to the precision and accuracy of the pool of like apparatuses. This Instrument Performance Index (IPI) is a novel and unexpected derived indicia, more specifically, a dot or a point, that denotes the combined influence of previously independent system performance indicators.
As discussed hereinabove, this invention can provide an improvement in IQAP for hematology analyzers which measure several blood parameters. Each analyzer (generic term being apparatus) in the peer group pool, is of the same type and utilizes the same test sample or specimen, examples of which are 4C~ and 4C~ Plus cell control. To be the "same" test sample would mean that such a cell control is from the same manufacturing lot, so that the multiparameter blood constituents therein are identical for all apparatus in the pool.
Likewise, our invention advantageously can be employed in an IQAP for chemical analyzers, one example being the COULTER~ DACOS~ chemistry system, which can analyze for numerous different analytes; and control solutions exist which can act as the sample material.
The resulting plotting of the Instrument Performance Index (IPI) point on each of several graphs is shown in Figure 4, with respect to seven different blood parameters, as can be measured by a COULTER
COUNTER Model S type analyzer, using as the test or specimen material 4C or 4C Plus cell control lot number 443; the participating laboratory being PRH 205. Portions of the individual graphs can be color keyed, shaded, or otherwise differentiated to enhance the visual recognition of desirable and undesirable IPI indicia positions. The desirable/acceptable performance limits shown in Figure 4 lie within a rectangle defined by +2 SDI along the baseline and 2 CVI, which is differently "framed" from the areas exterior of such rectangle on the graph. The closer the IPI dot is to the "MEAN" line which is midway between +1 SDI and -1 SDI, the greater i5 the accuracy of the instrument. The closer the dot is to the baseline, the greater is the precision. In coordinate system technology, a middle or mean distance along the ordinate or X-axis designates best accuracy; whereas, a zero value for the abscissa or CVI is optimal. Hence, the closer that the IPI single indicia is to both the MEAN line and the baseline, the better is the system performance with respect to both accuracy and precision, respectively.
In the WBC plot of Figure 4, the IPI dot exactly is centered on the MEAN line and is extremely close to the baseline. Hence, this IPI
plot represents a near perfect performance, both accurate ~SDI) and precise (CVI) relative to the pool data. The RBC graph in Figure 4 shows the single indicia centered on the mean line, but at a height of 2.5 CVI. Here the performance is highly accurate (SDI), but compared wi`th the pool of like apparatuses (CVI to baseline), it is imprecise.
The Hgb plot shows the dot close to the CVI baseline, but located at -2.5 SDI. This illustrates good precision, but unacceptable accuracy.
The performance of the PRH 205 instrument with respect to Hct is both poor in accuracy and poor in precision, since the IPI dot is located too high on the rectangular plot and too far from the MEAN line. As easily seen, the IPI indicia in all three plots RBC, Hgb and Hct lie outside of the acceptable performance rectangle. The IPI resultant for each of MCV, PLT, and MPV is within acceptable precision and accuracy limits as defined by the pool data, with the MPV correlation being the best of these three.
The interpretative value of these performance matrix plots is immense. One can tell at a glance the relative accuracy and precision of his intru~ent, or conversely its inaccuracy and imprecision, compared to all other participants in the pool.
Notwithstanding the significant improvement our IPI provides to IQAP when the SDI and CVI values are a single point on a Cartesian coordinate system, the SDI and CVI values themselves are an improvement over the prior art IQAP and can be presented as a pair or set of values for operator consideration. This set of SDI and CYI
values can be listed on a table of resultant values, as will be expained with respect to Table III herein below.
Additionally, the novel Coefficient of Variation Index (CVI~, by itself, provides precision information not previously available and it can be derived easily, without need of calculations 4 and 8 for display, printout, etc.
The data needed from each instr~nent of the peer group pool can be obtained by various means, depending upon the sophistication of the instruments and the IQAP processing center. Fully automatic data transfer by telemetry or remote data acquisition, semi-automatic data transfer by telephone-modem or facsimile equipment, and primarily manual data transfer all are within the skill of the art. Next will be described, with reference to Tables I and II, a primarily manual mode of data commun;cation from the laboratories to the IQAP
processing center, followed by the data handling and resultant development at the processing center, and then transmittal of the IQAP
report to each laboratory.
To facilitate raw data transmittal from each IQAP laboratory, there can be a simple data entry form, the top portion of an example of which is shown partially completed in Table I. Depending upon the apparatus, the identity of the test parameters can be in columns, with the individual entries, for example one each day of the month, manually listed therebelow. A daily entry for approximately one month can provide statistically sufficient data. It is to be remembered that all numbers of the IQAP pool will be using the "same" test sample for the same reporting period. At the end of the reporting period, the data entry form would be mailed from each pool laboratory to the IQAP processing center. The IQAP processing center then would compile all of the data, perform the computations to obtain the CVI, SDI and IPI, preferably generate the IPI plots shown in Figure 4, as well as the data of Table II discussed below and, where needed, give guidance to each laboratory for improving the precision and accuracy of its apparatus and its system. The term "system" is used herein to include variables beyond that of the apparatus per se and includes laboratory operations which can influence the performance of the apparatus.
~.~9~
Table II presents the individual and comparative I~AP statistics and the ultimately derived values for SDI and CVI. The term "Y~UR"
designates the one apparatus being compared to all of the apparatuses in the pool. From left to right, the first columns of Table II list:
each tested parameter, the assay values of the sample or specimen material, the mean (a) for your apparatus, and the mean (b) for the pool apparatuses. The next three columns list the Standard Deviations: for your apparatus (SD), the pool (PSD), and the Standard Deviation of the Mean (MSD). The next four columns depict the Coefficient of Variation ~CV) for your apparatus for the present reporting period and the two previous reporting periods, and that of the pool apparatuses (PCV). Next listed in Table II is the number of laboratories (LABS) including your laboratory and the number of test values (n) included in the calculations. The last two columns present the Standard Deviation Index (SDI) and Coefficient of Variation Index (CVI), respectively.
The data presented in Table I, Table II and the IPI indicia of Figure 4 are not based upon a common set of information, nor derived from one another. However, Table II raw data was obtained from a peer group of hematology instruments and the derived data in Table II was developed from such raw data by use of the procedure of this invention.
Now with reference to the top row of information in Table II, which concerns the white blood cell count (WBC) parameter, and reading from left to right, the assay value 7.4 provides a reference for the reader of Table II in considering the adjacent values of the mean of his apparatus and the pool apparatuses. However, this assay value is not utilized as part of the data base of the statistics for the procedure of our invention. In this example, the mean value (a) of your instrument is 7.8 and pool mean (b) is 7.28, which when compared to the assay value of 7.4 would indicate that your mean value not only is further from the assay value than is the pool value, but also that your mean value is greater than the assay value; whereas, the pool mean value is less than the assay value. Hence, your WBC data initially is suspect. Your SD is .11, the PSD is .15, and the MSD is .12. This set of data would indicate that, with respect to Standard ~'~90~
Deviation, the operation of your instrument is better than that of the pool. Your current CV is 1.41, that of the pool is 2.00, and your previous two months were 1.76 and 2.15, respectively. Inspection of this set of data shows that your WBC parameter CV is acceptable and has improved in each of the recent three months. The number of laboratories (L) 300 and your tests (n) 28 indicates that your data, the pool data, and therefore the derived data for the WBC analysis are ; statistically sound.
Your SDI of 2.50 and CVI of .71 show that your system performance, with respect to WBC test analysis, is not acceptable, because it is inaccurate, hav;ng a high SDI of 2.50, even though your system performance is precise with a small CVI of .71. If these SDI
and CVI values had been plotted as the IPI point for W~C in Figure 4, that IPI point would have been near the middle of the lowest right end part or box and outside of the acceptable "framed" area.
A similar analysis of the RBC values in the second row of Table II would indicate favorable data, resulting in a SDI of -.88 and a CVI
of .84, both of which are very acceptable and would result in an IPI
plot point lying in the box which is in the lowest row and just to the left of the MEAN line; ie, an indication of high precision and high accuracy and lying well inside the "framed" portion.
Hence, a simple listing of the SDI and CVI values, as in Table II, can be interpreted relatively easily to advise a laboratory of the accuracy and precision, respectively, of its instrument and system for each tested parameter, without need for the ultimate physical plotting of the IPI point. Therefore, this invention permits a subscriber of our improved IQAP to ascertain quickly by visual observation of the single IPI indicia and/or its SDI and CVI components the accuracy and precision of a particular analytical apparatus, instrument and system in comparison to the identical or substantially identical pool of such apparatuses and instruments, with respect to each separate test parameter. Additionally, our novel CVI value itself provides precision information and the SDI value provides accuracy information.
Although there has been disclosed an embodiment directed to blood analyzing apparatus and hematology parameters, it now should be 12,90~
readily apparent that the disclosed invention readily is applicable for determining the accuracy and/or precision of other typPs of analyzers, for example, colorimeters, chromotography apparatus, etc.
Now therefore, variations of our invention lie within the reach and capability of those skilled in the art without departing from the scope and spirit of the invention as next claimed.
a ,~
0 00 _ O ~ _ O
a ~ 0 ~ o 00 ~ O
U~ C~ I _ _ ~q o o oo o o o~
C~ ~ ~; oo o o o o s~
g ~ o~ o U~
o ~ . . . . .
o o o o p ~ ~ _ o U`D ~ ~ ~ ~
~æ ~
3 ~ ~ o 3 a ~1 ~ ~ ~ ~ o a oo ~ o ~ . .
Is --, -- _ -- e 4 ~
~ ~ a ~ ~ ~ oo 0 D ~
u~ o o C~l c~ N C O
~ ~J O -- oo ~~ ~ ;t OD
a æ
~ c~ ~ ~ o ~
, 0 oo ol~ ~ 8 0 P 0c~ J _ O ~
'l:
~ ~ ~ ~ ~ s
CV
CVI =
PCV
The Coefficient of Variation Index is a novel comparative measure of your system imprecision relative to that of the pool imprecision.
This invention utilizes this CVI measurement to identify systems with poor precision when referenced to the pool precision.
~Q(~6~
The thus obtained Standard Deviation Index (SDI) and new Coefficient of Variation Index (CVI) are combined by our invention to constitute the set of coordinates in the Cartesian coordinate system of a graph. This set is plotted as a dot on a graph as illustrated in Figure 4. This focal point constitutes the novel Instrument Performance Index (IPI) of our invention. The location of the IPI
visually indicates the precision and accuracy of the instrument or apparatus resulting from the performance of one apparatus as compared to the precision and accuracy of the pool of like apparatuses. This Instrument Performance Index (IPI) is a novel and unexpected derived indicia, more specifically, a dot or a point, that denotes the combined influence of previously independent system performance indicators.
As discussed hereinabove, this invention can provide an improvement in IQAP for hematology analyzers which measure several blood parameters. Each analyzer (generic term being apparatus) in the peer group pool, is of the same type and utilizes the same test sample or specimen, examples of which are 4C~ and 4C~ Plus cell control. To be the "same" test sample would mean that such a cell control is from the same manufacturing lot, so that the multiparameter blood constituents therein are identical for all apparatus in the pool.
Likewise, our invention advantageously can be employed in an IQAP for chemical analyzers, one example being the COULTER~ DACOS~ chemistry system, which can analyze for numerous different analytes; and control solutions exist which can act as the sample material.
The resulting plotting of the Instrument Performance Index (IPI) point on each of several graphs is shown in Figure 4, with respect to seven different blood parameters, as can be measured by a COULTER
COUNTER Model S type analyzer, using as the test or specimen material 4C or 4C Plus cell control lot number 443; the participating laboratory being PRH 205. Portions of the individual graphs can be color keyed, shaded, or otherwise differentiated to enhance the visual recognition of desirable and undesirable IPI indicia positions. The desirable/acceptable performance limits shown in Figure 4 lie within a rectangle defined by +2 SDI along the baseline and 2 CVI, which is differently "framed" from the areas exterior of such rectangle on the graph. The closer the IPI dot is to the "MEAN" line which is midway between +1 SDI and -1 SDI, the greater i5 the accuracy of the instrument. The closer the dot is to the baseline, the greater is the precision. In coordinate system technology, a middle or mean distance along the ordinate or X-axis designates best accuracy; whereas, a zero value for the abscissa or CVI is optimal. Hence, the closer that the IPI single indicia is to both the MEAN line and the baseline, the better is the system performance with respect to both accuracy and precision, respectively.
In the WBC plot of Figure 4, the IPI dot exactly is centered on the MEAN line and is extremely close to the baseline. Hence, this IPI
plot represents a near perfect performance, both accurate ~SDI) and precise (CVI) relative to the pool data. The RBC graph in Figure 4 shows the single indicia centered on the mean line, but at a height of 2.5 CVI. Here the performance is highly accurate (SDI), but compared wi`th the pool of like apparatuses (CVI to baseline), it is imprecise.
The Hgb plot shows the dot close to the CVI baseline, but located at -2.5 SDI. This illustrates good precision, but unacceptable accuracy.
The performance of the PRH 205 instrument with respect to Hct is both poor in accuracy and poor in precision, since the IPI dot is located too high on the rectangular plot and too far from the MEAN line. As easily seen, the IPI indicia in all three plots RBC, Hgb and Hct lie outside of the acceptable performance rectangle. The IPI resultant for each of MCV, PLT, and MPV is within acceptable precision and accuracy limits as defined by the pool data, with the MPV correlation being the best of these three.
The interpretative value of these performance matrix plots is immense. One can tell at a glance the relative accuracy and precision of his intru~ent, or conversely its inaccuracy and imprecision, compared to all other participants in the pool.
Notwithstanding the significant improvement our IPI provides to IQAP when the SDI and CVI values are a single point on a Cartesian coordinate system, the SDI and CVI values themselves are an improvement over the prior art IQAP and can be presented as a pair or set of values for operator consideration. This set of SDI and CYI
values can be listed on a table of resultant values, as will be expained with respect to Table III herein below.
Additionally, the novel Coefficient of Variation Index (CVI~, by itself, provides precision information not previously available and it can be derived easily, without need of calculations 4 and 8 for display, printout, etc.
The data needed from each instr~nent of the peer group pool can be obtained by various means, depending upon the sophistication of the instruments and the IQAP processing center. Fully automatic data transfer by telemetry or remote data acquisition, semi-automatic data transfer by telephone-modem or facsimile equipment, and primarily manual data transfer all are within the skill of the art. Next will be described, with reference to Tables I and II, a primarily manual mode of data commun;cation from the laboratories to the IQAP
processing center, followed by the data handling and resultant development at the processing center, and then transmittal of the IQAP
report to each laboratory.
To facilitate raw data transmittal from each IQAP laboratory, there can be a simple data entry form, the top portion of an example of which is shown partially completed in Table I. Depending upon the apparatus, the identity of the test parameters can be in columns, with the individual entries, for example one each day of the month, manually listed therebelow. A daily entry for approximately one month can provide statistically sufficient data. It is to be remembered that all numbers of the IQAP pool will be using the "same" test sample for the same reporting period. At the end of the reporting period, the data entry form would be mailed from each pool laboratory to the IQAP processing center. The IQAP processing center then would compile all of the data, perform the computations to obtain the CVI, SDI and IPI, preferably generate the IPI plots shown in Figure 4, as well as the data of Table II discussed below and, where needed, give guidance to each laboratory for improving the precision and accuracy of its apparatus and its system. The term "system" is used herein to include variables beyond that of the apparatus per se and includes laboratory operations which can influence the performance of the apparatus.
~.~9~
Table II presents the individual and comparative I~AP statistics and the ultimately derived values for SDI and CVI. The term "Y~UR"
designates the one apparatus being compared to all of the apparatuses in the pool. From left to right, the first columns of Table II list:
each tested parameter, the assay values of the sample or specimen material, the mean (a) for your apparatus, and the mean (b) for the pool apparatuses. The next three columns list the Standard Deviations: for your apparatus (SD), the pool (PSD), and the Standard Deviation of the Mean (MSD). The next four columns depict the Coefficient of Variation ~CV) for your apparatus for the present reporting period and the two previous reporting periods, and that of the pool apparatuses (PCV). Next listed in Table II is the number of laboratories (LABS) including your laboratory and the number of test values (n) included in the calculations. The last two columns present the Standard Deviation Index (SDI) and Coefficient of Variation Index (CVI), respectively.
The data presented in Table I, Table II and the IPI indicia of Figure 4 are not based upon a common set of information, nor derived from one another. However, Table II raw data was obtained from a peer group of hematology instruments and the derived data in Table II was developed from such raw data by use of the procedure of this invention.
Now with reference to the top row of information in Table II, which concerns the white blood cell count (WBC) parameter, and reading from left to right, the assay value 7.4 provides a reference for the reader of Table II in considering the adjacent values of the mean of his apparatus and the pool apparatuses. However, this assay value is not utilized as part of the data base of the statistics for the procedure of our invention. In this example, the mean value (a) of your instrument is 7.8 and pool mean (b) is 7.28, which when compared to the assay value of 7.4 would indicate that your mean value not only is further from the assay value than is the pool value, but also that your mean value is greater than the assay value; whereas, the pool mean value is less than the assay value. Hence, your WBC data initially is suspect. Your SD is .11, the PSD is .15, and the MSD is .12. This set of data would indicate that, with respect to Standard ~'~90~
Deviation, the operation of your instrument is better than that of the pool. Your current CV is 1.41, that of the pool is 2.00, and your previous two months were 1.76 and 2.15, respectively. Inspection of this set of data shows that your WBC parameter CV is acceptable and has improved in each of the recent three months. The number of laboratories (L) 300 and your tests (n) 28 indicates that your data, the pool data, and therefore the derived data for the WBC analysis are ; statistically sound.
Your SDI of 2.50 and CVI of .71 show that your system performance, with respect to WBC test analysis, is not acceptable, because it is inaccurate, hav;ng a high SDI of 2.50, even though your system performance is precise with a small CVI of .71. If these SDI
and CVI values had been plotted as the IPI point for W~C in Figure 4, that IPI point would have been near the middle of the lowest right end part or box and outside of the acceptable "framed" area.
A similar analysis of the RBC values in the second row of Table II would indicate favorable data, resulting in a SDI of -.88 and a CVI
of .84, both of which are very acceptable and would result in an IPI
plot point lying in the box which is in the lowest row and just to the left of the MEAN line; ie, an indication of high precision and high accuracy and lying well inside the "framed" portion.
Hence, a simple listing of the SDI and CVI values, as in Table II, can be interpreted relatively easily to advise a laboratory of the accuracy and precision, respectively, of its instrument and system for each tested parameter, without need for the ultimate physical plotting of the IPI point. Therefore, this invention permits a subscriber of our improved IQAP to ascertain quickly by visual observation of the single IPI indicia and/or its SDI and CVI components the accuracy and precision of a particular analytical apparatus, instrument and system in comparison to the identical or substantially identical pool of such apparatuses and instruments, with respect to each separate test parameter. Additionally, our novel CVI value itself provides precision information and the SDI value provides accuracy information.
Although there has been disclosed an embodiment directed to blood analyzing apparatus and hematology parameters, it now should be 12,90~
readily apparent that the disclosed invention readily is applicable for determining the accuracy and/or precision of other typPs of analyzers, for example, colorimeters, chromotography apparatus, etc.
Now therefore, variations of our invention lie within the reach and capability of those skilled in the art without departing from the scope and spirit of the invention as next claimed.
a ,~
0 00 _ O ~ _ O
a ~ 0 ~ o 00 ~ O
U~ C~ I _ _ ~q o o oo o o o~
C~ ~ ~; oo o o o o s~
g ~ o~ o U~
o ~ . . . . .
o o o o p ~ ~ _ o U`D ~ ~ ~ ~
~æ ~
3 ~ ~ o 3 a ~1 ~ ~ ~ ~ o a oo ~ o ~ . .
Is --, -- _ -- e 4 ~
~ ~ a ~ ~ ~ oo 0 D ~
u~ o o C~l c~ N C O
~ ~J O -- oo ~~ ~ ;t OD
a æ
~ c~ ~ ~ o ~
, 0 oo ol~ ~ 8 0 P 0c~ J _ O ~
'l:
~ ~ ~ ~ ~ s
Claims (16)
1. A procedure for evaluating statistically the performance of at least one apparatus in a pool of like apparatuses, by comparing the performance of said one apparatus with the performance of said like apparatuses, and by providing at least one comparative performance indicator of said one apparatus, said procedure comprising the steps of:
performing at least one substantially identical operation by each of said apparatuses in said pool;
producing, as a result of said performing operation, associated resultant data by each of said apparatuses, for each separate operation;
collecting said resultant data from said one apparatus and said like apparatuses;
calculating from said resultant data the arithmetic mean data of all of the pool apparatuses;
translating mathematically said resultant data from said one apparatus and said arithmetic mean data from said pool apparatuses into at least one performance value for each said operation; and providing said performance value as an indicator specific to the precision of performance of said one apparatus as compared to the precision of performance of the pool apparatuses.
performing at least one substantially identical operation by each of said apparatuses in said pool;
producing, as a result of said performing operation, associated resultant data by each of said apparatuses, for each separate operation;
collecting said resultant data from said one apparatus and said like apparatuses;
calculating from said resultant data the arithmetic mean data of all of the pool apparatuses;
translating mathematically said resultant data from said one apparatus and said arithmetic mean data from said pool apparatuses into at least one performance value for each said operation; and providing said performance value as an indicator specific to the precision of performance of said one apparatus as compared to the precision of performance of the pool apparatuses.
2. A procedure according to claim 1, wherein said step of translating includes computing mathematically from said resultant data and said arithmetic mean data the standard deviation and the coefficient of variation for said one apparatus and said pool apparatuses, and deriving therefrom a Coefficient of Variation Index, which is said performance value for apparatus precision.
3. A procedure according to claim 2, wherein said deriving of said Coefficient of Variation Index is from the formula CVI = wherein CV is the coefficient of variation, as exemplified by the formula 6 of the Specification, and PCV is the pool coefficient of variation, as exemplified by formula 7 of the Specification.
4. A procedure according to claim 1, wherein said one performance value is an index specific to apparatus precision, as defined by formula 9 of the Specification, and said translating includes mathematically calculating such precision index in accordance with formulas 1, 2, 5, 6, and 7 of the Specification.
5. A procedure according to claim 4, wherein said translating provides a second performance value which is an index specific to apparatus accuracy, as defined by formula 8 of the Specification, and said translating includes mathematically calculating in accordance with formulas 1, 2, and 4 of the Specification.
6. A procedure according to claim 1, wherein said translating provides a pair of performance values, one specific to precision and the other specific to accuracy, both specific to said one apparatus as compared to the precision and accuracy of the pool apparatuses.
7. A procedure according to claim 6, wherein said precision and accuracy performance values, by said step of translating, are provided as a pair of values which define coordinates, and said procedure further comprises displaying said coordinates for visual interpretation.
8. A procedure according to claim 7, wherein said displaying of said coordinates is arranged such that the two coordinates define a single point indicating both the accuracy and precision.
9. A procedure according to claim 8, in which said displaying further includes the step of displaying graphically said single point in a Cartesian coordinate system.
10. A procedure according to claim 8, wherein said single point defines an Instrument Performance Index.
11. A procedure according to claim 8, wherein said displaying includes formatting the Cartesian coordinate system to define an area of acceptable values for said single point.
12. A procedure according to claim 5, wherein said precision performance value, by said steps of translating and providing, is defined as the Coefficient of Variation Index, and said accuracy performance value is defined as the Standard Deviation Index.
13. A procedure according to claim 12, wherein said Standard Deviation Index is defined by formula 8 of the Specification, and Coefficient of Variation Index is defined by formula 9 of the Specification.
14. A procedure according to claim 1, wherein said performing of said operation by said pool apparatuses includes analyzing a reference specimen having at least one known constituent.
15. A procedure according to claim 14, wherein said reference specimen is a whole blood cell control, and said apparatuses are blood cell analyzers.
16. A procedure for evaluating statistically the performance of at least one apparatus in a pool of like apparatuses, by comparing the performance of said one apparatus with the performance of said like apparatuses, and by providing comparative performance indicators of said one apparatus, said procedure comprising the steps of:
performing at least one substantially identical operation by each of said apparatuses in said pool, said operation including analyzing quantitatively a reference specimen having at least one known constituent;
producing, as a result of said performing operation, associated resultant data by each of said apparatuses, for each separate operation;
collecting said resultant data from said one apparatus and said like apparatuses;
calculating from said resultant data the arithmetic mean data of all of the pool apparatuses;
translating mathematically said resultant data from said one apparatus and said arithmetic mean data from said pool apparatuses into one pair of performance values for each said operation, said pair of performance values being indicators respectively specific to the accuracy and precision of performance of said one apparatus as compared to the accuracy and precision of performance of the pool apparatuses; and displaying graphically said pair of performance values as a single point, by using them as a pair of coordinates in a Cartesian coordinate system.
performing at least one substantially identical operation by each of said apparatuses in said pool, said operation including analyzing quantitatively a reference specimen having at least one known constituent;
producing, as a result of said performing operation, associated resultant data by each of said apparatuses, for each separate operation;
collecting said resultant data from said one apparatus and said like apparatuses;
calculating from said resultant data the arithmetic mean data of all of the pool apparatuses;
translating mathematically said resultant data from said one apparatus and said arithmetic mean data from said pool apparatuses into one pair of performance values for each said operation, said pair of performance values being indicators respectively specific to the accuracy and precision of performance of said one apparatus as compared to the accuracy and precision of performance of the pool apparatuses; and displaying graphically said pair of performance values as a single point, by using them as a pair of coordinates in a Cartesian coordinate system.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US06/848,887 | 1986-04-07 | ||
| US06/848,887 US4858154A (en) | 1986-04-07 | 1986-04-07 | Interlaboratory quality assurance program |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA1290060C true CA1290060C (en) | 1991-10-01 |
Family
ID=25304542
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA000533719A Expired - Fee Related CA1290060C (en) | 1986-04-07 | 1987-04-02 | Interlaboratory quality assurance program |
Country Status (7)
| Country | Link |
|---|---|
| US (1) | US4858154A (en) |
| EP (1) | EP0263870B1 (en) |
| JP (1) | JPS63503088A (en) |
| AU (1) | AU599104B2 (en) |
| CA (1) | CA1290060C (en) |
| DE (1) | DE3781233T2 (en) |
| WO (1) | WO1987006340A1 (en) |
Families Citing this family (66)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0575966A (en) * | 1990-12-13 | 1993-03-26 | Nikon Corp | Electronic still camera |
| US5196997A (en) * | 1991-01-22 | 1993-03-23 | International Business Machines Corporation | Method and apparatus for quality measure driven process control |
| US6509192B1 (en) * | 1992-02-24 | 2003-01-21 | Coulter International Corp. | Quality control method |
| US6362003B1 (en) | 1992-02-24 | 2002-03-26 | Coulter Corporation | Hematological reference control composition containing leukocyte analogs, methods of making, and uses thereof |
| US5351202A (en) * | 1992-02-27 | 1994-09-27 | International Business Machines Corporation | Evaluation and ranking of manufacturing line non-numeric information |
| US5440478A (en) * | 1994-02-22 | 1995-08-08 | Mercer Forge Company | Process control method for improving manufacturing operations |
| US5532941A (en) * | 1994-07-08 | 1996-07-02 | Lin; Lawrence I. | Inter-laboratory performance monitoring system |
| US5619428A (en) * | 1995-05-31 | 1997-04-08 | Neopath, Inc. | Method and apparatus for integrating an automated system to a laboratory |
| DE60042105D1 (en) * | 1999-11-30 | 2009-06-10 | Sysmex Corp | Quality control method and device |
| JP4728277B2 (en) * | 1999-11-30 | 2011-07-20 | シスメックス株式会社 | External quality control system |
| JP4690284B2 (en) * | 1999-11-30 | 2011-06-01 | シスメックス株式会社 | Clinical laboratory analyzer support system, management apparatus, and clinical laboratory analyzer |
| JP4690286B2 (en) * | 1999-11-30 | 2011-06-01 | シスメックス株式会社 | Management device and accuracy management method |
| US7027931B1 (en) * | 2000-02-24 | 2006-04-11 | Bionostics, Inc. | System for statistical analysis of quality control data |
| US6549876B1 (en) | 2000-07-25 | 2003-04-15 | Xyletech Systems, Inc. | Method of evaluating performance of a hematology analyzer |
| US6748337B2 (en) * | 2001-03-14 | 2004-06-08 | Wardlaw Partners, Lp | Method and apparatus for providing quality control in an instrument for medical analysis |
| US7440856B2 (en) * | 2002-03-13 | 2008-10-21 | Becton, Dickinson And Company | System and method for determining clinical equivalence of test methods |
| US7467054B2 (en) * | 2003-05-02 | 2008-12-16 | Bio-Rad Laboratories, Inc. | System and method for integrating the internal and external quality control programs of a laboratory |
| US9123077B2 (en) | 2003-10-07 | 2015-09-01 | Hospira, Inc. | Medication management system |
| US8065161B2 (en) | 2003-11-13 | 2011-11-22 | Hospira, Inc. | System for maintaining drug information and communicating with medication delivery devices |
| EP1855221A3 (en) * | 2003-10-07 | 2008-03-12 | Hospira, Inc. | Method and system for evaluating the performance of medical devices from a medication management unit |
| JP4951216B2 (en) * | 2005-07-05 | 2012-06-13 | シスメックス株式会社 | Clinical examination information processing apparatus and system, analyzer, and clinical examination information processing program |
| JP4746471B2 (en) * | 2006-04-21 | 2011-08-10 | シスメックス株式会社 | Accuracy management system, accuracy management server and computer program |
| JP4817251B2 (en) * | 2006-09-22 | 2011-11-16 | シスメックス株式会社 | Quality control system |
| WO2008057729A2 (en) | 2006-10-16 | 2008-05-15 | Hospira, Inc. | System and method for comparing and utilizing activity information and configuration information from mulitple device management systems |
| US9026370B2 (en) | 2007-12-18 | 2015-05-05 | Hospira, Inc. | User interface improvements for medical devices |
| US8271106B2 (en) | 2009-04-17 | 2012-09-18 | Hospira, Inc. | System and method for configuring a rule set for medical event management and responses |
| US8538727B2 (en) * | 2009-07-21 | 2013-09-17 | George S. Cembrowski | Method and apparatus for calibration and testing of scientific measurement equipment |
| US11042605B2 (en) | 2009-07-21 | 2021-06-22 | Ccqcc Corp. | Method and apparatus for calibration and testing of scientific measurement equipment |
| AU2012299169B2 (en) | 2011-08-19 | 2017-08-24 | Icu Medical, Inc. | Systems and methods for a graphical interface including a graphical representation of medical data |
| ES2959510T3 (en) | 2011-10-21 | 2024-02-26 | Icu Medical Inc | Medical device update system |
| WO2013090709A1 (en) | 2011-12-16 | 2013-06-20 | Hospira, Inc. | System for monitoring and delivering medication to a patient and method of using the same to minimize the risks associated with automated therapy |
| ES2741725T3 (en) | 2012-03-30 | 2020-02-12 | Icu Medical Inc | Air detection system and method to detect air in a pump of an infusion system |
| EP2879733B1 (en) | 2012-07-31 | 2019-06-05 | ICU Medical, Inc. | Patient care system for critical medications |
| WO2014138446A1 (en) | 2013-03-06 | 2014-09-12 | Hospira,Inc. | Medical device communication method |
| JP2014202608A (en) * | 2013-04-04 | 2014-10-27 | 日本光電工業株式会社 | Method of displaying data for evaluation of external precision management |
| AU2014268355B2 (en) | 2013-05-24 | 2018-06-14 | Icu Medical, Inc. | Multi-sensor infusion system for detecting air or an occlusion in the infusion system |
| ES2845748T3 (en) | 2013-05-29 | 2021-07-27 | Icu Medical Inc | Infusion system and method of use that prevent oversaturation of an analog-digital converter |
| ES2838450T3 (en) | 2013-05-29 | 2021-07-02 | Icu Medical Inc | Infusion set that uses one or more sensors and additional information to make an air determination relative to the infusion set |
| CA2922425C (en) | 2013-08-30 | 2023-05-16 | Hospira, Inc. | System and method of monitoring and managing a remote infusion regimen |
| US9662436B2 (en) | 2013-09-20 | 2017-05-30 | Icu Medical, Inc. | Fail-safe drug infusion therapy system |
| US10311972B2 (en) | 2013-11-11 | 2019-06-04 | Icu Medical, Inc. | Medical device system performance index |
| AU2014353130B9 (en) | 2013-11-19 | 2019-09-05 | Icu Medical, Inc. | Infusion pump automation system and method |
| CA2939302C (en) | 2014-02-28 | 2021-12-28 | Hospira, Inc. | Infusion system and method which utilizes dual wavelength optical air-in-line detection |
| EP3138032A4 (en) | 2014-04-30 | 2017-12-20 | ICU Medical, Inc. | Patient care system with conditional alarm forwarding |
| AU2015266706B2 (en) | 2014-05-29 | 2020-01-30 | Icu Medical, Inc. | Infusion system and pump with configurable closed loop delivery rate catch-up |
| US9724470B2 (en) | 2014-06-16 | 2017-08-08 | Icu Medical, Inc. | System for monitoring and delivering medication to a patient and method of using the same to minimize the risks associated with automated therapy |
| US9539383B2 (en) | 2014-09-15 | 2017-01-10 | Hospira, Inc. | System and method that matches delayed infusion auto-programs with manually entered infusion programs and analyzes differences therein |
| US11344668B2 (en) | 2014-12-19 | 2022-05-31 | Icu Medical, Inc. | Infusion system with concurrent TPN/insulin infusion |
| US10850024B2 (en) | 2015-03-02 | 2020-12-01 | Icu Medical, Inc. | Infusion system, device, and method having advanced infusion features |
| WO2016189417A1 (en) | 2015-05-26 | 2016-12-01 | Hospira, Inc. | Infusion pump system and method with multiple drug library editor source capability |
| EP4085944A1 (en) | 2016-05-13 | 2022-11-09 | ICU Medical, Inc. | Infusion pump system with common line auto flush |
| AU2017277804B2 (en) | 2016-06-10 | 2022-05-26 | Icu Medical, Inc. | Acoustic flow sensor for continuous medication flow measurements and feedback control of infusion |
| US11574737B2 (en) | 2016-07-14 | 2023-02-07 | Icu Medical, Inc. | Multi-communication path selection and security system for a medical device |
| EP4116984A1 (en) * | 2016-08-29 | 2023-01-11 | Beckman Coulter, Inc. | Remote data analysis and diagnosis |
| US10613081B2 (en) | 2016-11-27 | 2020-04-07 | Alireza Palangi | Quality control system and kit for automated ELISA devices |
| JP6875137B2 (en) * | 2017-01-31 | 2021-05-19 | シスメックス株式会社 | Quality control method, quality control system, control device, analyzer and quality control abnormality judgment method |
| US10089055B1 (en) | 2017-12-27 | 2018-10-02 | Icu Medical, Inc. | Synchronized display of screen content on networked devices |
| NZ793485A (en) | 2018-07-17 | 2023-06-30 | Icu Medical Inc | Systems and methods for facilitating clinical messaging in a network environment |
| US11152109B2 (en) | 2018-07-17 | 2021-10-19 | Icu Medical, Inc. | Detecting missing messages from clinical environment |
| EP3824386B1 (en) | 2018-07-17 | 2024-02-21 | ICU Medical, Inc. | Updating infusion pump drug libraries and operational software in a networked environment |
| US10861592B2 (en) | 2018-07-17 | 2020-12-08 | Icu Medical, Inc. | Reducing infusion pump network congestion by staggering updates |
| US10692595B2 (en) | 2018-07-26 | 2020-06-23 | Icu Medical, Inc. | Drug library dynamic version management |
| AU2019309766A1 (en) | 2018-07-26 | 2021-03-18 | Icu Medical, Inc. | Drug library management system |
| US11278671B2 (en) | 2019-12-04 | 2022-03-22 | Icu Medical, Inc. | Infusion pump with safety sequence keypad |
| WO2022020184A1 (en) | 2020-07-21 | 2022-01-27 | Icu Medical, Inc. | Fluid transfer devices and methods of use |
| US11135360B1 (en) | 2020-12-07 | 2021-10-05 | Icu Medical, Inc. | Concurrent infusion with common line auto flush |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US2398988A (en) * | 1943-03-11 | 1946-04-23 | Askania Regulator Co | Apparatus for detecting trends in observed data |
| CH374407A (en) * | 1957-07-30 | 1964-01-15 | Hrabak Jaroslav | Process and device for regulating a quantity that determines the quality of products using a static method |
| US3520316A (en) * | 1963-12-12 | 1970-07-14 | Bowles Eng Corp | Pressure-to-pressure transducer |
| US3812335A (en) * | 1970-12-02 | 1974-05-21 | Coulter Electronics | Apparatus for obtaining statistical indices and analyzing the same and method |
| US4071891A (en) * | 1976-04-30 | 1978-01-31 | Barrows George H | Electronic calculator - register for hematology differentials |
| US4202033A (en) * | 1977-12-27 | 1980-05-06 | Royco Instruments, Inc. | Apparatus and method utilizing calculator for quality control of hematology sample analysis |
| US4299726A (en) * | 1979-05-07 | 1981-11-10 | Coulter Electronics, Inc. | Process for preparing whole blood reference controls having long term stability, preconditioning diluent and media therefor |
| US4320463A (en) * | 1980-02-25 | 1982-03-16 | S. Himmelstein And Company | Production control system |
| US4405719A (en) * | 1981-05-29 | 1983-09-20 | Coulter Electronics, Inc. | Method of stabilizing platelets for determining multiple platelet parameters in reference control and calibrator compositions; diluents therefor; and combination stabilization procedures |
| US4596464A (en) * | 1983-10-14 | 1986-06-24 | Ortho Diagnostic Systems, Inc. | Screening method for red cell abnormality |
-
1986
- 1986-04-07 US US06/848,887 patent/US4858154A/en not_active Expired - Fee Related
-
1987
- 1987-03-30 AU AU72852/87A patent/AU599104B2/en not_active Ceased
- 1987-03-30 WO PCT/US1987/000717 patent/WO1987006340A1/en active IP Right Grant
- 1987-03-30 DE DE8787903008T patent/DE3781233T2/en not_active Expired - Fee Related
- 1987-03-30 JP JP62502513A patent/JPS63503088A/en active Pending
- 1987-03-30 EP EP87903008A patent/EP0263870B1/en not_active Expired - Lifetime
- 1987-04-02 CA CA000533719A patent/CA1290060C/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| WO1987006340A1 (en) | 1987-10-22 |
| DE3781233D1 (en) | 1992-09-24 |
| EP0263870A1 (en) | 1988-04-20 |
| AU7285287A (en) | 1987-11-09 |
| EP0263870A4 (en) | 1988-08-24 |
| JPS63503088A (en) | 1988-11-10 |
| DE3781233T2 (en) | 1993-04-15 |
| US4858154A (en) | 1989-08-15 |
| AU599104B2 (en) | 1990-07-12 |
| EP0263870B1 (en) | 1992-08-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA1290060C (en) | Interlaboratory quality assurance program | |
| England et al. | Guidelines for the evaluation of blood cell analysers including those used for differential leucocyte and reticulocyte counting and cell marker applications | |
| INTERNATIONAL COMMITTEE FOR STANDARDIZATION IN HAEMATOLOGY (ICSH) et al. | Protocol for evaluation of automated blood cell counters | |
| Glick et al. | Unreliable visual estimation of the incidence and amount of turbidity, hemolysis, and icterus in serum from hospitalized patients. | |
| Whitby et al. | Quality control in routine clinical chemistry | |
| Davis et al. | Flow cytometric reticulocyte analysis: multiinstitutional interlaboratory correlation study | |
| Małecka et al. | A comparison of Sysmex-XN 2000 and Yumizen H2500 automated hematology analyzers | |
| Longanbach et al. | Automated blood cell analysis | |
| Knoll et al. | Clinical hematology: in-clinic analysis, quality control, reference values, and system selection | |
| Lewis et al. | Quality assurance in haematology | |
| Nagaraj et al. | Evaluation of quality control in clinical hematology laboratory by using six-sigma | |
| Hallawell et al. | An evaluation of the sysmex NE-8000® hematology analyzer | |
| Stirn et al. | Quality Management of Hematology Techniques | |
| EP1144979A2 (en) | A new hematological parameter | |
| Ross et al. | Neutropenia: the accuracy and precision of the neutrophil count in leukopenic patients | |
| Khodaiji et al. | Performance evaluation of fully automated 5-Part differential hematology analyzer ELite 580 (ERBA) | |
| Cembrowski | Hematology quality practices | |
| Broughton et al. | Improving the quality of plasma cholesterol measurements in primary care | |
| JPH0137692B2 (en) | ||
| Lakos | Calibration and QC guidance for the Abbott Alinity HQ hematology analyzer | |
| Westgard et al. | Relationship of quality goals and measurement performance to the selection of quality control procedures for multi‐channel haematology analysers | |
| Briggs | Evaluation of hematology analyzers | |
| Berta et al. | Analytical performance evaluation of Hematology analyzers using Sigma metrics | |
| Carlson et al. | Evaluation of the Sysmex CC-800®: an automated eight-parameter hematology instrument | |
| Fares | Performance evaluation of two haematology analysers: The Sysmex KX-21 and the Beckman Coulter AC. T diff |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MKLA | Lapsed |