|Publication number||US5608814 A|
|Application number||US 08/341,470|
|Publication date||Mar 4, 1997|
|Filing date||Nov 17, 1994|
|Priority date||Aug 26, 1993|
|Publication number||08341470, 341470, US 5608814 A, US 5608814A, US-A-5608814, US5608814 A, US5608814A|
|Inventors||Robert S. Gilmore, Patrick J. Howard|
|Original Assignee||General Electric Company|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (12), Non-Patent Citations (4), Referenced by (33), Classifications (8), Legal Events (5)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is Continuation of application Ser. No. 08/118,337, filed 26 Aug. 1993, now abandoned.
The present invention relates to the field of image processing, and more particularly to a dynamic thresholding method for converting an ultrasonic C-scan image into a binary flaw-no flaw image.
Ultrasonic pulse-echo inspection of titanium and other large grain metal objects is plagued by grain noise produced by ultrasonic reflections from large grain interfaces. Grain noise occurs because the microstructure of metals such as titanium or the like can be coarse which causes the grains of the microstructure to return signals during a raster scan of the object with an ultrasonic transducer. Due to the extruding and forging processes used in forming titanium parts, grain structure, and therefore grain noise, can vary significantly between different regions of an object. Grain noise can typically vary between 6-20 dB. Grain noise can mask or conceal flaw indications in a defective region or produce false indications of flaws in defect free regions. False flaw indications can result in a defect free object being rejected for use in a particular application. Masked flaw indications can result in the unintentional use of a flawed part. Obviously, either masked flaw indications or false flaw indications can result in significant waste of time and materials in the manufacture of metal-parts and/or an undesirable increased risk of part failure.
Previously in ultrasonic C-scan imaging, a technique known as global thresholding has been used to convert an ultrasonic inspection data into a flaw-no flaw decision. In the global thresholding technique, an initial calibration is performed which provides an estimate of grain noise which can be expected to occur in a particular object during the ultrasonic inspection. This initial estimation is based on a sample of grain noise in one particular region of the object. Based on this estimation, an alarm threshold or global threshold is set for the duration of the inspection. For example, a sample of the grain noise of an object may indicate that grain noise can be expected to occur up to a value of 40, wherein the value of 40 is proportional to the ultrasonic amplitude received by the ultrasonic transducer used in inspecting the object. From this estimate noise value, a global threshold value is set which is used to determine whether the signals received during inspection of the object are to be indicated as a flaw or no-flaw. If noise is estimated to occur up to a value of 40, and the threshold value is set at 40, any ultrasonic signal received having a value greater than 40 will be considered to indicate a flaw or defect in the object, while any signal received having a value less than 40 will be considered as noise and not a flaw. Thus, when ultrasonically scanning an object utilizing the global thresholding technique, a global threshold level is set once prior to scanning the object and this threshold level remains constant throughout the entire duration of the inspection.
A major disadvantage of global thresholding is that it does not take into account variations in grain noise levels which may exist between different regions of the object. Regardless of whether a region has high or low grain noise, the same threshold value is used in flaw-no flaw decision making in all regions of the object. As a result, global thresholding results in a high occurrence of masked flaw indications and/or false flaw indications when used for inspecting an object in which noise levels vary significantly in different regions of the object.
For example, if a high noise region is used in determining the global threshold level, flaws in a low noise region are masked by the high global threshold level, and therefore go undetected. Conversely, if a low noise region is used in determining the global threshold level;, defect free regions which have high noise levels will produce many false flaw indications.
A primary object of the present invention is to provide a method for converting an ultrasonic C-scan image into a binary flaw-no flaw image which enables a high probability of flaw detection and a low probability of false flaw indications.
A more specific object of the present invention is to provide a method for converting an ultrasonic C-scan image into a binary flaw-no flaw image which enables dynamic thresholding based on the particular characteristics of each region of an object.
A further object of the invention is to provide a method for converting an ultrasonic C-scan image into a binary flaw-no flaw image which maximizes the probability of detecting flaws and minimizes the probability of false flaw and minimizes the probability of false flaw indications in each region of the object.
Another object of the invention is to provide a method for converting an ultrasonic C-scan image into a binary flaw-no flaw image which takes into account particular characteristics of an associated data acquisition system used to obtain data values of the C-scan image.
Another object of the invention is to provide a method for producing a binary image from an ultrasonic C-scan of an object which enables a pixel threshold level to be obtained for each pixel data value of the C-scan image.
Yet another object of the invention is to provide method of ultrasonic scanning which enables a greater probability of flaw detection and less probability of false flaw indications than has heretofore been achieved with global thresholding methods.
A further object of the present invention is to provide a method of ultrasonic scanning of an object which insures continuity of pixel threshold levels at regional boundaries of a binary flaw-no flaw image of the object.
Still another object of the invention is to provide a method for generating a binary flaw-no flaw image from an ultrasonic C-scan image that enables threshold values to be dynamically determined for each pixel data value in the C-scan image based on grain noise present in each particular region of a scanned object.
These and other objects and advantages are achieved by the present invention by providing a method of producing a binary flaw-no flaw image of an object, which includes employing an ultrasonic data acquisition system to obtain data values f(i,j) which define a C-scan image (F) of the object, dividing the C-scan image into a plurality of subimages Gk for k=1,2, . . . ,K, determining regional threshold levels y(k) for each of the plurality of subimages, using the regional threshold levels y(k) to determine pixel threshold levels t(i,j) for each pixel (i,j) of the C-scan image, and generating a binary flaw-no flaw image (B) by assigning binary values thereto based on a comparison between the pixel threshold levels t(i,j) and data values f(i,j), respectively.
In accordance with one aspect of the invention, the step of determining the regional threshold levels includes initializing a first subimage Gl with a region threshold level y(l), and determining for each of the subimages Gk, for k=2, . . . ,K, a regional threshold level as a function of the regional threshold level y(k-1) for subimage GK-1.
A further aspect of the invention involves determining the regional threshold level as a function of at least one adjustment function which is defined based on the characteristic a data acquisition system. In accordance with this aspect of the invention, an upper and lower saturation point, wherein the values of the adjustment function above and below the upper and lower saturation points, respectively, remain constant.
In accordance with the present invention, determining the regional threshold levels further includes calculating an initial regional value for each subimage from the regional data values of each subimage, respectively, and inputting the initial regional value for each subimage to the adjustment function to obtain a first adjusted value for each subimage, respectively.
In a more particular embodiment of the invention, the initial data value may be the maximum, minimum, mean, median, standard deviation or some combination of the above of the data values in each region, respectively.
In accordance with a further aspect of the invention, calculation of the regional threshold levels for each subimage Gk further includes subtracting the regional threshold level y(k-1) from the first adjusted value, to obtain a second adjusted value, operating on the second adjusted value with another adjustment function to obtain a third adjusted value, and adding the third adjusted value to the regional threshold level y(k-1) to obtain regional threshold level y(k).
According to a more particular object of the present invention, the method includes adjusting the regional threshold levels y(k) by averaging each of the regional threshold levels with regional threshold levels of subimages adjacent thereto, respectively.
A further aspect of the invention includes converting the regional threshold levels into pixel threshold levels for each pixel of the C-scan image using interpolation.
In accordance with another aspect of the invention, the binary flaw-no flaw image is produced by assigning for each pixel (i,j) of the binary flaw-no flaw image a binary pixel value indicating a flaw if the data value f(i,j) is greater than the pixel threshold level t(i,j), and a binary pixel value indicating no flaw otherwise.
Other objects and advantages of the subject invention will become apparent from a study of the following specification when viewed in light of the accompanying drawings, in which:
FIG. 1 is a flow chart of the steps which define the dynamic thresholding method of the present invention;
FIG. 2 depicts a preferred embodiment of the data acquisition system used in the method of the present invention; and
FIG. 3 is a graph of a preferred embodiment of the adjustment function used in the method of the present invention.
Referring now to FIG. 1, the first step 10 in the method of the present invention is to obtain a C-scan image F of an object using ultrasonic pulse-echo inspection. The present method has particular utility in the inspection of metal objects or manufactured parts made of large grain metals such as titanium or the like, to identify indications such as actual flaws in the metal, abnormally large grains, or any other indications which are identifiable over the grain noise in the part, such as hard alpha defects or the like.
As shown in FIG. 2, the C-scan image is obtained by a process known as C-scanning, wherein an ultrasonic transducer 50 is used to raster scan the part 52 with sound along a surface thereof. During the raster scan, the part 52 is immersed in a tank 54 full of couplant 56, such as water or the like. At each location (x,y) on the part 52 the ultrasonic transducer 50 is pulsed by pulser 58 to send a sound wave through the part 52 which reflects off or echoes back signals from the part to the transducer 50. A gate or time zone determines an area of interest over which data is going to be collected. A receiver 60 is provided to receive and amplify the echo signals and transmit the signals to a gated peak detector 62 via connection 61. Within the gate, the gated peak detector 62 is operable to output the maximum amplitude values of the signals received via line 63 to processor 64. The maximum amplitude values are then digitized and stored in a data processing processor 64, thereby defining a C-scan image of the object which may be outputted via connection line 65 to a suitable image viewing output device 66.
Thus in C-scanning, pixel data values f(i,j) are obtained for each pixel (i,j) of the C-scan image, thus defining a M1 ×M2 pixel image. The data processing device 64 includes image processing means which enables the pixel data values f(i,j) to be converted to binary flaw-no flaw values b(i,j) as will be described in detail below. Output device 66, such as a video monitor or printer, is provided to enable the binary flaw-no flaw image to be displayed or printed once generated. Inasmuch as C-scanning is well known in the art, particular details of C-scanning will not be further discussed herein.
Once the pixel data values f(i,j) are obtained for the M1 ×M2 C-scan image F, a dividing step 12 is performed which logically divides the image F into K subimages or regions of dimension N1 ×N2, denoted Gk, k=1, . . . ,K, wherein each subregion includes pixel data values g(i,j). For example, if a 2048×1024 pixel image F is used, the image may be broken down into 128 subimages each having 128×128 pixels therein. The size and shape of the subimages is a design parameter which can be selectively chosen relative to the size of the image to achieve a desired level of performance. Generally, the smaller the subimage, the smaller the indication which can be identified by the method of the present invention.
Preferably, the first subimage G1 is defined in a corner of the image, and the remaining subimages Gk for k=2, . . . ,K, are selected using a raster scanning convention, thereby defining the subimages in a manner which preserves the spatial correlation of the image.
Once the subimages are defined, a computing step 14 is performed which calculates a regional threshold level y(k) for each of the subimages Gk, using the following equations:
y(k)=y(k-1)+A.sub.3 (k) (1)
A.sub.3 (k)=h.sub.3 (A.sub.2 (k)) (2)
A.sub.2 (k)=A.sub.1 (k)-y(k-1) (3)
A.sub.1 (k)=h.sub.1 (G.sub.k) (4)
wherein, A1 (k) is a first adjusted value which is calculated from the pixel data values g(i,j) in each subimage Gk using the function denoted h1 (•) as shown in equation (4). Preferably, the first adjusted value is the mean plus some multiple of the standard deviation of the pixel data values g(i,j) in each subimage, but depending on the particular application, the maximum, minimum, mean, median, or other suitable first adjusted value may be used. The choice of the first adjusted value is a design parameter which can be selectively chosen based on the type of metal used or indications one desires to identify.
A second adjusted value, A2 (k) may then be calculated by subtracting from the first adjusted value A1 (k) the preceding regional threshold level, y(k-1) as shown in equation (3).
A third adjusted value, A3 (k) can be calculated from the second adjusted value A2 (k) using the function h3 (•) which is selectively chosen to match the particular characteristics of the data acquisition system used in C-scanning the object to obtain the pixel data values f(i,j). A preferred embodiment of the function h3 (•) is shown in FIG. 3, which is particularly suited for use with data values collected from an 8-bit C-scan data acquisition system. As shown in FIG. 3, the functional h3 (•) is preferably a non-linear function which includes upper and lower saturation points 100 and 102, respectively, and is linear between the saturation points. The saturation points 100 and 102 operate to provide upper and lower limits for the third adjusted values of each subimage.
While the function h3 (•) of FIG. 3 represents a preferred embodiment of the adjustment function h3 (•), the saturation points 100 and 102 and/or the shape of the function h3 (•) may be different depending on the particular characteristics of the system in which the method is used. For example, the functional h3 (•) may be determined based on the type or dynamic range of the data acquisition system used, the amount of attenuation or gain which is applied to the ultrasonic signals, the length of the gate in the metal, and/or the size or type of indication to be identified. In other words, the particular adjustment function h3 (•) is a design parameter which can be selectively defined based on the particular inspection procedure and/or requirements of the application in which the present invention is used.
In addition to incorporating information about the data acquisition process, function h3 (•) is a weighting function which acts similar to a forgetting factor in an adaptive filter. The function h3 (•) determines the "memory" of the procedure by defining how to weight the information from the present subimage Gk (contained in the second adjusted value, A2 (k)) relative to information from the previous subimage Gk-1 (contained in y(k-1)) in calculating the threshold level y(k) for the present subimage Gk. Preferably, the function h3 (•) is a constant function such as h3 (×)=0.5×in the linear range, but any other suitable weighting function may be used. Thus, the function h3 (•) is a design parameter which can selectively be defined to adjust the performance of the present method.
As can be seen from equation (1), the regional threshold level y(k) for subimage Gk is determined by adding the third adjusted value, A3 (k), to the previous regional threshold level y(k-1) for subimage Gk-1. Maintaining consistency between the defining and numbering of the subimages and the spatial correlation of the object, as discussed above, enables the present method to take advantage of the invention present in a previous threshold level when calculating the next threshold level. Thus, equations (1)-(4) act like a moving weighted average in calculating the threshold levels for each region.
In order to enable initialization of the threshold procedure of equation (1), a threshold level y(1) must be defined for subimage G1. This can be achieved by using the value of the adjustment function h1 (•) corresponding to an initial regional value calculated from data values g(i,j) in subregion G1, or by any other suitable means which enables initialization of the procedure.
Once the regional threshold levels y(k) are determined for each subimage Gk, an average step 16 is performed which neighborhood averages the regional threshold levels y(k) into a new set of regional threshold levels z(k) using an l1 ×l2 neighborhood Lk, as follows: ##EQU1## where y(j) ε Lk is each of the y(j)'s included in Lk. The neighborhood averaging is performed to take into account the fact that the raster scanning convention used to define the subimages is causal, and noise correlation in C-scan images is typically non-causal. Since the procedure of equation 1 only takes into account information from subimages which are prior in time to the present subimage in calculating the regional threshold value of the present subimage, the neighborhood averaging enables the method to take into account all of the information near or around the present subimage, regardless of whether it is prior in time or not. For example, the neighborhood Lk may be defined such that the regional threshold level y(k)of subimage Gk is averaged with all of the regional threshold levels of subimages which are directly adjacent to subimage Gk, thereby determining a new regional threshold level z(k) for subimage Gk.
While the step of neighborhood averaging is preferably used in the present method, it is an optional step which, when used, can provide a higher probability of flaw detection and/or a less probability of false flaw indication in some applications. However, it has been found that neighborhood averaging may have only a small or negligible effect on flaw identification in some applications. Thus, in certain applications the neighborhood averaging step can be eliminated to simplify the present method.
Once the regional threshold levels y(k) or z(k) are determined, an interpolation step 18 is performed in which pixel threshold values t(i,j) are computed for each pixel (i,j) of the image F from the regional threshold values using interpolation. The interpolation step is performed to ensure continuity between subimage boundaries by smoothing the threshold levels out between the regions, thereby eliminating the occurrence of large differences in the threshold levels between adjacent pixels at the boundaries of subimages. Preferably, linear interpolation is used to determine the pixel threshold levels t(i,j).
Once pixel threshold levels t(i,j) are determined, the final step of the present invention is a decision step 20 in which binary values b(i,j) are determined which define a binary flaw-no flaw image B based on a comparison between the pixel data values f(i,j) and the pixel threshold values t(i,j). Thus resulting in M1 ×M2 binary data values which make up the flaw-no flaw image B. For example, the binary values b(i,j) may be determined as follows: ##EQU2## Thus, a binary value of 1 would identify an indication such as a flaw or a large grain in the metal at the corresponding location thereon, and a binary value of 0 would indicate that no flaw in the metal exists at that particular location.
In accordance with the invention, the particular criteria for selecting the binary values in equation (6) may vary depending on the particular application. For example, in some application a binary value of 1 may be selected if the data value f(i,j) is greater than, rather than greater than or equal to the pixel threshold values t(i,j). Conversely, in some data acquisition systems in which the present method could be employed, it may be desirable to identify an indication if the pixel data value f(i,j) is below, rather than above the corresponding pixel threshold value t(i,j).
Thus, the present invention provides a flexible and reliable method for generating a binary flaw-no flaw image from an ultrasonic C-scan image that enables threshold values to be dynamically determined for each pixel data value in the C-scan image based on grain noise present in each particular region of a scanned object.
As can be seen from the foregoing description, the present method provides an increased probability of flaw detection and a decreased probability of false flaw indications compared to that which has heretofore been achieved with global thresholding techniques.
While the preferred forms and embodiments of the invention have been illustrated and described, it will be apparent to those of ordinary skill in the art that various changes and modifications may be made without deviating from the inventive concepts and spirit of the invention as set forth above, and it is intended by the appended claims to define all such concepts which come within the full scope and true spirit of the invention.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4442544 *||Jul 9, 1981||Apr 10, 1984||Xerox Corporation||Adaptive thresholder|
|US4468704 *||Oct 28, 1982||Aug 28, 1984||Xerox Corporation||Adaptive thresholder|
|US4501016 *||Nov 22, 1982||Feb 19, 1985||U.S. Philips Corporation||Device for the dynamic adjustment of a black/white discrimination threshold for the processing of images containing grey values|
|US4561104 *||Jan 16, 1984||Dec 24, 1985||Honeywell Inc.||Automated inspection of hot steel slabs|
|US4751846 *||Oct 3, 1986||Jun 21, 1988||Kontron Holding A.G.||Reducing noise in ultrasonic images|
|US4809349 *||Jul 29, 1987||Feb 28, 1989||U.S. Philips Corporation||Method of automatically extracting a contrasting object in a digital image|
|US4823194 *||Jul 30, 1987||Apr 18, 1989||Hitachi, Ltd.||Method for processing gray scale images and an apparatus thereof|
|US4837846 *||Sep 18, 1985||Jun 6, 1989||Minolta Camera Kabushiki Kaisha||Method of the image processing|
|US4869109 *||May 11, 1988||Sep 26, 1989||Thierry Miglianico||Automatic process for identification of defects by ultrasound and corresponding system|
|US4887306 *||Nov 4, 1987||Dec 12, 1989||Advanced Technology Laboratories, Inc.||Adaptive temporal filter for ultrasound imaging system|
|US4908875 *||Mar 21, 1989||Mar 13, 1990||Hughes Aircraft Company||Adaptive thresholding technique|
|US5047851 *||Feb 26, 1990||Sep 10, 1991||Isotopen-Technik Dr. Sauerwein Gmbh||Process and device for detecting and evaluating surface cracks in workpieces|
|1||*||Gonzalez et al. Digital Image Processing, Addison Wesley Publishing Co. 1992, pp. 447 452.|
|2||Gonzalez et al. Digital Image Processing, Addison-Wesley Publishing Co. 1992, pp. 447-452.|
|3||*||Yanowitz et al. A new method for image segmentation, 9th Int. Conference of Pattern Recognition, IEE comput. Soc. Press, 1988, pp. 270 275.|
|4||Yanowitz et al. A new method for image segmentation, 9th Int. Conference of Pattern Recognition, IEE comput. Soc. Press, 1988, pp. 270-275.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US6014465 *||Mar 19, 1996||Jan 11, 2000||Christian Bjorn Stefan||Method for transforming a gray-level image into a black-and-white image|
|US6041132 *||Jul 29, 1997||Mar 21, 2000||General Electric Company||Computed tomography inspection of composite ply structure|
|US6408695 *||Dec 3, 1999||Jun 25, 2002||General Electric Company||Ultrasonic inspection method and system for detecting critical flaws|
|US6549820||Jul 27, 1999||Apr 15, 2003||The Boeing Company||Method and system for providing feedback from a non-destructive inspection of a composite part|
|US7783433||Jul 21, 2007||Aug 24, 2010||Honeywell International Inc.||Automated defect detection of corrosion or cracks using SAFT processed Lamb wave images|
|US7795349||May 24, 2006||Sep 14, 2010||Z Corporation||Material systems and methods of three-dimensional printing|
|US7905951||Dec 7, 2007||Mar 15, 2011||Z Corporation||Three dimensional printing material system and method using peroxide cure|
|US7968626||Feb 22, 2008||Jun 28, 2011||Z Corporation||Three dimensional printing material system and method using plasticizer-assisted sintering|
|US7995829||Aug 1, 2007||Aug 9, 2011||General Electric Company||Method and apparatus for inspecting components|
|US8108168||Mar 12, 2009||Jan 31, 2012||Etegent Technologies, Ltd.||Managing non-destructive evaluation data|
|US8157908||Jan 28, 2011||Apr 17, 2012||3D Systems, Inc.||Three dimensional printing material system and method using peroxide cure|
|US8167999||Jan 10, 2008||May 1, 2012||3D Systems, Inc.||Three-dimensional printing material system with improved color, article performance, and ease of use|
|US8473236||Sep 10, 2009||Jun 25, 2013||Etegent Technologies, Ltd.||Managing non-destructive evaluation data|
|US8506862||Feb 2, 2011||Aug 13, 2013||3D Systems, Inc.||Three dimensional printing material system and method using plasticizer-assisted sintering|
|US8521480||Nov 12, 2009||Aug 27, 2013||Etegent Technologies, Ltd.||Managing non-destructive evaluation data|
|US9804997||Jun 25, 2013||Oct 31, 2017||Etegent Technologies, Ltd.||Managing non-destructive evaluation data|
|US20030095144 *||Nov 21, 2001||May 22, 2003||Trevino Scott E.||Method and apparatus for prescribing an imaging scan and determining user input validity|
|US20040138336 *||Aug 26, 2003||Jul 15, 2004||Z Corporation||Three dimensional printing material system and method|
|US20050003189 *||May 19, 2004||Jan 6, 2005||Bredt James F.||Thermoplastic powder material system for appearance models from 3D prinitng systems|
|US20050087016 *||Oct 24, 2003||Apr 28, 2005||Gilmore Robert S.||Inspection method and apparatus for determining incipient mechanical failure|
|US20060208388 *||May 24, 2006||Sep 21, 2006||Z Corporation||Material systems and methods of three-dimensional printing|
|US20060230984 *||Jun 15, 2006||Oct 19, 2006||Z Corporation||Three dimensional printing material system and method|
|US20070241482 *||Apr 3, 2007||Oct 18, 2007||Z Corporation||Production of three-dimensional objects by use of electromagnetic radiation|
|US20080138515 *||Dec 7, 2007||Jun 12, 2008||Z Corporation||Three Dimensional Printing Material System and Method Using Peroxide Cure|
|US20080187711 *||Jan 10, 2008||Aug 7, 2008||Z Corporation||Three-Dimensional Printing Material System With Improved Color, Article Performance, and Ease of Use|
|US20080281019 *||Feb 22, 2008||Nov 13, 2008||Z Corporation||Three dimensional printing material system and method using plasticizer-assisted sintering|
|US20080289423 *||Jul 21, 2007||Nov 27, 2008||Honeywell International, Inc.||Automated defect detection of corrosion or cracks using saft processed lamb wave images|
|US20090034828 *||Aug 1, 2007||Feb 5, 2009||Andrew Frank Ferro||Method and apparatus for inspecting components|
|US20100235111 *||Mar 12, 2009||Sep 16, 2010||Sheet Dynamics Ltd.||Managing non-destructive evaluation data|
|US20100235112 *||Nov 12, 2009||Sep 16, 2010||Sheet Dynamics Ltd.||Managing non-destructive evaluation data|
|US20100235153 *||Sep 10, 2009||Sep 16, 2010||Sheet Dynamics Ltd.||Managing non-destructive evaluation data|
|US20110130489 *||Jan 28, 2011||Jun 2, 2011||Z Corporation||Three dimensional printing material system and method using peroxide cure|
|WO2001071337A1 *||Mar 12, 2001||Sep 27, 2001||Howmet Research Corporation||Method of detecting hard alpha inclusions in a titanium casting|
|U.S. Classification||382/141, 382/270|
|International Classification||G06T5/20, G06K9/38|
|Cooperative Classification||G06K9/38, G06T5/20|
|European Classification||G06K9/38, G06T5/20|
|Jun 27, 2000||FPAY||Fee payment|
Year of fee payment: 4
|Jun 14, 2004||FPAY||Fee payment|
Year of fee payment: 8
|Sep 8, 2008||REMI||Maintenance fee reminder mailed|
|Mar 4, 2009||LAPS||Lapse for failure to pay maintenance fees|
|Apr 21, 2009||FP||Expired due to failure to pay maintenance fee|
Effective date: 20090304