CA2264867C - Systems & methods with identity verification by streamlined comparison & interpretation of fingerprints and the like - Google Patents

Abstract

Preferably a sensor receives a print image from an authorized person to form a template, and from a candidate to form test data. Power spectral density (PSD) data for the template and candidate are compare d, to read out rotation & dilation; these are used to adjust the template or candidate preparatory to a correlation to find translation. After applying the translation, and refinement of the rotation and dilation, normalized spatial correlation values (NSCVs) are used as a measur e of quality of the match - and thresholded to make an early rejection or acceptance decision in very clear cases. Where the question is closer, isomorphic adjustment is applied to the entire template or candidate for a fairer comparison in their overlap area. Such comparison proceeds by the same type of PSD analysis - but for multiple subregions in the overlap area. Resulting NSCVs are averaged to obtain a measure of quality of the match, which again is thresholded for a final decision in the closer cases. Noise variance from the test data, vs. position in the image, is used to weight the importance of comparison with the template in each subregion. Nonvolatile memory holds instructions for automatic operation.

Description

102025303540CA 02264867 1999-02-23WO 98146114 PCT /U S98/07 260-1-SYSTEMS & METHODS WITH IDENTITY VERIFICATION BY STREAMLINEDCOMPARISON & INTERPRETATION OF FINGERPRINTS AND THE LIKE RELATED U. S. PATENT DOQUMENTSA closely related coowned, copending application isserial O8/709,302 of Lawrence R. Thebaud, filed September 9,1996, and issued , 199__, as U. S. Patent5, , Two other coowned, copending applications arerelated: serial 08/382,220 of J. Kent Bowker and Stephen C.Lubard, Ph. D., filed January 31, 1995, and issued, 199__, as U. S. Patent 5,____,____; and serialO8/709,785 of J. Kent Bowker et al., filed September 9,1996, and issued , 199__, as U. S. Patent5,____,____ All three applications are wholly incorporatedby reference into the present document.FIELD OF THE INVENTIONThis invention relates generally to systems and methodsfor verifying identity of people, by comparison and inter-pretation of skin patterns such as fingerprints; and moreparticularly to novel firmware and software stored in appa-ratus memories, as portions of apparatus, for interpretingwiththe inventionsuch patterns and controlling utilization devices.respect to certain of the appended claims,further relates to systems that include such utilizationdevices.A utilization device is, for example, a facility,apparatus, means for providing a financial service, or meansfor providing information. The phrase “utilization device”thus encompasses, but is not limited to, businesses, homes,vehicles, time-and-attendanceautomatic teller machines,systems, database—searching services, and a great many otherpractical systems. An apparatus memory for such storage is,for example, a programmable read-only memory (“PROM”), or acomputer-readable disc.SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260BACKGROUND OF THE INVENTIONClassical methods for evaluation of fingerprints. toe-prints, palmprints and like skin patterns entail location,Efforts to adaptthese classical techniques for automated print verificationcategorization and tabulation of minutiae.have received great attention and elaboration, but arefundamentally limited by their sensitivity to measurementnoise at the location of the minutiae.Automated analysis based on minutiae also is inherentlyvery dependent on image enhancement —— which often breaksdown when initial data quality is marginal. For thesereasons some workers have explored other methodologies.Some seemingly promising efforts employ holograms -either direct three-dimensional images of prints, or holo-graphic Fourier transforms (which have the advantage ofbeing position invariant). Some of these techniques, forbest results, impose costly demands on special memory de-vices for storing the holograms. These holographiccorrelators are in essence modern refinements of muchearlier two—dimensional direct-optical-overlay correlatorssuch as that described by Green and Halasz in U. S. Patent3,928,842.An intermediate ground is represented by a fewrelatively sophisticated patents that use digital computersto (1) automatically select one or more distinctive smallregions —— not necessarily minutiae —— in a master print or“template”, and then (2) automatically look for one or moreof these selected small regions in a print provided by aperson who purports to be the maker of the template. Theseearlier patents particularly include U. S. 5,067,162 ofDriscoll, 5,040,223 of Kamiya, 4,982,439 of Castelaz,4,805,223 of Denyer, and 4,803,734 of Onishi.All of these latter patents describe making finalverification decisions based upon such comparisons of smallregions exclusively —- although in some-cases a small numberof such regions are considered concurrently. We haveconfirmed that many fingerprints can be analyzed veryquickly and accurately using just one or two regions, but weSUBSTH1H1ESHEET(RULE25)0!102025303540CA 02264867 1999-02-23wo 93/451 14 PCT/US98/07260-3-have also found that provision must be made for asignificant number of prints in which such short-formefforts are indeterminate or at least not adequatelyreliable.Thus the patents listed just above are flawed in theirultimate dependence upon isolated, small amounts of data -more specifically, very small fractions of the availableinformation in a candidate user's print —— for allfingerprints, regardless of the character of the print. Theabove-mentioned related patent document of Thebaud, on theother hand, takes into account essentially all the availableinformation in a candidate print.Thebaud’s system does so for all prints. We haverecognized that for some types of systems this thoroughnessand the accompanying time consumption can represent asignificant drawback, because —— in a large majority ofcases —— small regions contain sufficiently distinctiveinformation for a reliable analysis.Some of the patents in the above list do describe soundtechniques for one or another part of their respectiveprocesses. Some workers, such as Driscoll and Kamiya, usecorrelation methods (but electronic-data correlationmethods, not optical correlation methods) to choose thesmall reference sections in the enrollment process —— i. e.,in forming the template —— and also in comparison of thoseregions with features in a candidate user's print. Denyersimilarly uses an approximation to such correlationtechnique.These patents do generally allow for the possibilitythat the authorized user's template may be shifted, or inother words translated, in placement of the print image onthe sensor. some (particularly Driscoll and Denyer) allowfor the possibility that the template may be rotated too.Driscoll discusses finding a least—squares—fit betweenplural reference regions and a potentially correspondingHesuggests that departures from an ideal rotated pattern ofplurality of test regions in the candidate print.the reference regions is to be accounted for by distortionof the fingertip in the course of placement on a sensor, butby his reliance on a very small number (typically three, asSUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-.,(-understood) of well-separated reference regions hisallowance for distortion —— and his overall verificationdecision as well ~— inherently make use of only a very smallfraction of the available information. Denyer, too, brieflymentionsway) theAllhowever,(though in a much more generalized and tangentialpossibility of somehow accounting for distortion.of these patent documents except Thebaud’s,fail to take account of dilations which anauthorized user's fingertip may undergo —— relative to the(By the term “dilations”we mean to encompass dilations or contractions as the casesame user's established template.may be.) Such dilations may arise from variations in thepressure with which the finger is applied to an optical orother sensor (capacitive, variable—resistance etc.).Such dilations may be expected to have at least acomponent which is invariant across the entire image, inother words a dilation without change of fingerprint shape-— an isomorphic dilation. Furthermore all the above-mentioned patents fail to make systematic, controlledallowance for dilations and other forms of distortion thatare differential —— which is to say, gggisomorphic.Correlation methods, matched-filter methods, and(loosely speaking) related overlay—style techniques ofcomparison all fail totally in any area where a referenceprint is mismatched to a candidate print by as little as aIt has been foundthat dilations and other distortions can and commonly doquarter of the spacing between ridges.sizable areas -that is, manycorrelation andproduce spurious mismatches locally —- overexceeding twice the spacing between ridges,times the minimum disruption which destroysthereby recognition.Therefore, failure to account properly for eitherdilation (isomorphic distortion) or distortion (differentialdistortion) results in unacceptably high rates of failure toverify or recognize an authorized user —— i. e., high ratesof the so—called “false rejection” or “type 1 error”.Artificial measures aimed at reducing this failure rate leadinevitably to the converse: unacceptably high rates offailure to reject unauthorized users, impostors —— i. e.,SUBS'lTl’UTE SI-IEEF (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260.3.high rates of the so—called “false acceptance” or “type 2error”.For those few cases in which abbreviated analysis isunreliable, it appears unlikely that adequate account ofdistortions can be made merely by allowing for randomvariation as between two or three cores or distinctiveregions. The full— coverage paradigm of the Thebauddocument, by virtue of its ability to use all informationavailable in the entire area of overlap between thereference and test images, has an imunity to such error,but at the cost of a relatively long analysis time -currently several seconds (after the fingerprint data arefully acquired) per determination —— even for prints whichhave very distinctive regions.Similarly none of the prior-art patents noted makes useof decisional downweighting of data from areas that are lesscertain or noisier; rather, to the extent that anyconsideration at all is given to such matters, noisy dataare simply discarded -— a very undesirable way to treatexpensive data. Bandpassing of test data is not seen inthese references, although certain other forms of filteringare used by Driscoll and others. Normalizing is likewiseabsent —— except for trivial forms implicit in binarizationNone of thenoted patents teaches expression of test and template data,or trinarization, used in many print analyzers.or comparison of such data with one another, in terms oflocal sinusoids.Another problem which the art has not adequatelyaddressed heretofore is that of image—data coverage andquality.to find that images are acquired and accepted for analysisIt is common in commercial devices in this fieldbased only upon occluding of the acquisition port by afinger or some other object —— i. e., the presence ofsomething at the acquisition port —— without regard for theusability or reliability of the image, or indeed evenwhether it is an image of a fingerprint or other skinpattern.Still another difficulty is that analysis systems arenot necessarily attuned to the peculiarities of the skin-SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23WO 98/46114 PCT/US98/07260.6-pattern data encountered. When analysis proceeds onassumptions (for example, the size of the fingerprint-ridgespacing) that are not applicable to the actual specimenpresented, reliability of the results is impaired.Another challenge not adequately met heretofore is thatfingerprint acquisition systems labor under severely adverseconditions of low skin—pattern contrast and high variationin lighting across the pattern -— so that a relatively highdynamic range in light intensities is present —— but yet thecost, time consumption and in some cases sheer space andbulk required to store or transmit the resultant,correspondingly high—dynamic—range signal data.A very closely related problem is that attempts toreduce the required cost, time consumption and space or bulkassociated with data storage and transmission run into acontrary requirement that the data must be fairly comparableto original data. It must not have anomalouscharacteristics that can be associated with, for example,commonplace data-compression techniques. A particularlyknotty problem is the need for smoothness along ridges, lestthe analysis system be unable to recognize their essentiallycontinuous character.Thus the skin—pattern verification field has failed to—— in a time-effective manner —— make good use of allavailable data, take adequate account of dilations ordistortions, make suitable allowance for known statistics ofplacement variation, and apply modern decisional and signal-processing tools. As can now be seen, prior art in thisfield remains subject to significant problems, and theefforts outlined above —— while praiseworthy -— have leftroom for considerable improvement.SUMMARY OF THE DISCLOSURETheperformspresent invention introduces such improvement, andfingerprint verifications in a remarkably shorttime and with an outstandingly high accuracy not availableheretofore. The invention has several facets or aspectsSUBSTITUTE SHEET (RULE 25)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260.7-which are usable independently —— although for greatestenjoyment of their benefits we prefer to use them together,and although they do have some elements in comon.Comon parts will be described first. In preferredembodiments of the first three independent facets which willbe discussed below, the invention is apparatus for acquiringpersonal skin-pattern print data for use in comparison toverify the identity of a person.In preferred apparatus embodiments of the next seventhe fourththe present invention is apparatusindependent facets to be discussed (i. e.,through tenth aspects),for verifying the identity of a person. It operates by com-paring (1) test data representing a two—dimensional testimage of that person's skin-pattern print with (2) referencedata derived from a two—dimensional reference skin—patternprint image obtained during a prior enrollment procedure.Certain additional aspects or facets of the inventionwill be described following the first ten. Each of theapparatus embodiments includes some means for holdinginstructions for automatic operation of the other elementsof the apparatus; these instruction-holding means include ormake use of a nonvolatile memory device, and may be termedthe “nonvolatile memory means”.Now in preferred embodiments of a first of itsindependent aspects, the apparatus includes some means forphysically receiving contact by the skin of such a personand for, during that contact, forming an optical image ofthe skin pattern. For purposes of breadth and generality indiscussion of the invention we shall refer to these meanssimply as the “optical means".In addition the apparatus includes some means forreceiving the optical image from the optical means —— andgenerating in response a series of electronic signal arraysderived from such pattern during the contact.generality and breadth we shall call these theAgain for“optoelectronic means”.thefor monitoring the series of electronictheIn addition the apparatus includes some means,“electronic means”,signal arrays during the contact; and some means,SUBSTITUTE SHEET (RULE 26)1015203035CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-8-“saving means”, for saving at least one of said electronicAlso in theapparatus are some means for deferring operation of thesignal arrays for use in such comparison.saving means until at least one of the electronic signalarrays in the series satisfies a particular conditionrelated to a characteristic of the pattern; these last“deferring means” are responsive to the electronicmonitoring means during the contact.The foregoing may represent a definition or descriptionof the first aspect of the invention in its broadest or mostgeneral form; however, even in this form this facet of theinvention can be seen to importantly advance the art of fin-gerprint acquisition and analysis. In particular the systemis more reliable than heretofore, in that an image isaccepted for analysis only if its quality (and as will beseen its areal coverage) are adequate for analysis.Nevertheless we prefer to practice the first aspect ofthe invention —— and others mentioned below as well —— withcertain further features and characteristics that enhanceenjoyment of the benefits of the invention. Accordingly weprefer, for example, that the “particular condition”mentioned above includes a test for adequacy of skin-patternimage area, based not merely upon occlusion of the opticaldata port but actually upon spatial-frequency content of theelectronic signal arrays.That is to say, the signals should contain energy atspatial frequencies characteristic of skin-pattern prints,even when our invention is simply testing for arealcoverage. A like more but stringent criterion is appliedwhen the system is at a more advanced stage of testing forquality of the print.Furthermore we have found that details of a skincontact settle during contact, and the electronic signalarrays in the series tend to improve as said skin contactsettles. We accordingly prefer to collect sequential imagesover an extended period of time when necessary, halting theprocess only if and when an image is acquired that passesusability criteria.SUBSTITUTE SHEET (RULE 25)101520253035CA 02264867 1999-02-23WO 98/46114 PCT/US98/07260-9-Numerous other preferences will appear in regard tothis first aspect (and the others as well) of the invention,in the “DETAILED DESCRIPTION” section that follows.In preferred embodiments of a second main facet oraspect of the invention, the apparatus includes some meansfor receiving or generating an electronic signal arraycorresponding to the skin pattern —— these will be calledthe “receiving or generating means” —— and also some meansfor defining a plurality of signal wavenumber bands.Included moreover are some means - the “comparingmeans” —— for comparing wavenumber content of the electronicsignal array with the plurality of definedFurther the system includes some means fora particular one band of said plurality tosaid electronic signal array to verify thewavenumber bands.selectinguse in analyzingidentity of suchperson; these selecting means are responsive to thecomparing means.Even as defined thus broadly, a system in accordancewith this second aspect of our invention operates in asignal spatial waveband that is specifically chosen to matchthe data encountered. Reliability of the resulting analysisis accordingly enhanced.In preferred embodiments of the third main aspect ofthe invention, the apparatus include some means forreceiving or generating a multilevel electronic signal arraycorresponding to such skin pattern. It also includes somemeans for preliminarily evaluating or preprocessing, orboth, the multilevel electronic signal array.By “multilevel” we mean that the dynamic range of thesignal is at least four binary bits —— i. e., a factor ofsixteen times the smallest signal shift which the system cancomprehend —— and preferably five bits or more. Our presentthough this isprimarily a matter of economically available components.preferred embodiment is an eight—bit system,In addition the apparatus includes some means forexpressing the preliminarily evaluated or preprocessed, orboth, signal array in two- or one-bit form. It alsoSUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-10-includes means for storing or exporting the signal arrayexpressed in said two- or one-bit form.The foregoing may provide a definition or descriptionof the third aspect of the invention in its most general orbroad form, but even as thus broadly couched the thirdaspect of the invention significantly advances the art.particular, preferred embodiments according to this thirdInfacet of the invention can now be seen to provide a fulldynamic range for the signal, to allow for lighting varia-tions —— and other variables such as whether the skin-pattern contrast is high or low. At the same time thisaspect of the invention does not compromise as to the time,space or cost of storage or data-export capacity.As mentioned earlier, several preferences areapplicable to even still further enhance the benefits ofthis third aspect of the invention.As to the fourth main aspect of the invention, theapparatus includes some means for extracting reference datafrom storage or from an imported data set for use inverification. The apparatus also includes preprocessingmeans for bandpassing, normalizing and smoothing theextracted data for use in verification.Further included are some means for comparing the datafrom the preprocessing means with the test data to verifyidentity. The foregoing presentation may represent thefourth facet of the invention in its most broad and generalmanifestation, but even so it does meaningfully promote theart, particularly in that economically and quickly exportedor stored data with only one or two bits (in accordance withthe third aspect of the invention) is readily andeconomically rendered completely adequate for use inanalysis.Turning to a fifth major facet of the invention, theapparatus includes some means for deriving from referencedata or test data, or both, a respective form of a vectorgradient field.smoothing such reference or test data, or both, underThe apparatus also includes some means forcontrol of the vector—gradient—field form so that theSUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-11-smoothing is substantially along the directions ofIn addition theapparatus includes some means for applying the smoothed datarespective ridges of the skin pattern.in making an identity-verification decision.In this way data quality needed for entirely reliableanalysis is simply reconstituted, when stored or transmittedimage information is readied for use. No compromise need bestruck between storage/transmission economies andreliability of verification.This broadest form of the fifth facet of the invention,too, is subject to additional preferences. For instance weprefer to find a vector gradient field from a fast Fouriertransform of the data, screening the vector gradient fieldto account for phase jumps.In addition, merely as a matter of practicalities wecurrently prefer that the form of vector—gradient-fieldemployed be a vector wavenumber field. This preferencearises simply from the availability of a finished routinefor accomplishing this task, as our current most highlypreferred form of the apparatus does not actually use thescalar magnitudes that are part of the wavenumber field.We nonetheless prefer, in view of the availability justmentioned, that the deriving means further comprise somemeans for calculating from the gradient field a covariancematrix, and from the covariance matrix in turn a scalarmagnitude field for the wavenumber. In this case we alsoprefer that the system include some means for constructingthe vector wavenumber field as the scalar magnitude fieldwith directedness of said vector gradient field.In preferred embodiments of yet a sixth main facet oraspect of the invention, the apparatus includes some meansfor computing power spectral density of at least a portionof the test image. In addition it includes some means forapplying the power spectral density to estimate an assumeddilation of the test image relative to a reference image.Also the apparatus includes some means for comparingthe test data with the reference data,the estimated dilation.taking into accountFurther included are some means,SUBSTITUTE SHEET (RULE 25)10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/U S98/07 260-12-responsive to the comparing means, for making an identity-verification decision.The foregoing may represent the sixth main aspect ofthe invention in its most broad or general form. Even asthus formulated, however, this facet of the inventionprovides an extremely important contribution to the art offingerprint andAs will bedocument, powerreveal both theridges —— particularly within a small area of a pattern.other skin-pattern-print analysis.explained in greater detail later in thisspectral density (or “PSD”) can be made tospacing and directionality of skin-patternBYcomparing the spacing portion of a PSD for a test image withthe like portion of a PSD for a reference image or“template”, the present invention is thus able to read offthe relative dilation of a test image relative to areference image, for corresponding areas.In this way, as will shortly be seen, the inventionproduces excellent approximations to the results of not onlythe early global search of the Thebaud patent document butalso the later gradient search with its holisticnonisomorphic distortion fields. The PSD technique, oncesystematized and made efficient, also can be used forscreening prints at acquisition to be certain that energy ispresent in spatial wavebands characteristic of skin—patternprints.A calculation of two PSDs, however, unlike the time-consuming procedure taught by Thebaud, for a small regiontakes a yggy small fraction of the time which his apparatusrequires —— most typically between one and two orders ofmagnitude faster overall. Thus the present invention isable to achieve very nearly the same results in considerablyless than a tenth the time.In preferred embodiments of a seventh of its major as-pects, the invention is closely related to that of the sixthbut with respect to rotation rather than dilation. Onceagain a good estimate of relative rotation is accomplishedmerely by comparing the orientational portion of a test-image PSD with the corresponding portion of a reference-image PSD.SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23WO 98/46114 PCT/US98/07260-13-Through finding both the relative dilation and rotationin this way, the invention is able to estimate the entireisomorphic distortion with surprising accuracy. Through useof a multiple set of regions covering a full print area, orat least the area of overlap between reference and testimages, the invention also approximates adequately thenonisomorphic distortions found by Thebaud.Preferred embodiments of an eighth main aspect of theinvention are also related to analysis by PSD comparisons.The apparatus here includes some means for computing powerspectral density of at least a portion of the test image andof the reference image, respectively.In this main facet of the invention, the apparatus alsoincludes some means for transforming the respective computedThrough thisinnovative tactic, the transformed power—spectral—densitypower spectral densities to polar coordinates.information —— which now can be interpreted as rectangular-coordinate data —- has the form of power-density valuesplotted on a rectangular grid of ridge spacing andorientation.In addition the apparatus includes means forconsidering the transformed power spectral densities forsuch test and reference images together. These means alsohave the further function of reading off from the“considered—together” power spectral densities an estimateof such assumed relative rotation and dilation.The particularly favorable result, in the case of thiseighth aspect of the invention, is that even greater timesavings and efficiency can be gained by expressing the ridgeorientation and spacing as fields defined within the samerectangular grid and rectangular—coordinate mathematics asare applicable to most of the other procedures in thefingerprint analysis. All these advantages are furtherenhanced by preferred operating modes such as ratioing therespective ridge-spacing and orientation values, orcorrelating the two transformed power spectral densities -within a hypothesis range of relative rotation and dilation—— to find an estimate of the most probable relativerotation and dilation.SUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23WO 98/46114 PCT/US98/07260.].I.In preferred embodiments of a ninth of the main aspectsor facets of the invention, the apparatus includes somemeans for estimating relative translation, and at least onecomponent of relative isomorphic distortion, between thetest and reference images. It also includes means foradjusting the test or reference image, or both, to allow forthe estimated relative translation and component of relativeisomorphic distortion.Further the apparatus includes some means for comparingthe test and reference images, after the adjustment, withintoAs will be noted,common with thesubstantially all area that is common to both images,make an identity—verification decision.this advantageous operating scheme is ininvention set forth in the previously mentioned Thebaudpatent document —— and represents a potent advance over theprior art.We prefer, however, to practice this ninth aspect ofthe invention in conjunction with certain other facets oraspects that maximize enjoyment of the benefits of theinvention. For example we prefer that the comparing meansinclude means for analyzing power spectral densities withinthe common area to estimate remaining distortions.In this regard we prefer that the comparing meansinclude some means for dividing one of the images into amultiplicity of substantially overlapping subregions that inthe aggregate cover substantially the entire said one image;and additional means for evaluating the degree of similarityof said test and reference images, with respect tosubstantially every one of said subregions of which a sig-nificant fraction is within said all area comon to bothimages.In this way the capability of the PSD to yield rotationand dilation information very quickly and efficiently forsmall areas is exploited to obtain an estimate of suchinformation for quite large areas. Preferably theevaluating means include some means for estimating, withineach of the subregions in the comon area respectively, afurther component of relative distortion between test andreference images.SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-15-Preferably evaluating means form a composite measure ofthe “further components" for all of said subregions in thecommon area; and means for thresholding that compositemeasure to make said decision. Also preferably theapparatus extracts from the test data an estimate of noisevariance in the test data as a function of position in thetest image; in this system the composite-measure formingmeans take into account the estimated noise variance —— andpreferably weight the further component of distortion, foreach of the subregions in the comon area, in an inverserelation with the noise-variance estimate for thatsubregion.In preferred embodiments of yet a tenth major aspect ofthe invention, the apparatus includes some means forreference data withItthat first measurecomparing a first small region of thethe test data to form a first measure of similarity.also includes first means for testingagainst a first threshold to verify such person's identity.In case the first measure is not high enough foracceptance, the apparatus also includes some means for thencomparing a second small region of the reference data withthe test data to form a second measure of similarity —— andassociated second means for testing said second measureagainst a second threshold that is higher than the firstAs will benoted, it would be more expectable after failure of thethreshold, to verify such person's identity.first measure to test against a second measure that islower, but that is not the case in the tenth major aspect ofthe present invention.The reason for this anomaly is that the second test em-ploys a smaller window. This strategy is adopted on thebasis of the reasoning that the first test may have failedin recognition merely because too much distortion is presentto allow recognition over the area of the first test: it isa small area, but that of the second test is relatively evensmaller.Several other preferences will appear. In particular,the first testing means also test the first measure ofsimilarity against a first, relatively low, rejectionSUBSTITUTE SHEET (RULE 26)1015202530CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-16-threshold to deny verification. The second comparing meansoperate only if the first measure of similarity is betweenifthe first measure of similarity is neither high enough forthe first acceptance and rejection thresholds —- i. e.,acceptance nor low enough for rejection.Preferably the second testing means also test thesecond measure of similarity against a second rejectionthreshold,smallthreshold that is higher than the first rejectionto deny verification. As noted above, the secondregion is smaller than the first small region.is betweenso thatthe second measure is neither high enough for acceptance norAlso preferably, in event the second measurethe second acceptance and rejection thresholds -low enough for rejection —— then the system comparessubstantially the entire comon area of the test andreference images to make a verification decision.Preferred apparatus embodiments of yet an eleventhindependent facet or aspect of our invention divergesomewhat from the first seven. The apparatus here is forreceiving surface-relief data from a sensor that acquiressurface-relief data from a relieved surface such as a finger—— and in response controlling access to facilities,equipment, a financial service, or a system for providing orreceiving information.The apparatus is for use in the presence of an assumeddilation of the relieved surface. The apparatus includes asystem for processing the received data to determineIn addition to thepreviously mentioned instruction-holding memory means, thisidentity of the relieved surface.system includes:means for calculating and comparing power spectraldensities of at least a portion of the receiveddata and test data respectively, and analyzing thepower spectral density comparison to estimate theassumed dilation,SUBSJTflIflESHEET(RULE25)1015203040CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-17-means for comparing the test data with reference data,taking into account the estimated dilation, andmeans, responsive to the comparing means, for making anidentity-verification decision.In addition, the overall apparatus includes some means forapplying the determined identity to control access to suchfacilities, equipment, financial service, or source orreception of information. Thus this aspect of theinvention, while specifically incorporating the dilation-estimating feature mentioned above in connection with thefifth independent aspect, particularly focuses on andincludes, as part of the invention, components that actuallycontrol access to various types of utilization means.A twelfth independent facet of the invention involves afurther divergence, in that it is a secured system subjectto access control based upon surface—relief data from a re-lieved surface such as a finger. This system is for use inthe presence of an assumed distortion of the relievedsurface.The system includes utilization means, susceptible tomisuse in the absence of a particular such relieved surfacethat is related to an authorized user. The utilizationmeans being selected from the group consisting of:a facility,apparatus,means for providing a financial service, andmeans for providing or receiving information.In addition the system includes sensor means for acquiringsurface—relief data from such a relieved surface.The system also includes some means for processing thedata to determine identity of the relieved surface, and forapplying the determined identity to control access to theSUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-18-utilization means. These processing and applying means in-clude, in addition to the instruction-holding memory means:means for calculating and comparing power spectraldensities of at least a portion of the receiveddata and test data respectively, and analyzing thepower spectral density comparison to estimate theassumed dilation,means for comparing the test data with reference datarelated to the particular relieved surface relatedto the authorized user, taking into account theestimated distortion, andmeans, responsive to the comparing means, for making anidentity-verification decision.Thus this aspect of the invention includes the utilizationmeans themselves, as well as the access-control intermediarythat is included in the eighth aspect of the invention.While thus focusing on and including the utilizationmeans, the invention makes use of the distortion-estimatingfeature discussed earlier in connection with the sixthindependent facet of the invention.In yet another of its independent aspects or facets,preferred embodiments of the invention take the form of amethod,the identity of a person.rather than apparatus. This method is for verifyingThe method does so by comparingtest data representing a two—dimensional test image of thatperson's skin—pattern print with reference data derived froma two—dimensional reference skin—pattern print imageobtained during a prior enrollment procedure.The method includes the step of ratioing or correlatingpower spectral densities of corresponding regions of thetest and reference images to determine relative isomorphicdistortion between the images. Another step is using anormalized spatial correlation value as a measure ofsimilarity between corresponding regions of the test andreference images.SUBSTITUTE SHEET (RULE 25)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-19-Furthermore the method includes the step of making anidentity-verification decision based on said normalized spa-tial correlation value. Another step is, in nonvolatilememory, holding instructions for automatic operation of theforegoing steps.Thus the method partakes of the advantageousness of theapparatus embodiments of the power—spectral-density aspectsof the invention, discussed earlier. Preferably this methodis optimized by incorporation of other features orcharacteristics, particularly the steps of operating asensor to acquire the test data and —— responsive to thedecision-making step -— operating a switch if identity isverified.All of the foregoing operational principles andadvantages of the present invention will be more fullyappreciated upon consideration of the following detaileddescription, with reference to the appended drawings, ofwhich:BRIEF DESCRIPTION OF THE DRAWINGSFig. 1 is a flow chart or block diagram showing at aconceptual level, for certain preferred embodiments of ourinvention, how different portions of the programmed firmwareperform the processes of the invention;Fig. 2 is a like chart or diagram of details within aparticular routine or module of Fig. 1 -— namely, a routinethat is used in block 34 and again in block 51, and also ina preliminary procedure that prepares the authorized user'sfingerprint data (“template”) for use;Fig. 3 is a rough conceptual diagram of a candidateuser's fingerprint superposed in position on the authorized-user template, as linked with the template by a particularisomorphic distortion found by the processes of the presentinvention;Fig. 4 is another conceptual diagram, conveying thegeneral principle of determining relative dilation androtation from a power spectral density graph;SUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23wo 93/451 14 PCTIUS98/07260-20-Fig. 4a is a like diagram conveying the use of arectangularized (polar-coordinate) form of the Fig. 5 graph;Fig. 5 is a rough conceptual diagram, conveying thegeneral principle of applying a distortion field to modifythe template;Fig. 6 is a conceptual diagram illustrating the findingof such a distortion field piecemeal from power spectraldensity analyses as in Figs. 4 and 4a;Fig. 7 is a highly enlarged conceptual diagram of awhorl area in a fingerprint, particularly illustratingchanges of interridge phase in the area;Fig. 8 is an overall block diagram showing theembodiment of our invention in a hardware system;Fig. 9 is a diagram showing search areas in a sensor ofthe present invention, as used for acquiring the authorizeduser's template data;Fig. 10 is a flow chart showing procedures for use indata;11 is a darkness-level (or brightness-level)acquiring both authorized and candidate users’ andFig.diagram conveying data compression used for storage of thetemplate data.DETAILED DESCRIPTIONOF THE PREFERRED EMBODIMENTSThe first parts of this section set forth the operationAn “APPENDIX”then follows presenting the basic mathematics for actualof the system in purely descriptive terms.practice of the invention.Inputs —— Preferred embodiments have at least threegroups of inputs: one group of inputs from the candidateuser of a weapon or other apparatus, another from theauthorized user (or that person's surrogates), and the thirdThecandidate's inputs include a fingerprint—image data array 11implicitly from generalized population data.(Fig. 1) and a command 57 (at bottom left in the drawing)that the apparatus operate.SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23wo 93/451 14 PCT/US98/07260-2]-In Fig. 1 the general format of the first illustrationof the Thebaud patent document has been retained, to moreplainly highlight both the similarities and the differencesParallelreference to Thebaud's Fig. 1 is suggested, and the reader'sfamiliarity with the disclosures set forth in the Thebaudpatent document is assumed.between the present invention and that of Thebaud.The data array 11 originates from a skin—patterndetector, which is most representatively an optical sensorarray but may instead be of another type such as capacitive,variable—resistive or high—frequency acoustic. In anembodiment that is now most highly preferred, the comand 57takes the form of the optical signals that result fromplacing the user's finger etc. on the sensor contactsurface, though in other embodiments the command 57 may comefrom operating a switch —— e. g. a microswitch actuated bypressure on the sensor contact surface.The authorized user's inputs include a fingerprint- image data array 21 (originating analogously to the array 11for the candidate user, discussed above), and implicitparameter settings such as thresholds 41, 53 which reflectthe desired certainty with which a fingerprint match must befound. The authorized user does not necessarily personallyenter these parameters into the system, but may insteadindicate to a technician a selection of the value, or acqui-esce in the value, of such parameters.The several threshold parameters 41, 53 etc. are re-lated to the relative numbers of false positives and falsenegatives to be tolerated —— but not necessarily in anIn the Thebaud system therelationship is rather complicated and statistical as hearithmetically direct way.explains; but for purposes of the present more streamlinedsystem design, relatively direct and straightforwardthresholding is preferred.The values used as thresholds 41 are actually dual ——one controlling rejection 42 of the candidate and anothercontrolling acceptance 43. (As will be seen shortly, thethresholding is truly quadruple rather than dual, since twodifferent values are used at different points in theprocedure for each of the two thresholds just mentioned.)SUBSTfi1H1ESHEET(RULE1flfi10152025303540CA 02264867 1999-02-23wo 93/451 14 PCT/US98/07260-22-All these values are selected to reflect the type of usageanticipated. In particular, they control respectively theprobability of false negatives (establishing the “desiredcertainty” of acceptance for the authorized user) and theprobability of false positives (establishing in an inverseway the desired certainty of rejection for an ggauthorizeduser).For example, if the apparatus is to control access toan advance—fee—based gymnasium, the primary objective may bemerely to discourage occasional cheaters. In this case thethreshold for acceptance 43 (for the prepaid customer ormember of the gym) may be set rather low, and that forrejection 42 very low —— accepting a significant chance ofletting in someone who has not paid.Similarly if the apparatus is a weapon to be used inthe field by military or police personnel, a primaryobjective may be to have use of the weapon available to theIn thiscase the thresholds for acceptance 43 and rejection 42 mayauthorized user without delay and without question.be set relatively low —— accepting some small chance thatthe weapon might be usable by an opponent who takes it fromthe authorized user. In this case, however, since there aresignificant risks associated with an opponent'sappropriation of a weapon, the acceptance threshold mightnot be set quite as low as in the first example above wherethe adverse consequences of admitting a cheater are minor.Now in a contrary example, for control of access to asecure area containing documents or apparatus of utmost sen-sitivity, a primary objective may be to exclude spies. Inthis case the certainty level for acceptance 43 (hopefully,of authorized personnel) may be set distinctly high, andthat for rejection 42 also quite high —— accepting somesignificant likelihood that an authorized individual may bedelayed in entry by having to repeat the verificationprocedure.Similarly if the apparatus is a weapon to be kept in ahome for protection against intruders, a primary objectivemay be to prevent unauthorized use by children or teenagersIn this case the thresholdsmay be set relatively high —— accepting some small degree ofwho live or visit in the home.SUBSTITUTE SHEET (RULE 26)101525303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-23-unreliability in the weapon’s availability for use againstintruders —— but perhaps not as high as in the immediatelypreceding example, since delayed availability of a weapon toan authorized user in an emergency is ordinarily much moreonerous than delayed entry to a secure area.A third, implicit type of input is a statistical set (not shown) preferably coming from neither the candidateuser nor the authorized user, but rather from a generalizeddatabase representing people in general.setting the specific levels of thresholds for variousThese are used inpurposes, and also for use of the variance estimates 15,etc.These statistical data are ordinarily derived withoutreference to the particular people known to be involved, andmay be called “prior statistics” or “a priori statistics”.They may be employed at certain points in the processing totake into account the known degree of variability in the waypeople place their fingers on a fingerprint-acquisitionimaging device. This variability may differ depending onthe position and orientation of the imaging device inrelation to the user.For example, variability in a panel—mounted imager atan automatic teller machine may be expected to have astatistical pattern that is different from variability in adesktop imager in an office. Variability in an imager thatis built into a tool (e. a., a weapon) may be expected to bedifferent still.In some cases, particularly where a user typically isstanding while applying a fingertip to a stationarilymounted imaging device, this variability may depend in partupon the height of the user. In any event it is preferableto collect a different a priori data set using the actualtype of imager and collection geometry for which aparticular apparatus will be_used.In special cases, initial data acquisition may showthat the authorized user's fingerprints have very unusualproperties or characteristics. In such extraordinary casesbetter performance may result from using a statistical set17 derived from input data 21 for the authorized user.SUBSTITUTE SHEET (RULE 25)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-24-Such provisions require a lengthier procedure forenrollment or registration of the authorized user, toestablish not only this user's fingerprint but also certainmeasures of the variability in this user's presentation ofthe print to the apparatus. For good results, furthermore,such a procedure should be deferred until the authorizeduser has acquired some familiarity with the apparatus, whichintrinsically tends to lead toward habits of handling —— andthereby not only to reduced variability but also toparticular patterns of variability.Such extra effort might possibly be justified in spe-cial cases, as for instance with a person who has an injuryor a handicap that affects the posture or the attitude ofthe arm or hand. Another possible special situation perhapsmay occur when a person of very unusually short stature, ora person in a wheelchair, will be placing a fingerprint on adevice to operate an automatic teller machine where mostusers stand. Such special problems of stature, etc., ifthey prove significant may be best managed by assemblingheight-correlated and other specially correlated statistics.In general the use of a priori statistics, ideally col-lected from users who have already formed habits in placingfingers on imagers, appears preferable.Procedural overview —— A glance at the bold verticallines 14, 22 in Fig. 1 reveals that the fundamental schemeis to direct signals 12-14-14’ from the candidatefingerprint image data 11, and signals 22-24 representingthe authorized user's preprocessed fingerprint image data or“template” 21,tain side calculations or signal paths 15, 28-38 along theto a common final comparison 41 or 53. Cer-way facilitate and enhance the comparison.One major departure from the Thebaud system is that thecomparison does not necessarily proceed all the way to thethreshold decision 53 near the bottom of the diagram.Rather, that route is reserved only for candidate data thatpersistently fall between thresholds 41 for rejection 42 andacceptance 43 ——-in other words, for candidate informationthat are indeterminate 44, within the earlier procedures 32-37, 41-45 in the upper portion of the diagram.SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23wo 93/45114 PCT/US98/07260-25-In the case of an early rejection 42 or acceptance 43in response to the dual or quadruple thresholds 41, theasterisked bypass path 42/43 is followed to the finaldecision 42. Here the system physically implements thedenial 55d or enablement 55e of access to some utilizationmeans which the system is set up to guard.Results 42/43 of the early thresholds 41, or the resultof the later thresholding 53, interact with signals 59generated by the candidate's command 57 —— to determinewhether the command 57 produces no perceptible action at all¢, or produces operation 56. (The invention encompasses in-cluding a no-function warning light or tone, rather noperceptible action, if utilization is denied 55d.)Initial acquisition and preprocessing of image data,whether for candidate 11 or authorized user 21, is describedin a later section of this document. There is some overlapbetween that later discussion and the sections followinghere immediately —— which relate only to what appears inFig. 1.Preliminary processing of the candidate's data —— Pro-cessing of the candidate image data 11 begins with analysis12 of the dynamic range of signals which represent groovesand ridges within the image. The result includes forming anew image—data version 13, in which this dynamic range isnormalized, i. e. locally stretched or compressed to precisely match the overall range of the later processingstages.In addition the new version of the image is subjectedto Fourier analysis —— expressing the data as spatialsinusoids —— and bandpass filtering, to eliminate inap-propriate spatial frequencies in the image version 13. Inthe analysis 12, preferably (but not necessarily) spatialfrequencies are treated as “inappropriate” if they are Q9;spatial frequencies 21’ that could have originated from thesimilarly preprocessed print (template)-21 of the authorizeduser.Preprocessing of the authorized user's print to obtainthe template will be described later. In such original pre-SUBSTITUTE SHEET (RULE 26)1015203540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-26-processing, spatial frequencies can be rejected based on amore leisurely harmonic-content analysis of the authorizeduser's print.Closely associated with the range analysis 12 and re-sulting bandpassed, normalized sinusoidal data 13 is a down-sampling step 13’ which greatly reduces the amount of dataThisis important because it can make the differenceto be processed in all later stages of the procedure.step 13’between a procedure that is unacceptably time consuming anda procedure that is practical.To be sure it is also important that the procedure beaccurate. Properly controlled downsampling at this step,however, does not degrade overall performance. Morespecifically, it is known that the data 13 are representedsinusoidally, and that these data cannot have majorcomponents at finer spatial frequencies than the smallestspacing of troughs or ridges in the authorized user's print21.Accordingly, in downsampling 13’ it suffices topreserve representative values at a reasonable fraction lessthan half of that smallest periodicity -— or for exampleabout one third of the average periodicity. Once again thetemplate frequency content 21' is useful, in guidingselection of an optimum spatial frequency for use in thedownsampling step 13’.Philosophical overview —— Four importantcharacteristics of the invention can be gleaned already fromthe foregoing discussion of blocks 12 through 13’ in Fig. 1.First, the assumption is made throughout that the candidateuser is the authorized user —— and that this assumption canbe confirmed, if only we conduct a fair comparison.It might be supposed that this assumption will lead toSucha supposition would be incorrect, for it has been found thatan overwhelming number of false—positive test results.a fair comparison will only highlight the underlyingdifferences in information content between an unauthorizedcandidate (impostor) and the true authorized user.The present detailed description, as it unfolds, willmake progressively more apparent that each intermediate pro-SUBSTITUTE SHEET (RULE 26)101520303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-27-cess step of the invention —- when practiced upon a typicalimpostor’s print —— is most likely by far to lead to adecisive denial 55d.also confirmed fromif the candidatefair comparison isThe assumption under discussion isthe opposite perspective: what happensuser is in fact the authorized user? Aeffects of enormousto details ofSuch details include, in particular,absolutely essential to eliminating thevariation in fingerprint appearance dueoperating conditions.the physical and emotional condition of the user —— andthese considerations are especially important to avoidrejecting the authorized user.Thus the assumption that the candidate is theauthorized user only leads to a very great reduction in theamount of data to be processed, and a very great increase inreliability of the results.A second characteristic of the invention is a plan -but more a plan—in-reserve than an overriding plan as in AThebaud —— to form respective versions of the two data sets11 and 21 which are adjusted to be as much alike aspossible. This adjustment, however, is only with respect tocertain data properties that are known to be variable withinmultiple trials or instances of essentially any single userto form a print.These particular variable data properties, within theirknown degree of variability, are at best immaterial (and atTheinvention is accordingly fashioned to ferret them out, sothat they can be canceled out —— in a word, to ignore them.In doing so, it is necessary to accommodate the extremeworst misleading) to identification or verification.time pressure associated with the candidate-data processing.Conversely, relatively long times can be devoted toobtaining several instances of an authorized user's print -and selecting the most representative one(s) of them, andperforming image enhancement on the best instances.The shaded lines 58 enclose those portions of the datacollection and processing that can be performed in advance,before deploying the apparatus or receiving the candidateuser's command. These portions include establishment of anSUBSTITUTE SHEET (RULE 26)10152023035CA 02264867 1999-02-23wo 93/45114 PCT/US98/07260-23-implicit statistical set and the thresholds 41, 53, as wellas the authorized-user data collection and processing 21through 31’.A third characteristic of the invention is closely re-lated to the first two.invention makes the template as clean and as definite asThis characteristic is that thepossible —— and then exploits that fact by primarily relyingupon the template, rather than upon the candidate data,wherever feasible.An example of this is in the preferred use of thefor both theanalysis 12 and downsampling 13’ -— rather than relying upontemplate to provide periodicity criteria 21’statistics of the candidate data 11 for the bandpassingcriteria. This strategy is preferred even though theanalysis 12 does in fact extract those candidate-datastatistics 15 for other purposes.On the other hand, many instances of thischaracteristic of the Thebaud invention are absent in thepresent system, which has been very greatly streamlined andshortened. For example, the path 28, 29 for usage of vectorwavenumber fields 29 now ends with use of those fields insmoothing; in the present system they -— and their highlyspecialized gradient and quadrature forms —— are not used incomparisons of the distorted template with the candidatedata.A fourth characteristic of the invention is that itoperates on the data in terms of local sine—wave patterns,rather than as isolated binary data bits or linear (ridgeand groove) structures. Thus the initial noise and rangeanalysis 12 operates not only in positional space but alsoin Fourier space (in other words, in terms of the spatialfrequencies in the candidate image), and the new version orfiltered information 13 is presented as amplitudes ofsinusoids associated with each region of the original image.By virtue of this characteristic, while guided bydetection theory the invention can also take advantage ofthe high computational efficiency and fidelity of the FastSUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23WO 98146114 PCT/US98/07260-29-Fourier Transform (FFT).of the computationally intensive processes in the algorithm.Preprogessing of the authorized user's fingerprintimages —— During preprocessing 58 the authorized userprovides a fingerprint that will be refined to form atemplate 21.The FFT performs a large fractionDetails of the refinement will be discussedshortly.Where time permits, best results are obtained byacquiring several realizations, or successive trial images,of the authorized user's print —— and analyzing them todetermine which is most representative and whether they haveany extraordinary character that may require specialhandling. This information is very useful in controllingthe application of these data in the real—time processesthat follow.In some cases a user may appear to have more than onefamily or group of realizations —— perhaps due to divergent,separate habits of gripping or presenting a finger. In suchcases it is possible to assemble a composite of partial in-formation from each of plural realizations, or even to storeplural entire templates (with associated respectivelikelihoods of occurrence) to be tried alternatively inevaluating a candidate print 11, 13.In any event, from the representative authorized—userprint image or images 21, during preprocessing 58 the systemselects 31 several -— preferably exactly three -distinctive regions, subsets or windows 31'. These small,preferably circular regions 31’ may be stored separatelyfrom the full template 21 as in Thebaud —— but in thepresent system, in the interest of minimizing cost and timeof transmission or storage, it is preferred to avoid suchadditional storage. Due to use of PSD analysis, it is nolonger desirable to store numerous versions or variants ofeach region, prepared —— as described by Thebaud —— byapplying a variety of crosscombinations of various-sizedrotations and dilations.In addition, a so—called “matrix covariance estimator”is used to map 28 magnitude and direction of local ridgespacings in the template 21 -— to form vector wavenumberSUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23wo 93/451 14 PCT/US98/07260-30-fields 29,suggested earlier,(Asthe present invention uses only thewhich are used in smoothing the data.directionality and not the magnitude, related to ridgespacing, of the wavenumber fields, and accordingly aIn thepresent invention such smoothing guided by wavenumber fieldsgradient field may be successfully substituted.)or other gradient-field forms is performed for the candidatedata as well as the template; but after that has beencompleted, the wavenumber fields may be discarded.In addition, flags are set up in certain of the vectorwavenumber fieldstemplate data 22,flags are used in29 to warn of phase reversals in theas will be explained below. These warningavoiding adverse effects of allowing theprocessing to continuously traverse phase discontinuities.Using candidate-data variance estimates —— Thepreviously discussed initial noise analysis 12 in thecandidate-data (left) half of Fig. 1 may be consideredroughly as a data colander, which separates data from noise.Both the data and the noise are then suitably directed,respectively, for beneficial use.Fig. 1 shows that the data and the noise actuallyproceed to the same later stage 51 of the algorithm, in thesense that the later block 51 receives both data and noise.In that later processing module, however, these differentpieces of information are separately received and verydifferently used.Thus one of the above-mentioned side-calculation pathsis application of the noise information 15 abstracted fromthe candidate data to enhance later stages of processing.This information 15 is in the form of an array or field ofvariance estimates, in effect overlaid on the reformed imagedata 13 themselves.In other words the system constructs and keeps aseparate index 15 of the reliability of the image data 13 ineach region of the image, respectively. These reliabilityindices are used to weight the respective significance thatis attributed to a comparison 52 based on data in thecorresponding image regions.SUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-31-Thus the final test decision 52, 53 depends moreheavily on portions of the candidate data 11 that arerelatively cleaner. The test is thus made to depend moreIt will benoted, however, that this information is not needed for thelightly on portions that are relatively noisier.early thresholding decisions 41-44 —— simply because thethresholding 41 is set so that those decisions representextremely clear cases.Such use of downwgighted information, where theinformation is of lesser reliability, is fig; superior - inmaking maximum use of available information —— to merelysetting an arbitrary criterion of reliability and thendiscarding questionable information. The latter technique appears, for example, in Driscoll’s selection of a verysmall number of “best—match” regions, and then proceedingdirectly to final decision based on such regionsexclusively, regardless of the difficulty of the decision.For any given intensity of calculation, and any givennoisiness and distribution of noisiness in the candidatedata, the downweighting maximizes the reliability of theresults.Global search and isomopphic adjustment: puppose -Another side calculation 31-38 has a dual function. It isthis section which:(1) in cases that present very clear decisions, leadsdirectly and swiftly to a final answer and tocontrol ¢, 56 of the utilization means; and(2) in cases that present hard decisions, provides anecessary measure of simple (shape-invariant)geometrical mismatches in the formation, orrealization, of the candidate print image 11, relative to the template 21.By the terms “formation” and “realization” we mean todistinguish variations in placement of a fingerprint fromthe information content of the candidate print itself.SUBSTfl1fl1ESHEET(RUUE26)10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-32-Preferably for certain embodiments this second set oflike the first,These calculations 31-38 accountcalculations 31-38, is partially performedin preprocessing time 58.for displacements or translations of the entire image,rotations of the entire image, and also dilations orcontractions of the entire image resulting from variation inpressure with which the entire fingertip is pressed againstthe sensor. As will be understood, when increased pressuresquashes the whole surface of the fingertip against thereceiving surface, the whole fingertip surface may expand.slightly —— but preserving the original shape, i. e.isomorphically.Of course the authorized user's initial print is takenwith sgmg applied pressure, so each candidate-printrealization may be made with either more or less pressurethan applied in making that initial print. Hence the amountof size change if characterized as “dilation” may be eitherpositive or negative ——-or, if multiplicative, as a factorgreater or less than unity.The global search is “global” in two senses: first,the entire candidate print is canvassed to find one or moreregions that most closely match certain preidentifiedportions of the template. Second, once the one or morebest—match regions are found the remaining mismatch istreated as a positional/dilational error with respect to theentire useful area of both prints.Identifying comparison regions for the qlobal search —-The comparison regions 31, also called “local subsets” andmost preferably called “cores” of the template 21, are firstidentified 31 (and if desired their data separately stored)during preprocessing 58. They are identified 31 as regionsthat have some particularly distinctive character.Such distinctiveness may be defined for example inIfpreferred, within the scope of the invention they mayterms of high rates of change of harmonic content.instead be defined in more conventional ways —— such asclosely adjacent plural/multiple ridge or groove endings.In the preferred embodiment, the choice of subset ismade by locating a circular subset window in such a way asSUBSTITUTE SHEET (RULE 26)101520303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-33-to minimize the values of the crosscorrelation function ofthe windowed subset versus the entire template image —— atnonvanishing offset values. Preferably plural (ideallythree) windows are established 31 in this way, each underthe assumption that any already-established windowed regionis unavailable.In any event it is important that the selected windowscontain essentially the most distinctive features of theauthorized user's print, since they will be used either (1)to make a final decision 41-43 or (2) to guide the processof adjusting 23 the template 24 to match the candidate 14.If the features used were instead relatively comon, thesystem would be more likely to make the decision or performthe adjustment incorrectly even if the candidate is theauthorized user —— resulting in a55d.false—negative finding Each of the cores selected 31 represents a view of asmall part of the template, as seen through a small window.The size of the window is important: it must be largeenough to contain a moderately complex and therefore trulydistinctive set of features.Nevertheless, it must be small enough to preservecorrelation -— which is to say, enable ultimate recognition —— of its distinctive features when allowance is made forisomorphic translations, rotations and dilations, and evenif the fingerprint has undergone more general locally-varying distortions. Furthermore, the core must be smallenough to enable a preliminagy recognition even with minimal(or no) preliminary isomorphic adjustment.It is also desirable that the several identified 31Ifthey are too close together, they may not be independentsubsets be reasonably well separated from each other.enough to complement each other in the ways to be described.As suggested earlier, if a particular authorized useris found to have more than one discrete way of placing afinger on the apparatus then special provision may be madefor accommodating this idiosyncrasy. (This case is to bedistinguished from the normal range of positioning variationabout a single mode of placement.) For instance it isSUBSTITUTE SHEET (RULE 25)101520253035CA 02264867 1999-02-23W0 98/461 14 PCT /U S98/07260-34-possible to incorporate auxiliary memory, perhaps at addedcost, to cover the extra storage requirements —— for such anauthorized user who has two or more fingerprintpersonalities.Alternatively, and particularly if the authorized userhappens to be interested in minimizing false positivesrather than false negatives (incorrect acceptances ratherthan incorrect rejections), then an adequate solution maylie simply in planning to test fewer variations about eachof two discrete placements.The currently preferred embodiments of the inventionfocus upon automatic operation even in the authorized user'senrollment stage. Hence these special processing tacticsare not part of the most highly preferred form of theinvention.Isomogphs and the Thebaud system -— In later real-timecomparison processing, the invention will search through the14'to find a closest available match for at least one of thedownsampled sinusoidal data 14, from the candidate user,cores from the authorized user. The way in which thesubsets are prepared for such a search, during preprocessing58,requirements for the system and (2)strongly influences both (1) the data-storagethe time which passeswhile the prospective user is waiting for the invention tomake its decision.A tradeoff between these two factors, data storage andreal-time processing, leads to two major alternativeapproaches to managing the subset preprocessing. At presentthe limiting consideration is time; however, in the futureif much higher processing speeds become available it maybecome desirable to instead opt for solutions that reducestorage at the expense of time.will be outlined here.Therefore both approachesFor minimum data storage, it is possible to simply saveeach selected core in the original form that appears withinits respective small—window portion of the template. Inthis case, the cores shown as rectangles 31’ in Fig. 1 maybe identified on a one-to-one basis with those selectedSUBSTITUTE SHEET (RULE 26)1520253540CA 02264867 1999-02-23WO 98146114 PCT/US98/07260-35-windows, though actually there are likely to be only threeor four such windows.This minimum—data-storage case is in fact an extremelyimportant one, so that actually it is highly desirable tosave each subset -— and indeed the entire data set for anauthorized user —— in an abstracted or abbreviated formrather than in its original form. Accordingly these optionsare associated with one major preferred embodiment of theinvention.They are important in particular when a compact, self-contained system either must store many templates, for eachone of many (e. c., a few thousand) authorized users, or must read in a template from a remote data bank -— or froman identification card (e. o., with magnetic strip or bar code) carried by the user. Either of these cases puts apremium on smallness of the data file for each user, sincefull data (and even more emphatically preprocessed fulldata) are very costly to store within the system forThisfirst major preferred embodiment is particularly applicablemultiple users, or to transmit or store on an ID card.in environments where a short additional delay, perhaps ahalf second to a second, for calculations is acceptable -automatic tellers, office doors, etc.In later real—time processing, however, if a subset ispresented for comparison only in its original form, siftingthrough the candidate data 14’relatively unlikely to succeed.for a particular subset isThis is true even if thecandidate is in fact the authorized user, since there is afairly strong likelihood that the subset of interest hasbeen rotated or dilated, or both.Therefore a fair test requires, to begin with, someequivalent of the Thebaud process of checking each region ofthe candidate data 14’ against several rotated forms of thesubset under test —— rotated through different angles. Inaddition to a nonrotated subset, his preferred system checkseight nonzero rotations, ranging from negative (clockwise)through positive angles.A fair test also requires some equivalent of Thebaud’sprocess of checking each such region against several dilatedforms of that same subset —— dilated by different factors,SUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-36-ranging from values below unity through values above unity.His preferred system checks, in addition to a nondilatedsubset, six nonunity dilations.Furthermore each region of the candidate data 14’should be subjected to some equivalent of Thebaud’s processof checking against forms of that subset which have beenbgth dilated and rotated -— covering most or all crosscom-binations of those same rotation angles and dilation fac-tors. Taking into account the zero—rotation, unity-dilationcases, his preferred system uses nine rotations and sevendilations, for a total of sixty—three cases to be checked.Each case represents rotation and dilation isomorphi-Each ofAscally —— that is to say, without change of shape.the sixty-three variants may be termed an “isomorph”.will be understood, for a representative three subsetwindows this works out to nearly two hundred isomorphs to bechecked against each region of the candidate, and Thebaud’spreferred system preforms and stores the resulting onehundred eighty-nine isomorphs as illustrated in Thebaud’sFig. 2 -— to which, again, reference is suggested.Isomorphs and the present invention -— None of thoseprocedures is part of the most highly preferred embodimentsof the present invention. We instead utilize a verydifferent, 51 (Fig. 1)which is shown in greater detail in Fig. 2 of the presentstreamlined estimating routine 34,document. (This procedure is used not only in theestimating block 34 of the global search but also in thelater analogous block 51 used in hard decisional cases.)The candidate data 14’enter the global search in what we call “stage 1" of thefirst encounter this block as theyprocessing.In our streamlined system, the template is windowed 34aabout the location of the most highly distinctive core 31’,33 -— which is served to the estimator module 34 by thecore—stepper block 32. (The other cores 31' are used onlyif no close relative can be found in the candidate data -since such an occurrence might be due to skin changes orfolding as discussed later.) A spatial correlation 34b isSUBSTITUTE SHEET (RULE 25)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-37-then performed to locate the most similar region 38 (Fig. 3)in the candidate 14, 14’.With the data expressed in sinusoidal terms, thedesired output information —— best-match location andquality of match —— can be found efficiently by search forFirst theFourier transform of the candidate print is multiplied bythe Fourier transform of the particular rotated, dilatedsubwindow of the template.the correlation in Fourier space, as follows.Then the resulting product is back—transformed, and theresulting real array holds the quality of correlation foreach position of interest -— i. e., for each position in thecandidate print, just as if found by stepping across anddown the candidate. In this array, the location of themaximum correlation value represents position in thecandidate print, and the value itself is the quality ofcorrelation at that position. Thus the procedure yields thebest—match position of the subset in the candidate, and thequality of the match.That best—match region of the candidate is thensimilarly windowed 34c. Now the system calculates andcompares the power spectral densities, PSDs, of the imageportions that are within the candidate and template windowsrespectively.Power spectral density, ridge spacing & orientation.and templatezcandidate dilation & rotation —— We digress toexplain the basis of our use of the power spectral densityanalysis. Each PSD initially is found in two-dimensionalreal space, in the rectangular-coordinate grid of the imagedata (Fig. 4).The PSD appears in the form of power—spectral-densityvalues in that grid. It is central to the PSD techniquedisclosed in this document that distance on a PSD graph suchas shown has the dimensions of spatial frequency —— i. e.,(The PSD graph isconsidered in Fourier space, which is to say spatial-wavenumber, or reciprocal ridge spacing.frequency space.)A particular PSD of interest manifests itself as aregion in which a cluster of relatively high PSD valuesSUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-38-appears in the grid. Associated with such a cluster of highvalues is a vector E, pointing from the origin into thecluster. For example, a general vector E1 of this sort,shown in the drawing, lies at an angle 61 to the abscissa andhas a lengthi1§1|-This illustrated vector is representative of a vectorpointing to a cluster of high values (not shown) in the gridin the region where the symbol “E1” appears in Fig. 4. Byvirtue of the above-mentioned significance of distance on aPSD graph,characteristics in the analyzed window region.such a vector is closely related to the ridgeIn fact such a vector E1 is a wavenumber vector, whosemagnitude by definition is the reciprocal of thee. |E1[ E 1/X,Furthermore the angle 01of inclination of the vector E1 to the abscissa is thesupplement of the angle at which the ridges in the windowedregion are inclined to the abscissa.characteristic, periodic ridge spacing X, i.in the skin pattern being analyzed.The latter relation is shown for another vector K2(Fig. 4), which as will be noted is perpendicular to theseveral parallel line segments at lower left in the drawing.These line segments represent local ridges, spaced apart atthe regular periodic spacing K2 shown. As already suggested,the magnitude (length) of the vector, [E11 E 1/X2 —— and itsangle is: 180° — (the ridge angle to the horizontal).Hence the location of a cluster of high values in the PSDgrid reveals both the spacing and orientation of the ridges.For example, horizontal ridges in the §—y plane producea cluster of high PSD values only along the axis ofordinates, i. e. at top and bottom of the graph, but only ata radius equal to the reciprocal of the spacing of thosehorizontal ridges. Vertical ridges conversely produce acluster of high values only along the abscissa —— at rightand left of the graph,spacing reciprocal.also at a radius equal to the ridge-Furthermore an incremental distance, such as theannular thicknessAAlEfl marked in the drawing, represents aband of spatial frequencies (wavenumbers) A (1/K) —— or, toSUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-39-put it another way, a range of ridge spacings A. The lengthof a particular vector 2 can therefore be compared withpreestablished circular annuli representing common rangesfor spatial wavelengths, to determine which preestablishedThis factwe employ in preprocessing as will be set forth later inthis presentation.spatial bandwidth to use in fingerprint analysis.In practice this form of analysis is subject to a re-dundancy: energy-value clusters appear at diametrically op-posite sides of the origin, and spaced equally from it. Onemay think of this duplicate information as corresponding tothe fact that an array of parallel straight lines —— such asthat illustrated —— is symmetrical; that is to say, such anarray is equivalently traversed in either direction by anormal path. In any event, consideration of both sides ofthe array is not necessary, and we prefer to analyze justthe right half.Since each PSD represents the ridge spacing andorientation of the respective specimen, a suitablecomparison will yield the difference, or the ratio, or anyother desired relationship between the respective ridgespacings —— which may be recognized as related to dilation—— and between respective angular orientations of theridges, which may be recognized as related to rotation.prefer to correlate the two PSDs, which has an effectclosely related to reading out 34d the ratio of ridgespacings and the difference between the angularWeorientations.To facilitate this process, as mentioned earlier weprefer to apply a conventional transformation that yieldsthe vector characteristics in polar coordinate These inturn may then be interpreted as rectangular coordinates, sothat the two variables [KI and 3 become, e. g., the abscissaand ordinate respectively of a new grid (Fig. 4a) —— but onein which the high-value PSD cluster can still be plotted, atlEl,9-With both the template and candidate data expressed inthis form, dilation and rotation can be obtained by anyprocess that has the effect of (for example) ratioing thevalues along the abscissa, |Ec[/IETI, and subtracting theSUBSTITUTE SHEET (RULE 25)101520303540CA 02264867 1999-02-23W0 98l46114 PCT/US98/07260-40-values along the ordinate, BC - GT. If only PSD values fromthe right half-plane in Fig. 4 are recorded, only the righthalf-plane in Fig. 4a has data.Continuing stage 1 of processing —— With this insightinto the PSD technique in mind, we resume now our discussionof the estimator routine 34 (Fig. 2). In the first passthrough this procedure —— i. e. in the initial pass throughthe global-search estimator block 34 —— the system refinesthis determination by applying 34f the found rotation anddilation to the template.The template 21 is thereby reoriented, and expanded orcontracted, to form a manipulated version 24 (Fig. 3, butpp; yet the like—numbered block in Pig. 1). This is theversion that offers the fairest possible comparison with thecandidate image in the respective particular regionsselected (the core in the template, and the correlation-chosen region in the candidate).When the candidate is actually the authorized user, aparticular matching isomorph 31’m (see also Fig. 2 of The-baud) —— clockwise-rotated and rather strongly dilated, inthe particular example chosen -— will in general appear in adifferent position in the candidate data as compared withthe template (Fig. 3). The association of such a structure31’m with both the template 21 and candidate data 11 thuslinks the two data sets together.In our invention, the correlation that corresponds tothe amount of distortion necessarv to obtain the match isused as one indicium of the plausibility of the propositionthat the candidate and authorized user are one and the same.This is our test mechanism for the preliminary thresholdtests 41-44.Fig. 3 also demonstrates how an isomorph 24 of theentire template 21 can be used for a fuller and most—faircomparison. Just such information 38 is what is sought bythe global search 32-37,that the preliminary threshold tests 41 are indeterminate.for later use in the possible caseFig. 3 is repeated from the Thebaud patent document,demonstrating that the present process provides anequivalent (though significantly more efficient andSUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-41-economical) to his selection of isomorphic adjustment bystepping through an array of isomorphs. Both proceduresproduce an isomorphically adjusted template 24 which has aregion 124 that is overlapping or in common with the11, 14.adjustment may have an extremely pronouncedcandidate imageSince thisimpact upon the selection of matching region in thecandidate, the template is next rewindowed 34g and thespatial—correlation step repeated 34h to locate a hexclosest-corresponding (i. e. similar) region 38 in thecandidate. The newly selected region, like the earlier-selected one, has an associated spatial correlation valuewhich represents the closeness of that closestcorrespondence. It is suitably normalized, to eliminate theconfounding effects of different variance levels.This normalized spatial correlation value or “NSCV” 34iis saved 35 to the quality-of—match block 36 —— and is thendirectly thresholded 41 (Fig. 1) to determine, as mentionedearlier, whether the processing up to this point is to becalled a rejection 42, an acceptance 43, or indeterminate44.is the closest—corresponding-region information 34k, as willAlso saved for possible reuse in the indeterminate casebe explained momentarily.A rejection 42 or indeterminate result 44 may arisebecause a critical portion of the candidate user's skinpattern has been damaged or obstructed (as by smalladhesions of dirt etc.). To accomodate this possibility,the core stepper 32 —— responding to a return path from thethreshold block 41 which is not shown in the diagram of Fig.1 -— cycles to the second (next most distinctive) templatecore, and the system proceeds again through the stage-1thestepper can cycle to still a third core -— in event theloop. The same loop may repeat yet again -— i. e., thresholding concludes with a rejection or indeterminacy inthe second pass through the loop.The processing to this point concludes “stage 1" and,in case of rejection 42 or acceptance 43, essentially alsoleads to the decisional conclusion 54-56 of the entireoperation. The results of rejection 42 or acceptance 43‘SUBSTTflIflESHEET(RULE25)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-42-have already been described, and it should be appreciatedthat some ninety-five percent of all verification proceduresterminate with rejection or acceptance at this stage 1.Stage 2 of processing —— In the case of anindeterminate route 44, the processing is bifurcated: ifwhat has just been completed was “stage 1”, the processmakes a looping return 45 to the core stepper 32 -If not, thenprocessing is different as will be described shortly.initiating “stage 2” of the processing.In the second, stage—2 pass through the global-searchloop 32-37, operation is essentially the same —— includingstarting with the first core, as before, and cycling to theothers as needed —— except that the window sizes arethe thresholds 41 higher,regions 34h found in stage 1 are reused 34k rather thanAsthe window-size and threshold settingssmaller, and the closest-matchfound again by initial spatial correlation 34b.mentioned previously,reflect concern that the system may have failed, because ofrelatively strong isomorphic distortion, to give a properlyhigh score 34i to the right region in the candidate in thecorrelations 34b, 34h.Such distortion is considered to have somewhat lesserdisruptive influence on that spatial correlation step 34b ifthe window is smaller. On the other hand, with a smallerwindow the observed “normalized spatial correlation value”NSCV should be better.evaluation with smaller windows and higher thresholds.Hence the repetition of search andReuse 34k of the previously found best-match locationin the candidate image is based on the reasoning that thesystem may be unable to function effectively in selecting abest—match location when operating with smaller windows.Another important difference is that the isomorphicdistortion parameters 34d and new best—match region 34h are(to be describedin case of a final indeterminate finding in stage 2.saved out 34e,below)34j for use in a “stage 3”The cases in which our system is able to reach aconclusive rejection 42 or acceptance 43, by the finaliteration of stage 2, amount to a fraction on the order ofSUBSTITUTE SHEET (RULE 25)10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-./3-ninety-nine percent. In all these cases total processingtime for stages 1 and 2 is considerably less than one second—— after image acquisition is complete.When the system either exhausts these efforts orreaches an indeterminate finding with a core that doessatisfy the minimum correlation requirement, then processingbranches 46 (Fig. 1) from the indeterminate path into anisomorphic adjustment module 23 that is the beginning of“stage 3”. At the same time the best saved-out data 38 fromthe various passes through the estimator routine 34 are alsodirected to the same isomorphic adjustment module 23 ——which also receives the template 22, as shown.Stage 3 of processing —— The object of the isomorphicadjustment 23 is to form an adjusted version 24 of thetemplate, from which isomorphic distortion has beennominally removed —— so that as to the isomopphicdistortions the two images 14, 24 match. The isomorphicadjustment module 23 applies the best—match position, angleand scale 38 as adjustments to the template 22, to yield theadjusted template 24 - which now may be taken as thetemplate 24 of Fig. 3.In the isomorphic adjustment, as the name conveys, nochange of shape occurs —— but the entire template signal 22(i. e., the template throughout all of its regions) is shif-ted, cocked, and dilated or contracted, to form an isomorph24 of the entire template 21, that matches as nearly aspossible the corresponding features 38 of the candidateWhile Fig. 1shows adjustment or perturbation of the template 21, 22, forprint as found in the selected window 31'.purposes of the present invention an adjustment orperturbation of the filtered candidate data 13, 14preparatory to the comparison is essentially equivalent -and in fact that is what is done in a now—preferredcommercial embodiment of our invention.This step thus isolates for consideration one lastperturbation that may be the reason for failure of the twoimages 11, 21 to pompletely match:distortion, and more particularly anonisomorphicdistortion field 45SUBSTITUTE SHEET (RULE 26)(VI10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-44-(Fig. 4).of displacements 45a.This field is roughly conceptualized as a fieldThe structure of the frame of refer-ence itself, i. e. the web of the skin as distinguished fromits pattern of ridges and troughs, may be symbolized byinitially “fixed” horizontal and vertical grid lines 45”.The distortional displacements 45a are movements of,for example, the vertical grid lines 45” locally (i. e.,nonisomorphically) to left in some places and to right inothers, so that those grid lines 45” assume new forms 45b.Fig. 5 is taken from the Thebaud document, highlighting thefact that what is shown is the physical reality underlyingthe final stages of analysis in accordance with bothinventions. (Arrowheads 45a representing the individualdisplacements are in many portions of the drawing veryshort, and so can be seen as only very small arrowheadtips.)Even if the candidate is in fact the authorized user,there still exists a crucially important potential formismatch between the adjusted template 24 and candidate data14.twisting or other deformation in the candidate print.That potential resides in the possibility of suchIn other words, the candidate user's finger may havebeen applied to the sensor surface in such a way as todistort the overall pattern.differential offsets,Such distortion consists ofrotations and dilations internal tothe pattern.No single isomorphic adjustment can possibly takeaccount of such internal differential effects. It isbelieved that this internal—distortion phenomenon may bemost to blame for failure to reliably verify presence of anauthorized user —— false negatives —— in even the mosthighly sophisticated of prior—art systems.In the accompanying illustration for tutorial purposesthe distortion field 45, 45a has been drawn very simplified,so that there are no displacements of the horizontal gridlines 45’ although naturally in practice displacements ofboth sets of grid lines 45',does show, however,45” are expected. The drawingthat on balance the overall amount ofleftward and rightward shifting is about equal —— as itshould be, since any isomorphic dilation or contractionSUBSTITUTE SHEET (RULE 26)102025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-45-should have already been incorporated into the isomorphicadjustment 23 which formed the first-adjusted template 24.The symbol“x” in Fig. 4 is not to be misinterpretedliterally as an actual multiplication. Though some complexmultiplication is involved, that would be anoversimplification; rather the symbolism of multiplicationis only intended at a rough conceptual-analogue level torepresent application of a distortion field 45.In the process described by Thebaud, the most probabledistortion field 45 —— on the assumption that the candidateand the authorized user are one and the same person —— isextracted from the data. That field is then applied to makeone final correction to the template 24, yielding a NONiso—mogphic distortion-corrected field which is later used as amatched filter in a final comparison.Thus in his final analysis what is of interest is theridge/groove pattern rather than the nonisomorphicdistortion. He focuses upon that component of distortion Aonly temporarily and only for purposes of isolating and thencanceling it —— just as the global search isolatedplacement/dilation so that it could be globally (butisomorphically) canceled.In the present invention the isomorphic distortionitself is estimated, still on the same identity assumptionmentioned just above —— but then its associated spatialcorrelation value (NSCV) is used as a measure ofplausibility of the assumption. Where relatively largedistortions 51, 52 are needed to equalize the candidate andtemplate images, comparison with a preselected threshold 53yields a decision 54 to refuse 55d access. Thus no attemptis made to form a nonisomorphic—distortion-correctedtemplate.For purposes of estimating the isomorphic distortion,the common or overlay area 124 (Fig. 3) is then dissected 50(Fig. 1), enabling use of the PSD technique one last time,to measure the overall distortion needed to fairly call thetwo images fingerprints of a single person.The total area imaged in the candidate print 11, 14cannot be closely controlled to match the template 21, 24 -SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-46-and the template furthermore is shifted, angled and dilated.Naturally when the two data fields 14, 24 are eventuallyoverlaid for comparison some areas of each data field willfall outside the other,As in all print—analysis systems, comparison will thenand therefore be unusable.proceed on the basis of the remaining areas, those areaswhich coincide, or in other words what may be called the3 and 6). In thethe coarse hatching is“usable” or “overlapping” data 124 (Figs.conceptual illustration (Fig. 3),only intended to help identify the overlap region 124, notto suggest fingerprint ridges or the like -— which of courseare much finer.The above-mentioned dissection consists of dividing upthe entire primary data region of the template 24 into amultiplicity of overlapping subregions 138, 238 (Fig. 6) -preferably a set of twelve partially overlapping circles.Of these circles, some 138 are within the common or overlayregion 124 and so qualify for use in the analysis to follow.Thus in “dissecting the overlay area” 50 (Fig. 1)include those circles 138.weOthers 238, entirely or (asshown) mainly outside the candidate region 14, we discard asineligible for inclusion in the final procedure.Next for each qualifying subregion 238 we compute PSDsThese PSDcomparisons as before yield relative dilation and rotationfor both images, and compare the two PSDs.needed to fit together the two images for each subregion;and based on these we also calculate a relative translationneeded for such a best alignment.Now, each of these distortions for a qualifyingsubregion 238, considered individually, is in essence takenas isomorphic —— but the overall distortion being assessedat this point is assumed to be nonisomorphic (per Fig. 5).Hence it is to be expected that the multiple individualdistortions will in general differ from one another.What we require, however, is a single unitary measureof the degree or extent of overall isomorphic distortion -for comparison with the final threshold 53. This unitarymeasure we construct as an average of the NSCVs associatedwith the individual isomorphic distortions for the severalqualifying subregions.SUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-47-since we know that the candidate image data in somequalifying subregions is noisier —— i. e., less reliable -—than in others, we form the average as a weighted average52.proportion to the noise level.In this process the noisier regions are downweighted inIn the Driscoll and Denyer patents, very small data ex-cerpts from template and sample are used in proceeding di-rectly to a final decision for all cases. As pointed outearlier, reliability of such hasty conclusions appearsquestionable.In the present invention, by contrast, similarly smallamounts of template data 31’ have been used, in the globalsearch and isomorphic adjustment, but in very different ways-—-namely only to obtain a final result in extremely clearThatresult is a “once-adjusted” template 24 which is more fairlycomparable with the candidate image data 11-14.cases, or an intermediate result in all other cases.All of the overlapping data in this adjusted template24, which is to say essentially all the overlapping data inthe original template 21, are eventually used for stage 3.Furthermore, all of these data are used in comparison withessentially all of the overlapping data 14 from the candi-date —— i. e., excepting only the data points removed 13’ asredundant.Utilization —— In operation a candidate user's fingeror toe 90 —— or palm, or any other surface having acomparable skin pattern —— is applied to the sensitivesurface 91 of a sensor module 92 (Fig. 8). The system maybe programmed to start when a skin pattern is thus applied57 (see Fig. 1, bottom left) to the sensitive surface, or ifdesired may be provided with a separate start-up switch (notshown).The sensor module 92 develops an electronic image 111).an optical detector array -— e. g.,(see also Fig. The sensor unit 92 advantageously may beone of the types de-scribed in the Bowker and Lubard patent document mentionedearlier -— or may be any other type that yields a suitableSUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-48-candidate—user image data set 11, for instance a capacitive,variable-resistance, or ultrasonic detector.We prefer to use an optical—fiber prism as described byBowker and Lubard.large sensors and optical—fiber tapers,In view of the current economics ofhowever, wecurrently prefer to use a relay lens (rather than such ataper) to focus the image from the output end of that prismonto a small sensor.Associated with the sensor module is a read-only memoryor ROM (or a programable ROM, EPROM) 93, which holds theauthorized user's template 21, 22 (Fig. 1) and associateddata 22”, 29 -— as well as the desired-certainty threshold27 and the a priori statistics 17. (In Fig. 8 these severalcallouts are abbreviated “2l &c.”)The candidate data 11, template data 21,data sets all flow to a programmed or programmable(CPU) 94.or partly in each,and relatedmicroprocessor or “central processing unit”Stored in the ROM 93 or in the CPU 94,the program described in this patent document.isThe portions 91-94 of the apparatus discussed so far ——and certain other portions if desired —— are advantageouslymade self—contained and for certain applications also madeportable. Accordingly a battery or other portable powersupply 95 may be included with the sensor module 92, ROM 93and CPU 94, and interconnections incorporated, all within ahousing 96.In such a case the output enablement signal 55e (alsosee Fig. 1) might be the only output from the apparatus.That output passes to access-control module 97, which mayfor99.include a suitable local or remote switching devicepassing an actuation signal 98 to utilization meansThe utilization means 99 represent a facility,apparatus, means for providing a financial service, and/ormeans for providing or receiving information. Merely by wayof example, and without any intent to limit the types oftheutilization means may be and/or may include a cabinet, home,these devices which can be controlled in this way,office, military or other governmental installation,educational institution, weapon, computer, vehicle ignitionSUBSTITUTE SHEET (RULE 25)10152025303540CA 02264867 1999-02-23W0 98/461 1 4 PCT/US98/07260-49-and/or entry, automatic teller machine, credit system, time-and—attendance system, or database information service.As shown the self—contained unit 96 may provide an en-ablement or decisional signal 55e to a discrete access-control unit 97. the access-In many systems, however,control module 97 is preferably integrated into the self-contained unit 96 —— in accordance with security—enhancingintegration principles described in the aforementioneddocument of Bowker and Lubard. Similarly the whole of theprint-verifying and access—control devices 96, 97 isadvantageously integrated into the utilization means 99.In both cases the general idea of such integration isto make the security aspects of print—verifying controlrelatively invulnerable to bypassing. That is to say,integration of the whole system can provide resistance toinsertion of a jumper, short, or other form of injectedsimulated access-control signal 98 at the utilization-means99 input.Thus for instance in a weapon, bidirectionalinformation flow between the CPU 94 and a detonator 99within each projectile (bullet etc.) can prevent tamperingwith the intermediate firing mechanism. In a vehicle thathas a distributor or other ignition module 94 directlyassociated with the combustion system, automatic exchange ofinformation between the CPU 94 and that ignition module candeter bypassing of the security system.In a credit, time-and—attendance, or information-dispensing database-access system, similarly, the CPU 94should be programmed to participate in a dialog with thecentral computer 94 of the credit etc. system. Such adialog ideally is conditioned to verify not only theidentity of the user but also the integrity of theconnection between the CPU 94 and the central system.In view of the foregoing, further examples will nowoccur to those skilled in the art.Initial data acquisition & preliminary processina -Before entering the procedures of Fig. 1, our inventionperforms several image—acquisition tests for differentsensor regions (Fig. 9), and data—premassage steps (Fig.SUBSTITUTE SHEET (RULE 25)10152025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-50-10).the accompanying drawings —- to a person of ordinary skillFor the most part these will be self explanatory fromin the art, i. e. a senior technician or programer familiarwith fingerprint analyzers and algorithms —— but a fewcomments may be helpful here.theinitial processing includes seeking adequate signal contentin the “Central” but if thisis not available then the “Lower” Ifadequate signal energy is found only in the latter regionFor enrollment of a newlv-to—be—authorized user,(Fig. 9) section of the sensor,section is tested.then the user is asked to move the finger “up” (forwardalong the sensor surface), to center on the sensor thoseportions of the finger that are providing usable signaldata.This part of the enrollment procedure appears in theflow chart (Fig. 10) at lower right. For candidate usersthis assistance in ensuring good data quality is omitted,since the authorized user is presumed to have learned duringenrollment how to position the fingertip etc. to provide agood image.The three large test triangles at upper center in theflow chart represent cascaded screenings for coverage firstand image-quality next. When the coverage test has not yetbeen passed, the processing branches into the lower-righttriangle, representing a module that tests whether thesignal energy in expected spatial wavebands is adequate forall octants of the downsampled image.If not, the system loops back (along the upward returnpath along the right edge of the drawing) to the image-capture block, and thus continues for up to five seconds(the smaller test triangle just down and to the right) toIf still nosatisfactory image is obtained the user is prompted to try asnap pictures of the skin-pattern presented.different finger position —— and the processing again loopsback to image capture.Once this all—eight—octant coverage test is satisfied,the routing shifts to the lower-left triangle, representinga module which applies a more stringent threshold butrequiring passage for only any six octants. This qualitytest then proceeds in the same vein as the coverage test,SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-51-looping through thebefore.five-second clock test and the prompt asWhen eventually the quality test is passed, severaldata—massage blocks follow, and then processing divergesdepending upon whether the skin pattern representsenrollment of an authorized or a verification of a candidateuser. In the former case, considerably greater efforts arepursued to define, prepare and store a clean, usabletemplate.For best results some positions in the candidate print—- in other words, some values in the array —— are excludedfrom consideration. The apparatus should not be allowed toselect regions that are subject to edge effects, inparticular, or any other systematic major corruptinginfluence. For this reason as may be seen from the Appendixwe prepare the data for Fourier transformations by taperingor beveling a few pixels near the edge of the image—dataregion to be used.The Fourier—transform procedure itself has alternativeversions. In particular, for greatest efficiency, ratherthan a two—dimensional Fourier transform the invention cancalculate two successive transforms of the so-called “Fast(FFT) type,dimensions of the candidate print.Fourier Transform” one for each of the twoThe smoothing procedure included in the enrollmentsection of the flow chart is guided by the inherentdirectionality of skin-pattern ridges, expressed in the formof “local wavenumber fields” as mentioned earlier. Thesefields must be carefully prepared to account for thereentrant or whorl—like structures found in mostfingerprints and the like.If not for this reentrant character of most skinpatterns of interest, the mathematics of analysis would be7) andwhorls familiar in fingerprints, however, render simplefar simpler. The typical closed patterns 62 (Fig.representations inadequate for the following reasons.In a generally linear region 61 of a print, of courseif one could monitor, along a path very generally from ridge75 to ridge 75’, it would be natural to expect continuity ofSUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23WO 98146114 PCT/US98/07260-52-phase—gradient direction 65, 65' —— i. e., the directionlocally perpendicular to each ridgeline, in which phaseincreases. By the phrase “continuity of phase—gradientdirection” here is meant the property of the direction beingconsistent, or in other words not reversing (except perhapswhere it is near zero).Such continuity as illustrated by phase-directionarrows 65, 65' near the left end of the drawing, is expectedirrespective of the fact that two adjacent, parallel ridges75, 75' happen to open up into a whorl 62, and as shown evena whorl which includes distinctly closed loops.The phase—gradient directions 65, 65' for both suchadjacent parallel ridges 75, 75' -— which happen to spansuch an enlargement 66 —— can be traced toward the rightalong the respective ridges 75, 75’. Eventually a point 72is reached at which the “two” ridges 75, 75’ are found tohave been actually different tails of just one common ridge75-75’.At some such place along the way, therefore, theinitially comon phase—gradient directions 65, 65' are foundto be directed oppositely 68, 68'. If this phenomenon isnot somehow controlled, the entire phase field becomesunmanageably ambiguous as can be seen by tracing the upward-pointing phase arrows 65 around the loop.Such tracing finds likewise upward-pointing arrows 67across the top of the whorl 62, rightward pointing arrows 68along the right end 64 of the pattern, and downward-pointingarrows 69 back across the bottom to the left end 61.in this latter region 61 of generally parallel and75'oppositely directed from not only the upper set of phaseEvenrectilinear ridges 75, the downward arrows 69 arearrows 65 above the division line 66 but also the lower set65'.with the identical ridge line 75’ below the division line.That lower set, as will be recalled, is associatedTo deal with such potentially troublesome discontinui-ties, the Thebaud invention forms and maintains severalwavenumber vector fields, quadrature forms of those fields,gradient-times-wavenumber product, and various otherparaphernalia required to maintain all necessary internalinformation about the template.SUBSTITUTE SHEET (RULE 25)101520253035CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-53-The present invention monitors for phasediscontinuities of the sort illustrated, but only once at anearly stage of preliminary processing -— and can discard allsuch information as soon as it has completed the process ofThemonitoring is preferably carried out by detection of suddenreversals in the sign of the wavenumber field 29.smoothing along template ridges as mentioned earlier.These sign reversals can be found during creation oflocal wavenumber fields (lower right in Fig. 10), and theirlocations marked by warning flags 73, 74 a specified dis-tance from each discontinuity 72 —— in each direction alongan axis transverse to the discontinuity. Preferably thiswork is done during preprocessing, in formation of thetemplate, moving one step at a time in either the 5 or ydirection, in real space —— while requiring neighboringvalues to be substantially continuous, and setting up theflagging in an associated wavenumber field 29.In terms of the Fig. 7 example, continuous processingalong a vertical (y) direction locates the discontinuity 71at a height 72 in the pattern.Then in smoothing the system watches for the flags 73,74 only at right angles to the direction y selectedpreviously for imposition of continuity. This strategyenables the processing to stay some distance away from adiscontinuity.As mentioned earlier, many realizations of skinpatterns are subject to distortions which amount, locally,to more than a half wavelength or even one or more fullwavelengths in the pattern. If such a distortion is allowedto develop too rapidly, the only portion of it which is ineffect “visible” is the fractional part remaining afterdeduction of an integral number of wavelengths.It is essential to realize that correlation goes tozero in any region of the template that is misaligned byonly a Quarter of a wavelength. Hence, avoiding errors of ahalf wavelength, or of course anything larger than that, isof extremely great importance to successful practice of ourinvention —— at least in those cases where sizabledistortions are in fact present.SUBSTITUTE SHEET (RULE 26)101520253035CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-54-Scaling of the steps to avoid falling into suchambiguities is preferably achieved by limiting thealgorithm’s rate of stepping.Moving or storing data —~ Storage of templates inabstracted or abbreviated form (e. g., level—downsampled totwo—bit or binary data) does require care to avoid loss ofultimate performance. Storage need not impair accuracy ifthe data are properly processed after retrieval.In particular, routine template—data steps of bandpass-ing, normalizing and smoothing should be performed bothbefore downsampling and afterward upon retrieval of theabstracted data to as nearly as feasible reconstitute theoriginal information set. These steps respectively ensurethat what is about to be stored is properly representativeof the data before storage or transmission, and later beatdown the high frequencies introduced by storage in one- ortwo—bit form.The downsampling of levels is preferably performed onthe basis of statistical distribution in the data, forexample based on unit variances as suggested for four levels11,signal levels.in Fig. rather than arbitrarily on the basis of initialAs the drawing suggests, this preferredprocess of discriminating between levels will result in useof rather high set points, as well as an algebraic signbut with the advantage that discriminations are therebydataIRDWintroduced which distinguish between levels among thosepoints that are the most important. A like schema willbe clear if for instance an eight-level system is desired.It will be understood that the foregoing disclosure,and that of the following Appendix, are intended to bemerely exemplary, and not to limit the scope of theinvention —- which is to be determined by reference to theappended claims.SUBSTITUTE SHEET (RULE 25)101520253035CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-5-APPENDIXThis describes the skin-pattern print (e. g.,fingerprint) verification algorithm that operates inpreferred embodiments of the present invention. Thismaterial is included, though perhaps in an excess offullofcaution, to ensure satisfaction of the obligation ofenabling disclosure, including that of the best modepractice of the invention. _Whereas the body of the foregoing patent disclosure isintended as a complete conceptual presentation, thisAppendix provides all additional information necessary for aperson of ordinary skill in the art —- namely, a seniorprogrammer or programming technician experienced in thefield of fingerprint analysis using higher mathematics —— toprepare firmware needed for successful practice of theinvention. other hardware aspects of the invention aretaught at a like level of detail in the Bowker et al. patentdocuments mentioned in the disclosure text.A sumary of the processing steps appears first. Inthe sections that follow, the algorithm is broken down intocomponents which are described in detail.The major components of the algorithm are FingerprintAcquisition and Preparation, Enrollment, and Verification.Acquisition and preparation involve image capture, digiti-zation and filtering the image to enhance ridge contrast.Enrollment is the process of entering a new authorized userinto the system; normally this is only done once perauthorized user, and is performed by a trained operator ortechnician. Verification is the process of matching acandidate user's fingerprint to the stored information aboutthat candidate. "Enrollment," "Verification," and otherterms are defined in the glossary at the end of thisAppendix.Minutia—based algorithms are susceptible to errors dueto the presence of scarring over time, or to dirty fingersthat produce false minutia points or hide true minutiaSUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23WO 98146114 PCT/US98/07260-56-points. The invention exhibits robust performance in thesescenarios.PROCESSINGThe software waits in a dormant state until action isrequired. This action is requested by either a command fromthe unit to perform enrollment or verification, or a commandfrom a network controller. The first step in the processingis image acquisition. A fiber-optic prism, through aprocess called frustrated internal reflection, provides animage of the fingerprint to a CCD camera. This image isdigitized and placed in memory for the software to read.The software repeatedly reads a subsection of thefingerprint area until the image subsection passes twotests. An image coverage test verifies that the fingerprintimage is relatively centered in the optics. An imagequality test requires significant ridge contrast in the datacollection window. When these tests are passed, a newsimilarly sized central sub-section of the image isdownsampled to 128 by 64 pixels (referred to as the b-grid)Thesignal field is normalized to unit variance and the noiseand filtered to generate signal and noise fields.field is adjusted accordingly.Enrollment processing uses the normalized signal fieldand aexclusively. The field is smoothed along the ridges,search is conducted for "cores" (see glossary forThe ambiguity statistic ofeach core is analyzed, and for the image to be considereddefinition) in a central region.useful, the least ambiguous core's statistic must not exceeda threshold. then itstored as a template with its related information. If thethe algorithm searches a lower regionIf,If the image is considered useful, isimage is not useful,(below the central region previously used) for cores.in this region, the least ambiguous core's statistic islower than the threshold, the user is prompted to move theirfinger forward on the platen and a new image is acquired.Otherwise, the user is informed that enrollment has failedand the unit waits for further commands.SUBSTITUTE SHEET (RULE 26)1015203035CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-57-After theimage has been acquired and prepared, a global searchVerification processing is more complicated.procedure is initiated for each core in turn. The purposeof the global search is to find a good estimate of the largescale rotation and dilation between the candidatefingerprint and the stored template information with respectNext,between the two images with respect to each core is made byto each of the cores. an estimate of the translationrotating and dilating the template about each core accordingto the estimates. The peak correlation as a function ofposition between the candidate and the manipulated templateis identified for each core. The position corresponding towithThe magnitude of thiscorrelation is a measure of the match between the twothis peak represents the best estimate of translationrespect to the core being examined.images. Based on the magnitude of the correlation of thebest-matched core,(Pass/Fail/Maybe).If the answer is Maybe, "stage 2" is entered. Stage 2a "stage 1" decision is madeprocessing is similar to stage 1 with the main exceptionthat the processing window is smaller. A conditionaldecision, similar to stage 1, is also made at the end ofthis stage.If again the decision is Maybe, "stage 3" is entered.In stage 3, the candidate fingerprint isrotated/dilated/translated to align with the template.central region of the template is divided into twelveAt this point, the candidateand template have a region of overlap.Theoverlapping circular regions.only the circularregions that lie within the overlap of the manipulatedcandidate and the unmanipulated template contribute to thedilation andBachrotated/dilated/translated template sub—region is correlatedstage 3 statistic. An optimal rotation,translation is calculated for each sub-region.to the corresponding candidate sub-region. Thesecorrelations are weighted and averaged, producing a teststatistic that is used to make a final decision (Pass orFail).SUBSTITUTE SHEET (RULE 26)101520CA 02264867 1999-02-23W0 98/461 1 4 PCT/US98/07260-58-FINGERPRINT ACQUISITION AND PREPARATIONI. Preprocessing (verifying the overall image quality)A. Define an input "mean ridge spacing" wavenumber ko,then define the following five signal bands to allow forvarious mean ridge spacings in the fingerprint population:4.1% L<k1< §iJ@:§12.‘/1+‘ 1.21.4-\/1.’5k0x/T3<F}:%-¢T§ko-1__‘2<H< kg-1.2-Mk”-L4< I/?|< /(0-1.4-\/GVLS «L5B. Capture a fingerprint image, then excise the centralregion of the fingerprint (about 1.6x1.25 inches currentlyfor efficiency). Real-space downsample this image to the b-grid for FFT processing.C. Divide up the b—grid window of data into eight equaloctants, then FFT each octant separately. Multiply eachtransformed field by its complex conjugate to get the powerspectral density (PSD).D. For each of the eight subwindows, form a sum of themagnitudes of the frequencies that lie in each signal band.Pick the signal band that corresponds to the maximal sum ofthe five sums generated for each subwindow.E. Perform the finger coverage test according to thefollowing steps:1.all eight subwindows in order for the fingerprint to beThe maximal sum must exceed a set threshold co indeemed as adequately covering the image platen.SUBSTITUTE SHEET (RULE 25)10152925303540W0 98/461 1 4F.CA 02264867 1999-02-23PCT/US98/07260-59-2. If the image does not pass this test, then a newimage is collected as described in (B) through (D) andThisprocess will continue for five seconds or until anthe test is performed again on the new image.image is deemed adequate, whichever comes first, withthe idea being that finger placement may be adjustedIf the test fails after fiveseconds, then the user is prompted to retry placing thefinger onto the platen. then thealgorithm continues on to the quality test.during this time.If it passes,Perform the print quality test according to thefollowing steps:1. Collect a new image as described in (B) through(D).2. This time the maximal sum for a signal band ofenergy in a given subwindow must exceed a threshold qo >co. If it does, then the subwindow of the image isconsidered to have adequate image quality.3. Six of eight subwindows must be considered asadequate in quality in order to declare the fingerprintimage as adequate for further processing. If the imageis not adequate, the process repeats in the same way asdescribed in (E.2) for five seconds, this time with therationale that fingerprint quality will improve overtime as the finger rests on the imaging platelet. Ifthe test fails after five seconds, then the user isprompted to retry placing the finger onto the platen.If it passes, then the algorithm continues on to rawprocessing.II. Raw Processing (preparing a fingerprint image forenrollment or candidate processing)A. Define the following signal and noise bands, which arecurrently:SUBSTITUTE SHEET (RULE 26)101520CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-60-k__"__< < k0 - ./2.6fl.6(signal)k7;< k0- < Il?1< ko-V5 (noise)k0(/276-B. Using the image that passed the pre—processing tests,excise the center region (currently 2.0x1.0 cm) of thefingerprint. FFT this window of data.C. Apply the signal bandpass to the transformed data,then back-rqfifl on the b-grid.truncate data to Nyquist in frequency space,transform to get the signal fieldD. Perform step (C) in the noise band to generate thenoise field n56) on the b-grid.Note that no 1/N factor is included because normalizationwill occur later.E. Define a smoothing operation as equivalent to a real-space smoother with positive weights (i.e. not a bandpass orlowpa§§_filter). For each of the two bandpassed fields,form mg. as the smoothed square of the data (withindependent parameter choices).Define Uyéi. as the bandwidth areas for the two bands.Construct an estimate of the signal variance as ‘‘ n-1' Z3 <R+ 6)’l I =0where the field R is defined byR: Wig)‘ - If(Mk), msSUBSTITUTE SHEET (RULE 26)10152025CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-61-and e is a small positive parameter that functions as a softlimiter in case R>1 (R should obey R<1, but is notmathematically required to do so). If the integer parameteris chosen as a power of two, then the summation can beperformed iteratively, with an advantage in efficiency.F. Normalize the signal field to unit variance, and createthe inverse noise field, by outputting the following fields:mgfl0,. (F) nzl (F) = (normalized signal field)These fields will be used in the rest of either enrollmentor verification processing.(NI),mgP I (F) = - 0'; (fl (inverse noisefield)ENROLLMENT PROCESSING I. The first step in enrollment processing is acquire thefingerprint image and perform the data preparation that isdescribed above in “FINGERPRINT ACQUISITION ANDPREPARATION”.II.processing, the vector wavenumber field,HB, is generated.Once the image (defined here as HWGD.) is available forThis is done in a multiple step process.of the image,Wn(fl,First the gradientis generated by performing an FFT onthe b-grid data, multiplying byib in Fourier space, andperforming the inverse FFT.A. Then the smoothed-dyadic 2x2 covariance matrix,QH§, iscalculated asQxfkfihzfihflSUBSTITUTE SHEET (RULE 26)10152025CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-62-where the smoothing is the same as applied to the data inthe preparation step. Finally, we calculatefifl from§U3 asfollows: The magnitude of RF) is given by|/<1: ‘/Cm, +(‘D_.B. Then, a complex field, Zf)is defined as2m 2 CH — (‘xv + 2 L7”: .This field is used to generate the phase of R3, Gk, asfollows:1. Start at the end of the b—grid farthest from thecore. Beginning at one corner, defineBk = phase (Z) _1o|~2. Now march to the other corner at the same end byupdating Gk viaA0:7Aphase Qf(ZZ' * )l\J|-*where Z’ represents the previous data point. Now marchaway from the end in the same manner (in a directionperpendicular to that end). The result is the a=1version of £63. Generate the a=2 version by repeatingthis procedure with the dimensions reversed.3. For each wavenumber map, march in the directionparallel to the original end and detect angle jumpsexceeding p/2. At these locations, and for someset a 1-bit flag field tounity. (This field will always be associated with fm’Jdistance (input) around them,C. Now we smooth the fingerprint image,nW@), along ridges(i.e. along loci perpendicular tofifl). Use nearest—neighbormapping and a step size in arc length equal to b—gridSUBSTITUTE SHEET (RULE 26)1015203035CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-63-resolution, Ax5. Call the smoothed result nfflfl.normalize this ridge-smoothed template by the RMSNowQSo (fl = m'3_ (7') n1‘x(r"))2>lD. The next step is to find a predetermined number of"core" regions. This is done by means of what is called an"automated core finder." First a central region (see Figure1) is searched for acceptable cores, that is cores withanambiguity less than the defined threshold (currently set at0.40).region,If no acceptable cores are found in the centrala lower region is searched for a single good core.If a good core is found in the lower region then the user isprompted to move the finger up on the platen and processingjumps back to Image Acquisition (see “FINGERPRINTACQUISITION AND PREPARATION”).that Enrollment has failed and processing jumps back toIf not, the user is informedImage Acquisition.E. The automated core finder has been implemented asfollows:1. Define a coarse sampling of trial core locationsin the template image staying a specified distance fromthe edges. For each trial location, determine a figureof merit as follows.2. Calculate the lagged cross-covariance between theridge-smoothed template S'o windowed about the triallocation (with a specified circular window) and thefull template. This is done by taking the FFT of theS’o and its windowed counterpart. The fields are thenmultiplied together and the result is inverse FFT'd toyield the lagged cross-variance. Normalize by windowarea so that the cross—covariance is ideally unity atzero (vector) lag. Compute the maximum of this fieldover lags with magnitude exceeding a specified amount(nominally, at least one—half ridge). This is anestimate of the degree of ambiguity for this trial corelocation.SUBSTITUTE SHEET (RULE 26)10152925303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-54-3. Now choose a core location, or possibly amultiplicity of locations, as follows:a. The location with minimum ambiguity is chosenand labeled number 1.b. If more than one core is desired, all corelocations within a specified distance of this core(nominally one core radius) are now excluded, andthe minimum ambiguity remaining location isdesignated the second core.c. The same procedure, excluding locations closeto the first two cores, results in a third core,and so on.F. The template is truncated to N bits of dynamic range(where N is currently equal to two) and compressed forminimum storage requirements The process of compressing thefingerprint image for template storage involves multiplesteps.1. The data are quantized to two bits (four graylevels)a. Histogram equalize the data -— define fourranges within the range of the data, each to beequated with a gray level.1. Find the median of the data (equalnumbers of data points on either side of themedian value).2. Determine values for two other thresholdsthat are equivalent to the medians of each ofthe two populations (one on either side ofthe data median). 11 shows thehistogram of the data and the three definedthresholds A, B, and C.Fig.SUBSTITUTE SHEET (RULE 26)10152025303540W0 98/461 14G.CA 02264867 1999-02-23PCT/US98/07260-65-b. Use the defined thresholds to transform thedata to a two-bit version of itself by assigningeither a 0, 1, 2, or 3 to the value depending onwhich section of the histogram the data originallyoccupied.2. Perform a shift (by two bits) and add for groupsof four values,eight—bit bytes.thus packing the two-bit data into3. Apply a standard compression technique to theresulting data (to be determined). At this point thetemplate (compressed image and related information) canbe stored in internal memory, reported to a centralcontroller, or saved to external memory.The final step of the enrollment process is to storethe template for future use in the verification process.The template storage format is currently defined as a headerand a variable number of bytes for the compressed templatedata.H.The header information consists of:0 Template Version Number° Personal ID Number (PIN) —— used to reference thetemplate° Thresholds defined to quantize the data0 For each core being generated and saved:- X location- Y location- Ambiguity Factor —— calculated in the automatedcore finder- Size of the compressed template data- Reserved words —— for future expansionIt is possible that more header information will berequired in the future, but these changes will becomprehended by the software through the template versionnumber.SUBSTITUTE SHEET (RULE 26)1020253035CA 02264867 1999-02-23 W0 98/46] 14 PCT/US98/07260.. 66 -VERI FI CAT ION PROCE SS INGI. Verification processing requires a template to testagainst. This is typically retrieved when the user entershis/her personal identification number (PIN).A. The PIN is used to reference a stored template inmemory.B. The template is made up of a compressed fingerprintimage and its related information. The compressedfingerprint information must be decompressed and filteredbefore it can be used in the verification process.C. Decompression of the data is the reverse process ofstoring the template (see Enrollment Processing).1. First the data decompression is applied that willretrieve the packed, two—bit template information.2. The packed, two-bit template is unpacked byconverting eight-bit bytes into four two—bit values.These are in turn converted to floating—point valuesusing the thresholds defined in Enrollment Processing.D. The data are still in a pseudoquantized form, and isnot suitable for processing. The data are bandpass filteredin a manner similar to the Signal band filter defined forthe candidate fingerprint.II. Acquire fingerprint as per Fingerprint Acggisition andPreparation. This will be our "candidate" print.III. Overview of the global search procedure:SUBSTITUTE SHEET (RULE 26)102025303540CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-67-A. Make some working copies of the template and candidatefields for the following steps. The copies of the candidatefield are modestly tapered to avoid aliasing.B. Define a circular window of a given size about one ofthe template's core locations by zeroing the b—grid dataoutside of the template window.C. Using a spatial correlation of the window with the 4whole candidate field, find the closest corresponding regionof the candidate.D. Define the corresponding window of the candidate as in(A).E. Determine the optimum rotation and dilation to apply tothe template window in order to match the correspondingcandidate window most closely. This is done by thefollowing:1. FFT the b—grid template and candidate windoweddata fields,corresponding PSDs,then truncate to Nyquist. Generate thewhich are in the shape of a half-annulus.2. Remap the PSDs from their XY coordinates to polarcoordinates, so they are now rectangular. Now, eachnew pixel in the b—grid corresponds to the signalenergy at a particular radius and angle in the spatialwindow region.3. Correlate these two new fields in the hypothesisrange (currently) of —1B° to 18° of rotation and i 10%of dilation to find the best rotation/dilation valuefor the template window region.F. Using the rotation/dilation values from (E), rotate anddilate the template field with respect to the core locationusing a real-space rotation and dilation procedure withbilinear interpolation.SUBSTITUTE SHEET (RULE 26)1015202530CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260- 68 -G. Window again about the remapped template's corelocation.assuming that the rotated/dilatedwindow may correspond to a slightly different location inH. Repeat step (C),the candidate because of its new orientation. The goal isto obtain a better estimate of the global translations thanthose obtained from (C).I. Thethe besttemplateoutputs from these steps include an estimation ofglobal rotation/dilation/translation of thewith respect to one of its cores in order toglobally align the template and candidate. Also included isthe normalized correlation value from step (H), i.e., thevalue of the correlation field at the optimum translation.Define this value as Lo.IV. Stage 1A. Perform (II) for all three cores of the template usinga relatively large window size for the circular windows(currently the window radius is twenty—four pixels).B. Save the rotation/dilation/translation values for thecore that gave the maximum (of the three) Lo value from step(II.H). Define this value as Lww.C. Make a pass/fail/maybe decision for the candidate basedupon the following formulas:L.m.¥ 7.9.43 (""59L.'.1«L\' ’ 7";m1 Uh")77..-.4 “ 11114;? ’’ 7P.4x< ("’“.V”“)D. If the decision is "maybe" at this point of processing,then continue to Stage II.SUBSTITUTE SHEET (RULE 26)1015202530CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260- ()9 -v. Stage 2A. Perform (II) for all three cores of the template withthe following detailsC.1. Use a smaller window size than in (III.A)(currently the window radius is 18 pixels). This maygive a better estimation of the global rotation/dila-tion/translation values for a given window region incase the larger window contained too much imagedistortion relative to the corresponding candidateregion.2. Omit step (II.C). For step (D) simply use thetranslation values associated with the Stage 1 Lw.The values are to be directly used for the one coreassociated with gum, otherwise for the other two coresthey should be modified to account for the Stage 1 Lwmrotation/dilation values not being identically 0°rotation and 0% dilation.Generate a new Lmm value for this stage.Make a Pass/Fail/Maybe decision for the candidate basedupon the following formulas:L>A {AX ypaxs + EpasxQxms)Qbfl)4mu'< Km1+ §m17?az1+ €fa:I < LA/A.\' < ypaxs + Gpaxs Note that the reason for the elevated thresholds is becauseof the smaller window size used in Stage 2.D.thenIf the decision is "maybe" at this point of processing,continue to Stage 3.SUBSTITUTE SHEET (RULE 26)10152025303540CA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-70-VI . Stage 3A. Rotate/Dilate/Translate the tapered candidate imageaccording to the appropriate inverse values of the bestvalues from Stage 2 processing. Zero—pad any data in the b-grid array that no longer contains the candidate print.B. Divide the center region of the template intooverlapping circular subregions. Currently we are using a112x48 region divided into twelve windows of radius elevenpixels each.C. For each template subregion, check if a considerablepercentage of the subregion corresponds to remainingcandidate data (as opposed to mostly being aligned in thezero-pad region).D. If a template subregion qualifies as viable forcomparison (see Fig. 6), do the following orientationprocedure:1. Generate various rotation/dilation hypotheses forthe template subregion. This is done by real-spacerotation and dilation (with bilinear interpolation) ofthe subregion. The allowable rotations/dilations forthe subregion are relatively small because smallvariations are allowed between a globally re—orientedcandidate and the template.2. Correlate each of the hypotheses with thecorresponding candidate subregion.3. Choose the best rotation/dilation hypothesis forthe template subregion, and rotate/dilate it by thatamount.4. Spatially correlate the newly oriented templatesubregion with the corresponding candidate subregion,allowing for only small translational shifts betweenthe two regions.SUBS1THIflESHEET(RULE1flflCA 02264867 1999-02-23W0 98/46114 PCT/US98/07260-7]-5. Save the final normalized correlation valueassociated with the optimum translation. For the ithsubregion denote this value as Li.E. After checking all of the template subregions, andprocessing through the qualifying ones, form a weightedaverage of the Li values according to the following:2 —1 . . .Z L, - P" (center pixel 0_f sub - region I)1 E qua/z_/iedz— 1 . . -\Z P" (center pixel of sub - regzom)IE z/uulzfiedzNote that the Li values are squared to boost the spread10 between low and high values of Li,. Also note that theinverse noise value for the candidate (assumed to berelatively constant over the subregion) is applied todownweight the contributions of subregions associated withhigh noise, because it is assumed that the general signal15 quality of these regions is poorer than in the rest of theprint.F. A yes/no decision is made based on the hmmflx statistic.11x-'51-1/.1-1A_x'> yfinal (Pass)LNEWA./A_x' < yfinal (fail)£0 GLOS SARYa~grid —— The raw fingerprint image is digitized as 329 by242 pixels and written to memory. The software reads in asubsection of this that is 256x128 pixels in size during25 image acquisition, and is 216x160 pixels in imagepreparation.SUBSTITUTE SHEET (RULE 25)101520253040CA 02264867 1999-02-23W0 98/461 14 PCT/US98/07260-72..b-grid —— The b-grid is currently 128 by 64 pixels.image acquisition,Inthis is achieved by performing a real-space downsample on the a-grid. theIn image preparation,b-grid is achieved by transforming the a—grid image,truncating at Nyquist, then inverse transforming.Core —— A region of the fingerprint image (calculated on thetemplate) that is relatively distinctive, or moreconspicuously unique, compared to all other regions in theimage. Distinctiveness is quantified by a lagged cross-correlation between the image and a tapered version of theimage.Enrollment —— The process of entering a person's fingerprintThe information canThisinformation will be used to verify a candidate fingerprint,into the stored memory of the system.be stored in the device or in a centralized area.and therefore, this procedure should be conducted by atrained operator for best results.Verification —— The process of determining the authenticityof a person's identity by means of matching theirfingerprint to the stored information collected duringenrollment. The accuracy of authenticating a user'sidentity is directly related to the quality of the storedtemplate information.Template —— The ridge—smoothed fingerprint image and itsThisand used duringassociated information make up the template.information is generated during enrollment,verification.Ambiguity -— This term refers to the distinctiveness of thecore region relative to other areas of the fingerprintimage. In finding cores, one desires a very low degree ofambiguity (i. e., highly distinctive, or more plainlyunique).Candidate -— This is the fingerprint image that is presentedto the unit during verification processing.SUBSTITUTE SHEET (RULE 26)

Claims (25)

WHAT IS CLAIMED IS:

1. Apparatus for verifying the identity of a person by comparing test data representing a two-dimensional test image of that person's skin-pattern print with reference data derived from one or more two-dimensional reference skin-pattern print images obtained during a prior enrollment procedure comprising:
means for computing power spectral density of at least a portion of the test image;
means for applying the power spectral density to estimate dilation of the test image relative to a reference image;
means for comparing the test data with the reference data, after adjusting one or both of the test data and the reference data based on the estimated dilation;
means, responsive to the comparing means, for making an identity-verification decision; and nonvolatile memory means for holding instructions for automatic operation of each of the foregoing means.

2. The apparatus of claim 1, wherein:
the computing means further comprise means for computing power spectral density of at least a portion of the reference image; and the applying means comprise means for comparing the power spectral densities of the test image and reference image to estimate their relative dilation.

3. The apparatus of claim 2, wherein:
the applying means comprise means for interpreting a radial component of power spectral density as a measure of relative dilation.

4. The apparatus of claim 1, wherein:
said applying means further comprise means for applying the power spectral density to estimate rotation of the test image;

and the comparing means further comprise means for adjusting one or both of the test data and the reference data based on the estimated relative rotation.

5. The apparatus of claim 4, wherein:
the applying means comprise means for interpreting an angular component of power spectral density as a measure of rotation.

6. The apparatus of claim 3, further comprising:
means for applying the estimated relative dilation and rotation to further estimate relative translation of the reference and test images.

7. The apparatus of claim 1, further comprising:
means for estimating relative rotation of the reference and test images; and means for applying the estimated relative dilation and rotation to further estimate relative translation of the reference and test images.

8. Apparatus for verifying the identity of a person by comparing test data representing a two-dimensional test image of that person's skin-pattern print with reference data derived from one or more two-dimensional reference skin-pattern print images obtained during a prior enrollment procedure comprising:
means for computing power spectral density of at least a portion of the test image;
means for applying the power spectral density to estimate rotation of the test image relative to a reference image;
means for comparing the test data with the reference data, after adjusting one or both of the test data and the reference data based on the estimated rotation;
means, responsive to the comparing means, for making an identity-verification decision; and nonvolatile memory means for holding instructions for automatic operation of each of the foregoing means.

9. The apparatus of claim 8, wherein:
the computing means further comprise means for computing power spectral density of at least a portion of the reference image; and the applying means comprise means for comparing the power spectral densities of the test image and reference image to estimate their relative rotation.

10. The apparatus of claim 9, wherein:
the applying means comprise means for interpreting an angular component of power spectral density as a measure of rotation.

11. The apparatus of claim 10 further comprising:
means for estimating relative dilation of the reference sand test images; and means for applying he estimated relative dilation and rotation to further estimate relative dilation of the reference and test images.

12. Apparatus for verifying the identity of a person by comparing test data representing a two-dimensional test image of that person's skin-pattern print with reference data derived from one or more two-dimensional reference skin-pattern print images obtained during a prior enrollment procedure comprising:
means for computing power spectral density of at least a portion of such test image and of such reference image, respectively;

means for transforming the respective computed power spectral densities to polar coordinates;
whereby the transformed power-spectral-density information interpreted as rectangular-coordinate data has the form of power-density values plotted on a rectangular grid of ridge spacing and orientation;
means for estimating relative rotation and dilation between said test image and said reference image from the transformed power spectral densities for such test and reference images; and nonvolatile memory means for holding instructions for automatic operation of each of the foregoing means.

13. The apparatus of claim 12, wherein:
the estimating means comprise means for ratioing the respective ridge-spacing and orientation values, or correlating the two transformed power spectral densities, within a hypothesis range of relative rotation and dilation to find an estimate of the most probable relative rotation and dilation.

14. Apparatus for verifying the identity of a person by comparing test data representing a two-dimensional test image of that person's skin-pattern print with reference data derived from a two-dimensional reference skin-pattern print image obtained during a prior enrollment procedure comprising:
means for estimating relative translation, and at least one component of relative isomorphic distortion, between the test and reference images;
means for adjusting the test or reference image, or both, to allow for said estimated relative translation and component of relative isomorphic distortion;
means for comparing said test and reference images, after said adjustment, within substantially all area that is common to both images, to make an identity-verification decision;
wherein the comparing means comprise means for analyzing power spectral densities within said common area to estimate remaining distortions; and nonvolatile memory means for holding instructions for automatic operation of each of the foregoing means.

15. The apparatus of claim 14, wherein the comparing means comprise:
means for dividing one of the images into a multiplicity of substantially overlapping subregions that in the aggregate cover substantially the entire said one image; and means for evaluating the degree of similarity of said test and reference images, with respect to substantially every one of said subregions of which a significant fraction is within said all area common to both images.

16. The apparatus of claim 15, wherein the evaluating means comprise:
means for estimating, within each of said subregions in said common area respectively, a further component of relative distortion between test and reference images.

17.The apparatus of claim 16, wherein the evaluating means further comprise:
means for forming a composite measure of said further components for all of said subregions in the common area; and means for thresholding said composite measure.

18. The apparatus of claim 17:
further comprising means for extracting from the test data an estimate of noise variance in the test data as a function of position in the test image; and wherein the composite-measure forming means forms said composite measure based in part on the estimated noise variance.

19. The apparatus of claim 18, wherein the taking-into-account means comprise:
means for weighting the further component for each of said subregions in the common area, in an inverse relation with the noise-variance estimate for that subregion.

20. Apparatus for receiving surface-relief data from a sensor that acquires surface-relief data from a relieved surface, and in response controlling access to facilities, equipment, a financial service, or a system for providing or receiving information comprising:
a system for processing the received data to determine identity of the relieved surface, said system including:
means for calculating and comparing power spectral densities of at least a portion of the received data and test data respectively, and analyzing the power spectral density comparison to estimate dilation, means for comparing the test data with reference data, after adjusting one or both of the test data and the reference data based on the estimated dilation, and means, responsive to the comparing means, for making an identity-verification decision;
means for applying the determined identity to control access to such facilities, equipment, financial service, or source or reception of information; and nonvolatile memory means for holding instructions for automatic operation of each of the foregoing means.

21. The apparatus of claim 20 wherein said relieved surface is a finger.

22. A secured system subject to access control based upon surface-relief data from a relieved surface comprising:
sensor means for acquiring surface-relief test data from such a relieved surface;
means for processing the data to determine identity of the relieved surface, and for applying the determined identity to control access to the utilization means, said processing and applying means including:
means for calculating and comparing power spectral densities of at least a portion of reference data related to said particular relieved surface related to the authorized user and said test data respectively, and analyzing the power spectral density comparison to estimate distortion, means for comparing the test data with said reference data after adjusting one or both of the test data and the reference data based on the estimated distortion, and means, responsive to the comparing means, for making an identity-verification decision; and nonvolatile memory means for holding instructions for automatic operation of each of the foregoing means.

23. The system of claim 22 wherein said relieved surface is a finger.

24. A method for verifying the identity of a person by comparing test data representing a two-dimensional test image of that person's skin-pattern print with reference data derived from a two-dimensional reference skin-pattern print image obtained during a prior enrollment procedure; said method comprising the steps of:
correlating power spectral densities of corresponding regions of the test and reference images to determine relative isomorphic distortion between the images;
using a normalized spatial correlation value as a measure of similarity between corresponding regions of the test and reference images;
making an identity-verification decision based on said normalized spatial correlation value; and in nonvolatile memory, holding instructions for automatic operation of the above-mentioned steps.

25. The method of claim 24, further comprising the steps of:
operating a sensor to acquire the test data; and responsive to the decision-making step, operating a switch if identity is verified.