CA1286030C - Character and pattern recognition machine and method - Google Patents

Abstract

CHARACTER AND PATTERN RECOGNITION MACHINE AND METHOD
Abstract of the Disclosure A pattern recognition system, particularly a handprint character recognition system in which electrical binary black/white "image" of one or more handprinted characters is formed and a plurality of centers of recognition (CORs) within said binary black/white images are selected as reference points for measuring the characteristic enclave of the black/white image immediately surrounding the CORs. A library of templates of said measurements around the CORs for a plurality of know exemplary character images is stored in a memory for comparison with corresponding measurements made around the CORs of images whose class is unknown to produce "template scores" proportional to the similarity of the enclaves of the know image to the enclaves measure by templates. The generic shape of a character in expressed as a "character equation" involving template scores developed on an unknown image, and each character equation is evaluated including comparing the values of such equations, and selecting the best value to determine the generic name of the unknown character.

Description

31[~

CHARACTER AND PATTERN RECOGNITlON MACHINE AND METHOD

~ACKGROUN~ OF THE INVENTION

The present lnvention relates generally to character and p~ttern recognltlon machines ~nd method~s, and more particularly.
to fe~ture extr~ction sy~tem~ for u~e ~ith optical readers for re~dlng characters whlch have been hand printed without any constralnts, such ~8, surrounding box limits, red center lines,or s1mllar artlicl~1 devlce~. One novel feature of this lnventlon ls in the method of chooRing the features and the hlghly normallzed method of measurin~ the individu~l featuxe parameters.
Th~ inventlon can be sald to perform a crude ~lmulatlon of a llttle known psychological phenomenon occuring ln prim~tes called the "saccadic flick".

Whlle there are gener~lly different views on the definition of the feature~ of patterns, many studies made on the recognition of characters ~s well ~s the recognition of p~ttern~ have proved that the so-called quasi-topological features of a character or pattern such a~ the concavlty, loop, and connectivity are very important for the reco~nltion. To d~te, many different methods 3~

hsve been proposed for the purpose o extractlng ~uch quasi-phasic featurss. Up until this invention these methods all u3e ~n~lysl3 of the progresslve slopes of the bl~ck pixals. Morl et ~1. patent 4,468,808 cla~lfles thoqe ~n~lyses lnto three type3. The flrst i~ the pattern contour tracking system developed by ~renl~ with IBM~ Mori calls thi~ a serisl system. The 3econd type 13 Morl'3 preferred, the earlle~t patented ex~ple of whlch 13 Holt called the "W~tchblrd". In this type of an~lysis ~equentlal rows Qnd column~ ~re compared. Another ex~mple of the sequentlRl rows and column type is Holt' 9 Center Referrenced U~lng Red Llne . Morl's third type in a parallel analysls system whlch Morl d~smlnses as elt~er taking too long or costing too ~uch. All system-n involvlng the sequential an~lysls of the slope of bl~ck plxel ~roupn suffers severely from smoothlng and llne thlnnln~ errors~ WorAe yet~ they Qre very llkely to produce ~ubstltution errors when the lines have voida or when unw~nted lines touch.

The present inventlon , whlle using qu~sl-topologlcal f~tures, employs a novel method of me~suring ~nd scorlng such fa~turos, re~ulting in great improvement in perform~nce o the readin~ m~chins. A comprahansive survey of prior art systems is found in an article by C.Y. Suen et al. entitled "Automatlc Recognition o Handprinted Ch~racters - The St~te of the Art", Proceedlng3 of the IEEE, Vol. 68, No. 4, Aprll ~9ôO, whlch 1~
lncorporAted hereln by reference. The techniqua uses none of the methods mentioned by Suen et al. or Mori t al.

Briefly, my lnvent~on employ~ mes~urement of the enclo~ure characteristlcs of each whlte pix~l independently of other white plxel~. Since the me~surement~ are made in two (or more~
dl~enslon~ rsther than ln ona dimenslon (~uch as ~lope), the result~ are lnsensitlve to first order aber~tlon~ ~uch a~
accldqntal vold~,touchlng lines and ~mall numbers o~ black pixel~
carrylng noi~e only. In the pre~erred embodlment, no noi~e proces~lng l~ performed lt ~ll since all form3 of nol~e processing are done at the expen~e of accuracy ln recognition.
As used hereln, a pixel la deined a~ an ima~e information cell con~tituted by the blnary states "on" and "of" or "black" ~nd "whlte`', respectively.

~ he S~ccadic Fllck phenomenon, which occurs in prlmqtes, h~s the purpoAe of focusing varlous sm~ll areae of the entlre retinal field o view upon the"fove~". The"fovea centralis" is a RmQll p1t or depres~ion at the bsck of the retina ~orming the point of ~harpest vl~ion. Recent re~e~rch has shown thqt the fovea~ in additlon to providlng the hi~he~t re~olution in the retina, more importantly provide~ lmportant information processing on the vlsual data. In partlcular, it ~eems to"recognlze" a multiplicity of gener~l patterns or sm~ll features which it has been trained to reco~nize ~t earller periods in its exi~tence.

According to one aspect of the present invention there is provided a pattern recognition system ln which an array of cells store information in two or more levels representing two or more changes of value of each parameter defining in pattern to be recognized, the system having m~ans for examining cells in the array to locate a centre of recognition cell, means for determining the boundaries of all cells having specific relationship to the centre of recognition cell and producing boundedness measurements to recognize the pattern.
Another aspect of the invention resides in a p~t.ern recognition method in which information in two or more levels representing two or more changes of 1~ val~es of each parameter defining a pattern to be recogni~ed is stored in an array of information storage cells. The method includes the steps of examining cells in the array to locate a centre of recognition cell, determining the boundedness of other cells have specific relationship to the centre of recognition cell, and producing boundedness measurements to recognize the pattern.
According to yet another aspect of the invention, there is provided an artificial fovea which has an array 2~ of storage cells and loading means for loading an image information which has one or more enclaves in black/~hite pixel formation, into said storage cells, ~ne pixel per storage cell. Means is provided for salecting one of the white pixels at a center of 3~ ~eco~nition (COR) to constitute a test COR location and lo~i~ means for determining whether each white pixel in the image is bounded in predetermined directions by black pixel in the array of storage cells. Summing means is provided for summing the number of pixels which are bounded in each of the predetermined direction, respectively, and producing enclave measurements proportional to each respective percentage of the total number of white cells in the array.
B

Still another aspect of the invention resides in a hand print character recognition system, the system having means for creating an electrical binary black/white "image" of one or more hand printed charactersJ with means for selecting a plurality of centers of recognition (COR's) within the binary black~white image as reference points and measuring the characteristic enclave of the black/white image immediately surrounding the CO~'s. Means is provided l~ ~or storing a library of templates of the measurements around the COR's for a plurality of known exemplary character images, and means are provided for comparing the library of templates to corresponding measurements made around the COR's of images whose class is unknown to produc~ "template scores" proportional to the similarity of the enclaves of the known image to the enclaves measured by templates. Means are provided for expressing the generic shape of a character as being a "character equation" involving template scores developed 2~ ~n an unknown image. Means are also provided for evaluatin~ each character equation, including comparing ~he values of such equations, and selecting the best value to determine the generic name of the unknown character.
7~ My invention has the following desirable characteristics and features:

1) It recognizes hand~rinted characters ra~ardless o~ their absolute size, except when the size 3~ or relative size of the enclaves is necessary for discrimination between classes.

2) The invention recognizes characters essentially independently of small variations in line thickness, independently of small voids, and independently of small numbers of extraneous black pixels in and around the character; this is accomplished - 4a -~ 2~33[~because the algorithm is based on the normalized number of white cells within the enclave bounds, but not on the easily poisoned curvature of the black lines.

3) The invention is able to pick out well known characters from an image which includes a great many patters of types not previously taught to the machine:
this is accomplished because the measurements of unknown enclaves must have high correlation with previously 1~ taught enclaves before they are even considered.

- 4b -~ It is able to achleve a ~ub~ltution rate ( the percentage of wrongly chosen cla~ name~ divided by the number of cl~ss names correctly cho~en by a human ~udge working ~rom only the sa~e images ) of zero. This rem~rk~le char~cteristic 1~
~ccompllshsd becau~e the algorithm uniquely allow3 for continous llne~r scoring ~nd compariaon throughout ~11 correlation~. This ch~r~cterlstlc i~ to be ~tringently observed a9 being ~paci~ lly dliferent ~rom all ~orm~ of declslon makln~ ln which cholces ~re m~de on a ye~no ba~i~ all form~ o~ tree ba~ed loglcal recognition method~ have the inherent Achilles h~el o~
~klng an absolutely wrong deci~ion wlth absolute certainty~ Thls i~ often cauced by an lnsigniflcant varlation in ths lmage p~ttern.

5> Another great ~dvantage of my invention i~ th~t it i~
~ble to Judge tha ~ccept~bility of ch~r~cters on the ~usls o~
comp~rl~ons to an absolute Mlnlmum Acceptable Level (MAL) ~nd ~130 to ~ Mimimum Double~ Ratio <MDR). Theae vlrtue~ ag~in -~pring ~rom the linear scores which ~re contlnuously generated.
These c~p~bllltles provide great ~dvantage ~ecause they ~llow the mQchina ~or the oper~tor~ to vary the acceptability crlteria dapendlng on context, character set, quality of lm~ges, etc.

6) Furthermore, my invention ls ~daptive ln the sen~e that lt can learn the names o~ previously unknown pattern~ and automatically generate new enclave~ ~nd new recognitlon equations ,~ .

3~

which are c~refully crafted to be non-conflicting with previously learned patterns. Typically the operator ~ill provide ~ correct clsssificatlon or a reJected charactar non-supervi~ed le~rning CQn a130 take pl~ce,e.g.,learning using an automatic dictlonary.

7~ One of the most import~nt characteri~tics of my invention ls it3 ~bility to recognize touching and overlapplng ch~racters.
Th~s hQ3 been, up until now~ an lmpossible ta~k for any h~ndprlnt r~a~ar. I ~ccomplish this by two methods, which can be mutually supportive in their decisions. The first is my use of "test sagmentation and analysis`'. The ~econd method is by the use of "superclass" tralning,e.g., "three" touchlng ~"four'', for ax~mple 18 tr~ined to be recognized as a new class called a"three-four". Likewise a blob in whlch a"one" touches a"zero"
whlch ln turn overlaps a"nlne", for example, ls recognlzed a~ the clQs~', "one-zero-nlne".

B~IEF DESCRIPTION OF THE DRAWINGS

FIG lA ls a simplifled block diagram of a pOEttern recognltlon machlne which utilizes my invention of an Artiflci~l FOV~A, 3~

FIG lB ls an example of ~ typ~cal pattern stored ln Pattern Source, FIG lC shows an exemplary output from ~n Artifici~l Fovs~, FIG 2 is a ~lock Dl~gram of an Artiflcial Fo~ea, FIG 3 is a block di~ram of ~ Recognition Scoring Device, ~IG 4 15 a Block Dlagr~m of an enclave Me~suring Device, FIG 5A is ~ di~gram showing the FOVQ~ Matrix r whlch cont~in~
~peciflc clrcuitry capa~le of performing most of the functions of an Encl~va Maasurlng Device, FIG 5~ is ~ circùit di~gr~m showing ~ sin~le Polyp In the Northe~st Qu~drant, ~ "Polyp" being defined ~s one element in t~e Fove~ M~trlx, FIG 5C 1~ ~ Table showing how the n~me~ of the g~te ~unctionA of FIG SA are modified for Polyp~ appearlng ln other quadrants, FIG 5D iq A Diagr~m ~howing ~ Single Typic~l Polyp In E~ch Qu~dr~nt, FIG SE i-q a diagram of the Simpllfied Connection~ in a Fovea M~trlx, FIG 6 illu~trate~ ~etails of a Recognltlon 5corlng Devlce, FIG 7A is a Table of Generic Enclave Names, ~ IG 7a illustrates five enclaves ln the closed top n~meral 4, FIG 7C is a table of the cardinsl template score for encl~ve No. 1 of FIG lA, FIGS 8A-8K compri~e a Table of Ar~bic Numerals, conventlonally drawn~ showing positlon6 of conventionAl generic ancl e~ves, FIG 9 is a Table of R~cognltlon equations. showin~
typical neg~tions and typical relative po~ition requirements, FIG lOA illustrates Progressive Samples of Membership Ou~lification in a Generic Character, FIG lO~ illustr~tas Pro~ressive S~mples of Membership ~u~lific~tion in ~ char~cter ~ith void~, FIGS llA-llC illustrHte Examples of Characters ~ith Vold~
that would fail to be recognized by line tracklng methods~

FIG llD illustrates the scoring pattern for FIG llA, FIG llE illustrata~ the scoring pattern for FIG llB, FIGS l~A and 12~ illustrate Examples of Characters with extr~ bl~ck pixels th~t would ~il to ba recognized by line tr~ckin~ methods, FIGS 13A-13E illustr~te ~'x~mples of Other Ch~r~cters whlch c~n be recognized, such as widely vsrying size, FIGS 14A-14K illu tr~te reco~nition of Touching ~nd , :.
, . ~

O~erl~pplng Characters in accordance with the in~ention, FIG 15 lllustrates Artificial Fovea Shown as a Computer Element, FIG 1~ illu~trate~ Block Diagram o+ P~rallel Plpeline C~n~i~uration showing a plurality o~ Artificlal Foveas .tn op~ration, FIG 17A l~ a block diagram of a partial recognitlon machlne which include~ a re~olution modifying component, FIG 17B illustrate3 Example~ of Resolution Conslderations, FIG 17C illustrates reduction and quantlzation of FIG 17B
by 3X3.
FIG 17D illustrate~ a quantl~atlon o~ FIG 17B with no red~ct~on.
FIG 18A ig a block dlagr~m o a recognltion system incorporating the invention and utilizing a learnlng module, FIG 18B illu3tr~tes Learnin~ Capabilities using 8 zero with 3n openlng, FIGS 19A and l9B lllu3trate C~pabillties for Re~ecting Nonsense Shapes and Disorderly Noise, FIG 20A and 20B illu3trate the analysi3 of pointyness trl~ngles for closed top "4" and a well formed "9", 3~i FIG 21A, 218 and 21C. illustrate a "period" in black whlte, inverse, black white "period" ~nd inverse displ~y of a blob eight, respectively, FIG 22A is an example of a numeral requiring a relative size function, and FI~ ~a i~ an example o a complex functlon derived from tlve enclave ~ize~.

~ESCRIPTION OF TH~ PREF~R~D EMBODIMENTS

The present invention wlll now be described in ~re~ter ~etail with re~erence to the illustrated embodiments.

Fi~ure l is a ~lmplifled block diagram of a pattern recognltlon machine which lncorporates my inventlon. Since my ~nvention may be used to recognize obJects from tot~lly unrelatad disclpllnes. The ~ir~t block is labeled Pattern Source the p~tterns may be from widely different kinds of sources, Such obJect~ m~y be handprinted numerals, or handprinted alph~betic characters, or machine printed characters. It can also be taught to recognize cursive script ~nd to recognize v~rious types of biological cell classe~. The principle can be applled to recognizing pictures o~~ airplanes to recognizing sketches of 3~

turtles.In general. the Artificial Fovea can usefully be u~ed to reco~nize any of the many klnds of lnformatlon that the humln eye recogni~es. This includes three dimensional informatlon thst ha~
been converted by 30me method to the two dimenslonal lnformation that the eye sees.

The Pattern Source 10 shown in FlG 1 therefore repre~ent~ ~
source of data containing a pattern to be recognized. Stnce thls lnventlon is not concerned with methods oi' scanni~g original p~tterns, the ~rdware of the pattern source ls ln this embodiment can be some form of well known type of computer memory, such as Random Access Memory (RAM). Pattern source 10 csn also be part of an on-llne system 3uch as a gr~phic tablet wherein each resolvable element can bbe deemed a pixel and ~he movement of the stylu~ (or fln~er) on or over the tablet forms the character to be recognl~ed.

Block 11 shown in FIG 1 is labeled Arti~~icial Fovea. This block represants apparatus which emulates the beh~vlour of a hum~n foveA, includlng temporary ~torage of` the image, shifting of the im~ge, measuring of image sections (called "enclaves">, and scorlng of these measurements against earlier stored measurements (called "templates"). The best of these scores are called "feature~".

Block 12 shown in FIG 1 1~ labeled ~ecognltion Scoring.

3~

~lock 1~ represents apparatu~ w~ich stores and evaluates a number of equations, called "Recognition Equations". These equations call for various feature values to be combined ln ~uch a way that the highest scoring equation reveals the most probable class to which the unknown image belongs.

~ lock 13 shown in FIG 1 1~ labeled Utilization Device. This block 13 represents a device which i8 an "end user'` of the recognition process. Examples of such devices are co~lputer~, printers, displays, sorting machines, speech output devices, etc.

FIG lB i~ an example of a typical pattern stored in a P~ttern Source. Note th~t it has been shown in a two dimensional ~rray and that the values of the pixels ~hown are binary, i.e., black and whlte. The two dimensionality and binary value~ are n~rmal for mo~t of the appllcations of my invention, although it i~ n~t ra~tricted to tho~e characterlstic~. The number o~ plxels par pnttern i~ not to be limited to the number ~hown in this ax~mple: one of the great advantages of my invention is that the absolute 3i~e of the pattern~ is normalized as an inherent part o~ the measurements. The pattern shown is called a ~eneric handprinted 'two' beca~se the shape satisfies the b sic requirements for u handprinted '`two" without including any other extr~ information. Such extra information ~normally conusing to analysis> consist of extra loops (at the beginnin~ or middle of the "two"), white pixel~ in sections of black line plxels (voids~, and extra non-connected blAck pixels and variations ln the width of the line forminq the character. In most of the examples herein, the "line~ width is shown as a aing1e pixel but lt will be eppreciated tha~ in many charQcters this wlll depend on the width oi the wrltin~ implement.

Flyur~ lC ~hows an e~emplary output from ~n Artificlal Fovea. Note that the Artliicial Fovea 11 has found two excellent Center~ of Recognition <CORs) and ha~ labeled them"A" ~nd"B". It h~ also L~beled all the pixels belonging to ~nclave ~1 wlth ~
nu~ric`'l". ~imilarly the pixels in the second enclave ~ave been l~beled with 8 "2". The Artificial Fovea 11 has also ~cored both encl~ves against known templates stored in its memory. It has ~ound that a Feature called "Be~t-in-West" ha~ the best score for Enclave ~1, and that score ls 100. Similarly the Artificlal Fovea 11 h~s ound that the best ~core for ~nclave ~2 i~ developed by ~eature c~lled "Best-In-East", which also has ~ value o 100.
Sco~e v~lues run between zero and 100, an enclave with a ~trange shapa, or a pattern contalnlng a vold would have lower scores.

FIG 2 ls a Block Dlagram o an Arti~lcial Fove~. It includes four blocks. ~hey are &n Enclave Me~suring Device 14, a Template Storaqe Device lS~ A Compurator 16, and a Best 1`emplute Sorting Device 17.

~2~

The term "enclave-', as used in this inven~ion, means ~n area of white pixels which are more or less bounded by black pixels or the edge of the image. An example of an enclave is an area of white pixels surrounded by black pixels: another ~xample is an areu of white pixel~ bounded by black: pixel.~ except ln one or more directions. Some use~ul enclave shape~ are shown in FIG 7.

It i~ important to emphasize at hls ~uncture that almost any whlte area can be deflned 85 an enclave, and that the most use~ul enclaves Qre memorlzed by humans and by thls apparatus of my lnventlon.

Referrlng a~aln to FIG ~, Encl~ve Measuring ~evlce 14 is shown in much more det~ FIG 4. Stated slmply, enclave measuring devlce 14 produces a set of measurementa which descrlbe the shape of the area withln the enclave. These measurement~
~mphasl2e the dlfference~ between enclaves that are nece~s~ry to ~ep~rate pattern cl~sses from each other, but they "norm~lize"
the dlferences between enclave.~ whlch are not neces~ary for s~p~ratlon o cla~ses. These measurements prlmarily describe the positions and abund~nce of tho~e ~hite pixels which ~re bounded by black plxel.~ on the edges of the enclaveO In one embodiment the area of the enclave is divided into quadrant~, desi~nated NorthEast (NE~, SouthEast (SE), South~West ~SW> ~ and NorthWest ~NW~ In each quadrant there are ~our possible pixel type~: those th~t are not bounded (wlthin that quadrant) by bl~ck pixels in 3q3 either the vertical or horizontal direction, tho~e those that ~re bounded vertically but not horizont~lly, tho3e that are bounded horizontally but not vertlcally, and those that are bounded both horizontally and ~ertlcally.

Templ~te Storage D~vice ~TSD~ 15. The purpose o~ the TSD i~ to 3tore hundreds of ~elected sets of mea~urement~ so a~ to compare tima l~ter to measurements t~ken from new snd unknown enclAves.
As soon ~ these set~ ~re selected ~nd stored they are known ~g `'Templ~tes". The physic~l methods used to store the Templste~ can be ~ny type of memory th~t h~s re~onable acces~ tlme ~uch ag RAM, ROM, magnetlc di~k~, optic~l dl~ks, etc. If the memory is dyn~mlc or vol~tile, procedure~ must be provlded to m~intain the ln~ormatlon or to reloQd.

Compar~tor 16 correlates the ou~put of the encl~ve mea3urln~
davice 14 with e~ch one of the hundreds of Templates ~tored in t~e TSD 15. The result o~ each correl~tlon i~ ~ score running line~rly between zero and 100.

On~ embodlment o~ Comparator 16 develops it~ score by considerlng each quAdrsnt lndependently; the absolute difference3 between the EM mea~urement3 and the Template v~lues ~re summed and normali~ed. The hardware used ln ~he comp~r~tor may con~lst of an Rbsolute value subtraction clrcuit, plu~ a sumning mechanlsm and a low accurAcy dlviding circuit to take - 16 ~

6~3~

percen~a~es.

~ est Template Sorting Device (Bl'S~> 17. accepts each new score produced by Comparator 16 and stores the value o that score in ~ 11st which has been ordered by the value of the score.
The ldanti~ying nu~ber of the Template i5 to be stored in such a w~y that it c~n be identified ag belon~ing to each score. and the coord1nate~ o the Center o~ Recognition ~COR~ used by the EMID
l~ must likew se be ~sociated with the score. In pr~ctice~ only the scores assoclated wlth the best two Templates must be kept by the BTSD 17. When an end to the partlcular ~coring cycle ha~
occurred, the BTSD 17 will output the Be~t Template Number~ the ~est Template Score, and the coordinates of the Test COR which ~a~lnes the winning enclQve.

The concept of orderly reJection of non~en~e shapes and.
~l~or~erly nolse is ~trongly clRlme~ in my lnvention. An l~portAnt output o~ the BTSD 17 is the REJECT cap~billty. In order to make intelllgent reJectlon of test enclave~ two input~
must be ~ddltion~lly provided to ths ~TSD 17: these nre the Mlnlmum Acceptable Template Level ~MATL~ on line 18, and the Mlnimum Template Doubles R~tio (MTDR) on line 19. The ob~ect of these inputs is to force the output o~ the BTSD to be a reJect i~
either ~> the ~bsolute ~core o~ the be~t templ~te is less thsn the MATL, or b) the ratlo o the best score to the next best score (o~ another template) ~g le~s than the MTDR.

33~

FIG 3 shows a block dia~ram of ~ Recognition Scorin~
Device. It~ purpose i3 to perform the second and fin~l stags of recognition of indlvidual patterns. It does this by usin~ the e~ture scores to evaluate the ch~racter equations, sorting the resulting equation scores. and performing ~cceptability testing.

~ ith reference to FIG 3, Feature Stor~ge Unit 20 storss th~ inform~tlon provlded by the operstions of the artificial ~ovaA shown ln FIG 2. This inform~tion consists of ~ ~et of d~ts descrlbin~ the best correlstion or the best templ~te number on line 21-A obt~ined for eQch of ~ plur~lity of encl~ves within the imsge; the set also lncludes the Best Templ~te Score inputted on line 21-B for e~ch of the winning ~core~ ~nd a short description ~f the location of each enclave, in the form of the COR
coordln~tes on line 23, with respect to the other encl~vesO
Phyalcslly the Feature Storage ~0 comprlses a digltal memory whlch has an acces~ time rea~onibly mstching the speed requlrements of the system.

Equation Stora~e Unit 2~ stores recognitlon equations whlch have been developed by previoue hum~n analysls ~nd machline experlence. According to the invention, These recognition equations typically are the sums of terms. Each term consists o~
slgn, a weightlng factor (w). and the value of ~ p~rticulsr feature. The memory or ~torage hardware hardware perorming Equ~tion Storage Unit 2~ is similar to the hardware chosen for the Feature Storage Unit 20.

Equ~tion Evaluation Device 26 performs the additions.
subtractions! multiplicstion~, and dlvision~ which Rre called out by the recognition equations ~tored in the equatlon Storage 24.
It must also perform any logical operations called out by the aqu~tion~, such a~ relative location requlrement~. Physically.
th~ Equ~tion Evaluation Device 26 is preferrably ~ ~et of ~dicated ha~dwAre chips w~ich perform high ~peed arithmetlc and logical ~unctlon~. It may also consist of a fairly general purpo~e computer chip.

The Be~t Score Sorting Device 27 and the Acceptability Taatlng Devlce 2~ ~re almost exactly simllar in function to the Ba~t Tamplate Sorting Device 17 shown in FIG 2. It'~ output con31sts of the name of a recognlzed ch~racter if the ~cceptQbility criteria Cminimum acceptabla char~cter level and mlnim~m character doubles ratio) are passed: if the criteriA ~re not p~s~d, ~ REJECl` code is produced.

FIG ~ is a B10ck Diagram of an Enclave Me~surlng Device 14 ~hown in FIG 2. Storin~ and Shifting (S&S) device 30 ~ccepts a p~ttern or imAge from the Pattern Source 10 shown in FIG l. This pattern may be transferred from P~ttern Source lO either by ~ny 3~

one of several parallel lnformstion tr~n~er~ or ~erlally , pixel by pixel. Note that at thi~ point in the proces~lng the plxels have only two ~tates or "color~": black und white, "on" or "off", "l" or "O". Because the pattern will be shi~ted often during the operation of the Artificial Fovea it will be convenient to have the pattern loaded into the S&S Unit 30 using a serial method.
The S~S Unlt 30 provlde~ information directly to almost all of`
tha othar blocks in FIG 4. The pattern is initially shited so that a white plxel falls right on the center o~ the ~torage area o~` the S~S Unit 30. This center of the stora~e ~rea is abbrevlated CSA.

Element 31 is a Boundedne~s Determining Unit. The meanlnq of "boundadnass" ln this lnventlon ls that each white pixel is c~lled "bounded" if any black pixel exists in the same row or ~olumn at a distance further away from the Center of the Storage Area ~CSA> than the locatlon of the white pixel. A plxel may be bounded vertically only. horizontally only, or bounded both vertically and horl~ontally. It wlll be appreciated that the 3t~te3 m~y be lnverted wherein a white pixel becomes a black p~xal and A black plxel become~ a white pixel.

Pixel Type Determining and Type Counting Unit 3~ performs the functlons of labelin~ each of the plxel~ wlth label~

3~

describing their boundedness ch~ract~ristics. and then counting the absolute number of pixels of each different type. For nomenclature purpo~es, the ~rea around the CSA lS divided into quadrlnts named NorthEast! South~ast, SouthWest, and NorthWest (see FIG 5A). There are four types of pixel in each quadrant , so the tot~l number of descriptor3 per encl~ve ls 4 types tlme qu~dr~nts, making l6 descrlptors.

Encl~ve Membership auallfying Unlt 3~ speclfies, ~ccording to speclfic rules, which of the white pixels surrounding the C~A are to be included a~ belonging to an enclave. This block performs this qualiiic~tion primarily by using information obtained from the ~oundedness Determining oper~tion of unit 31.

Pe~entsqe of Plxel rype ~etermlnin~ Unit 34 performs ~
3imple low accuracy (1~) dlvision procedure in which the absolute number o~ each pixel type in e~ch quadr~nt is multiplied by 100 ~nd dlvided by the number o~ enclave members in that quadrant.
Tha~e percentages are, in fact, the enclave measurements.

Finally, ~illing of ~est Enclave Unit 36 performs a function whlch occurs after final determination of which encl~ve be~t matches R stored templ~te, as described in FIG 2. This operatlon shit~ in codes to the storage matrlx ~0 which are ~tored along with the color (bl~ck or white) of the pixel. These codes will prevent ~ach pixel thus coded from becominq a member of another encl~ve.

FIG 5A is illustrates a Fove~ Matrix. It shows a preferred embodiment of most of the func~ions of an Artlficial Foves. FIG
SB, FIG SC, FIG 5D,FIG 5E, alld FIG 5F colltain additional det~ils tha embodiment.

The Fovea Matrix 40 ~hown in FIG 5A i8 a ~3 by 13 square ~rray of elements I call"Polyps". T~e exact number of Polyps ~l ~8y vary from applicatlon ~o application and i~ only crltlcal in the sense that t~e number of polyps be greater than any enclave whlch i3 nece3sary to be measured in a ~iven application. The odd number of polyps on each edge is signiicant only because an odd number yields a central symmetry about both the vertical and horizontal axes. The system I have chosen to number the Polyps is onQ which l~blels all quadrants ~ymetrically. except for minu~
slgns. Thu3. the central Polyp is labeled POO, the furthest NE
P41yp is labeled P6~,the furthest SE Polyp is labeled P6,-6; the ~urtha3t SW Polyp is lQbeled P-6,-6; and the furthest NW Polyp ls l~beled P-6,~

FlG 5B is called"Polyp in NE Ouadrant". Thi~ figure lllu3trates actual logical electronLc circuitry which will perfor~ many of the complex function~ required of an Artifici~l Fovea. This fi~ure describes the circuitry t~at every PoLyp in ~8~i~3~

the NE Quadrant will contain. (With some sign changes ~shown in FIG 5C~ this circuitry will also apply to the Polyps of all other quadrants.> The numbering of the Polyps is important to the underst~nding of the operations. The qeneralized NE Polyp of FlG
5D is labeled Pi, J the subscript"i" stands for the number of the vertic~l column of which the Polyp i5 a member, and the subscript"~" is the number of the horizontal row r This numbering ~ystem i~ consistent with the Pi,J elements shown in FIG 5A.

FIG SB contains 5 groups of circuits which are closely related to the blocks shown in FIG 4. The first group is labeled"Polyp Image register" 43, and its function ls to perform the storage and shifting functions (S&S~ of the Fovea Matrix da3cribed earlier. The second and third groups perform the '`B~un~na3s Determlnlng" descrlbed in connection with FIG 4. The ~nurth group per~orms ~he qualiflcation of enclave member~hlp for that Polyp and also store~ the member~hip statu~. lhe fifth group ~c~lled the"~lll Reglster"~ stores a blnary one lf that Polyp has baan previously ~elected as part of an enclave.

The Polyp Image Register (PIRi,J~ 43 perfor~s the functlons o~ both stor~e o the pixel color and that of ~ shift register.
This type of e~ement is well known in the art, being basically a flip-f`lop wlth a shlfting gate 63 having enough dynamlc storage characterlstics to allow it to act also as one stage of a shift reglster. It recelves its color input from PIR~ t~] ,which is loc~ted on the s~me roW directly to the left it~ ~hlfting output ~oes ~o PIRti~l~[~ which is loc~ted on the same ro~
dlrectly to the rlght. Polyps on the le~t edge of a row receive their inputs from the rightmost Polyp in the next lowest row~
while Polyps on the right ed~e of a row shift their outputs to the leftmost element ln the next highe~st row. Thls is illustr~ted in FIG 5E.

Referrlng 3till to FIG 5B, the Vertical Closure Regi~ter ~4 twhose output ls VCR[i]~] ~ becomes a binary "one" if any of the Polyps further up in the vertical column cont~in a black pLxal description. This is accomplished by using the"OR" gate 64 whose inputs are labeled 31 and 32. Input ~1 is true if the Vertic~l Cl03ure Register 44 of the Polyp immedi~tely above Pi~ is true, and thl~ sets VCRlJ to a true ~tatu~. Input ~2 lq true if the Polyp immedl~tely above Pi~ is storing a black pixel color; if true, it ~190 sets VCRi~ to true status. Thl~ matrix arran~ement of OR gates provlde~ a "rlpple" such that within R small portlon o~` a mlcrosecond the presence of a black pixel ~t the top of ~ny m~trlx column will cause the VCRs 44 below it to propA~te a "true" statu3 downw~rd. The Horlzont~l Closure Register VCRl~ 45 has a 3imll~r logic~ et of ~ates 65, and its function is the ~mq except for the fact that it detects boundedness to the right o~ Polyp PiJ.

The Enclave Member~hip Regi~ter ~6 of FIG SB u~e~ many of the ~2~ 3~

outputs o~ surrounding Polyps to determine whether the the pixel represented ~y Pil is qu~lified for encllve membership. Inputs 5 and 54 connect to AND g~te 5S which become~ true if the Polyp Just to the left of Pi~ is a member ~nd if Pi~ itself is bounded horizont~lly. Inputs 56 and 57 connect to AND g~te 58 which becomes true lf the Polyp ~ust under Pi~ i~ a member AND lf Pi~
i~ it~elf bounded vertic~lly. OR gate 5~ becomes true lf either ~te S5 or g~te 58 becomes true, ~nd this will c~use the encl~ve mamb~r~.tp Re~lster 46 to be true unles~ there ~re ~ny"inhiblitions". Inhibitions are applied to the EM ~6 via~OR ~te ~, lf lt is true~ then the ~MOR remalns f~lse. Gate ~O becomes true if ~ny of its inputs become true. Inhi~iting lnputs are ~s follows:.

~LRlJ

PIRl~

PIRtl-1~ t~-l]

PIRti-l] t~-2 PIRti-2] t~

PIRti-2] tJ-2]

Fill Register FLRi3 61 requires that the Polyp may not be ~ member of s new enclave if lt ha~ already been chosen ~ part o~^ snother encla~e. PIRl~ requlres that lf the plxel represented is black, thst Polyp mly not be a member of any enclave. The other fDur PIR lnhlbltions repre~ent the lnhibiting effect o-f black plxels at points clo~er to the Center of the Storage Area (C5A~. The output of OR Gate 40 collect~ the~e signalq, whlch ~ra collectively called "Llne-of-Sight Inhibition~"~ They ~ecti~ly prevent enclave membership from propa~ating in ~n ~ra~ which ls mo~tly isolated from the main encl~ve. Other ~unctlons beside~ the functlon repre~ented by OR Gate ~O may be u~ed to sccompllsh this result.Note that ~uch functions must not prevent legitimate propa~ation through"pepper noise" within an enclRve.

The l~t o~ the circuitry groups in FIG SB is the Fill registar 61 and its output i~ called FLRi~ . It is loaded through shlftlng gate 62 whose input is from the Fill Reglster directly to tha laft. The Fill signals are supplied and shifted through the FOVQ~ M~trix each time ~ ~est ~ncl~ve h~s been selected. The 3hl~tlng is accomplished by exactly the same technique ~s that usad for lo~ding the patterll itself.

As mentioned before, FIG 5A ~pplies in det~il only to - 2~ -36~

those Polyps in the NE quadrant. When modlfied by the information ln FIG 5D, however, a design for all four quadrants can be obtalned ~rom FIG 5A. Specifically, FlG SC is a table showlng the gate input designations for each o~ the dif~erent quadrAnts.

Uslng a Fovea Matrix nomenclature which has a zero column number and a zero row number create~ confuslon as to which qu~drant the zero column and zero row 3hould be as~l~ned during en~lave mea3urements. From a theoretical point of view lt does not m~tte- ~o long aq the cholce ls conslstent. Furthermore, lf th~ r~solution of the data is sufficiently hlgh, the data of this row ~nd thl~ column can be dl~carded. From A c03t point of vlew, however, re~olutlon must be kept ~s low ~ po~slble consi~tent with good results. In this de~cription the zero row to the la~t of the CSA is considered part of the NE quadrant, the zero column to the South of the GSA i~ considered part of the SE quadrantJ
the zero row to the West 1~ treated AS part of the SW quadrant, and the zero column to the North 18 treated a~ pArt of the NW
quRdr~nt .

FIG 5D 3how~ a Slngle Polyp ln each quadrant. The maln purpo~e of thi~ flgure ls to ~how the addltional circuitry~ whlch 13 u~ed to calcul~te the percentageq used in the measuraments. Of the m~ny pos~lble ways of electronically generatlng percentages I
h~e chosen an ~n~log method ~s the preferred embodlment. In order to do this it is first necess~ry to gener~te slgnals th~t 3~

are propor~ional to the ~bsoluts number of pixels. In th2 NW
qu~dr~nt these sign~ls ~re NWM (NorthWest Membership), NWM
~NorthWest HorizontHl> , and NWV (NorthWest Vertical). They are respectively EMR (Encl~ve Membership Register) through a high v~lue of re~istance,HCR (Horizontal Closure Rayister~ through high value o resistance, and VCR (Vertical Closure Register~
through a high value of resistance. All of the NWM polnts are to be tied together ~nd also connected to a very low imedance device 3hown ln FIG SE a~ an operational amplifler 70. The voltage output of the oper~tional ampllfier wlll be proportlon~l to the ab301ute number of enclave members in the NW qu~drant. The sum of NWH and NW~ are similarly generated.

FIG SE also shows analog circuitry for gener~ting x~ and xH
~or the NW quadrant. The circuitry uaes operational amplifiers 71V ~nd ~lH with appropriate lnputs~ a~ s~own. Circuitry for generating similar signals are to be provided for each o tbe quadrants.

FI~ SE additlonally shows the preferxed method of shifting th~ pattern PIR and fill FLR information through the Fovea Matrix.

FIG 6 shows detall~ of ~ recognition scoring device 12. This is an expansion of FIG 3, which di~cusses the functions from a block diagram point of veiw. The preferred embodiment of the 3~

Reco~nition Scorlng Devlce 12 is Q serial computer o the clas3ical ~on Neum~n type. It lnclude~ a Be3t Feature Storage davlce 60, a Central Processing Unit 61. an Equation and Control Storage Devlce 6~, ~qnd sn Output Stora~e device 63.

When the Artificial Fovea of FIGS 1,~,and 5 has finished lts work it outputs ~he Best Features ~ound in the pP~tern to the CPU
61, which in turn stores them in the Random Access Memory tRAM~
callad Best Festure Storage 60. CPU 61 then procede~ to evaluate tha Equations which are stored in Read-Only-Memory (ROM~ 62. This ~11 ls done under the control of the Progr~mr wich al~o reside~
in R~M 6~. The sorting of the equation scores and the acceptibility testing is ~lso done under control of the program in CPU 61. The name of the Accepted Class, plus instructions about what to do if the character is re~cted, are ~11 stored in ~ section of RAM called Output Storage 63. The separation of RAM
shown in FIG 6 ls mude only for illustr~tive purpose~, and many other asslgnmentR may be used. There i~ no speclflc reason, ~or ex~mple, why Equation and ~ontrol Storage 62 cannot also be RAM":
since the informat.Lon stored in that memory does, in fact, ch~nge la~3 o~`ten than the in~ormation stored in memories 60 and 63, the u3e of ROM" is indicated on grounds of hi~her performance at le5s C08t. Although the use o special purpose hardware designed speci~ically to perform the recognltion ~coring function is parfectly possible, the preferred embodiment is ~ general purpose computer because of the economies possible. Its hlgh degree of ~8~

~le~ibility i5 also v~lu3bl~. The only drawback to using gener~l purpose computer here is its speed, which is slow compared to dedic~ted hardware. If ~peed becomes a problem, it is quite easy to ~dd more microproce3sor computers in p~r~llel to perform the Recognition ~coring function.

FIG 7A is a Table o~ Generic Enclave NRmes~ In the ~ketches shown in this table~ the black pixels are represented by'X marks, whlla the whlte pixels are represented by "dot" or "perlod"
m~rk3.

The Table shows four dif~`erent ma~or clRssific~tions of n~mes o~ enclaves. The first classific~tion is that of "Loop'-.
There ~re no aub-classific~tions. A score of 100 on the generlc loop template would be achieved b,v ~ symetrical area of white pixels whlch ls tot~lly surrounded by black pixels. Thls also implies th~t eQch o the quRdrRnts scored lOOX of plxels bein~
bounded ln both the vertical and horizont~l ~xes. (It will be sppreclated that the boundedne~s determination~ can be made on ~l~gon~ls ~ust as easily and the terms ''north", "~outh", e~st", ~nd "west" are merely terms of reference.> The numeral zero, if no volds exist, will score 100 on the Loop generic tQmplate.
~ncl~e ~1 of FIG 7B will also score 100 on the Loop generic templ~te, even though the shape is trlangular.

The next classification is called "C~rdinal Conc~vities', 3~

and the sub-classes are called ''Generic Nor-th", '`Generic E~et","Generic South", snd"Generic West". ln order to achieve a 3core o~ 100 on t~e Generic ~orth template, ~n encl~ve must h~ve 1) every white pixel in the NE be bounded to the Ea~t but not to the North, 2) every white pixel in the SE be bounded to the East and the South 3> every white pixel in the SW be bounded to the South and the West, ~) every white pixel in the NW be bounded to the W~st but not bounded to the North. Any plxels devi~tlng ~rom tha~ ~pecl~ications wlll cause the score of the enclave on this t~mplate to be proportionAtely reduced.

In order to score 100 on the Generic East templ~te, the white pixel~ in the encl~ve must be bounded in ~11 directions e~cept the East~ In other words, this generic template has the ~ame requirements as the Generic North templ~te except the requirements are rotated 90 degrees clockwise.

Similarly, the Generic South ~nd Generic West templstes h~ve requlrements which are lookin~ for white pixels th~t Qre bounded ln three directlons but unbounded in the fourth direction.

Two example~ o~ Cardinal Concavities are shown in FIG lB.
Encl~ve ~1' will score 100 on the Generic West templ~te, ~nd Enclave ~ will score 100 on the Generic East templ~te.

The second m~or clRssific~tion of generic templ~tes shown ~2~

in FIG 7A ~re called"Corner Conc~vities". Four sub-cl~sses are illustrAted. They are called "NE Square", "SE Square", "SW
Square", snd"NW Squ~re". Three examples of good corner concsvlties are shown in FIG 7B, whlch is a m~trix sketch of a closed top numeral four. They are enclaves 2D 3, ~nd ~4. ~2 will score 100 on the NE Square template, ~3 wlll score 100 on the SE
Square templste, and ~ will ~core 100 on the SE Square generic template.

The thlrd ma~or classiflcation of generic template~ are c~lled corner convexltle~. Four subcl~sses are called NE Vex`',SE' Vex, "SW ~ex", and "NW Vex"~ An lllustratlon o an enclave which scores 100 on the NW Vex template is enclave ~5 o~ ~IG 7A.

FlG 7C ls tltled"Cardinal Template Scores for ~nclave ~1 of FIG lA". In thl~ table, the various template~ are glven numbers ~or exemplary purposes. The Generic North template is glven the number TlO Tll 18 a specllc template wlth maJor openlng to the north. All templates numbered TlO through Tl9 are north templ~tes. Slmilarly, T20 ls the Generlc Eest templ~te, with T21 being ~ templ~te for some specif`ic unusual east opening encl3ve.
Ag~ln the numbers T20 through T29 ~re reserved or ea~t openlng template. South and West templates are similarly numbered.

The scores shown in FIG 7C ~re the ~cores that the measurements obtained from enclave 1 would have ~ttained on ~2~6~3~

TlO,Tll,T20,T2l,1'30,T3l,140, and T~l. 1`he name "Bsst-in-North" 18 given to the best ~core of TlO through Tl9. The name "Be~t-in-~ast is given to the best score of the templates numbered 20 through 29. The names Best-ln-South and Best-ln-West are similarly derived. These inclusive tltles for ~cores derived from ~eneral characteristics of templates is a powerful tool .
Particularly this method reduces the number of Recognition equations by a large factor. The size of this factor ls applic~tion dependent, but is at least equal to lO or better.

If the scores for Enclave ~2 of FIG lB were computed, they would show that T20 had obtained a score of lO0, and that the Best-in-East score was lOO.Other scores will ~ll be 75 or le~s.

.:

.

FIG. 8A through EIG.8K comprise a table of eleven Arabic numerals which can be descr1bed using generic enclave~.
The Recognition Equations (REq) shown contain only the Assertion terms and the Relative Position Ratio terms. They do not include Negation terms, which wlll be dL~cus~ed in a ~nother section.

FIG 8A shows a loop a~ being the best enclave, and a generic zero is de~ined as being the score of the Best Loop template .

FIG ~B shows a slngle ~troke ONE~ slopad le4t. The score of this generic shape is the score of the best NW VEX plus the score of the Be~t SE VEX, the sum divided by two. A right sloped ONE is similar but uses NE VEX and SW VEX.

FIG 8C shows the mo~t common shape o a handprinted TWO. It has no extra loops. The slmplest form of the Recognition Equ~tlon i8 equal to ~BIW ~ BIE)~2 tlme~ the Rel~tive Po~ition Ratio Two (RPRC2~). RPR~2] ls a function which is equal to unity if all the ~IW enclave pixels are above or to the left o4 the BIE
pixels. RPRt2~ goes to zero very quickly if a large percent~ge of enclave pixels violate this requirement.

:
FI~ 8D shows a generic THREE. It has two ma~or enclaves. These are labeled Best In West ~BlE) nd Second Best In - 34 - ;
!

. , . . ~

' , ' ~2~6~

h'eat (SBIW~. Its Recognition Equation is equal to ~BlE ~ SBl~>/2.

PIG 8E shows an open top FOUR. It~ important enclaves are Best In North (BlN), ~est SW S~, Best NE sa, and Best SE S~
Its Recognition Equation is equal to the sum of these scores divided by 4.

FlG 8F shows a closed top FOUR.Its important enclaves are Best Sharp Loop, Best SW SQ, Best NE SQ, Best SE SR, and Best NW VEX. The Shsrp Loop function will be defined later in the speclfication. The "sharpness" function helps separate the closed top FOUR from a NlNE.

FIG 8G shows a generic FlV~'. It~ ~ecognition Equation is equal to <BIB ~ BIW)~2 times RPR[5], whers RPR[5l is the Relati~e Position Ratio function for the numeral FIVE.

FIG 8H shows a generic SIX. Three acceptable enclaves are shown. Encl~ve 1 i~ accepted as "Best-In-E~st", Enclave ~2 as "Loop", and Enclave ~3 as "Best NWVEX-'. Enclave ~3 is only marginally acceptable, ~ince the number of pixels in it i~ a amall percentage of the total number of pixels in khe total white pixel ima~e. Thls illustrates the point that m~rgin~l encla~es do not need to be speciied when writing a Recognltlon Equ~tion (REq) for generic ch~racter~. Thus the REq for a generic SIX is equal to CBIE ~ Loop>/2 times RPRL6]: where RPR~6] is the .' .: ' , , ' '' ' 3~

rel~tive Position Ratlo function for the numeral SIX.

FIG 8I shows a generic SEV~N. Its ~ccept~ble enclaves ~re Best-In-West ~nd Best SEVEX. Its recognition equ~tion is equ~l to ~DIE ~ 5EVEX)/2. Note -that none of the Recognition Equ~tLons discussed in connection with FIG 8A through 8K show ~ny o the"negation" terms. ~or the SEVEN. one of the appropriate negatlon terms would be some fraction of Best-In-East score; this wo~lld prevent the psttern shown in FIG 8G from producing a good score on the REq for SEVEN.

FIG 8J ~hows a generic ElGHT. Its acceptable Enclaves show a Best Loop <BL), Q Second De~t Loop (SDL) and a Be.~t-In West.
Because double loops appe~r coincidentally in m~ny other handprlnted numersal~, the BIE term must be u~ed together with R
Relstive Posltlon Ratio whlch i~ unity if one loop ls above the BIE which in turn ls above the second loop. RPRt8] fall~ qulckly to zero for numeral shapes whlch do not meet thls function. The recognition equatlon ~or ~IGHT 1~ equal to (DL ~ SBL + BIW~/3 time~ RPRt8~.

FIG 8K shows a gener.ic NINE. Its ma~or enclsves ~re a Be~t Loop ~BL), a Best-ln-We~t (BIW) and a SEVEX. Although no other re~l numer~l should have a BIW o~or ~ BL, it is good pr~ctice to protect again~t garbage pattern~ by addlng the RPRt9~ term which speclfiies that the DL mu~t be over the DIW to be ~ high scorlng :

, ', ~ . - ' :,, '' .
,: .

NINE. The REq for NINE, twithout negation terms) is equal to (BL
BIW~2 tlmes RPRt9~.

DISC~SSION OF RECOGNITION EOUATIONS

FIG 9 shows ~n exemplary set of eleven Recognltion Equ~tions for the Arabic numerals ZE~O through NINE, includi~g sep~r~te c~ses or the open top FOUR and the closed top FOUR.

The nomencl~ture used i~ designed to facilitate the logic~l sorting and inclu~ion of generic templates which ha~ already been discus~ed. Thus, RE~ 0-0 mean~ the Recognition Equation which represents the most common sh~pe of a ZERO. Its shape is shown in FIG 8A. Simil~rly, RE~ 1-0 me~ns the Recognition Equation for the most common shape of ~ ONE. The first sub~cript is the numer~l CLASS, whlle the ~econd subscript belng the ~pecific shape type within the CLASS. RE~ 4-0 is the generic equation for ~n "open top" FOUR, whlle REQ ~-1 is the generic equ~tion for ~ "clo~ed top" FOUR. ~ote th~t the best scores ~rom each cl~s5 must compete ag~inst e~ch other to excede the MINIMUM ACCEPlABL~ DOUBLES RATIO
~MADR) ~Refer to FIG 3), but MADR does not ~pply within the same CLASS n~me. Tha~, ~ score of 90 for REQ 4-1 is ~cceptable over a Bcore of 87 for REO 4-0 the best CLASS FOUR Bcor~ must h~ve ~

Doubles Ratio of better th~n the Best Class score of every~other - 37 -~

, . :

3~

cla~s, howe~er. In a typical case ~ the most likely class to be competing closely with the closed top FOUR is CLASS NINE. Thus, u~ing a typical MADR of l.2 , all o the other clas~ scores mu~t be less than 72 for a Best Cl~ss score of 90 to be accepted.

Three types of terma are used in FlG 9. They are the As3ertlon term~ ~such ~ BL, BIW, etc.), the Nega~ion Terms (such ns NEGtSBL], and the Relative Position Ratio terms tsuch as RPRC2]~. In each Recognltion Equation the Assertlon score mu~t be normalized by dividing each Assertion 1`erm by the number o such term~ in the equation. This results in producing s score of 100 and no score ~ubtractea due to Negations and RP~s. Thi~ i~
b~ic to the capability of comparative scoring.
~ he negation terms may be any desired function which produces a deslrable diminution of the score if some undesireable encl~ve is present in the shape. For REQ 0-0, for example, the ~core should be reduced lf there are two enclaves in the shape whlch ~core high on a loop template. 1`he best loop ~core is c~lled BL, while the Second Best Loop is called SBL. In order tokeep scores normalized, it is alsonecessary to have ~ negation unctlon which does not add to the REQ score if the unwanted encl~ve is absent. Another ch~racteristic of ~ good Negstion function is that it should not subtract too much: if 100 points were subtr~cted in a case where S~L was 100, the re~ulting REQ
0-0 score would be unable to compete ln the very important double~ reJect comp~rison. One of the use~ul functions is that .
' .:

~8~

shown ln FIG ~: the amount to be subtracted is zero so long e3 the argument is lower than 75, but becomes equal to 100 minu~ the argument for argument values greater th~n 75.

MEMBERSHIP ~UALIFICATION

A method of qualifylng whlte plxels for ~ember~hlp in an enclsve ia illustr~ted ln Figure lOA. This showa a seriea of sketches lllustrRtlng progres~lve phases of member~hlp quallficatlon ln a clean ch~racter. The phrase "clean character"
means the image oi an alph~-numeric char~cter which does not have voids or extra black plxels.

For purposes of the following explanatlons, each pixel is identified by its x-y coordinates relative to ~ pr~determined polnt in the im~ge: for FIG 10 the test COR A ia Rt location O,O.

FIG lOA show~ four phaaea of the progreaalve qualifylng activity. FIG lOA.l. (Ph~se 1~ shows the choice o~ Pixel A as a Test Cor location, ~nd it ~lso shows three white pixels that h~ve been qu~lified for membership; these pixel3 heve been labelsd "I". They quallfied beceu~e they"touch" Pix~l A."Touching" is delined ~s being next pixel neighbors on the same row or s~me colmun. A further requirement for qu~lification is thet the whits pixel must be bounded by a bl~ck plxel in the same row or column.

~- 39 -': :

.

3~

This bl~ck pixel must be in the same quadrant as the candldate plxel and must be located a a distance further from the Test COR
than the candidate. Note that the plxel dlrectly to the West of Pixel A is not qualifled because it ls not bounded ln its qu~drant. (As noted above, the boundedness evaluation3 c~n also be made in diagonal directions).

FIG lOA.2 ~Phase 2~ shows additional pixels h~ving been quallfied as a result of "touching" already quallfied pixels and being "bounded" in thelr respective quAdrants. Plxel~ on quadrant boundaries are defined to be ln both quadrants.

FIG lOA.~ (Phase 3~ show~ a further progression of the qualificatlon "wave", and the Final phase is shown ln FIG lOA.~.

While the black line elements in the four phase~ of FIG lOA
~re ~hown as being only one black plxel wlde, one of the important advantage-~ of thi~ lnvention ls that the width of a llne formlng ~ ch~racter (and therefore the number of black pixels ln the width of a bl~ck line element) is irrelevant to the basic operation of ldentlfylng the character which it represents.

FI~ lOB contalns three sketches labeled FIG lOB.l, FIG

lOB.2, and FIG lOB.3. FIG lOB, ls called "Membershlp ~ualification with Voids ( Salt) and ~xtra Slack Plxel~
(Pepper)". Sketch 1) "Ponetratlon of Vold'' abows a ~lngle pixel :

.

`

~ , `', ~ ' ~` . ` .
~, ~

qualifylng at the location of the void. Sketch 2)"Llmitatlon of Penetration Due to Black Inhibitions". Inhibltions extendlng the lnfluence of black plxels occur ln human vlsion RS well a~ ln the Artlflclal Fovea. The rule illustrated ls expressed by the following statement: any black pixel at coordinates i,J inhibits membership of pixels located at i~l,J~ 2.1~1 : 1l1 ,J~;
i~2,J(2. The coordinate numbers are po~ltive in the directions away from the Test COR~ The lnhibited pixel~ of particular lnterest ln FIG lOB.l and FIG lOB.~ are labeled with black dots.
Note th~t the inhibited plxels prevent urther spread of quallfled plxels in the particular lmage shown ln FIG lOB 2>.

The hum~n fovea pays a llnear price for inhibiting penetratlon of enclaves through volds. This also occurs in my Artlficial Fovea, as shown in FIG lOB.3 labeled "Loss of Membershlp Due to Pepper Noise". Note there ~re two pixels lost to membership due to Black Inhibition (plus the loss of the black pepper plxel ltself).

It is important to realize that the "labeling" o~ plxels and the slze of enclaves is roughly proportional to the general influence of neighboring pixels in my lnvention~ ~nd not to some disasterous slngIe accident of noise. Thus, it must be noted that the result of comparing the Final Phase enclave o~ FIG lOAl.~ to FIG lOB.2 will produce a high degree of correlation in spite of the void. Thi~ happy situatation occurs more often for my area ': ' '~ '" ' ~ ' , , ' ., ... .

~2~ 3~

.
bRsed invention than lt occurs ~or methods based on the curvature of black pixel aggreg~tes.
Figures 11 and 12 illustrate characters which will be increaslngly more dificult for u method based on"line tracking"
to succes~ully recognize the ones which are recognizable and reJect the ones which are not"s~ely" recognizable. Note ln FIG
12 the varlatlons in the width of the "line" whlch is easily handled by this invention but cre~tes difficult problem~ in prlor art systems which use line thinninq or skeletonlzatlon steps. The term"line tracking" is her~ intended to mean all those methods whlch ~ttempt to measure the dlrectional characteristics of ~roup~ of black pixels. A classic example is Grenias (REF 3).
Another term for line trackin~ lS "p~ttern contour tracklng".There are an enormouq number o~ example~ of methods which expand upon this technique a number of which require "line thinnlng" or skeletonizatlon for them to be operatlve.

The llno tracking method 1~ to ~e contrasted with the "~accAdlc 11ck" and "enclave measurement" techniques of thls invention, as previously described. lt cannot ~e emph~sized too strongly that the measurements be~in with the closure ~` characterlstics of each white plxe1 independently o other white pixels. The agglomeratlon of white pixels lnto enclaves ls per~ormed in a system~tic way, complete1y independently of the slope char~cteristics o black pixel segments. The measurements of enclaves treat llne segment voids ("salt noise") only in terms . . . .

'.

; ' ~' ' ' ' - , ,' ,~ , . .

3C~

of how they affected the closure chsracteristlcs of of the prlmal white pixels. Irre~ularity of the edges of blac~ Line se~ments have only a minute effect on the scorlng of enclave~. Black pixels which ~re separated from the main line segments ("pepper noise"> afect the acoring primarily ln terms of how they change the closure characteristics of the primal white pixels.

In a llne trackin~ machine there may be extensive "pre-processlng" performed on the pattern before trackiny is attempted. The various transformations, which often lnclude "filling" ~change white plxels to black>, "smoothlng", and "line thinning" are all conceptually part of the technique o~ line tracking. On patternA whlch are free of nol~e, these efforts are redundant and the line tracking ~achine generally performs well ~
for familiar ~hapes>. On patterns which con~ain a small amount of noise, the pre-processing generally helps. On pattern~ which contaln a large amount of nolse, the pre-processing and llne tracklng oten lead to that most heinou~ of errors: a aubstitutlon.

I wlll now procede to lllu~trate the above general discussion of the problems of line tracking with references to Figures 11 snd 12. ~'igure llA shows the slmplest possible case of a ~oid. The lmage contains a single wldth black pixel line p~ttern. A completely unsophisticated pre-processor/line-tr~cker would decide that the pattern has composed of only one loop - 43 ~

.
.' -, - ' : .. : ~ : .` : .

,' ~ : j' ~36~3~

inste~d of two. My invention will gi~e ~ score of 100 out of a po~sible 100 to euch loop, and 8 Recognition Equ~tion ~core of lOO for an EIGHT. See Figure llD for detalls of the scoring. If the void in the crossbar W~5 wider, th~ score would decre~se u~lng my invention.

An obvlous"improvement" to the pre-proce~sor~line-tracker would be to autom~tlcally 111 ln all single volds. This then wlll produce ~ correct decls~on for Figure llA. Thls would le~d the ever hopeful promoter of line tr~cklng to demand of hls engineer th~t the plttern oi Flgure llB be al~o recognized a8 ~n EIGHT. In order to do this, the engineer mlght add to the pre-processor the following ~lgorithm: if two and two only bl~ck pixels ~re present in any three-by-three pixel matrix, then ill ln the ahortest path between them. This then would recognlze the pattern of Flgure llB as an EIGHT. My lnvention will produce very low scores for Loops, ~est in West, ~est in E~st, etc., because the percent~ge of white plxels bounded by black pixels is 40 low.
See Flgure llE for some detalls of the scoring u~lng my lnvention. The outputs of ~11 of the Recognltlon equatlons will therefore also be very low, producing both absolute level reJects as well as doubles reJectS. My invention would call for another sampllng of the original pattern, using ~ dlfferent quantizing level or a larger s~mpling ~rea, or both. Thls example may lead the unwary re~der to ~ume thst the pre-processing~llne-tracking method i~ superlor. This is not so, bec~use the p~ttern o Figure - .

.. .: . , . ~ . .... .

,: , . .

~2~ 3~

llC wlll be re~ognlzed ag a satl~factory THREE by llne trscklng, thua produclng the worst posslble error: a substltutlon.

Referrlng now to the pattern of Fl~ure llC, we observe lt to be slmost ldentical to that of Flgure 11~. Only two black pixels have been moved, but the void fillln~ algorithm now used the the pre-proces~or produces continuous line~ everywhere except at the critlcal We~t Sides of the loops. My inventlon wlll output a REJECT for this pattern, but a sophisticated line trackin~
machlne may very well produce ~ substitution. Engineers can contlnue to add speclal case exceptlons ~ called AD HOC ~olutions by the professlon) whLch fix special case but invarlably end up makln~ matters much wor~e for cases that have not yet been trled.

The addltion of "pepper noise" ~extra black pixels whlch are not connected to line segments~ compounds ~he problems facing the line tracklng machlne. Figures 12A and 12B show two lllustratlons o modest llne thlckening and pepper nolae which wlll drlve any llne tracklng machine crazy. The problem is that too many llne ~egment~ have been started and went nowhere uaeful.
Becau-~e even a few extra branches cause an exponential rise ln the number of permutations and combinations, ~he line tracking machlne quickly runs out o~ ability to express ths allowable pattern shape~. The problem i8 that a "computable" parameter doe~
not exist for line tr~cking machines. Conversely, my invention usea contlnuously computable parameter~, rather than"decl~lon , : ' , ''' ~ , . ' , ': ' . ;

, . , ' ~B~q~3~

tree" recognition logic. These continuously comput~ble par~meters ~r~ the scores that are continuously generated ut every correlation step of the proces~. The patterns of Figure 12 A and 12 B produce excellent scores for the ZERO and the TWO using my inventlon.

The examples of Flgure 12 A and 12 B have been chosen to lllustr~te salt nolse in Flgure 12 A ~nd pepper nolse in Flgure 12 B, wlthout comblning the nolses. The dlfflculties whlch oc~ur within ~ line tracking machine when faced with a comblnation of these nolaes ~an only be described ~s"awe~ome".

~XAMPLES OF ~ETAILED QUADRANT MEASURMENTS

Flgures 13A ~nd 13B lllustrate the baslc working of the me~surements whlch provide ~or recognltlon which i~ slze lndependent. Flgure 13A "BIG ZERO" show~ a character whlch ls approxlm~tely 16 plxels hlgh by 13 pixels wlde. Utllizing the point "A" for the COR, ~lI of the white pixels in the NE quadr~nt are bounded in both the vertic~l and horizont~l direction, thus warranting the designation "s". There are ~1 of these pixel~.
~ote that white pixels on the vertical and horizontal axes of the COR are counted in both of the qu~drants whlch they divide.) Thus "NE x ~" = 100 %". Flgure 13B "SMALL ZERO" h~s only a total of 11 white pixels in the NE quadr~nt of it~ ~n~ly~is diagr~m.

.

., ~ ~ , ' : : ' ' .~' : ': .' :
.
.: ,~ - . ' ~ , '. , :
~ . .
, ', .: ' :
.

~Note Rgain tha~ white plxel~ on the vertical and horlzontal axes of the COR are counted in both of the quadrant~ whlch they dlvlde.~ Nevertheless lts percentage of white pixels which are bounded both in the vertical and horizontal direction is 100 %;
thus "NE X 5 = lOOx", the ~ame a~ in ~he big zero. In the OC~
Indu3try thls is called "Size Normalization". Slmllarly the SE, SW, and NW quadrantQ show a similar analysls for the big and small zeroes. Note that Figure 13B i~ not an èxactly scaled down version of Figure 13A. Thl~ wa~ done on purpo~e to also illustrate the independence ~rom line noise.

Figures 13C "SMALL SEVEN" and 13D "BlG SEVEN" go ~urther in illustrating size normalization. Both these figure~ h~ve two ~atisf~ctory encl~ve~. The COR for Enclave A is ~hown as Plxel A, whlle the COR for Enclave B is called Pixel B. (Note that the white pixels which separate the quadrants are scored in each quadrant, as previously e~plalned.) For Enclave A, the number of NE ~ember~ ia 11, and "NE x~ = 100". SE Members = 6, and "SE xs =
100". In the SW quadrant there are only three member3, and these are dlrectly below the COR. These members are all bounded vertlcally only, and thus carry the deslgnation o~ "v": this fact is expres~ed a~ "SW %v = 100". In the NW quadrant there are 11 member~, and all of them are vertlcally bounded only: thus "NW Xv = 100". These four percentage~ are exactly the s~me a5 the peFcentagss in the perfect Best-In-West template: ~hus the BIW
feature ~core is equ~l to lOO. Enclave B, whose COR is at Pixel :, ~ .
,' ' :
:

~3~3 B, has it~ member~ bounded in a very different set of way~.
~uadr~nt NE has s=lOOX. Quadrant SE has * = lOOx, since is has no white pixels bounded. ~u~drant SW h~s 1~ members and ~h = 100.
Qusdrant NW has only ~ members, snd ~ll of them are bounded vertically and horizontally X5 iS therefore 100 x for NW. The templ~te called "SE~EX" c811s for exactly this set of relationships, and feature SE~X has therefore a ~alue of 100.
The generlc Recognitlon Equatlon RE~ 7-0 equals ~BIW + SEVEX>~2 vNEG[SBIW] . Slnce BIW is 100, SEV~X is 100~ and there i~ no second best-in-west enclave, these score for REQ 7-0 ls 100.

BIG SEVEN, shown ln Figure 13D illustrates ~ize normalization and ls to be compared with Figure 13C. The figure has been deliberately drawn, however, to give a slightly different score ln the NW quadrant. Because of the sllght loop tendency ln the NW, the NW xv is only 70x, while xg has 30%. The ~core for the BIW template is therefore reduced, becomlng only 90 instead of 100. Thls produces a REQ 7-0 score only only 95 lnstead of 100.

The method of ~coring a set of me~urements ag~inst a known template 18 uniform throughout this specification. There are three terms in each quadrant descrlption of each template~ These are Xs, xv, Xh. The equation I use in scorlng is as follows:

: ' '. ' '' , ~' 33~

SCORh OF ENCLAVE ~ AGAINST TEMPLATE T

= (100-NE Absolute Dlfference~/4 ~ ~loo-SE Absolute Diff~rence)~4 + (100-SW Absolute Difference~/4 + (100-NW Absolute Difference)~4 where each quadrAnt Absolute Difference = ~b~olute dlfference (XsE-v~sl'~3 + ~baolute difference (XhE-vxh'r)~3 + absolute dlfference (XvE-vXvT)~3 Flgure 13E 111ustr~tes a "pertect" Europe~n SEVEN. Thl~
numer~l, whose R~cognition Equ~tion is designAted REO 7-1, requires our encl~ves insteAd of the two required for ~ norm~l SEVEN. The an~lysis ia shown in detail and does not have any anomalies. Note th~t white pixels not claimed ~s members by one enclave m~y well be claimed by another enclave whose Analy3is was cArrled out l~ter. (The order of anAly~l3 never ~ffects the me~3urements or scoring. Similsrly the tempor~ry ~ppellation , ' ' : :, ,' .'.': ''," ,"'' ' :, ~;
-: ,- ' ' : ' - , ', ~'- ' ,-. :~.

. : .

3~

~tt~ched to ~n enclave. such ~s Encl~ve A. or Encl~ve ~2, never affects the measurements or scoring.~

The NW quadrant o Enclave C shows ~ ca~e of "line-of-sight" inhlbition in qualifying white pixels as members of Enclave C; notice th~t the crossbar oi the SEVEN ls between COR C ~nd ten whlte plxels. Line-of-slght membershlp inhibition is a useful and lmportant tool for the prevention of unwsnted "bloomin~" of enclave membershlp.

; - 50 -:
:
:, . ~
: . ..

,. . . . . . . . . . ..

. - ~ " ' ' ' ' ' 3~

METH~VS FO~ HANDLIN~ Uv~RLAppE~/Tou~HING CHARACTERS

~ Rndprlnting from uncontrolled sources contalns a grest many p~tterns which sre overlappln~-not-touching, touching-not-overlRpping, and touching-overlapping. Thi~ ~ection de~cribes some of t~e ways this invention can uniquely recognize ~uc~ chsrscters, although most h~ndprint reco~nition slgorithms are unable to copé with these defects.

` One rea~on thl~ invention 1~ ~uperior to most other methods is that lt is not neces~ary to obtain precise ~egmentatlon. Since the scorlng ls linear and carries highly accurate measures of segmentation auality 1~ is theoretically possible to perform measurements on all posslble segmentations and then choose the best set of segmentations sfter the entire image hss been snRly~ed in comple~e detail. In practice, ~owever, it saves time and money to use a combination of measurements which can be used to generate a plot of the Probability of Segmentation ~POS Plot) as a +unction of the hori~ontal axis.
1`here are occurence~, however, o hsndprinting in which two or more characters are so much intertwined that the POS Plot is not u~eiul. For these ca~es. this invent~on has the cap~bility of using "Dy~dic Recoqnition" and "Trladic Recognltion. These concept~ wlll be illustrated after discussing the~baslc POS Plot~

. ~ .
' '' ' ' .' :

.
- '. , ' . . , . :,~ ' : , :' . ~

: . ., . :
- : . . . ' :
:: , .: . . ~. : . `~: : . . .

3~

Xnitially, when the control unit is presented with a new image, the first function performed is Ssgmentation Analysis.
This is an an~lysis of the entire image to ~ind out whether more than one character occurs within the image and,if so, where the probability of separation is nigh. In the case of ~ourtesy Amount Recognltion (CAR~ ~the arabic numerals entered onto a bank check~
the image presented i~ the Courte~y Amount Fleld (CAF), and the CAF alw~ys contains more than one numeral. When n~merals are separated by a vertical column of white pixel~, this fact is recorded to~ether with the horizontal coordinate of this occurence, It is called a Vertical Column Clear Occurence (VGCO~.
The reco~nition of this occurence is basic to any OCR scheme and methods for accomplishing this ~re well represented in the patent literature. Figure 14A shows one example of this feature.

Figure 14A al~o shows a more interesting occurence called an overlappin~-not-touchin~ occurence ~ONTO). The handprinted TWO
and THREE are overlapping but not touching at both the top and bottom of the numerals. An "ONTO Membership Pulse" ls initiated at the point P in the Artificlal Fovea ~AF~ durinq Seymentation Analysis. Point P may be the center of the AF so lon~ as the black lmage is kept roughlY centered vertically a5 the image is shifted from ri~ht to le~t. Figure 14B shows an Exemplary ONTO
Stage within the Artificial Fovea~ A matrix of such sta~es is simply added to the Artiflcial Fovea previously descrlbed. This ,'~ - , ' '` '' ~ ~', : ' ' ' ' ~ , C33~

new ONTO matrlx wlthin the Artiflcial Fovea is connected to the main AF only by the Pl,m lnput which lnhibits an ONTO stage from firlng i~ the Polyp at the sa~e locatlon represents a black pixel. If the Polyp repre~ent~ a whlte pixel, the ONTO st~e ~t locatlon l,m ls flred if any of the eight surroundlng ONTO stages have flred. The circuit to do this i~ the eight input"OR GATE"~
The result of this logic is that a kind of brushflre occurs in which each new ONl'O member become~ the source for a new brushfire. Continuous lines of black pixels will act as "firebreaks", but the brushfire will reach the top and bottom boundaries of the AF if there is any possible route. OR ~ate detectors (not shown) along the top snd bottom boundaries fire when the brushflre reaches each boundary. The horlzontal extent of the ONTO matrlx shoul~ no~ b0 as great at for the other functions of the AF, since lt is a waste of resources to try to detect segmentations in whlch the characters are woven together but not touching for more than half of an avera~e character width.

Figurs 14A shows a possible ONl`O feature between the TWO ond the THREE. The important route followed by the brushfire is indicated on the d.r~wing by a symbol composed of a dot inside a circle. Most o~ the brushfire is not portrayed, in the lnterest of maklnq the principle more obvious.

The value of the ONTO feature will be recorded if both - 53 - ~

.

. ' ' :
.~: .

~ ~ . ` . ' 1;~ 3~

the top houndary detector ~n~ the bottom bound~ry detector fire within a reasonably ~hort time. Assuming the clrcuitry is asynchronous the length of time required for the brushfire to fini~h its propagation should be less than a microsecond. The probability of segmentatlon (P05~ is higher the shorter the length of time: the ~pread of firings is al~o significant, and the v~lue of the ONTO feature will be some unction of these measurements. Note that the ONTO feature becomes 8 way of mea~uring the VCCO.

Another lnput to the POS Plot may be the values o what are called Upper Segmentation Templates ~UST>and Lower Segmentation Templ~tes (LST~. Figure 14~ illustr~tes ~ number of pairs of numerals with upper and lower segmentation enclaves indicated by down arrows and up arrows, respectively. Figure 14D
shows a detailed example of a lower segmentation enclave and its somewhat generalized template. ~ically. my lnvention is making uQe o+ the same fact that humans use if they have to perform segmentation, nam~ly that there are enclaves and combinations of enclaves which typically occur between characters, even if they ~re touchlng. The mo~t u~eiul, for numeral~ is fact that mo~t numer~ls do not have enclave~ which are ~outh openin~ or North openlng. Therefore, if South and North opening enclaves have high scores, there is a high pro~bability of a segmentation point being close by. (Note that the open top FOUR i~ a maJor exception to this rule -- no matter, a high POS merely generates a detailed ~: ~' ' ' .
. . ' ~ , . .

3~

sxamination, and the linear scoring reveals the correct structure.~

The e~ample o~ Flgure 1~ is instructive. It shows an enclave whose CO~ l~ at point A. A Lower Segmentatlon Template is ~lso presented which calla for the N~ and NW measurements to have lOOxO of the plxels bounded both ver-tLcally and horlzontally, while the SE and SW quadrant~ are to have zeroX bounded both vertically and horizontally. Any deviation from these template specificationa will result in a score of less than 100. When a Template is desiyned by a human operator, the Template may o~ten be improved by the human to the extent of specifying only those parameters which are required to produced the desired filterin~ action. Thus, in Fiqure l~D. only the ~s values are specified in the rempl~te. In evaluatin~ the correlation scoring equation only the terms specified by the Template are to be used.
When no operator is present, thia inventlon calls for machlne learnln~ of a Template whlch must have all the parameters llste.
The le~rning capabilities of my machine sre discussd in mora detil later ln this speci~ication.

Flgures 14E and 14F illustrate a totally different and novel way of recogni~ing character~ which are touching~overlapping. This method is an alternate to the POS Plot method, or it may~be used to confirm ~ marginal acceptance. Both these figures~contain a "dyad" of the same two ~ 55 ~ ~

: .':~ . ;, - ' -' .
' ' ,'', .- . . :

.
, ',, , ;' : ' , ~2~

touching/overlapping characters, but the individual numeral shspes ~re different and the posltional relatlonship of the numerals is different. The basic method for recognizil-g such characters is called "Multiple Character ~ecognltion'. The first time this invention is presented with this dy~d a reJect will occur. A human is called (either on-line or later in an off-line mode) . The human in the case shown identifies the dyad as a 3/6 double character and ~pecifies that some or all of the measured enclaves be made into templates. A new class of numerals is created called CLASS 36 and a new Recognition Equation is created.

A part of the Template correlation scoring ls shown in Figure 14F. One of the many possible methods of scoring the correlation between an Enclave E-1~F and a Template 1'l01 has been glven in an earlier paragraph. It is fully normalized equation, ln that its maxlmum value is always 100 and its minimum value is always zero. The following computation i~ presented in order to demonatrate the detail~ of thl4 exemplary scorlng technique~
There are our terms to compute initially, one for each quadrant.

NE absolute difference = ~100-60)/3 (5-0>/3 ~5-0)/3 ' ' ' :' ~ ', .

~ Z~ 3~

SE ~bsolute difference = ~100-100~/3 ~ O

+O

SW ~olute difference = (10-0>/3 l ~0-0)/3 + (100-90)/3 M ~bsolute dlfferenc~ 0~/3 (0-0) f~

+ (100-79)/3 = 14 '' ., ':
' ~ - . . ':' ' . ' ~ ' 6~

score = <100-27)/4 + (100-0)/4 ~ (100-6)~4 r <100-14~/4 Note that plxels may have a dif~erent symbolic notatlon in different qu~dr~tnts; this occurs only along the qu~tdrant boundaries. However. in the preferred embodiment, no pixel~ are member~ o more than One nclave, however. In order to improve the ea~e of understanding, some of the encl~ves in Figure 13E do not have their pixel notation shown ;these are shown insteetd in Figure 13G. Figures 13F and 13H are al~o separated into two . .
flgures for the same rea~on o clarlty.

The analy~is of Enclave E of Figure 13E i~ routlne. A Template T102~baaed on this enclave has descriptors as follows:

:`
NE %~ = lOO
;

SE %h = 100 ~: :
:
:
- 58 - ~ ~
;

, ~ :

3~

SW %h = 100 NW xs = 100 Analysi3 of Enclave E of Figure 13E ~the set of these measurements ara designated ME-14F~ yields the same descrlptors as T102. and the correlation of ~102 versus ME-14F is therefore 1 00~ .

Figure 14G shows the plxel design~tions within Enclave B.
~Note that there would have been many more memberQ of this enclave i this enclave had been analyzed prior to the analysis of the enclave directly above it.~ A Template T103 may be written as a result o choosing this dyad as the prototype of Class 36-0. Its descriptors are as follows:

NEXs = 100 SE ~S3 = lC)C) SW ,~v = 100 NW Xv = 100 Analysis oi Enclave B of Figure 14H shows exactly the sAme parameter6.The correlation of T103 versus MB-14H is therefore .

;` ,: ' , : .
-: ` ` ` : . .

3{~

1 00% .

Figures l~G and 1~ al~o show the pixel designatlons wlthin Enclave C for the two images. For Dyad 36-0 the north opening enclave (Enclave C) is f~irly standard, since there ~re no voids.
~Note however th~t four pixels in the southwest are~ have been denied membership in the enclave. l`his is because of inhlbition by black pixels closer to the Test COR. This has been dlscussed in the section descrlbing membership rule~.) A Template T104 may be wrltten dlrectly from the measurements. Its descrlptors ~re:

NE Xh = 100 SE Xs = 100 SW Xg = 1 00 NW Xh = 100 Enclave C measurements ~rom Figure 1~ H are somewhat dlfferent. They are:

NE Xh = 100 SE Xs = 100 :

.
.

, ' . :
,~ . : , , 3~

SW xv = 1/4 x 100 = 2S

SW x~ = 3/4 x 100 = 75 NW xh = 100 To get the correlation ~core of MC-14H versus T104, flrs~
get the quadrant absolute differences. For quadrants NE, SE~ and NW, the dlffsrence~ ~re zero.

SW ab~olute dlfference = (100-7S)~3 ~ ~25-0~/~

.
~, ( 0 - 0 MC~14H vs. T104 = (100-0)/4 ~ (100-0)/~

-l (100 33)~4 = 91 .
61 - ;

:: '' ~ " .
;' ' :' , '. , :
- , . . .
' . ,: ' , ': ' , ' ' :.:: ' ' ' : -' - ' : : ' . , . ' . .
- ': ' :. . , ' ' A "bang-bang" recognition equation using specific templ~te numbers as as3ertions, no negatlon term~, and no other qualifiers is e~slly wrltten a~ follow~:

REQ 36-0 = (T101+T102~T103~T10~TlO5~T106~/4 where each T term means the best corelation score yielded by that template against any encl~ve of the image.

This equation yield~ ~ score of 100 for Dyad 36-0 and 96 for Dyad 36-1~ This illustrates that one dyad can be used to recognlze many. An even broader recognltion equation can ~e wrltten uslng assertion terms llke "Be~t-In-West". The wrltlng o "smRrter" recognition equations will be discussed under the headlng of Learning Techniques.

, , ~ , . , .: ' .~, : :, , ~ .

- .... : . ~ : , .
.

.
, ' ~ ' ' .' ' '~ ' ,. ' - , ' . , ~ : ..

3~

An extenslon o the dyadic reco~nition method ls the trladic method. Thls i~ again particulArly u~eful for recogni~lng touchlng and o~erl~pping numerals. Figures 14F, 14J, and 14K
lllustrate an ~ctusl case whlch occurred during the computer based development of thls invention.Three Examplea of almost ldentlcal sets of numerals here ~ubmitted for snalysls. The flr~t set conslsted of the numer~ls ONE7ZERO,and NINE touching,wlth no overl~p. The image ls ~hown in Figure 14F. The anslysis was msde Qnd three slgnlficant enclaves were' found. Thelr measurements formed the basis for a Recognition Equatlon named REa 109-Oi REQ
109-0 = ~Best Loop ~ Next Best Loop ~ 8e~t-In-South)~ -NEG~Be~t-In-West~

The image whlch i~ named Triad 109-0 scored 100 on REQ
109-Oi The next best score was produced by REQ O-l whlch scored 71 polnts.

Figure l~J shows a very closely related im~e in which the NINE overlaps the ZERO conslderably. Thls image wa~ also analyzed by the lnvention and REQ 109-0 scored 97 points. The next best ~core was 85 points. produced by RE~ a-Qi Figure 14R shows a related image in which the ONE overl~ps the ZERO and the NINE ~lso overl~ps the ZERO. This ima~e was - 63 ~
. . . -', ',, ' " ~ ,' " ' . ': - "' ' ~ ` ' ' .
, ' ., , : ,- . . , :
: :
., .

,:

,'. ' "'''~ .' . , 3~

analyzed and REQ 109-0 scored ~7 points. The next best score was 84 points, again produced by R~ 8-0.

Thsse hi~h performance results ~re by no means unexpected in terms of informatlon theory. A great de~l of information remains even though images may be mangled in complex w~ys~ In commercial practice a recognition equation for a triad such as the example ~ust glven will include references to many more than JUSt three enclaves, and there will be ~dditional negations to prevent lmsges contalnlng super-sets to score well.

Note thst in Flgures 14J snd l~K sn sddltions1 loop hss been formed by the overlapplng ZERO and NINE. Thls loop ha~s been essenti~lly ignored by the line~r scoring and directed vector technlque used throughout. Althou~h the absolute number o plxels in an enclave hss been reduced to an~unimportant parameter for the most part, the ~ize of an enclave rel~tlve to other close enclaves i~ to be carried through snd used for scoring and for negstion purposes where nece~say. This technique is the sub3ect o an importsnt feature in thls invention. The qusdr~nt scoring bresks down when the number of pixels is small, ~nd it ls therefore de~irRble to h~ve higher r=~olution in the images used with SQccadlc Fllck th~n i3 required ~or constr~lned lm~ge recognition. Encl~ve C of Figure 14H is an example of margin~l re~olutlon. The lnventlon tends to be ~elf recovering , however, since msrginsl resolutlon typically produces reJects ln this :

. , ' ' ' ~ ' ~ ~ ' '' ,. :

. , ~ :
-~
.
' ~ - . ' ~a~o lnvention, and re~ects ~utomstlc~lly csuse the m~chlne to re-proce~ the chsrscter~ uslng 8 higher re~olution.

::
~-~65;~

- : ~ , . , ' , ~:'; '.. ' ' - ~ . ' ' ' :

.. , ., . :: , .
. : : . . . . .

12~ 3~

ARl'IFIClAL FOV~'A AS A C~MPUTE~ ELEMENT

Flgure 15 illustrates the use of an Artlficlal Fovea 309 as an element in a computer. Within the dotted line 300 are shown in block form the maJor components of a modern serial ~an ~euman type computer. These components are ~ B~ckplane ~us 301, ~ R~ndom Access Memory (RAM) 302~ a Re~d Only Memory ~ROM) 303, an Arithmetic Unit 30~, a Peripheral Channel ~05, a Dlsk Controller ~06 ~including ma~netic disks and optical disks) and a Tape Controller 307, (including various forms of magnetic tape transports~. Such a general purpose computer is often augmented by ~pecial purpose processors, such as a "Vector Processor 30~
~examples are the Vector Proces~ors 310 which are attached to the Cray and Control " Data "supercomputer~"), and the Fast Fourler Transform Processor (offered comercially as a single card whlch plugs into the backplane bus ). These speciHl purpose proce.ssor5 typlcally are interfaced to the general purpo3e computer bD using ~ny one o several standard "backpl~ne bu~' protocols such as the -'MultiBus" and the "VM Bus. " They are typically sent a relatively ~m-ll amount of data and assigned a highly complex set of operations to perform on this data. Communication back and forth i~ almo~t invari~bly on an"interrupt" basi~ usl.ng the bu~
protoGol.

'' ' ' ` : . , . . .. . ~ : . . ..
. !, ~ . ~ ; , . ' . . .
. '..''~

' ' ' '. ' ~ ' " : ' An Artlficial Fovea may also be used aa a speci~l purpose processor in con~unction wlth a gener~l purpose computer. Figure 15 shows a block containing an Artificial Fovea "procsssor" ln the same configuratlon. Thus an Artlficla1 Fovea c~n be used in close con~unctlon wlth a ~eneral purpose computer and can be assigned ~obs by programa wrltten in a great many of the ~tsndard higher level langua~es auch as FOR'rRAN , "C", PA5CAL, ADA, etc. Special purpose compilers can also be written to utilize completely the peculi~r capabillties of the Artiflcial Fovea.

PLURALITY ~` A~TIFlClAL FOVXAE

Flgure 16 shows ~ block diagram of a psrallel conflguration showing a plurallty of Artl~ici~1 Foveae ~01, 402, 40~, 404, ~ON.
Thl~ flgure ls ~eslgned dellber~tely to be almllar to Fl~ure lR.
Pattern Source 10'. Thls may be any klnd of a scanner or image lift which can accept pictorial informatlon and output electrical sl~nal. whlch ~re responalve to the pictorial informatlon. These electrical signals ~re distributed to any one of the fi~e Artifici~l Foveae shown by a control block (not shown). The criterion for deciding to which AF 401...40Nto send the pattern is simply whether tho AF is busy or not. The plura1ity of AF

; '' ' ' ' `
'' : `,. ~ . ~ :
~'` ' ~' -` ` ' .

.
..
., , .. ~, ' .

' ~2~

send their outputs to the Recognition Scoring block 410 and thence to the Utilization Device 411.

The normal reason for having ~ plurality of foveae in the system is that the complexity of the electronic functions to be performed in this invention lS so gre~t as to make each Arti1cial Fovea almost invariably slower in the completion of a unit t~sk than the functions surrounding it. Thus, an image llft consisting of a paper transport and a column of photocells may very easily scen flve hundred alpha numeric characters per second, while a first generation Artlficial Fove~ mey be only ~ble to analyze 100 per second. Thus five AF are needed to keep up with the input. The same situation applies to the Recognition Scoring ~lock ~10. Recognition Scoring is much simpler ~nd more stralghtforward then the Artlficial Fovea and a five-to-one ratlo may also be appropriate for these functions.

It may be illuminating to consider why humans do not seem to have a plurality of Artiflcial Foveae. Two reasons may be glven for this: fir~t, the human fovea centralis may very well u~e an internsl parallel technology which makes it much faster than our serial electrical technology secondt the functlons following the fovea in human recognition may be much ~ore complex th~n the simple Recognition Scorlng shown hero~ For e~ample, the human br~ln is requlred to per~orm the entire set of three dimen~lonal computations in addltion to character recognitlon.

:

:

It 18 contemplated th~t later c~eneration Artlficial Foveae may very well per~orm much ~aster than the AF dlsclosed herein with thi~ lnventlon. A ma~or improvement may ~e realized by the use of "optical computing components, +or example.

RESOLUTION MODIFICArION

Figure 17A is a block diagram of a partial Reco~nition Machlne 500 showlng a Resolution Modiflcatlon component SOl.
Other element~ also shown are the Image Lifting 502, the auantlzlng 503, the Art~flcial Fovea 50~, the Recognltion Scorln~
505, and the Controller 50~ unlts.

The ob~ect of the Resolution Modlficatlon component ls to modify the number of pixels contained ln a p~rtlcular lma~e ~o that the lmage may ~e recognl2ed a~ ~peedlly ~nd wlth the le~at cost possible. Since the lenqth of time necessary for an~lysis goes up e~ponentially as a function of the num~er of pixels ln a charscter~ the Resolution ~odification element initially reduces the resolution as much as possible, consistent with lts recent experlence with earlier images.

:

~lgure 17B shows an exemplary orlglnal full slze scan, with ~ ... ..

~: .

, .
- , .

. ' : `' . ' ~ -~6~3~

gray scale shown as hexadecimal symbol~ In the ex~mple shown~ the original ima~e was scanned with a set of photocells which generated an analog output. The analog output was converted to a sixteen level digital representation. Conventionally, -the 16 levels are printed by using the codes O to 9, with A =10, B 11, C
= 12, D = 13. E = 14. and F = 15. Again by convention, a pixel label F is the blackest pixel. These symbols repre~ent the original conversion which is typically made within microseconds of each analog read-out. Many research proJects use 256 levels of gray scale for the initial conversion, but 16 levels i5 aatiafactory to lllustrate the theory. The term quanti~ation la reserved in this discussion for the ~inary choice which decides whether a pixel is to be considered black or whit~.

Contlnuing now with Figure 17B, let us assume that three resolution~ are available: they are a reductlon of three in the x dlrectlon and 3 ln the y directlon, or two ln x and two in y, and no reduction at all. Let us designate these reductiona as 3x3, 2x~ and 1x1. Fl~ure 17C ahowa a black~white quantization at the 1x1 re~olution level. There are many me~hod~ of determining the grayness of a pixel which should be optimally be called black only one o~ them will be dlscu~ed here. One of the ~impleat i~
to add up all the gray scale valu~s of the pixels in ~ particular im~ge and divide by the humber of pixels. I`he resulting quotient is the black~white quantlzing level.
The reason for this discussion of re~olution modification 3~

is to demonstritte that some images cannot be ~nalyzed properly u~ingg a reduced resolutlon. The image of Figure 17C is such an example. It should be clear to the reader that no "Test COR" can be found that will generate measurements that will correlate well with a "Best Loop" Template. The machinery shown in Figure 17A
wlll generate a ReJect in this case and the Controller block will trlgger the Resolution Modiflcation block to gener~te a new image at a higher resolutlon and send that lmage downline for an~lysis~ Since the only hlyher resolution available in this example is lx1, the machinery will quantize each pixel of Figure 17B independently of its neighbors. The result is ~hown in figure 17D.
The reader should be able to ob~erve that a number of good "Test CORS-' points are possible in the image shown in Fi~ure 17D, and therefore high correlation is po~sible with the "Best Loop" Template, and an acceptable score for the numeral "ZERO"
will be obtained.

: ' , '' ' ~ ' ' ' '' ' ' '. . , ' ' ' ' 3~

LEARNING CAPABIL~TIES

~ igure 18A shows a block diagram of a machine employing a Le~rning Module. This discussion will primarily deal with the methods by whlch a human teacher can be efficiently employed to help thi~ invention to le~rn from experience. However7 this doe~
not preclude the later description of a machine whlch can learn from experience without the uqe o~ a teacher.
The machine of Flgure 1~A lncludes mo~t of the functions that h~ve been discu~ed already, such as Image Liftlng, Quantlzing~ Resolution Modlflcation, Segment~tion Plotting~
Artificial F`ovea. Reco~nition Scoring~ and ReJec-t Storing.
The simple~t and most direct w~y or learning to occur i~
by having a human operator on iine with the SF reader. The operation o~ thi~ simplest mode is a~ follow~: when a re~ect occurs the help of the operator is soliclted by the machine by fla~hing the image of the unrecognlzed data on a screen ln front of the operator. The operAtor hit~ ~ key on the keyboard indicating to the machine what aymbol or symbols should be asslgned to the image. This information enables the reader to continue with its processing, including the printiny of Magnetic Ink Characters on the hottom of the check. Thus f~r~, the steps are well known and are utilized in the present st~ee-of-the-art.
The new steps are added by the Learning Module o~ Figure ~ 72 -i .

. ~ . '.

. . :

, ~2~3~

l~A. Brle+-ly, these steps lnclude sddlng one or more new Templates lf neces~ary and new Recognltlon Equations lf necess~ry. These new features will allow the SF machine to recognlze the im~ge autom~tically the next time it or ~ ~imil~r image appears.
Figure 18B shows the first simple illustr~tlve example.
Let us ~ssùme that one of our clients characteristically writes his zeroes with a large void on the right side. The Recognition Equation for generic zeroes requires a high score on the Best Loop templ~te. Such ~ hi~h score would norm~lly be generated using a COR located approximately at Pixel A in Figure 18B. Due to the 1~rge void in the NE quadrant, the Best Loop templ~te ~roduces a score of lesa than 80, and REO O-O likewise produces a score less than the re~ect level. In fact, no Recognition Equation3 can produce an acceptable level output. The image is re~ected, ~nd the lmage comes on line. The oper~tor indic~tes that the machine ~hould "learn" the new shape. The Learning Module sends the imsge b~ck through the Artl~lcl~l Fovea again, but rsquires more detailed reporting. It determines that, if COR loc~ted at Pixel A' is used, ~ high output from the BEST-IN-EAST template is generated. If this output is high enough, no new templ~tes need to be le~rned. All th~t is necessary ls for the Learning ~odule to generate a Recognltlon Equatlon for a new sub-class of zeroes~ called RE~ O-1. Thls equatlon will consist of the term BEST-IN-EAST, and several negatlons to prevent iA output from belng hlgh for the numeral~

; . ' :, ' , ' . , , -'~ ' . ,~ ' . -' ' TWOD FIVE, and others containing enclave3 opening to the East.
A Recognition Equation, (somewhat simplified) for this east opening zero iq ~8 follow~:
REQ 0-1 - BIE - NEGt~lW] ~ NEG~S~L]
See Figure 9 and related discus~ion for a review of these terms lf necessary. Note that a first ~est Loop (BL) is not negated, because a good score on ~L is ~till quite likely; note ~lso, however, that a high score on a Second Best Loop (SBL) must be negated bec~use the image may be An EIGHT with an ea~tern void on the top loop.
In ~ddition to negations, lt msy be necess~ry to add other terms which describe the relative positions required of the v~rious encl~ves. The single enclave case shown in Figure 18A
has been deliberately picked to be an initial simple introduction to operational learning problems whose solution~ require complex computations and much time.
The operator/teacher will be occasionally asked to make dlficult deciaions. These dlfficult decislons fall lnto several categories. First, if the re~ected lmage is ~o bad that the normal amount field (not the Courtesy Amount Field) must be consulted, the operator should probably not attempt to teach the SF machine the meanlng of the distorted image. Second, the image may seem readable enough to the operator~teacher but it may still be in confllct wlth some character unknown to the operator/te~cher. For example, if the alphanumeric character "c"
had been added to the SF machine's list of recognizable :, characters, the image of Figu~e 18B would be clearly ~ dangerou~
~hape to be called a "ZERO". Such hidden confllcts must be discovered and resolved before any new shapes are uccepted into the oper~tional recognition set. if the operator/teacher is ~lso a researcher ~kllled in the SF art, t~en it may be possible to m~ke such a quick decision. What is really necessary, however, 18 allow the Learnln~ Module to conduct ~ nearly exh~ustive con11ct check uslng a great many lmages h~vlng some enclave~
common to the new candidate. This conflict ch~sck will~ in general~ tske so long to perform that it cannot be performed ''on-line" while the reading of other bank checks is delayed.
Thus an economically viable m~chine will likely have the reJect corrected for the tally tape, but the image and correct symbol will be saved for off-line learning. Such off-line learning is called "Dream Learning".
The shape ~hown in Figur~ 18B has lllustr~ted a conditlon in which the teaching proces~ must provide a new Reco~nition Equation but does not have to provide any new Template~, slnce at least one high scoring template already existed for the enclave in question. In the early months o operation ln a new wrlting environment many encl~ve3 will be found that do not score well on the Templates which were installed at the factory. An example of thls may be drawn from Flgures 14E and l~F. the ~outh-openLng Enclave, whose COK is l~beled ''E" is an Enclave who~e measurements would not typlcally be installed at the factory. In order to write a use~ul Recognition Equation ~or the Dyads shown, .

.~
. ,' ' . . -..
,' ' '. : -it would be necess~ry to te~ch the m~chine a Templ~te whose correlation with similar measurements would produce a high score. The oper~tor/teacher would observe that Encl~ves A, B, C, and D produced good scores on slready recorded templates, but Enclave E did not. ~`here is a manual way and an automatic way to select the best COR for this new template. The manual way is to have the operator~tescher call up a program within the Lesrning Module which allows the operator~teacher to set a COR location into the machine using a "mouse", or "Joystick", or by uslng keyboard coordinate entry. The operatorJteacher should know tha~
the most distinctive scores for ~ three-slded enclave are generated when the COR is placed on the edge of the open side, near the middle. The automatic way is to have the progr~m generate the measurements for all the possible COR locations with the ~nclave and then pick the locstion which produces the most "useful" measurements for a new templa~e.
The definition of "useful" must now ~e discussed. If` the enclave being measured is fairly similar to many alre~dy stored, but ~ust happens to have a little quirk which causes it to score poorly on the standard templates. then the deflnltion of "useful"
should be to write a new Template which can be added to an existlng class of templates: ln this case that class of templates 1~ the South-openlng "~est-ln-South" cIa~. In thls case. the criterlon should be to choose the COR which correlates best with other templates already members o that class, while st the same time correlatlng the worst with templates which are members o~

`

3~

other classes. In other words, the new template ~hould have some generallty unless the enclave ls an absolute "oddball".
The case of the ibsolute oddball is more easily dealt wlth. The be~t COR will be the location whLch produces measurements whlch are the most dlfferent from any templates already stored. The new te~plate should be assigned a templAte number which is not part of a recognized class of templates, and a new Recognition Equation can be written automatically which calls for that specific template number (in addition to other terms in the equation).
In the case of the enclave which is made part of an existing class of templates, the existing Recognition Equation wlll be sati~factory, and a new equation ~hould not be wrlttsn.
It may be recognized that careless addition of new templates m~y cause conflicts in the same way that new Recognition Equations may do. New templates must also be checked against a known lmage data ba~e st length, preferably during "dream learning".

CAPA~ILlTlLS FOR REJECTING NONS~NS~ SHAPES
AND DISORDERLY NOISE

This invention has two ~ntrinsic levels of capability for reJecting shapes which have no meaning and images which contain only random or near-random noise. The first level is at the Template scoring level, and the second is it the Recognition ' . .
' , ' .

.
. ~ .

Equation le~
Consider the lmage shown ln Figure l9A. lhis is intended to represent a nonsense shape which scores poorly at the Template level. if the machine has been t~ught only Templates derived from the generic enclaves shown in Figure 7A then every Template Cros~s Enclave Score tTXES) will be low for all possible COR
locatlons. Since criterla for acceptlng a Best TXES can be made dependent upon sbsolute level and distance from the next best TXES, the machine can be adJusted to actually ignore shapes before even computin~ the Recognition Equ~tion Score (RESC).
Note, however, that situations in whlch a shape can be i~nored ~olely on the basis of poor TXE ~core~ will tend to be limited to images which have mostly short disconnected sections of black pixels. Thus, random or nearly random noise will be often skipped over quite quickly by a machine well-constructed from the ldeas of thi~ lnvention. When the fragmented lines are not random, however? a~ ls shown in Figures llB and llC, fairly good TXE s~ore~s occur and the images may be ~tisfactorlly recognlzed.
A more interesting case is illustrated in Figure l~B.
which shows a nonsense shape which scores richly at the TXE
level. Good scores would be developed for Best Loop (BL~, Second ~est Loop (SBL~, Best-In-East (BIE~, ~est-In-West (BIW~, and the four square corners. If this ~OLLAR SIGN (S> w~s part of the deslre~ set of characters to be recognlzed, ~ Recognitlon Equation (REa) would exist and would score highly. If on the ~ 78~-.

~ 3~3 other hand, the DOLLA~ SICN was not lntended to be recogni7ed, the REQ would not exist; REQs for shapes which are sub-sets of "~" must have negation terms included or else conflicts will occur. Examples of such sub-sets are the TWO, the FIVE, and the EIGHT. If 911 the sub-set REQs are properly negated, the DOLLAR
SIGN can be lgnored and treated as if it didn't exist.
In any structured appllcatlon to whlch my lnvention may be put, however, the system wlll be much improved by carrylng the re~ect logic one stage further. In a banking application where the Courtesy Amount Fleld ~CAF~ is to be recognized, thl-~ thlrd level of reJect control will consist of logical statements which wlll consider the loc~tion of control symbols, rectangular boxes, lines, decimsl points, cents indlcators, and nonsense images.
For exAmple, a really high confidence CAF should consist of R
recognized rectangular box surrounding a"S'' heading ~ string of hlgh scoring numerics. ~ollows by a DECIMAL POINr, followed by ~ome form of fractional dollar symbol. Many combln~tions of tbese elements exist ln todsy's acceptable handprlnted CAFs, ~nd the Field Acceptance Loglc must be able to handle these vsrlatlons. It mu~t also be ~ble to recognlze that R poorly scoring nonsense shape may be proper cause to re~ect the whole CAF in one c~se~ whereas in a different case a garbage sh~pe mAy be allowed.

,~

., ~' .:
'''-, '`

33~

SPECIAL MEASUREMENT FOR CLOSED TOP FOUR

ln the field of Optical Character Xecognition, ~ust as in the better known fie)ds o~ Physic~ and Philosophy, no single ull-encompassing formula has been found which can be used to solve all problems. It is the mar~ of a really good Philosophy that it provides a matrix in which unusual methods can be nurture~ and exercised.
Such an unusual method i8 required to help dlstinguish between the closed top FOUR ~nd the normal NINE. Because the moat lmportant new measurementa of thls inventlon are primarlly used to "generallze" or normallze the dlfferences between handwrltten topologlcal features, these new encl~ve me~urements must be supplemented by Special ~easurements when the topology of two clasae~ 19 too simllar. The human Fovea ha~ an enormous number of mea~urements which are not primarlly based on topology.
The "straightness" of a line is ohe maJor example~
In separating with high confidence an arbitrary exampla of a clo~ed top FOU~ from a normal NINE, this invention uses as many topologic~l features as lt can. Figure 7B lllustrMtes the use of the NE~ SE~ and SW Square Corners~plus the ~W TFlangulor Corner and the Best Loop. O these eatures, only the NE Square~
Corner and the SE Square Corner are reliably dl~erent. The Be~t Loop la ln~arlably the stronge~t feature presentr however, and the human fo~ea almost certainly measures more details about the ::

shspe of the Best Loop. One o the v1rtues of this lnvention is that it m~kes pos~lble accurate ~sses~ment~ of the 3hapes of selected enclaves as well ag thelr topology, A method called the "Polntyness Triangle" (PT Method~
will be expl~ined to illustrate the versatility of my invention.
The PT Method starts with the coordln~tes of the COR from which the BL feature was mea~ured. Three points are then established.
The first one is c~lled Pne. It is the further point from the COR within the NE quadrant. The dist~nce mea~urement i5 computed using the sum of the squares. The second polnt, ~lled Psw is loc~ted by finding the position of the enclave member which is furt~est away from the COR in the SW quadrant. The third point, c~lled Pae~ is similarly loc~ted in the SE quadr~nt. the line~
~re connected between the three points and they are called the Polntyne~s Trlangle. The Pointyness Ratio i~ the number of members with the snclave as a whole dlvided by the number of member~ within the tri~ngle. For Figure 20A, the Polntyness Ration is unlty.
Figure 20B shows the Pointyness Tri~ngle ~uperimpo~ed on ~ normal well formed NINE. The Pointyness Ration is approximately 2.5. Decisions as to which pixels ~re inside or outside can be made prstty much at the whim of the machin~
designQd this can be done by using equ~tion~ ~nd tntegral arithmeti~, or lt can be done by t~ble lookup. rhs import~nt thing 1~ to get the Pointynes~ Rat1o into the Recognition Equation~ for the closed top FOUR and the NINE in such ~ lineHr - 81 - ~

.
.

; ''', :

3~

w~y th~t ~ numer~l scoring on the borderline between the two cl~sse~ can be gracefully reJected.
The technique used in successfully demonstrating this feature w~s to cre~te two fe~tures used for neg~tion only: these festrures are called trl[4] ~nd trit9]. They are clipped and off-set functions of the Pointness RRtios, where the clipping and the off-set values are parameters that can be varied accordlng to learning perform~nce. Referring to Figure 8F, the Recognltion Equation uses a feature called Best Sh~rp Loop ~BSL). We now define BSL as equal to BL - tri[~]. Similarly~ Figure 8K uses ~
feature called Be~t Round Loop CBRL)~ We now define ~RL as equal to BL - trit9~.

SPECIAL MEASUREMENTS FOR PERIODS AND UNRESOLVED BLOBS

There are some type~ of images and defective lmages whlch mlght aeem to be dif~icult or impossible to reco~ni~e uAing the encalve measurement technique. The PERIOD (".") la an example of this derived from the OCR industry, since it normally has no lnterlor whlte pixel~. NINES, EIGHTS And SIXES are exAmples o~
numerals whlch often have signific~nt loops filled in due to cArele~sness or the use of too wlde ~ writing in trument~
Contrary to expectation, these images provide some of the most novel and provocative examples o~ the Saccadic Flick ~nd Artiflcial Fovea.
Figure 21A show~ A per ectly round PERlOD u~ing the 3~

conventional Black/White display. While it is true that four triangular corner type enc1aves are present, these are pretty small with respect to the area of the char~cter.
A much more interesting solution is to invert the color of the plxels the ch~racter wlll then appe~r ag in Flgure ~lB, and a hlgh quallty Best Loop enclave can be measured using the methods prevlously taught by thls invention. In order to separate the PERIOD c1aas from the BLOB-ZERO clas~ a term whlch compares blob slze~ can be used, ln additlon to contextual lnformation. The most slgnificant use o the PERIOD is as a DECIMAL POINT ln the Courtesy Amount Field of checks. BLOB ZEROs sometlmes occur ln the cents section of the amount, slnce that sectlon ls often written sm~ller ~nd with less care th~n the dollar amount.
Figure 21C lllustrates ~n EIGHT with ~ blob lower loop.
Thls condltlon ls fairly characteristlc o right handed peopl~ in a hurry. The lower loop becomes slanted and thin and narrow enough so th~t few or no whlte plxel~ can be resolved. The upper loop oten has a vold ln the NW, and a distinctlve Template, not lncluded ln the north openlng feature class or the esst opening eature class, should be taught to the machine. The resultlng RE~ should contaln ~t least the ollowing terms:
REQ BLOB EIGHl = ~ INVERTED BL ~ TtNW] ~ BIW )/3 Sl~nlficant extensions c~n be made of this "color fllp"
technlque ~way from the world of black/white ~nd into the world of gray scale images. Such lmages aFe most prev~lent in , .
.~ ' - ,. :
.: - ' '. ~ ' ~ ' 3~

so-called "scene analysis" ~nd "bin-picking"~ In these worlds the inform~tion ln the im~ges c~nnot e~sily bs displ~yed using only two levels o intensity, as i~ dons in OCR. In the "bin-picking" ~pplication ~bin-picking is tha #utomatic selection of a single part out o a bin containing many diverse shapes ~trewn in random placing~ signific~nt features may often be discovered by checking the images for encl~ves which occur only wlthin certain gray scale "wlndows". For example, a bowl may be illuminated in ~uch a way that the center of the bowl and the rlm show specular reflection, while being connected by means of are~s which can be recognized by selectlng only those pixels havlng an intermedi~te intenslty.

SPECIAL MEASUREM~NTS USING ABSOLUl`E
AN~ RELATIVE SIZES OF ENCLAVES

Figure 2~A, however, a shape i~ illustrated which may cau~e some conflict between the ZERO class ~nd the EIGHT class since it has a Best Loop and a Second Best Loop and Q potential Be~t-In-West arl~lng from the dimple on the left of the man loop.
My ln~ention provldes method~ for treatlng quch shapes in very much the same way that humans probably do. Fir~t, no COR can be ound in the dlmple that produces four good quadrant~: secondly, lf a marglnal sized enclave is found, it can be compared to the sizes of other encl~ves assoclated with the image:and either entirely lgnored, or may be used as a "~poiler" to prevent . , 3~

substltutlons.
Figure 2~ ls an example of a complex function derlved from relatlve enclave sizes. Thls p~rticular function i~ zero when the ratio is le~ than 15~., and zero again when the ratio is between 65 and 100%. In between, the TLLL Function peaks at 30~.
Such a function lS useful as an Assertion term in a Recognition Equstion psrticulsrly designed or recognizing this shape, and it may also be used as a Negatlon term ln other RE~s.
Whlle there has been shhown and described the preferred embodiments of the invention, it will be appreclated that numerous modlfications snd adaptations of the invention will be readlly apparent to those skilled in the art. For example, in early forms of the invention~ I used computer software techniques to ~uccessfully perform substsntially all of the fuunctions dlsclosed herein. It is intended to encompass all such modifications and adaptations as may come within the spirit and scope of the claims appended hereto.
WHAT IS CLAIMED lS:

. ~ "''' :

Claims (59)

1. In a hand print character recognition system comprising, a) means for creating an electrical binary black/white "image" of one or more hand printed characters, b) means for selecting a plurality of centers of recognition (CORs) within said binary black/white image as reference points and measuring the characteristic enclave of the black/white image immediately surrounding the CORs, c) means for storing a library of templates of said measurements around the CORs for a plurality of known exemplary character images, d) means for comparing said library of templates to corresponding measurements made around the CORs of images whose class is unknown to produce "template scores" proportional to the similarity of the enclaves of the known image to the enclaves measured by templates, e) means for expressing the generic shape of a character as being a "character equation" involving template scores developed on an unknown image, and f) means for evaluating each character equation, including comparing the values of such equations, and selecting the best value to determine the generic name of the unknown character.

2. The handprint character recognition system defined in claim 1 wherein said b) means for selecting includes means for measuring the enclosure characteristic of each pixel within an enclave which is roughly related to the COR so each pixel has its own enclosure descriptions in four quadrants.

3. The handprint character recognition system defined in claim 1, including means for choosing and manipulating measurement parameters such that very important characteristics, including the degree of black enclosure around the COR are normalized to be independent of relatively unimportant characteristics, such as size and distance of the black enclosure from the pixel or COR under consideration in the enclaves.

4. The handprint character recognition system defined in claim 3, including means for measuring and recording for each white pixel in the quadrant of each said enclave whether that pixel is bounded by a black pixel to the north, bounded by a black pixel on the north-west diagonal, bounded to the east, all the eight combinations of that bounding.

5. The handprint character recognition system defined in claim 4, including means for counting pixels with similar boundedness and for normalizing their number by computing ratios of their number to the total number of white pixels in the enclave with which it is associated.

6. The handprint character recognition system defined in Claim 4, including means for using said measurement parameters to choose the location for the centers of recognition, to locate templates themselves to recognize useful CORs.

7. The handprint character recognition system defined in Claim 4, including means for relaxing said measurement parameters so as to use them to select centers of recognition which are less narrowly defined than the measurement parameters needed in the character equations.

8. In a hand print character recognition system comprising, means for transporting media bearing hand print characters, a photosensitive device, an optical system for focusing images of said hand print characters upon said photosensitive device, scanning means for converting the optical signals focused on said device to electrical signals, analog-to-digital converters for changing electrical grey scale levels associated with each individual small picture elements (pixels) in said image to digital values, and decision means for quantizing said pixels to be either black or white and creating a binary black/white "image" of a character or group of characters, the improvement comprising a) means for selecting a plurality of centers of recognition (CORs) within said binary black/white image as reference points for measurement of the characteristic enclave of the black/white image immediately surrounding the CORs, b) means for storing a library of templates of said measurements around the CORs for a plurality of known exemplary character images, c) means for comparing said library of templates to corresponding measurements made around the CORs of images whose class is unknown to producing "template scores" proportional to the similarity of the enclaves of the unknown image to the enclaves measured by the templates, d) means for expressing the generic shape of a character as being a "character equation" involving template scores developed on an unknown image, and e) means for evaluating each character equation, including comparing the values of such equations and selecting the character equation which matches the shape of the unknown image to determine the generic name of the unknown character.

9. The handprint character recognition system defined in claim 8 wherein said means for selecting includes means for measuring the enclosure characteristic of each pixel within an enclave (which is roughly related to the COR so each pixel has its own enclosure description in a measurement space in four quadrants).

10. The handprint character recognition system defined in claim 8 including means for choosing and manipulating measurement parameters such that very important characteristics, including the degree of black enclosure around the COR are normalized to be independent of relatively unimportant characteristics, such as size and distance of the black enclosure from the pixel or COR under consideration of the enclaves.

11. The handprint character recognition system defined in claim 10 including means for measuring and recording for each white pixel in the quadrant of each said enclave whether that pixel is bounded by a black pixel to the north, bounded by a black pixel on the north-west diagonal, bounded to the east, all the eight combinations of that bounding.

12. The handprint character recognition system defined in claim 11 including means for counting pixels with similar boundedness and for normalizing their number by computing ratios of their number to the total number of white pixels in the enclave with which it is associated.

13. The handprint character recognition system defined in claim 11 including means for using said measurement parameters to choose the location for the centres of recognition, to locate templates themselves to recognize useful CORs.

14. The handprint character recognition system defined in claim 11 including means for relaxing said measurement parameters so as to use them to select centers of recognition which are less narrowly defined than the measurement parameters needed in the character equations.

15. In a hand print character recognition method wherein an image of a character is converted to an electrical binary black/white "image" of a character or group of characters is the improvement comprising, a) selecting a plurality of centers of recognition (CORs) within said binary black/white image as reference points And measuring the characteristic enclave of the black/white image immediately surrounding the CORs, b) storing a library of templates of said measurements around the CORs for a plurality of known exemplary character images, c) comparing said library of templates to corresponding measurements made around the CORs of images whose class is unknown to produce "template scores" proportional to the similarity of the enclaves of the known image to the enclaves measured by templates, d) expressing the generic shape of a character as being a "character equation" involving template scores developed on an unknown image, and e) evaluating each character equation, including comparing the values of such equations, and selecting the best value to determine the generic name of the unknown character.

16. The handprint character recognition method defined in claim is wherein said step a) selecting includes measuring the enclosure characteristic of each pixel within an encalve so each pixel has its own enclosured descriptions in four quadrants.

17. The handprint character recognition method defined in Claim 15, including choosing and manipulating measurement parameters such that very important characteristics, including the degree of black enclosure around the COR are normalized to be independent of relatively unimportant characteristics, such as size and distance of the black enclosure from the pixel or COR
under considerattion of the enclaves.

18. The handprint character recognition method defined in Claim 17, including measuring and recording for each white pixel in the quadrant of each said enclave whether that pixel is bounded by a black pixel to the north, bounded by a black pixel on the north-west diagonal, bounded to the east, all the eight combinations of that bounding

19. The handprint character recognition method defined in Claim 18, including the steps of counting pixels with similar boundedness and normalizing their number by computing ratios of their number to the total number of white pixels in the enclave with which it is associated.

20. The handprint character recognition method defined in Claim 18, including the steps of using said measurement parameters to choose the location for the centers of recognition, to locate templates themselves to recognize useful CORs.

21. The handprint character recognition method defined in Claim 18, including the step of relaxing said measurement parameters so as to use them to select centers of recognition which are less narrowly defined than the measurement parameters needed in the character equations.
21a. The handprint character recognition method defined in claim 15 including the steps of determining whether a selected pixel is a member of a selected enclave.

22. In a hand print character recognition method in which a document bearing handprinted characters to be recognized, said documents are transported through a reading station having a photosensitive device, an optical system for focusing images of said hand print characters upon said photosensitive device, scanning means for converting the optical signals focused on said device to electrical signals, analog-to-digital converters for changing electrical grey scale levels associated with each individual small picture elements (pixels) in said image to digital values, and decision means for quantizing said pixels to be either black or white and creating a binary black/white "image" of a character or group of characters, the improvement comprising the steps of:
a) selecting a plurality of centers of recognition (CORs) within said binary black/white image as reference points for measurement of the characteristic enclave of the black/white image immediately surrounding the CORs, b) storing a library of templates of said measurements around the CORs for a plurality of known exemplary character images, c) comparing said library of templates to corresponding measurements made around the CORs of images whose class is unknown to producing "template scores" proportional to the similarity of the enclaves of the unknown image to the enclaves measured by the templates, d> expressing the generic shape of a character as being a "character equation" involving template scores developed on an unknown image, and e) evaluating each character equation, including comparing the values of such equations and selecting the character equation which matches the shape of the unknown image to determine the generic name of the unknown character.

23. The handprint character recognition method defined in claim 22 wherein said step a) includes means for measuring the enclosure characteristic of each pixel within an enclave (which is roughly related to the COR so each pixel has its own enclosure description in a measurement space in four quadrants).

24. The handprint character recognition method defined in claim 22 including the step of choosing and manipulating measurement parameters such that very important characteristics, including the degree of black enclosure around the COR are normalized to be independent of relatively unimportant characteristics, such as size and distance of the black enclosure from the pixel or COR under consideration of the enclaves.

25. The handprint character recognition method defined in claim 24 Including the stop of measuring and recording for each white pixel in the quadrant of each said enclave whether that pixel is bounded by a black pixel to the north, bounded by a black pixel on the north-west diagonal, bounded to the east,, all the eight combinations of that bounding.

26. The handprint character recognition method defined in claim 25 including the step of counting pixels with similar boundedness and for normalizing their number by computing ratios of their number to the total number of white pixels in the enclave with which it is associated.

27. The handprint character recognition method defined in claim 25 including the step of using said measurement parameters to choose the location for the centers of recognition.
to locate templates themselves to recognize useful CORs.

28. The handprint character recognition method defined in claim 25 including the step of relaxing said measurement parameters so as to use them to select centers of recognition which are less narrowly defined than the measurement parameters needed in the character equations.

29. A pattern recognition system in which an array of cells store information in two or more levels representing two or more changes of values of each parameters defining a pattern to be recognized, means for examining cells in said array to locate a center of recognition cell, means for determining the boundedness of other cells having specific relationship to said center of recognition cell and producing boundedness measurements to recognize said pattern.

30. The pattern recognition system defined in claim 29 including classification means for each recognized pattern.

31. A pattern recognition method in which information in two or more levels representing two or more changes of values of each parameters defining a pattern to be recognized is stored in an array of information storage cells, examining cells in said array to locate a center of recognition cell, determining the boundedness of other cells having specific relationship to said center of recognition cell, and producing boundedness measurements to recognize said pattern.

32. An artificial fovea comprising, an array of storage cells, loading means for loading an image information which has one or more enclaves, in black/white pixel format, into said storage cells, one pixel per storage cell, means for selecting one of said white pixels as a center of recognition (COR) to constitute a test COR location, logic means for determining whether each white pixel in said image is bounded in predetermined directions by a black pixel in said array of storage cells, summing means for summing the number of pixels which are bounded in each said predetermined direction, respectively, and producing enclave measurements proportional to each respective percentage of the total number of white cells in said array, and means for transmitting said enclave measurement to a utilization device.

33. The artificial fovea defined in claim 32 wherein said utilization device includes, means for selecting and storing a plurality of sets of said enclave measurements, each selected enclave measurement constituting a template, means for comparing a subsequent said enclave measurement with said plurality of templates and producing a set of ordered values bases on the results of comparing said subsequent enclave measurements with said plurality of sets of templates, and means for storing said ordered set of values.

34. The artificial fovea defined in claim 33 wherein said means for selecting includes means for causing each white pixel in an enclave to be said test COR.

35. The artificial fovea defined in claim 34 including means for selecting the best scoring test CORs, and means for transmitting the selected test CORs to said utilization device.

36. The artificial fovea defined in claim 35 including means for preventing use of all white pixels in the highest scoring enclave in subsequent scoring of test CORs for remaining enclaves.

37. The artificial fovea defined in claim 32 wherein said logic means includes means for limiting the boundedness determination in diagonal directions from the selected CORs.

38. The artificial fovea defined in claim 35 including means for preventing use of all white pixels in the highest scoring enclave in subsequent scoring of test CORs for remaining enclaves.

39. The artificial fovea defined in claim 32 including terminating propogation of membership.

40. The artificial fovea defined in claim 32 including means for detecting when a column of white pixels is not bounded to the north and south and [excluding said column of pixels from being deemed a part of the enclave].

41. The artificial fovea defined in claim 32 including emans for detecting when a column of white pixels is not bounded east and west [and excluding said column of pixels from being deemed a part of the enclave].

42. The artificial fovea defined in claim 32 including means for initiating the propogation enclave membership.

43. The artificial fovea defined in claim 42 including means for terminating the propogation of said enclave membership upon detecting a predetermined configuration of black/white pixels.

44. A pattern recognition system comprising, an array of storage cells, loading means for loading image information containing a pattern to be recognized which has one or more enclaves, in black/white pixel format, into said storage cells, one pixel per storage cell, means for selecting one of said white pixels as a center of recognition (COR) to constitute a test COR location, logic means for determining whether each white pixel in said image is bounded in predetermined directions by a black pixel in said array of storage cells, summing means for summing the number of pixels which are bounded in each said predetermined direction, respectively, and producing enclave measurements proportional to each respective percentage of the total number of white cells in said array, means for receiving said enclave measurements and reconizing said patterns therefor, and means for transmitting said enclave measurement to a utilization device.

45. The pattern recognition system defined in claim 44 wherein said utilization device includes, means for selecting and storing a plurality of sets of said enclave measurements, each selected enclave measurement constituting a template, means for comparing a subsequent said enclave measurement with said plurality of templates and producing a set of ordered values based on the results of comparing said subsequent enclave measurements with said plurality of sets of templates, and means for storing said ordered set of values.

46. The pattern recognition system defined in claim 45 wherein said means for selecting includes means for causing each white pixel in an enclave to be said test COR.

47. The pattern recognition system defined in claim 46 including means for selecting the best scoring test CORs, and means for transmitting the selected test CORs to said utilization device.

48. The pattern recognition system defined in claim 47 including means for preventing use of all white pixels in the highest scoring enclave in subsequent scoring of test CORs for remaining enclaves.

49. The pattern recognition system defined in claim 4 wherein said logic means includes means for limiting the boundedness determination in diagonal directions from the selected CORs.

50. The pattern recognition system defined in claim 47 including means for preventing use of all white pixels in the highest scoring enclave in subsequent scoring of test CORs for remaining enclaves.

51. The pattern recognition system defined in claim 4 including terminating propogation of membership.

52. The pattern recognition system defined In claim 32 including means for detecting when a column of white pixels is not bounded to the north and south.

53. The pattern recognition system defined in claim 44 including means for detecting when 8 column of white pixels is not bounded east and west.

54. The pattern recognition system defined in claim 44 including means for initiating the propogation enclave membership.

55. The pattern recognition system defined in claim 44 including means for terminating the propogation of said enclave membership upon detecting a predetermined configuration of black/white pixels.

56. The pattern recognition system defined in claim 52 including means for excluding said column of pixels from being deemed a part of the enclave.

57. The pattern recognition system defined in claim 53 including means for excluding said column of pixels from being deemed a part of the enclave.

58. The handprint character recognition system defined in claim 1 wherein said handprint characters include one or more overlapping characters and said c) means for storing a library of templates includes templates corresponding to said one or more overlapping characters.

59. The handprint recognition system defined in claim 1 including means for determining whether a selected pixel is a member of a selected enclave.