CA1308486C - Video image processing - Google Patents
Video image processingInfo
- Publication number
- CA1308486C CA1308486C CA000566156A CA566156A CA1308486C CA 1308486 C CA1308486 C CA 1308486C CA 000566156 A CA000566156 A CA 000566156A CA 566156 A CA566156 A CA 566156A CA 1308486 C CA1308486 C CA 1308486C
- Authority
- CA
- Canada
- Prior art keywords
- image
- vectors
- array
- area
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/008—Vector quantisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/167—Detection; Localisation; Normalisation using comparisons between temporally consecutive images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/94—Vector quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
- Closed-Circuit Television Systems (AREA)
- Color Television Systems (AREA)
- Image Processing (AREA)
Abstract
ABSTRACT
VIDEO IMAGE PROCESSING
A video processor identifies the head from a head against background scene using Vector Quantisation operating on a composite vector codebook including "head"
vectors and "background" vectors.
The codebook is initially derived by storing several frames of an image training sequence (1); diffferencing adjacent frames (2); thresholding the difference data against a luminance threshold (3); median filtering (4);
clustering the data sets (5); determining the minimum rectangle which will contain all the remaining non-zero pels (6); generating a border around the rectangles (7);
generating a head codebook from the pels of the original frames of the image that fall within the borders (8);
s similarly generating a background codebook; and finally generating a composite codebook (9) in which "head" and "background" vectors are distinguished by flags. Then the head is tracked by Vector Quantising (10) the image with this codebook (9); and analysing the flags (11) (12) to obtain the head position (13).
VIDEO IMAGE PROCESSING
A video processor identifies the head from a head against background scene using Vector Quantisation operating on a composite vector codebook including "head"
vectors and "background" vectors.
The codebook is initially derived by storing several frames of an image training sequence (1); diffferencing adjacent frames (2); thresholding the difference data against a luminance threshold (3); median filtering (4);
clustering the data sets (5); determining the minimum rectangle which will contain all the remaining non-zero pels (6); generating a border around the rectangles (7);
generating a head codebook from the pels of the original frames of the image that fall within the borders (8);
s similarly generating a background codebook; and finally generating a composite codebook (9) in which "head" and "background" vectors are distinguished by flags. Then the head is tracked by Vector Quantising (10) the image with this codebook (9); and analysing the flags (11) (12) to obtain the head position (13).
Description
J
This lnYention r~la~es ~o analyBl~ ~nd pro~ ing of video i~age~.
A video ~nage (which wlll be undaratood to enco~pass fros~en images 8tlah as facslmlle imag~s~ in aDdition to movin~ image~) will in g~neral lnclude at le~st one object which ~6 of in~ere~t and a ~ackground~ of l~sser intere~
(and hence o~ 1~BCer i~portance).
To analyse the l~q~, e.g. d~tect the o presence/absen~e or po~l~ion of a partlc~lar ob~ect of intere~t, ~8 oft~n des~rahle in a variety of appli~ations.
In an lmage transml~ n ~ystem an improved picture quality might be ~chieved lf dat~ r~lating to importan~
parts o~ the ~cene, I.e. object6 of ~nter~st, is coded u6inq relatiYely more bit~ ~han data r~lating to unimportant ~l.e. baokground) parts. For e~ample, ln a videophone syste~ a typical ~mage comprises a head and ~ho~lders again~t a back~roulld, ~nd the face area o~ the he~d i~ v~ually th~ ~ost important i~ is thus desirable to ~e able to identi~ the head area from the ~houlders and ~ackground ~o as to be able to proces~ the head at a higher refre6hment rate than thB re~t, 60 ~hat the impre6sion o~ gmooth h~ad motion is conveyed. The ability to loc~te a head wi~hin a head and shoul~ers ~cene can Z5 thus be u~ed to mo~ifg the spatial allo~ation of video da~a, enablln~ a degree of vi~ual lmport~n~e to b~
attr~buted to blo~kfi within the data.
Also, if th~ position of an ob~ect 15 accur~tely t~acked with ti~e ~t wlll be possible to prQdict lt~
motion, thu~ ~llowin~ "~ot~on co~pensated'~
one way of identifying dlf~erent reglon~ of an ima~e ls to utili~e the method proposea ~y Nagao ~. Nayao -"Pictur~ recogni~lon and data struct~re", ~raphlc 17^~ o Language~ - ~d Nake and Rossen~ , 1972). Thig ~ethod ha~ ~een u~ed ~n a viBeophone type system, on an im~ge of a head and shoul~erG agai~t a backyroun~. Some eucces~
was achleved in determininq the 81~e5 of th~ hsAd when the sub~ect was clean shaven, ~ut very littl~ succe~ w~
achieved in other ~ases; so this m~thod ~ n~t considered relia~le enough for the ~ B of an area ldentlfylng method~
Conventlonal coders, for instance hg~rid discrete o c~sine transfor~ coders, use no ~scene ~ontsnt~
~nfor~atlon ~o code th~ da~a ~ithln the ~cene, so e~ch p~rt of tne ~ene i~ operat~d on a3 i~ lt has the ~am~
visual lmpor~ance a~ every other part.
Other i~age analysls appll~a~i~n~ are m~nifold (for ex~mple, ln auto~ted manufacturing 6ystem3).
It is also known to code video image~ for transml~sion using Ye~tor Quantl-~atlon (Y~). In ~Q
coding, the image 1~ r~presented initially by an array of digLt~l d~ta corre~ponding to ~he i~a~e frame. ~locks o~
array points ~nsub-array~") are compared wlth v~ctors from a codebook, ~nd the be~t~matching vector sel~Gted uglng a ~lQas~ s~uares" di~ference criterion. A code des.i~n~ting this vector i.s then ~ran~mitted ~o represent the sub-array~ At the receiving end the indicated vector is ~elected fro~ an identical codebook and displayed.
Ths un~erlying principle o~ the inven~ionr however, is ko u~e VQ as ~n ~dent~fica~ion (e.g. ob~ect location) me~h~d. The extent of ~h~ variouR aspec~ of the lnvention are d~ined in the clai~s appen~ed hereto.
The different areas of a vlaeo image, when vect~r quant~sed (VQ), can be operated on differently provided each entry in the VQ codebook ha~ an associated flag indica~ing which ~rea ~hat entry repr~RBn~s~ So in the example of the videophone two differen~ flag entries ~r~
required, one for the head and the other for the remainder of the scene.
An embodiment of the invention will now be described by way of non-limitative example concerned with the identification of a head in a head and shoulders against a background scene, with reference to the accompanying drawings in which:
Figure l is a block diagram illustrating the initial stages of operation of parts of a coder embodying the invention;
Figures 2a-g show schematically various stages in the training sequence used to derive the codebooX;
Figure 3 is a block diagram illustrating the operation of a coder embodying the invention;
Figure 4a shows schematically a frame to be analysed;
Figure 4b illustrates the sub-array blocks used in vector quantising figure 4a;
Figure 4c shows the state of flags corresponding to the vector quantised image of figure 4b, Figure 4d shows schematically the result of analysing the frame of figure 4a according to the invention;
Figure 5a shows schematically a coder embodying the invention; and Figure 5b schematically shows the differential coder of Figure 5a in more detail.
To enable the invention to operate, it is necessary to have provided a composite codebook which includes vectors flagged as being "head". Preferably others are flagged as being "background". It is possible to derive a "standard" codebook for either an average or a given speaker, but to allow flexibility and greater accuracy of identification, this codebook is derived at the start in an initial "training" sequence. A preferred way of implementing such a sequence will now be described.
- 3a -To generate "head" and "background" parts of the codebook, it is necessary to unambigously obtain some 1 ~,, , '1 ~,, ~head only" data an~ "~ac~ground onlyN data; a crud~
lni~ial head dete~tion algorlthm i~ requlred.
Ref~rring to figures l and 2r ln o~der to d~tect the head, d~glt~l data representing sa~eral ~ontig~oug ~ es of the head and shoulder~ imaga are ~aptur~d~ for ~nstan~e ln a ~ore l. One of thess ~rames i5 ~epicted in ~lg~re 2a. Tnis data doe~ not need to b~ extrem~ly accur~t0, bu~
rather repre6entatlve~
on the as~mption that the prime ~oving ~res~ withln th~ data sequence are ~iroc~ly associat~d Yith th~ head area, frame dl~ferenc~ng 2 is applled to the data repres~ntlng each ad~a~ent pair o~ rames. Thl~ process typically yields a SBt of difference data for ea~h adiacent pair r~presentin~ mo~ing areas together with random noi~e acros~ the whole image area.
For all pic~ure ~lements (pel~) reprQ~en~ed by each set of difference da~, each pel above a given ~hreshold value of intensity i8 ~et to maximu~ intensity (255) and ~ach pel below the thre~hold i~ se~ to m~ni~u~ intenslty 1O)~ This 'threshold~ngl 3 r~moves a large quantiSy of the random noise and some of the ~ov~ng areas.
~edian filtering 4 i5 ne~t applied to each s~t of .
diffQrence data, which very e~fe~tively re~ove~ ~os~ of the re~alning rando~ noise, but erodes only s~all amounts of the ~oving area~.
The lmage represent~d hy each set of d~ta at this stage will rarely provide a cl~r outline of the head;
unless the hea~ to ~ackground con~ra~t iB very hlgh and the mov~ment of the head between ad~acent ~rames i9 more than one pel. Often ~nly one ~i~e and the top of the head may he deplcted as ~hown in f~gure ~b.
General~, the movlng area~ ~111 be clu~t~red ln regions ~round the head a~ea, hut gome isolated clu~ters may arl6e due to moti~n in other areas of the image.
' r 1,,, 1~ ) ~ 5 ~
A clustering proc~s~ 5 iB u~ed to re~ove so~e of the lsola~ed clu~ters: two orth~onal hl~to~ra~s are qenerated, one representing the number o~ '~oving' pels in the columns of ~he ima~e represQnted by thQ d~ta ~nd one repre~entlng ~he number of ~oving pe~s in the ro~s of the imAqe represente~ by the data. The first order momen~
are calcula~d an~ ~he 'centre o~ qravity' of the lmage determined, a~ ~hown in f~gure 2c. A ~ect~ngl~ ls then generated, centred on these co-ordinate~, of ~ch o dimensions that a given pe~centage o~ ~ov~ng area iB
incl~ded wit~n 1~, see ~gur~ 2d. The pel~ ~emain~ng ou~side thi~ rectangle are ~t to zero in~en~i~y, figur~
2e. By a 6uit~1e choice o~ rectangl~ l~ola~ed clu~ter~
ars re~oved by this proce~s.
Constraint~ are impo~ed on the ~election of the rectangles in or~er to reduce the occuren~e of in~orre~t rectangle~. Since a vQry small movement of the head between one fra~e and the next may produce a very small rect~nqle, the r~te of change of ~i~e of the rect~n~le from ~ne s~t of d~t~ to the next ~ restricted: either each of ~he boundary lin~R of the rectangle a~e ~on~tra~ned to lie w1thin a ~mall di~tAnce of the corresponding boundary in th~ ~mmediately p~eceding set of data~ or the ~ ate of change o~ the slze of the rectangle is link~d to the frame difference energy (eg.
the ~4uare of the d~feren~e d~t~ if ~he d~fference energy fs ~mall the change is kept small, ~u~ ~ the difference energy 1~ large the rate of change may be greater.
The rectan~le ~ rectangle~ are used because they require very few bit~ of data to de~ine - i~ then ~hrunk if neces~ary, a~ 6 in f~gure 1, and as shown in figure 2f, ~o become the ~mallest rectangle ~hat c~n ~e placed around the da~ to enclo~e all ~he r~maining non-zero pels. This .11 () ~. J ' rect~nqle 1~ ~sum~d to represent an approxl~ate model ~f th~ head.
A border i~ then created, at 7 in figure l, around the final rect~ngl~, a8 ~hown ln flqur~ 2~. Thl~ border s defines an exclu5ion zone from where no data wlll lat~ be ta~en. This ensures tha~ ~hen the bo~der i appllsd to the reepective frame o~ the orlglnal Image the data outside the ~order will be exclu~lvely h~ad data and the data outslde the border will be e~clu6i~ely bac~groun~
o daea~
~ fiv~ fr~mes of data are initlally captured in the store 1, th~n four ad~ac~nt palr8 of frames are analysed and four set~ of da~a result. After th~ four borders have b~n 3et 7 the head area data and the b~ck~round area da~a 1.5 are extracted from the fir~t four frame of th~ origin~l i~age respectively and tbe Linde-Buso-Grey algori~h~ i~
applied to generate ~ VQ codebook for each area 8, for example, a g bit background codehook an~ lO bit head cod~book (i.e. cod~oo~ containing reep~tiYely 29 ~n~
21~ en~ries~. The t~o code~sok~ ~re then ~h~bined ~ to ~o~m one codebook, ~ach ~ntry of which has ~n a~ocia~e~
flaq indi~ating it~ origln.
R~ferrlng now to Plg~r~s 3 and 4a-d, af~er this trai~ing sequence 1~ completed, the composite codeb~ok lS
us~d to locate ~he head in succ~sl~e lmag~ frames. The VQ coder operates ~ust a~ it would in a prlor Art ~ystem u~ng YQ as the transmis~lon coding, bu~ ~or each bloc~ of pels coded lO, the code generated will include ~ flag (for exa~ple, ~he first digit3 indi,~ating whether that ~lock ls ~head" or "background't so that ~he head position i6 known for each frame.
It will of ~ourse be appre~iated ~h~t when the codebook i8 deri~ed ~t the cod~r as indic~ted above, VQ
cannot be used as the ~ans~ission code ~unles~ this '` ^ 11 ~
U ~) - 7 ~
~odehook is mad~ known to the ~ecoder fir~t by transmittlng an indLcatlon vf the ~ectorB)~
~lnce ~he quantl~ation proc~s 1~ inhQrently approxl~ate, it wlll be ~ppreciated tha~ occa3ionally blocks from the he~d p~rt of the im~ge may best match a vector fro~ ~he "background" part of the codebook, or Vice ver~a. The ~ctu~l ~dentification of the head will thu~
u8ually lnvolve i~noring i~olated nbead" ~locks U~ing erosion and clu~tering llr 12 (~or example, as ~e~cri~ed 0 ~bo~e~, or de~ignatinq the ~re~ with the hlghest concentrat~on of "lle~d~ ~loc~s a~ the ac~ual head.-Another method in~ol~e6 detec~ing lsolate~ "head~l block~ and ~hen ~xaminlng the error between the block and the "head" vector, and that hetwe~n the blo~k and the be~t-ma~ching "backqro~nd" vQctor, and if the t~o scor~s are ~imilar (i~e. there iæ ambl~ulty ab~u~ whether the ~lOCK ~g llheadn or "background~l), reflagging t}le block t~
bac~roundll ins~ead.
If the head blocks are too scattered, it may be that the ~odebook is insuff~c~ent to charac~erise the head. ~n this ~ase, ~ re~raintng seq~ence may bP emp~yed t~
regenerate th~ cod~ook.
This retraining sequ~nce may ~ither simp~y be a furthe~ sequence o~ the kind descri~ed ~ove, or it may 2s attempt to ilnpro~e trather ~han simply redefine~ the codebo~lc. Fnt~ ~x~m~l P, ~ ~ount n~ e kcp~ ~ tho numbe~
of ~lncorre~ e. scattered) as oppo~ed ~o "c~r~ect"
(l.e. conc~ntratod ln the head area) occurenes of each ~ector, and ~he scatter may this be reduced hy reiecting fro~ ths codebook vec~ors which occur incorrectly too often .
or, alt~rnativel~, the approxi~ate head loc~t.ion derived by lo~atlng the greate~t concentratlon of "head"
~ 8 blocks may be used, ln the sa~e ~anner a-q d~crlba~ above, a~ an area ~or generating a new "head" codeboo3~.
The~e latt~r approacha~, in which VQ ~oder "learn~"
from e~ch retraining ~e~U~nce, are pr~ferred on groun~s o~
s accura~y, Figure 5 shows a block di~gram of ~ ~ideo coding apparatus (eg ~or a video telephone ) embody~ng the inst~ntion. Yideo ~i~nal~ are fed ~rom an input ~0 to a frame 6tore 21 where ~nd~vidual p~cture element Ya~Ues are recorded ~n respectlY~ 6torP lo~tlon~ ~ that deslred sllb-array~ of pel~ are accessible for fu~t~ler p~oce~sing.
The sub-arra5r ~læes may typically be 8 x 8. In an ~nitial~ trainlng, phasQ of the app~tus a tralning control llnlt 22 - wh~h may for ~pl~ b~ ~ suitably lS progra~m~d microproce660r sy8t~ - carrie~ o~t the code~ool~ generatlon method descri~ed above, ~nd enter6 the ~e~tors (~nd flag~) in a VQ codebook store 23. It will be un~erstood that the YQ proce~ ~nYo~Ye~ rnatching 8 x 8 su~-array to th~ neare~ one of the store~ of ~rectors, viz. a nu~er of B ~ 8 patterns w~ich are consis~ently fe~er ln nwn~er Shan the maximum p~ssible n~ er (2~4) of such patt~rn~.
In the codlng phase of the apparatug, the matchinq i~ c~rried out by VQ c~ntrol logi~ ~4 ~hich re~ei~,re5 succe~ive sub-arrays frosn ~he frarQe ~tore 21 and compare6 each of the~e with ali the vectors ln the code~ook ~tore.
The simple~t form of comp~ri~on would ~e to comput~ the mean ~quare di~fer~nce between th~ two~ the ve~tor glving the lowe~t result being deemed ~o be the best match. q~he output from the VQ control logic is tlle 6equence of flag~
associated wlth the vectors thlls identifie~.
The actuctl loqic is carrled out in this e~ample by an inter fra~e d$ffererttial coder 25, in which an inter-framQ dlfference (in sub~ractor 2~ takQn ~irt conventlonal manner) ~etween pQl8 from the fra~e ~to,-e ~1 and ~ pr~vlo~s frame predlctor ~7. As is u~ual ln ~ch 3y5~e~S, a quantizer 28 and an output buffer 29 (to match ~hQ irregular rate of data g~neratiOn to a transmission S link opsr~ting at a constant ra~ re ehown. A re~iver (not showm) uses the dlff~rence-information to update reconstru~ted l~a~e ln a fra~ ore. ~he flag output from the YQ control logic Z4 1,~ conne~ted ~if requlred, v~a erode/cluQ~er ~irc~it6 ~0) ~o the dlf~arent~al coder 25. ~hen the flag indicate~ ~hat 11headn inormation is b~ing proces~ed, th~ coder opera~e6 normally. If however "backgro~ndl~ îs indicated, then the gene~ation of differQnce lnform~tion ~6 carried o~t le6s frequently ~e~
on alternate ~r~me6 onl~). This operatlon i~ lllu~trated by a switoh 31 which, when ~he flag ind~cate6 "~ack~round'l, breaks tha coaing loop on al~ernate fra~es, It will be apparen~ ~rom ~l~e foregoing that any vis~ally distinctlve object or ob~eGt~ may ~e accurat~ly d~tected, recognis~ or located using methods ac~ordin~ t~
~ the lnYent~on.,
This lnYention r~la~es ~o analyBl~ ~nd pro~ ing of video i~age~.
A video ~nage (which wlll be undaratood to enco~pass fros~en images 8tlah as facslmlle imag~s~ in aDdition to movin~ image~) will in g~neral lnclude at le~st one object which ~6 of in~ere~t and a ~ackground~ of l~sser intere~
(and hence o~ 1~BCer i~portance).
To analyse the l~q~, e.g. d~tect the o presence/absen~e or po~l~ion of a partlc~lar ob~ect of intere~t, ~8 oft~n des~rahle in a variety of appli~ations.
In an lmage transml~ n ~ystem an improved picture quality might be ~chieved lf dat~ r~lating to importan~
parts o~ the ~cene, I.e. object6 of ~nter~st, is coded u6inq relatiYely more bit~ ~han data r~lating to unimportant ~l.e. baokground) parts. For e~ample, ln a videophone syste~ a typical ~mage comprises a head and ~ho~lders again~t a back~roulld, ~nd the face area o~ the he~d i~ v~ually th~ ~ost important i~ is thus desirable to ~e able to identi~ the head area from the ~houlders and ~ackground ~o as to be able to proces~ the head at a higher refre6hment rate than thB re~t, 60 ~hat the impre6sion o~ gmooth h~ad motion is conveyed. The ability to loc~te a head wi~hin a head and shoul~ers ~cene can Z5 thus be u~ed to mo~ifg the spatial allo~ation of video da~a, enablln~ a degree of vi~ual lmport~n~e to b~
attr~buted to blo~kfi within the data.
Also, if th~ position of an ob~ect 15 accur~tely t~acked with ti~e ~t wlll be possible to prQdict lt~
motion, thu~ ~llowin~ "~ot~on co~pensated'~
one way of identifying dlf~erent reglon~ of an ima~e ls to utili~e the method proposea ~y Nagao ~. Nayao -"Pictur~ recogni~lon and data struct~re", ~raphlc 17^~ o Language~ - ~d Nake and Rossen~ , 1972). Thig ~ethod ha~ ~een u~ed ~n a viBeophone type system, on an im~ge of a head and shoul~erG agai~t a backyroun~. Some eucces~
was achleved in determininq the 81~e5 of th~ hsAd when the sub~ect was clean shaven, ~ut very littl~ succe~ w~
achieved in other ~ases; so this m~thod ~ n~t considered relia~le enough for the ~ B of an area ldentlfylng method~
Conventlonal coders, for instance hg~rid discrete o c~sine transfor~ coders, use no ~scene ~ontsnt~
~nfor~atlon ~o code th~ da~a ~ithln the ~cene, so e~ch p~rt of tne ~ene i~ operat~d on a3 i~ lt has the ~am~
visual lmpor~ance a~ every other part.
Other i~age analysls appll~a~i~n~ are m~nifold (for ex~mple, ln auto~ted manufacturing 6ystem3).
It is also known to code video image~ for transml~sion using Ye~tor Quantl-~atlon (Y~). In ~Q
coding, the image 1~ r~presented initially by an array of digLt~l d~ta corre~ponding to ~he i~a~e frame. ~locks o~
array points ~nsub-array~") are compared wlth v~ctors from a codebook, ~nd the be~t~matching vector sel~Gted uglng a ~lQas~ s~uares" di~ference criterion. A code des.i~n~ting this vector i.s then ~ran~mitted ~o represent the sub-array~ At the receiving end the indicated vector is ~elected fro~ an identical codebook and displayed.
Ths un~erlying principle o~ the inven~ionr however, is ko u~e VQ as ~n ~dent~fica~ion (e.g. ob~ect location) me~h~d. The extent of ~h~ variouR aspec~ of the lnvention are d~ined in the clai~s appen~ed hereto.
The different areas of a vlaeo image, when vect~r quant~sed (VQ), can be operated on differently provided each entry in the VQ codebook ha~ an associated flag indica~ing which ~rea ~hat entry repr~RBn~s~ So in the example of the videophone two differen~ flag entries ~r~
required, one for the head and the other for the remainder of the scene.
An embodiment of the invention will now be described by way of non-limitative example concerned with the identification of a head in a head and shoulders against a background scene, with reference to the accompanying drawings in which:
Figure l is a block diagram illustrating the initial stages of operation of parts of a coder embodying the invention;
Figures 2a-g show schematically various stages in the training sequence used to derive the codebooX;
Figure 3 is a block diagram illustrating the operation of a coder embodying the invention;
Figure 4a shows schematically a frame to be analysed;
Figure 4b illustrates the sub-array blocks used in vector quantising figure 4a;
Figure 4c shows the state of flags corresponding to the vector quantised image of figure 4b, Figure 4d shows schematically the result of analysing the frame of figure 4a according to the invention;
Figure 5a shows schematically a coder embodying the invention; and Figure 5b schematically shows the differential coder of Figure 5a in more detail.
To enable the invention to operate, it is necessary to have provided a composite codebook which includes vectors flagged as being "head". Preferably others are flagged as being "background". It is possible to derive a "standard" codebook for either an average or a given speaker, but to allow flexibility and greater accuracy of identification, this codebook is derived at the start in an initial "training" sequence. A preferred way of implementing such a sequence will now be described.
- 3a -To generate "head" and "background" parts of the codebook, it is necessary to unambigously obtain some 1 ~,, , '1 ~,, ~head only" data an~ "~ac~ground onlyN data; a crud~
lni~ial head dete~tion algorlthm i~ requlred.
Ref~rring to figures l and 2r ln o~der to d~tect the head, d~glt~l data representing sa~eral ~ontig~oug ~ es of the head and shoulder~ imaga are ~aptur~d~ for ~nstan~e ln a ~ore l. One of thess ~rames i5 ~epicted in ~lg~re 2a. Tnis data doe~ not need to b~ extrem~ly accur~t0, bu~
rather repre6entatlve~
on the as~mption that the prime ~oving ~res~ withln th~ data sequence are ~iroc~ly associat~d Yith th~ head area, frame dl~ferenc~ng 2 is applled to the data repres~ntlng each ad~a~ent pair o~ rames. Thl~ process typically yields a SBt of difference data for ea~h adiacent pair r~presentin~ mo~ing areas together with random noi~e acros~ the whole image area.
For all pic~ure ~lements (pel~) reprQ~en~ed by each set of difference da~, each pel above a given ~hreshold value of intensity i8 ~et to maximu~ intensity (255) and ~ach pel below the thre~hold i~ se~ to m~ni~u~ intenslty 1O)~ This 'threshold~ngl 3 r~moves a large quantiSy of the random noise and some of the ~ov~ng areas.
~edian filtering 4 i5 ne~t applied to each s~t of .
diffQrence data, which very e~fe~tively re~ove~ ~os~ of the re~alning rando~ noise, but erodes only s~all amounts of the ~oving area~.
The lmage represent~d hy each set of d~ta at this stage will rarely provide a cl~r outline of the head;
unless the hea~ to ~ackground con~ra~t iB very hlgh and the mov~ment of the head between ad~acent ~rames i9 more than one pel. Often ~nly one ~i~e and the top of the head may he deplcted as ~hown in f~gure ~b.
General~, the movlng area~ ~111 be clu~t~red ln regions ~round the head a~ea, hut gome isolated clu~ters may arl6e due to moti~n in other areas of the image.
' r 1,,, 1~ ) ~ 5 ~
A clustering proc~s~ 5 iB u~ed to re~ove so~e of the lsola~ed clu~ters: two orth~onal hl~to~ra~s are qenerated, one representing the number o~ '~oving' pels in the columns of ~he ima~e represQnted by thQ d~ta ~nd one repre~entlng ~he number of ~oving pe~s in the ro~s of the imAqe represente~ by the data. The first order momen~
are calcula~d an~ ~he 'centre o~ qravity' of the lmage determined, a~ ~hown in f~gure 2c. A ~ect~ngl~ ls then generated, centred on these co-ordinate~, of ~ch o dimensions that a given pe~centage o~ ~ov~ng area iB
incl~ded wit~n 1~, see ~gur~ 2d. The pel~ ~emain~ng ou~side thi~ rectangle are ~t to zero in~en~i~y, figur~
2e. By a 6uit~1e choice o~ rectangl~ l~ola~ed clu~ter~
ars re~oved by this proce~s.
Constraint~ are impo~ed on the ~election of the rectangles in or~er to reduce the occuren~e of in~orre~t rectangle~. Since a vQry small movement of the head between one fra~e and the next may produce a very small rect~nqle, the r~te of change of ~i~e of the rect~n~le from ~ne s~t of d~t~ to the next ~ restricted: either each of ~he boundary lin~R of the rectangle a~e ~on~tra~ned to lie w1thin a ~mall di~tAnce of the corresponding boundary in th~ ~mmediately p~eceding set of data~ or the ~ ate of change o~ the slze of the rectangle is link~d to the frame difference energy (eg.
the ~4uare of the d~feren~e d~t~ if ~he d~fference energy fs ~mall the change is kept small, ~u~ ~ the difference energy 1~ large the rate of change may be greater.
The rectan~le ~ rectangle~ are used because they require very few bit~ of data to de~ine - i~ then ~hrunk if neces~ary, a~ 6 in f~gure 1, and as shown in figure 2f, ~o become the ~mallest rectangle ~hat c~n ~e placed around the da~ to enclo~e all ~he r~maining non-zero pels. This .11 () ~. J ' rect~nqle 1~ ~sum~d to represent an approxl~ate model ~f th~ head.
A border i~ then created, at 7 in figure l, around the final rect~ngl~, a8 ~hown ln flqur~ 2~. Thl~ border s defines an exclu5ion zone from where no data wlll lat~ be ta~en. This ensures tha~ ~hen the bo~der i appllsd to the reepective frame o~ the orlglnal Image the data outside the ~order will be exclu~lvely h~ad data and the data outslde the border will be e~clu6i~ely bac~groun~
o daea~
~ fiv~ fr~mes of data are initlally captured in the store 1, th~n four ad~ac~nt palr8 of frames are analysed and four set~ of da~a result. After th~ four borders have b~n 3et 7 the head area data and the b~ck~round area da~a 1.5 are extracted from the fir~t four frame of th~ origin~l i~age respectively and tbe Linde-Buso-Grey algori~h~ i~
applied to generate ~ VQ codebook for each area 8, for example, a g bit background codehook an~ lO bit head cod~book (i.e. cod~oo~ containing reep~tiYely 29 ~n~
21~ en~ries~. The t~o code~sok~ ~re then ~h~bined ~ to ~o~m one codebook, ~ach ~ntry of which has ~n a~ocia~e~
flaq indi~ating it~ origln.
R~ferrlng now to Plg~r~s 3 and 4a-d, af~er this trai~ing sequence 1~ completed, the composite codeb~ok lS
us~d to locate ~he head in succ~sl~e lmag~ frames. The VQ coder operates ~ust a~ it would in a prlor Art ~ystem u~ng YQ as the transmis~lon coding, bu~ ~or each bloc~ of pels coded lO, the code generated will include ~ flag (for exa~ple, ~he first digit3 indi,~ating whether that ~lock ls ~head" or "background't so that ~he head position i6 known for each frame.
It will of ~ourse be appre~iated ~h~t when the codebook i8 deri~ed ~t the cod~r as indic~ted above, VQ
cannot be used as the ~ans~ission code ~unles~ this '` ^ 11 ~
U ~) - 7 ~
~odehook is mad~ known to the ~ecoder fir~t by transmittlng an indLcatlon vf the ~ectorB)~
~lnce ~he quantl~ation proc~s 1~ inhQrently approxl~ate, it wlll be ~ppreciated tha~ occa3ionally blocks from the he~d p~rt of the im~ge may best match a vector fro~ ~he "background" part of the codebook, or Vice ver~a. The ~ctu~l ~dentification of the head will thu~
u8ually lnvolve i~noring i~olated nbead" ~locks U~ing erosion and clu~tering llr 12 (~or example, as ~e~cri~ed 0 ~bo~e~, or de~ignatinq the ~re~ with the hlghest concentrat~on of "lle~d~ ~loc~s a~ the ac~ual head.-Another method in~ol~e6 detec~ing lsolate~ "head~l block~ and ~hen ~xaminlng the error between the block and the "head" vector, and that hetwe~n the blo~k and the be~t-ma~ching "backqro~nd" vQctor, and if the t~o scor~s are ~imilar (i~e. there iæ ambl~ulty ab~u~ whether the ~lOCK ~g llheadn or "background~l), reflagging t}le block t~
bac~roundll ins~ead.
If the head blocks are too scattered, it may be that the ~odebook is insuff~c~ent to charac~erise the head. ~n this ~ase, ~ re~raintng seq~ence may bP emp~yed t~
regenerate th~ cod~ook.
This retraining sequ~nce may ~ither simp~y be a furthe~ sequence o~ the kind descri~ed ~ove, or it may 2s attempt to ilnpro~e trather ~han simply redefine~ the codebo~lc. Fnt~ ~x~m~l P, ~ ~ount n~ e kcp~ ~ tho numbe~
of ~lncorre~ e. scattered) as oppo~ed ~o "c~r~ect"
(l.e. conc~ntratod ln the head area) occurenes of each ~ector, and ~he scatter may this be reduced hy reiecting fro~ ths codebook vec~ors which occur incorrectly too often .
or, alt~rnativel~, the approxi~ate head loc~t.ion derived by lo~atlng the greate~t concentratlon of "head"
~ 8 blocks may be used, ln the sa~e ~anner a-q d~crlba~ above, a~ an area ~or generating a new "head" codeboo3~.
The~e latt~r approacha~, in which VQ ~oder "learn~"
from e~ch retraining ~e~U~nce, are pr~ferred on groun~s o~
s accura~y, Figure 5 shows a block di~gram of ~ ~ideo coding apparatus (eg ~or a video telephone ) embody~ng the inst~ntion. Yideo ~i~nal~ are fed ~rom an input ~0 to a frame 6tore 21 where ~nd~vidual p~cture element Ya~Ues are recorded ~n respectlY~ 6torP lo~tlon~ ~ that deslred sllb-array~ of pel~ are accessible for fu~t~ler p~oce~sing.
The sub-arra5r ~læes may typically be 8 x 8. In an ~nitial~ trainlng, phasQ of the app~tus a tralning control llnlt 22 - wh~h may for ~pl~ b~ ~ suitably lS progra~m~d microproce660r sy8t~ - carrie~ o~t the code~ool~ generatlon method descri~ed above, ~nd enter6 the ~e~tors (~nd flag~) in a VQ codebook store 23. It will be un~erstood that the YQ proce~ ~nYo~Ye~ rnatching 8 x 8 su~-array to th~ neare~ one of the store~ of ~rectors, viz. a nu~er of B ~ 8 patterns w~ich are consis~ently fe~er ln nwn~er Shan the maximum p~ssible n~ er (2~4) of such patt~rn~.
In the codlng phase of the apparatug, the matchinq i~ c~rried out by VQ c~ntrol logi~ ~4 ~hich re~ei~,re5 succe~ive sub-arrays frosn ~he frarQe ~tore 21 and compare6 each of the~e with ali the vectors ln the code~ook ~tore.
The simple~t form of comp~ri~on would ~e to comput~ the mean ~quare di~fer~nce between th~ two~ the ve~tor glving the lowe~t result being deemed ~o be the best match. q~he output from the VQ control logic is tlle 6equence of flag~
associated wlth the vectors thlls identifie~.
The actuctl loqic is carrled out in this e~ample by an inter fra~e d$ffererttial coder 25, in which an inter-framQ dlfference (in sub~ractor 2~ takQn ~irt conventlonal manner) ~etween pQl8 from the fra~e ~to,-e ~1 and ~ pr~vlo~s frame predlctor ~7. As is u~ual ln ~ch 3y5~e~S, a quantizer 28 and an output buffer 29 (to match ~hQ irregular rate of data g~neratiOn to a transmission S link opsr~ting at a constant ra~ re ehown. A re~iver (not showm) uses the dlff~rence-information to update reconstru~ted l~a~e ln a fra~ ore. ~he flag output from the YQ control logic Z4 1,~ conne~ted ~if requlred, v~a erode/cluQ~er ~irc~it6 ~0) ~o the dlf~arent~al coder 25. ~hen the flag indicate~ ~hat 11headn inormation is b~ing proces~ed, th~ coder opera~e6 normally. If however "backgro~ndl~ îs indicated, then the gene~ation of differQnce lnform~tion ~6 carried o~t le6s frequently ~e~
on alternate ~r~me6 onl~). This operatlon i~ lllu~trated by a switoh 31 which, when ~he flag ind~cate6 "~ack~round'l, breaks tha coaing loop on al~ernate fra~es, It will be apparen~ ~rom ~l~e foregoing that any vis~ally distinctlve object or ob~eGt~ may ~e accurat~ly d~tected, recognis~ or located using methods ac~ordin~ t~
~ the lnYent~on.,
Claims (25)
1. A method of processing an image array, comprising the steps of:
a) comparing, using vector comparison, each of a plurality of identically shaped two-dimensional sub-arrays into which the image array is notionally divided with vectors from a code book set of vectors which code book includes a sub-set of vectors associated with an object, each vector of the sub-set having an associated flag indicating that it is associated with said object, and b) in the event of substantial similarity, labelling that sub-array of the image array as corresponding to the object.
a) comparing, using vector comparison, each of a plurality of identically shaped two-dimensional sub-arrays into which the image array is notionally divided with vectors from a code book set of vectors which code book includes a sub-set of vectors associated with an object, each vector of the sub-set having an associated flag indicating that it is associated with said object, and b) in the event of substantial similarity, labelling that sub-array of the image array as corresponding to the object.
2. A method according to claim 1, in which the set includes also a second subset comprising members corresponding to the background, and each vector has an associated flag indicating which of the object or the background that vector is associated with, so that each sub-array may be labelled as corresponding to the object or to the background by the flag.
3. A method according to claim 2 in which the location of the object is identified by finding within the image array the greatest concentration of sub-arrays labelled as corresponding to the object.
4. A method of detecting a plurality of different objects, according to claim 1, 2 or 3, in which the set includes members associated with each such object.
5. A method of detecting a human head within a video image, employing the method of claims 1, 2 or 3.
6. A method of encoding a video image signal for transmission comprising the steps of:
a) identifying an area of the video image corresponding to an object of visual importance, using a method according to claim 1 or 2; and b) modifying the spatial allocation of coding data so as to preferentially favour that area, whereby a greater degree of visual importance may be attributed to that area of the image.
a) identifying an area of the video image corresponding to an object of visual importance, using a method according to claim 1 or 2; and b) modifying the spatial allocation of coding data so as to preferentially favour that area, whereby a greater degree of visual importance may be attributed to that area of the image.
7. A method of encoding a video image signal comprising the steps of:
a) identifying an area of the video image corresponding to an object of visual importance by repeatedly;
i) comparing a sub-array of the image with vectors from a set including members associated with the object;
and ii) in the event of substantial similarity, labelling that sub-array as corresponding to the object;
and b) modifying the spatial allocation of coding data in favor of that area, whereby a degree of visual importance may be attributed to that area of the image;
wherein the video image signal is encoded so as to update the area corresponding to the object at a higher rate than other areas.
a) identifying an area of the video image corresponding to an object of visual importance by repeatedly;
i) comparing a sub-array of the image with vectors from a set including members associated with the object;
and ii) in the event of substantial similarity, labelling that sub-array as corresponding to the object;
and b) modifying the spatial allocation of coding data in favor of that area, whereby a degree of visual importance may be attributed to that area of the image;
wherein the video image signal is encoded so as to update the area corresponding to the object at a higher rate than other areas.
8. A method of generating a set of vectors for use in a method of identifying a moving object by repeatedly i) comparing a sub-array of the image with vectors from a set including members associated with the object, and ii) in the event of substantial similarity, labelling that sub-array as corresponding to the object;
and said method of generating a set of vectors comprising the steps of:
a) identifying an area of the image corresponding to at least a part of the object, and b) generating vectors from video-data representing that area of the image, in which the areas of the image corresponding to the object are identified by analyzing the difference between a pair of temporally separated image frames, whereby the object is detected by its movement.
and said method of generating a set of vectors comprising the steps of:
a) identifying an area of the image corresponding to at least a part of the object, and b) generating vectors from video-data representing that area of the image, in which the areas of the image corresponding to the object are identified by analyzing the difference between a pair of temporally separated image frames, whereby the object is detected by its movement.
9. An image analyser for identifying an object against a background within an image, comprising vector quantisation means arranged to compare sub-arrays of an image array with vectors from a codebook and select therefrom the most similar vector to each such sub-array, the codebook comprising a subset of vectors associated with the object and a subset of vectors associated with the background, each such vector having an associated flag indicating to which subset it belongs, whereby the analyser may identify the object from the flags of the vectors selected for the sub-arrays.
10. An image analyser according to claim 9, further comprising clustering means for determining the position of a boundary to enclose a given proportion of those sub-arrays flagged as corresponding to the object, whereby the image analyser may identify the position and spatial extent of the object.
11. A coder for encoding video image signals comprising:
a) an image analyser according to claim 9 arranged to identify an object within an image, and b) an encoder arranged to preferentially allocate video encoding data to the area of the image corresponding to the object.
a) an image analyser according to claim 9 arranged to identify an object within an image, and b) an encoder arranged to preferentially allocate video encoding data to the area of the image corresponding to the object.
12 12. A coder according to claim 12, in which the encoder is arranged to encode the unquantised image.
13. A coder according to claim 12, in which the coder is a discrete cosine transform encoder.
14. A coder according to claim 11, further comprising:
c) motion analysis means arranged to detect motion of the position of the identified object between temporally separated image frames, and to predict therefrom the position of the object in a subsequent image frame.
whereby the encoder may be a motion compensated DPCM encoder.
c) motion analysis means arranged to detect motion of the position of the identified object between temporally separated image frames, and to predict therefrom the position of the object in a subsequent image frame.
whereby the encoder may be a motion compensated DPCM encoder.
15. A coder arranged to employ a method of encoding a video image signal comprising the steps of:
a) identifying an area of the video image corresponding to an object of visual importance by repeatedly i) comparing a sub-array of the image with vectors from a set including members associated with the object, and ii) in the event of substantial similarity, labelling that sub-array as corresponding to the object;
and b) modifying the spatial allocation of coding data in favor of that area, whereby a degree of visual importance may be attributed to that area of the image.
a) identifying an area of the video image corresponding to an object of visual importance by repeatedly i) comparing a sub-array of the image with vectors from a set including members associated with the object, and ii) in the event of substantial similarity, labelling that sub-array as corresponding to the object;
and b) modifying the spatial allocation of coding data in favor of that area, whereby a degree of visual importance may be attributed to that area of the image.
16. A coder according to claim 11, 12, 13, 14 or 15, also arranged initially to generate vectors of the said subsets of the said codebook, further comprising identification means for identifying an area of the image corresponding to the object or to the background, whereby the respective vectors may be generated from data derived from the said area.
17. A coder according to claim 11 in which the identification means is arranged to analyse the difference between a pair of temporally separated image frames to identify areas of the image corresponding to the object.
18. A coder according to claim 15 in which the identification means is arranged to analyse the difference between a pair of temporally separated image frames to identify areas of the image corresponding to the object.
19. A coder according to claim 17, in which the identification means comprises:
a) means for generating from a pair of frames of the image array elements within a notional two dimensional field, the value of each position within the field indicating whether the difference between the luminance levels of the picture elements at corresponding positions in the two frames lies above or below a threshold; and b) clustering means for determining the centre of gravity within the said field of those array elements indicating a difference above the threshold and determining the position of a boundary about the centre of gravity which encloses a given proportion of those array elements, whereby all picture elements lying within a boundary so determined are identified as belonging to an area of the image corresponding to the object.
a) means for generating from a pair of frames of the image array elements within a notional two dimensional field, the value of each position within the field indicating whether the difference between the luminance levels of the picture elements at corresponding positions in the two frames lies above or below a threshold; and b) clustering means for determining the centre of gravity within the said field of those array elements indicating a difference above the threshold and determining the position of a boundary about the centre of gravity which encloses a given proportion of those array elements, whereby all picture elements lying within a boundary so determined are identified as belonging to an area of the image corresponding to the object.
20. A coder according to claim 18, in which the identification means comprises:
a) means for generating from a pair of frames of the image array elements within a notional two dimensional field, the value of each position within the field indicating whether the difference between the luminance levels of the picture elements at corresponding positions in the two frames lies above or below a threshold; and b) clustering means for determining the centre of gravity within the said field of those array elements indicating a difference above the threshold and determining the position of a boundary about the centre of gravity which encloses a given proportion of those array elements, whereby all picture elements lying within a boundary so determined are identified as belonging to an area of the image corresponding to the object.
a) means for generating from a pair of frames of the image array elements within a notional two dimensional field, the value of each position within the field indicating whether the difference between the luminance levels of the picture elements at corresponding positions in the two frames lies above or below a threshold; and b) clustering means for determining the centre of gravity within the said field of those array elements indicating a difference above the threshold and determining the position of a boundary about the centre of gravity which encloses a given proportion of those array elements, whereby all picture elements lying within a boundary so determined are identified as belonging to an area of the image corresponding to the object.
21. A coder according to claim 17, 18, 19 or 20, which means further comprises filtering means for median filtering the array elements within the notional two dimensional field prior to the determination of the centre of gravity.
22. A coder according to claim 17, 18, 19 or 20, wherein said boundary about the centre of gravity is of a finite number of elements in thickness.
23. A coder according to claim 19 or 20, wherein said boundary about the centre of gravity is of rectangular shape.
24. A coder according to claim 19 or 20, wherein the rectangular shaped boundary is centred upon the centre of gravity and each side of the rectangle is moved inward, if possible, until it abuts at least one of those array elements indicating a difference above the threshold.
25. A coder according to claim 19 or 20, wherein the rectangular shaped boundary is centred upon the centre of gravity and each side of the rectangle is moved inward, if possible, until it abuts at least one of those array elements indicating a difference above the threshold, and wherein said boundary about the centre of gravity is of rectangular shape.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB8710737 | 1987-05-06 | ||
| GB878710737A GB8710737D0 (en) | 1987-05-06 | 1987-05-06 | Video image encoding |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA1308486C true CA1308486C (en) | 1992-10-06 |
Family
ID=10616913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA000566156A Expired - Lifetime CA1308486C (en) | 1987-05-06 | 1988-05-06 | Video image processing |
Country Status (8)
| Country | Link |
|---|---|
| US (1) | US5086480A (en) |
| EP (1) | EP0313612B1 (en) |
| JP (1) | JP2809659B2 (en) |
| AT (1) | ATE81930T1 (en) |
| CA (1) | CA1308486C (en) |
| GB (1) | GB8710737D0 (en) |
| HK (1) | HK129196A (en) |
| WO (1) | WO1988009101A1 (en) |
Families Citing this family (73)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0330455A3 (en) * | 1988-02-22 | 1990-07-04 | Kabushiki Kaisha Toshiba | Image encoding apparatus |
| JP2921936B2 (en) * | 1990-07-13 | 1999-07-19 | 株式会社東芝 | Image monitoring device |
| JPH0771288B2 (en) * | 1990-08-24 | 1995-07-31 | 神田通信工業株式会社 | Automatic view adjustment method and device |
| US5148477A (en) * | 1990-08-24 | 1992-09-15 | Board Of Regents Of The University Of Oklahoma | Method and apparatus for detecting and quantifying motion of a body part |
| DE4028191A1 (en) * | 1990-09-05 | 1992-03-12 | Philips Patentverwaltung | CIRCUIT ARRANGEMENT FOR DETECTING A HUMAN FACE |
| EP0476603B1 (en) * | 1990-09-20 | 1997-06-18 | Nec Corporation | Method and apparatus for coding moving image signal |
| US5243418A (en) * | 1990-11-27 | 1993-09-07 | Kabushiki Kaisha Toshiba | Display monitoring system for detecting and tracking an intruder in a monitor area |
| US5218648A (en) * | 1990-12-17 | 1993-06-08 | Hughes Aircraft Company | Constellation matching system and method |
| JPH04334188A (en) * | 1991-05-08 | 1992-11-20 | Nec Corp | Coding system for moving picture signal |
| JP3513160B2 (en) * | 1991-12-06 | 2004-03-31 | キヤノン株式会社 | Video signal encoding apparatus and method |
| FR2690031A1 (en) * | 1992-04-14 | 1993-10-15 | Philips Electronique Lab | Segmentation device of images. |
| FR2693291A1 (en) * | 1992-07-01 | 1994-01-07 | Philips Electronique Lab | Image coding sub-system with refresh correction for use with video telephone |
| US5420638A (en) * | 1992-04-14 | 1995-05-30 | U.S. Philips Corporation | Subassembly for coding images with refresh correction of the data to be coded, and subassembly for decording signals representing these images and previously coded by means of a subassembly of the former kind |
| FR2689994A1 (en) * | 1992-04-14 | 1993-10-15 | Philips Electronique Lab | Image coding sub-system with refresh correction for use with video telephone |
| US7788008B2 (en) * | 1995-06-07 | 2010-08-31 | Automotive Technologies International, Inc. | Eye monitoring system and method for vehicular occupants |
| EP0648400A1 (en) * | 1992-06-29 | 1995-04-19 | BRITISH TELECOMMUNICATIONS public limited company | Coding and decoding video signals |
| US5386482A (en) * | 1992-07-16 | 1995-01-31 | Scan-Optics, Inc. | Address block location method and apparatus |
| US5835641A (en) * | 1992-10-14 | 1998-11-10 | Mitsubishi Denki Kabushiki Kaisha | Image pick-up apparatus for detecting and enlarging registered objects |
| JP3133517B2 (en) * | 1992-10-15 | 2001-02-13 | シャープ株式会社 | Image region detecting device, image encoding device using the image detecting device |
| WO1994017636A1 (en) * | 1993-01-29 | 1994-08-04 | Bell Communications Research, Inc. | Automatic tracking camera control system |
| GB9308952D0 (en) * | 1993-04-30 | 1993-06-16 | Philips Electronics Uk Ltd | Tracking objects in video sequences |
| CA2119327A1 (en) * | 1993-07-19 | 1995-01-20 | David Crawford Gibbon | Method and means for detecting people in image sequences |
| US5623587A (en) * | 1993-10-15 | 1997-04-22 | Kideo Productions, Inc. | Method and apparatus for producing an electronic image |
| US7859551B2 (en) * | 1993-10-15 | 2010-12-28 | Bulman Richard L | Object customization and presentation system |
| GB2283876B (en) * | 1993-11-09 | 1998-06-24 | Matsushita Electric Ind Co Ltd | Encoding and decoding code sequences and frames |
| US5512939A (en) * | 1994-04-06 | 1996-04-30 | At&T Corp. | Low bit rate audio-visual communication system having integrated perceptual speech and video coding |
| US5557298A (en) * | 1994-05-26 | 1996-09-17 | Hughes Aircraft Company | Method for specifying a video window's boundary coordinates to partition a video signal and compress its components |
| CA2145914A1 (en) * | 1994-05-27 | 1995-11-28 | Alexandros Eleftheriadis | Model-assisted coding of video sequences at low bit rates |
| US6707484B1 (en) | 1994-07-28 | 2004-03-16 | Semiconductor Energy Laboratory Co., Ltd. | Information processing system |
| JP3167865B2 (en) * | 1994-07-28 | 2001-05-21 | 株式会社半導体エネルギー研究所 | Information processing device |
| US5751377A (en) | 1994-11-04 | 1998-05-12 | Matsushita Electric Industrial Co., Ltd. | Picture coding apparatus and decoding apparatus |
| ZA959491B (en) * | 1994-12-21 | 1996-06-29 | Eastman Kodak Co | Method for compressing and decompressing standardized portait images |
| ZA959492B (en) * | 1994-12-21 | 1996-07-10 | Eastman Kodak Co | Method and apparatus for the formation of standardised image templates |
| US5610995A (en) * | 1995-06-06 | 1997-03-11 | United Parcel Service Of America, Inc. | Method and apparatus for compressing images containing optical symbols |
| US5812787A (en) * | 1995-06-30 | 1998-09-22 | Intel Corporation | Video coding scheme with foreground/background separation |
| US6307962B1 (en) * | 1995-09-01 | 2001-10-23 | The University Of Rochester | Document data compression system which automatically segments documents and generates compressed smart documents therefrom |
| US5889891A (en) * | 1995-11-21 | 1999-03-30 | Regents Of The University Of California | Universal codebook vector quantization with constrained storage |
| US5764803A (en) * | 1996-04-03 | 1998-06-09 | Lucent Technologies Inc. | Motion-adaptive modelling of scene content for very low bit rate model-assisted coding of video sequences |
| DE69713779T2 (en) * | 1996-09-12 | 2002-10-17 | Univ Bath | OBJECT-ORIENTED VIDEO SYSTEM |
| US6078619A (en) * | 1996-09-12 | 2000-06-20 | University Of Bath | Object-oriented video system |
| AU722393B2 (en) * | 1996-11-07 | 2000-08-03 | Broderbund Software, Inc. | System for adaptive animation compression |
| KR100251051B1 (en) * | 1997-07-14 | 2000-04-15 | 윤종용 | An arbitrary shape coding method |
| US6173069B1 (en) | 1998-01-09 | 2001-01-09 | Sharp Laboratories Of America, Inc. | Method for adapting quantization in video coding using face detection and visual eccentricity weighting |
| JP3765923B2 (en) | 1998-02-26 | 2006-04-12 | シャープ株式会社 | HARDWARE SYNTHESIS METHOD, HARDWARE SYNTHESIS DEVICE, AND RECORDING MEDIUM CONTAINING HARDWARE SYNTHESIS PROGRAM |
| ATE240022T1 (en) * | 1998-06-05 | 2003-05-15 | Innomedia Pte Ltd | METHOD AND DEVICE FOR EXTRACTING THE BACKGROUND TO REDUCE THE NUMBER OF CODED BLOCKS IN VIDEO CODING |
| US6404900B1 (en) * | 1998-06-22 | 2002-06-11 | Sharp Laboratories Of America, Inc. | Method for robust human face tracking in presence of multiple persons |
| WO2000016243A1 (en) * | 1998-09-10 | 2000-03-23 | Mate - Media Access Technologies Ltd. | Method of face indexing for efficient browsing and searching ofp eople in video |
| US6549652B1 (en) | 1998-09-11 | 2003-04-15 | Cirrus Logic, Inc. | Method and apparatus for reducing noise during lossy transformation processes |
| US6347155B1 (en) | 1998-10-01 | 2002-02-12 | Sharewave, Inc. | Method and apparatus for digital data compression |
| US7158681B2 (en) * | 1998-10-01 | 2007-01-02 | Cirrus Logic, Inc. | Feedback scheme for video compression system |
| JP2000293687A (en) * | 1999-02-02 | 2000-10-20 | Minolta Co Ltd | Three-dimensional shape data processor and three- dimensional shape data processing method |
| US7005985B1 (en) * | 1999-07-20 | 2006-02-28 | Axcess, Inc. | Radio frequency identification system and method |
| EP1109409A3 (en) * | 1999-12-17 | 2011-11-30 | Canon Kabushiki Kaisha | Digital signal coding with division into tiles |
| US6940545B1 (en) * | 2000-02-28 | 2005-09-06 | Eastman Kodak Company | Face detecting camera and method |
| US7768546B1 (en) | 2000-05-12 | 2010-08-03 | Axcess International, Inc. | Integrated security system and method |
| EP1250005A1 (en) * | 2001-04-12 | 2002-10-16 | BRITISH TELECOMMUNICATIONS public limited company | Video communication with feedback of the caller's position relative to the camera |
| US7257268B2 (en) * | 2003-02-28 | 2007-08-14 | Aperio Technologies, Inc. | Systems and methods for image pattern recognition |
| US7136506B2 (en) * | 2003-03-03 | 2006-11-14 | Lockheed Martin Corporation | Correlation based in frame video tracker |
| US7379559B2 (en) * | 2003-05-28 | 2008-05-27 | Trw Automotive U.S. Llc | Method and apparatus for determining an occupant's head location in an actuatable occupant restraining system |
| KR100543706B1 (en) * | 2003-11-28 | 2006-01-20 | 삼성전자주식회사 | Vision-based humanbeing detection method and apparatus |
| US20050175243A1 (en) * | 2004-02-05 | 2005-08-11 | Trw Automotive U.S. Llc | Method and apparatus for classifying image data using classifier grid models |
| US7471832B2 (en) * | 2004-02-24 | 2008-12-30 | Trw Automotive U.S. Llc | Method and apparatus for arbitrating outputs from multiple pattern recognition classifiers |
| US20050196015A1 (en) * | 2004-03-02 | 2005-09-08 | Trw Automotive U.S. Llc | Method and apparatus for tracking head candidate locations in an actuatable occupant restraining system |
| US7841120B2 (en) | 2004-03-22 | 2010-11-30 | Wilcox Industries Corp. | Hand grip apparatus for firearm |
| US20070205896A1 (en) * | 2006-03-02 | 2007-09-06 | Axcess International Inc. | System and Method for Determining Location, Directionality, and Velocity of RFID Tags |
| US20070285241A1 (en) * | 2006-03-20 | 2007-12-13 | Axcess International Inc. | Multi-Tag Tracking Systems and Methods |
| WO2007133690A2 (en) * | 2006-05-11 | 2007-11-22 | Axcess International Inc. | Radio frequency identification (rfid) tag antenna design |
| US8638194B2 (en) * | 2008-07-25 | 2014-01-28 | Axcess International, Inc. | Multiple radio frequency identification (RFID) tag wireless wide area network (WWAN) protocol |
| JP2013535733A (en) * | 2010-07-26 | 2013-09-12 | コーニンクレッカ フィリップス エヌ ヴェ | Get keywords for search |
| US8364865B2 (en) | 2010-09-28 | 2013-01-29 | Microsoft Corporation | Data simulation using host data storage chain |
| CN104484418B (en) * | 2014-12-17 | 2017-10-31 | 中国科学技术大学 | A kind of characteristic quantification method and system based on dual resolution design |
| US10997395B2 (en) * | 2017-08-14 | 2021-05-04 | Amazon Technologies, Inc. | Selective identity recognition utilizing object tracking |
| WO2019185150A1 (en) * | 2018-03-29 | 2019-10-03 | Tobii Ab | Determining a gaze direction using depth information |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| SE365325B (en) * | 1971-11-04 | 1974-03-18 | Rothfjell R | |
| US3761613A (en) * | 1972-06-20 | 1973-09-25 | Bell Telephone Labor Inc | Dual mode video encoder |
| GB1545117A (en) * | 1976-05-25 | 1979-05-02 | Nat Res Dev | Comparison apparatus eg for use in character recognition |
| SE422714B (en) * | 1979-01-16 | 1982-03-22 | Ibm Svenska Ab | DEVICE WITH OPTICAL SIGN SOLUTION COMPUTER, WHICH CONTROL COMPUTER CALCULATES STARTING POINTS FOR IDENTIFYING A SIGN |
| JPS5923467B2 (en) * | 1979-04-16 | 1984-06-02 | 株式会社日立製作所 | Position detection method |
| US4700401A (en) * | 1983-02-28 | 1987-10-13 | Dest Corporation | Method and apparatus for character recognition employing a dead-band correlator |
| FR2551290B1 (en) * | 1983-08-30 | 1985-10-11 | Thomson Csf | METHOD AND DEVICE FOR DETECTING MOVING POINTS IN A TELEVISION IMAGE FOR DIGITAL TELEVISION SYSTEMS WITH CONDITIONAL COOLING RATE COMPRESSION |
| JPS60152904A (en) * | 1984-01-20 | 1985-08-12 | Nippon Denso Co Ltd | Vehicle-driver-position recognizing apparatus |
| GB8528143D0 (en) * | 1985-11-14 | 1985-12-18 | British Telecomm | Image encoding & synthesis |
-
1987
- 1987-05-06 GB GB878710737A patent/GB8710737D0/en active Pending
-
1988
- 1988-05-06 WO PCT/GB1988/000357 patent/WO1988009101A1/en active IP Right Grant
- 1988-05-06 AT AT88903903T patent/ATE81930T1/en not_active IP Right Cessation
- 1988-05-06 CA CA000566156A patent/CA1308486C/en not_active Expired - Lifetime
- 1988-05-06 JP JP63503708A patent/JP2809659B2/en not_active Expired - Lifetime
- 1988-05-06 EP EP88903903A patent/EP0313612B1/en not_active Expired - Lifetime
-
1990
- 1990-10-05 US US07/594,670 patent/US5086480A/en not_active Expired - Lifetime
-
1996
- 1996-07-18 HK HK129196A patent/HK129196A/en not_active IP Right Cessation
Also Published As
| Publication number | Publication date |
|---|---|
| ATE81930T1 (en) | 1992-11-15 |
| JP2809659B2 (en) | 1998-10-15 |
| HK129196A (en) | 1996-07-26 |
| JPH02500079A (en) | 1990-01-11 |
| WO1988009101A1 (en) | 1988-11-17 |
| GB8710737D0 (en) | 1987-06-10 |
| EP0313612B1 (en) | 1992-10-28 |
| US5086480A (en) | 1992-02-04 |
| EP0313612A1 (en) | 1989-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA1308486C (en) | Video image processing | |
| US10499054B2 (en) | System and method for inter-frame predictive compression for point clouds | |
| US6934415B2 (en) | Visual attention system | |
| JP3373008B2 (en) | Image area separation device | |
| USRE37668E1 (en) | Image encoding/decoding device | |
| CN101036150B (en) | Apparatus and method for processing image data | |
| Bayar et al. | A generic approach towards image manipulation parameter estimation using convolutional neural networks | |
| EP1439491B1 (en) | Signal processing method and processor | |
| JP2000508488A (en) | System and method for multi-resolution conversion of digital image information | |
| CN110087083B (en) | Method for selecting intra chroma prediction mode, image processing apparatus, and storage apparatus | |
| US5124791A (en) | Frame-to-frame compression of vector quantized signals and other post-processing | |
| JPH07184058A (en) | Compression for standardization picture library | |
| CN114359333A (en) | Moving object extraction method and device, computer equipment and storage medium | |
| Moussa et al. | Forensic license plate recognition with compression-informed transformers | |
| CN115861922A (en) | Sparse smoke and fire detection method and device, computer equipment and storage medium | |
| JPH09187017A (en) | Calculation support type movement estimation method for pixels of images successively continuous in terms of time in video sequence | |
| Sehli et al. | WeLDCFNet: Convolutional Neural Network based on Wedgelet Filters and Learnt Deep Correlation Features for depth maps features extraction | |
| Cheng | Visual pattern matching in motion estimation for object-based very low bit-rate coding using moment-preserving edge detection | |
| Klima et al. | Image quality evaluation in security imaging systems | |
| Vinogradov et al. | Digital processing of video information of objects by the method of contour-structured lines | |
| JPH06187455A (en) | Face area extracting device for animation picture | |
| JP3420389B2 (en) | Image encoding method and apparatus | |
| Cai et al. | Lossless image compression with tree coding of magnitude levels | |
| Sexton | Automatic face detection for videoconferencing | |
| CN114915794A (en) | Point cloud encoding and decoding method and device based on two-dimensional regularized planar projection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MKEX | Expiry |