« PreviousContinue »
Rajeev Sharma, State College, PA (US);
Satish Mummareddy, Washington, DC
(US); Namsoon Jung, State College, PA
VideoMining Corporation, State
College, PA (U S)
Subject to any disclaimer, the term of this
patent is extended or adjusted under 35
U.S.C. l54(b) by 730 days.
U.S. Appl. No. 12/011,650, filed Jan. 2008, Mummareddy, et al.
H. Buxton, et al., “Advanced visual surveillance using Bayesian
Networks,” in International Conference on Computer Vision, 1995,
Cambridge, Massachusetts, 12pp.
A. Cohn, et al., “Towards an architecture for cognitive vision using
qualitative spatio-temporal representations and abduction,” Spatial
Cognition III, 2003, 17pp.
Primary Examiner — Andrew W Johns
The present invention is a method and system for automatically analyzing the behavior of a person and a plurality of persons in a physical space based on measurement of the trip of the person and the plurality of persons on input images. The present invention captures a plurality of input images of the person by a plurality of means for capturing images, such as cameras. The plurality of input images is processed in order to track the person in each field of view of the plurality of means for capturing images. The present invention measures the information for the trip of the person in the physical space based on the processed results from the plurality of tracks and analyzes the behavior of the person based on the trip information. The trip infonnation can comprise coordinates of the person’s position and temporal attributes, such as trip time and trip length, for the plurality of tracks. The physical space may be a retail space, and the person may be a customer in the
Dove et al. .................... .. 706/13 . . . . .
Lausch 4 348/143 retail space. The trip information can provide key measure-
Jung et 31‘ 707/790 ments as a foundation for the behavior analysis of the cus-
Choi et al. .. 348/77 tomer along the entire shopping trip, from entrance to check-
C1°°°1° e1 i11~ 340/5731 out, that deliver deeper insights about the customer behavior.
Sorensen " 705/26 The focus of the present invention is given to the automatic
Steenburgh et al. 382/103 b h . 1 . 1. . b d h . f h
Pavlidis et al‘ 382/103 e avior ana ytics app ications ase upon ‘t e trip romt e
Trajkovic et a1, 343/1 5 5 extracted video, where the exemplary behavior analysis com-
Guler ........ .. 345/716 prises map generation as visualization of the behavior, quan-
gzer e1ta1~1 titative category measurement, dominant path measurement,
rang e a . ~
Sorensen 705/10 category correlation measurement, and category sequence
Aoto et al 348/155 measllremeni
Ma et al. 382/190
Hua et al. ...................... .. 706/12 28 Claims, 22 Drawing Sheets
g 354 K 355 K 356
VIDEO FEED VIDEO FEED VIDEO FEED
‘I 2 N
l /351 l /352 I /353
/357 /351 /351
$CENE SCENE SCENE
BACKGROUND BACKGROUND BACKGROUND
LEARNING LEARNING LEARNING
I/ass I/ass I/ass
FOREGROUND FOREGROUND FOREGROUND
SEGMENT- $EGMENT- SEGMENT-
ATION ATION ATION
I/ase I/359 V359
I PERSON 4 I PERSO PERSON
DETECTION DETECTIQN DETECTION
I/360 l/aeo I /aeo
I PERSON 1 I PERSON 1 PERSON 1
TRACKING TRACKING TRACKING
J. Fernyhough, et al., “Event recognition using qualitative reasoning on automatically generated spatio-temporal models from visual input,” in IJCAI 97 Workshop on Spatial and Temporal Reasoning, 1997, Nagoya, 9pp.
J. Fernyhough, et al., “Constructing qualitative event models automatically from video input, Image and vision computing,” 18, pp. 81-103, 2000.
N. Friedman, et al., “Image segmentation in video sequences: A probabilistic approach,” in Thirteenth Conference on Uncertainty in Artificial Intelligence, 1997, l3pp.
A. Galata, et al., “Modeling interaction using learnt qualitative spatio-temporal relations and variable length Markov Models,” in European Conference on Artificial Intelligence, 2000, Lyon, Spp.
S. G. Gong, et al., “Bayesian Nets for mapping contextual knowledge to computational constraints in motion segmentation and tracking,”
in British Machine Vision Conference, 1993, Guildford, England, l 6pp.
W. E. L. Grimson, et al., “Using adaptuve tracking to classify and monitor activities in a site,” in IEEE Conference on ComputerVision and Pattern Recognition, 1998, pp. 22-29.
Y. A. Ivanov, et al., “Recognition of visual activities and interactions by Stochastic Parsing,” IEEE Trans. on Pattern Analysis and Machine Intelligence 2000, 22(8), pp. 852-872, 2000.
D. Moore, et al., “Recognizing multitasked activities using Stochastic context-free grammar,” in Workshop on Models versus Exemplars in Computer Vision held in conjunction with IEEE CVPR 2001, Kauai, Hawaii, l0pp.
C. Stauffer, et al., Adaptive background mixture models for real-time tracking, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’99), 1999 (2), pp. 246-252.