Search Images Maps Play YouTube News Gmail Drive More »
Advanced Patent Search | Web History | Sign in

Patents

Given a system which detects simple events, one can define a complex event by constructing a list of sub-events. In order to recognize a complex event, the system keeps a record of the sub-events that have occurred thus far and the objects involved in these sub-events. Whenever the first sub-event in a complex event's sequence is recognized, an activation for that complex event is created. The activation contains an indication of the identity of the object involved in the event. The activation also includes an index initialized to one. If a newly detected event matches the next sub-event in any of the currently open complex events, the index for that complex event is incremented. If the index reaches the total number of sub-events in that complex event, the complete complex event is recognized. Thus any desired alarm is generated. Since the complex event that was just recognized may also be a sub-event of another complex event, the activation lists are consulted again to see if the...

InventorsFrank Z. Brill, Thomas J. Olson
Original AssigneeTexas Instruments Incorporated
Primary Examiner: Mehrdad Dastouri
Attorneys: Robert D. Marshall, Jr., W. James Brady, III, Frederick J. Telecky, Jr.
Current U.S. Classification382/226; 348/155; 348/E07.09; 382/103
International Classification: G06K/968; G06K/900; H04N/718

View patent at USPTO
Search USPTO Assignment Database

Citations

Cited PatentFiling dateIssue dateOriginal AssigneeTitle
US5666157Jan 3, 1995Sep 9, 1997ARC IncorporatedAbnormality detection and surveillance system
US5828809Oct 1, 1996Oct 27, 1998Matsushita Electric Industrial Co., Ltd.Method and apparatus for extracting indexing information from digital video data
US5969755Feb 5, 1997Oct 19, 1999Texas Instruments IncorporatedMotion based event detection system and method
US6107918Nov 25, 1997Aug 22, 2000Micron Electronics, Inc.Method for personal computer-based home surveillance
US6295367Feb 6, 1998Sep 25, 2001Emtera CorporationSystem and method for tracking movement of objects in a scene using correspondence graphs

Referenced by

Citing PatentFiling dateIssue dateOriginal AssigneeTitle
US6970183Jun 14, 2000Nov 29, 2005E-Watch, Inc.Multimedia surveillance and monitoring system including network configuration
US7088846Aug 12, 2004Aug 8, 2006Vidient Systems, Inc.Video surveillance system that detects predefined behaviors based on predetermined patterns of movement through zones
US7127083Aug 12, 2004Oct 24, 2006Vidient Systems, Inc.Video surveillance system with object detection and probability scoring based on object class
US7131136Jul 10, 2002Oct 31, 2006E-Watch, Inc.Comprehensive multi-media surveillance and response system for aircraft, operations centers, airports and other commercial transports, centers and terminals
US7136507Aug 12, 2004Nov 14, 2006Vidient Systems, Inc.Video surveillance system with rule-based reasoning and multiple-hypothesis scoring
US7148912Aug 12, 2004Dec 12, 2006Vidient Systems, Inc.Video surveillance system in which trajectory hypothesis spawning allows for trajectory splitting and/or merging
US7173526Nov 4, 2005Feb 6, 2007Apparatus and method of collecting and distributing event data to strategic security personnel and response vehicles
US7197228Aug 28, 1998Mar 27, 2007Multifunction remote control system for audio and video recording, capture, transmission and playback of full motion and still images
US7228429Sep 21, 2001Jun 5, 2007e-WatchMultimedia network appliances for security and surveillance applications
US7319479Sep 21, 2001Jan 15, 2008Brickstream CorporationSystem and method for multi-camera linking and analysis
US7343598Dec 15, 2003Mar 11, 2008Microsoft CorporationCache-conscious coallocation of hot data streams
US7359622Feb 14, 2005Apr 15, 2008Multifunction remote control system for audio and video recording, capture, transmission and playback of full motion and still images
US7362351Dec 5, 2001Apr 22, 2008Vision IQMethod, system and device for detecting an object proximate to a water/air type interface
US7365871Jan 3, 2003Apr 29, 2008Apparatus for capturing, converting and transmitting a visual image signal via a digital transmission system
US7386105May 27, 2005Jun 10, 2008Nice Systems LtdMethod and apparatus for fraud detection
US7400249Feb 14, 2005Jul 15, 2008Networked personal security system
US7400745Jan 29, 2007Jul 15, 2008Brickstream CorporationSystems and methods for determining if objects are in a queue
US7428002Jun 5, 2002Sep 23, 2008Emergency telephone with integrated surveillance system connectivity
US7428368Nov 29, 2005Sep 23, 2008Multifunction remote control system for audio and video recording, capture, transmission and playback of full motion and still images
US7436887Feb 6, 2003Oct 14, 2008Playtex Products, Inc.Method and apparatus for video frame sequence-based object tracking
US7495562Dec 28, 2006Feb 24, 2009Networked personal security system
US7499571Sep 22, 2006Mar 3, 2009Vidient Systems, IncVideo surveillance system with rule-based reasoning and multiple-hypothesis scoring
US7511612Sep 15, 2005Mar 31, 2009Ground based security surveillance system for aircraft and other commercial vehicles
US7539357Aug 10, 1999May 26, 2009Method and apparatus for sending and receiving facsimile transmissions over a non-telephonic transmission system
US7546173Aug 18, 2003Jun 9, 2009Nice Systems, Ltd.Apparatus and method for audio content analysis, marking and summing
US7551075Dec 28, 2006Jun 23, 2009Ground based security surveillance system for aircraft and other commercial vehicles
US7561037Dec 28, 2006Jul 14, 2009Apparatus for and method of collecting and distributing event data to strategic security personnel and response vehicles
US7573421Sep 24, 2002Aug 11, 2009Nice Systems, Ltd.System and method for the automatic control of video frame rate
US7576770Feb 11, 2004Aug 18, 2009System for a plurality of video cameras disposed on a common network
US7577246Dec 20, 2006Aug 18, 2009Nice Systems Ltd.Method and system for automatic quality evaluation
US7583196Jul 28, 2004Sep 1, 2009Vision IQMethod and system for detecting a body in a zone located proximate an interface
US7583815Apr 5, 2005Sep 1, 2009ObjectVideo Inc.Wide-area site-based video surveillance system
US7587709Oct 24, 2003Sep 8, 2009Microsoft CorporationAdaptive instrumentation runtime monitoring and analysis
US7599475May 2, 2007Oct 6, 2009Nice Systems, Ltd.Method and apparatus for generic analytics
US7606425Sep 9, 2004Oct 20, 2009Honeywell International Inc.Unsupervised learning of events in a video sequence
US7607119Apr 26, 2005Oct 20, 2009Microsoft CorporationVariational path profiling
US7631046Oct 26, 2006Dec 8, 2009Nice Systems, Ltd.Method and apparatus for lawful interception of web based messaging communication
US7634334Dec 28, 2006Dec 15, 2009Record and playback system for aircraft
US7634662Nov 21, 2003Dec 15, 2009Method for incorporating facial recognition technology in a multimedia surveillance system
US7639841Dec 19, 2005Dec 29, 2009Siemens Corporation
Siemens VDO Automotive AG
System and method for on-road detection of a vehicle using knowledge fusion
US7640083Nov 21, 2003Dec 29, 2009Record and playback system for aircraft
US7643168Dec 28, 2006Jan 5, 2010Apparatus for capturing, converting and transmitting a visual image signal via a digital transmission system
US7683929Dec 26, 2002Mar 23, 2010Nice Systems, Ltd.System and method for video content analysis-based detection, surveillance and alarm management
US7698450Mar 3, 2005Apr 13, 2010Method and apparatus for distributing digitized streaming video over a network
US7698730Mar 16, 2004Apr 13, 2010Riverbed Technology, Inc.Service detection
US7702132Jul 14, 2008Apr 20, 2010Brickstream CorporationSystems and methods for determining if objects are in a queue
US7714878Aug 9, 2004May 11, 2010Nice Systems, Ltd.Apparatus and method for multimedia content based manipulation
US7716048Jan 25, 2006May 11, 2010Nice Systems, Ltd.Method and apparatus for segmentation of audio interactions
US7733371Nov 14, 2005Jun 8, 2010Digital security multimedia sensor
US7760908Mar 31, 2005Jul 20, 2010Honeywell International Inc.Event packaged video sequence
US7761544Mar 6, 2003Jul 20, 2010Nice Systems, Ltd.Method and apparatus for internal and external monitoring of a transportation vehicle
US7768566Dec 28, 2006Aug 3, 2010Dual-mode camera
US7770153May 20, 2005Aug 3, 2010Microsoft CorporationHeap-based bug identification using anomaly detection
US7770221May 18, 2006Aug 3, 2010Nice Systems, Ltd.Method and apparatus for combining traffic analysis and monitoring center in lawful interception
US7801288Feb 29, 2008Sep 21, 2010Nice Systems Ltd.Method and apparatus for fraud detection
US7801328Mar 31, 2005Sep 21, 2010Honeywell International Inc.Methods for defining, detecting, analyzing, indexing and retrieving events using video image processing
US7822605Oct 19, 2006Oct 26, 2010Nice Systems Ltd.Method and apparatus for large population speaker identification in telephone interactions
US7839926Apr 21, 2005Nov 23, 2010Bandwidth management and control
US7859396Nov 22, 2006Dec 28, 2010Multimedia network appliances for security and surveillance applications
US7868912Apr 5, 2005Jan 11, 2011ObjectVideo, Inc.Video surveillance system employing video primitives
US7881537Jan 31, 2006Feb 1, 2011Honeywell International Inc.Automated activity detection using supervised learning
US7912877May 20, 2005Mar 22, 2011Microsoft CorporationLeveraging garbage collection to dynamically infer heap invariants
US7926043Jun 20, 2006Apr 12, 2011Microsoft CorporationData structure path profiling
US7932923Sep 29, 2009Apr 26, 2011ObjectVideo, Inc.Video surveillance system employing video primitives
US7953219Jul 18, 2002May 31, 2011Nice Systems, Ltd.Method apparatus and system for capturing and analyzing interaction based content
US7962901Apr 17, 2006Jun 14, 2011Microsoft CorporationUsing dynamic analysis to improve model checking
US8005675Mar 17, 2005Aug 23, 2011Nice Systems, Ltd.Apparatus and method for audio analysis
US8046752Nov 15, 2005Oct 25, 2011Microsoft CorporationDynamic prefetching of hot data streams
US8060364Nov 13, 2003Nov 15, 2011Nice Systems, Ltd.Apparatus and method for event-driven content analysis
US8078463Nov 23, 2004Dec 13, 2011Nice Systems, Ltd.Method and apparatus for speaker spotting
US8184154Feb 27, 2007May 22, 2012Texas Instruments IncorporatedVideo surveillance correlating detected moving objects and RF signals
US8204884May 2, 2011Jun 19, 2012Method, apparatus and system for capturing and analyzing interaction based content
US8207848May 18, 2009Jun 26, 2012Google Inc.Locking system for shipping container including bolt seal and electronic device with arms for receiving bolt seal
US8243990Dec 31, 2008Aug 14, 2012Industrial Technology Research InstituteMethod for tracking moving object

Claims

1. A computer implemented method for recognizing complex events in a sequence of video images comprising the steps of:

defining at least one complex event as an ordered list of a predetermined number of events;
recognizing simple events in the sequence of video images;
upon recognition of a simple event
determining if said recognized simple event is a first event in said set ordered list of events of a complex event, and if so activating a complex event recognition process for said corresponding complex event and initializing an index for said complex event recognition process to 1;
determining if said recognized simple event is a next event in said ordered list of events of an activated complex event recognition process, and if so logging said recognized event in said complex event recognition process and incrementing said index of said complex event recognition process; and
determining if all events in an activated complex event recognition process have been detected by determining if said corresponding index equals said predetermined number of events in said corresponding complex event, and if so recognizing said corresponding complex event.

2. The computer implemented method of claim 1, wherein:

said step of defining at least one complex event comprises defining a complex event including another defined complex event;
said method further comprises upon recognition of a complex event
determining if said recognized complex event is a first event in said ordered list of events of another complex event, and if so activating a recognition process for said corresponding complex event and initializing an index for said other complex event recognition process to 1;
determining if said recognized complex event is a next event in said ordered list of events in an activated complex event recognition process, and if so logging said recognized event in said complex event recognition process and incrementing said index of said complex event recognition process; and
determining if all events in an activated complex event recognition process have been detected by determining if said corresponding index equals said predetermined number of events in said corresponding complex event, and if so recognizing said corresponding complex event.

3. The computer implemented method of claim 1, wherein:

said step of defining a complex-event comprises defining a complex event includes at least one negated event;
said method further comprises the steps of upon recognition of an event determining if said recognized event matches a non-negated event corresponding to a next negated event in said ordered list of an activated complex event recognition process,
if so deactivating said complex event recognition process, and
if not determining if said recognized event matches a next non-negated event of an activate complex event recognition process, and if so logging said recognized event and all intervening negated events in said complex event recognition process and incrementing said index for said recognized event and for each intervening negated event.

4. The computer implemented process of claim 3, wherein:

said step of defining a complex event includes an initial negated event; and
said method further comprises determining if said recognized event matches a non-negated event corresponding to said initial negated event of a complex event, and if so disqualifying a corresponding tracked object relative to said corresponding complex event.

5. The computer implemented process of claim 3, wherein:

said step of defining a complex event includes a final negated event; and
said method further comprises upon detection of a final non-negated event in said corresponding complex event, recognizing said complex event if no detection of said negated event occurs within a predetermined period of time.

6. The computer implemented process of claim 3, wherein:

said simple events include an EXIT event wherein an object previously present in said sequence of video images is no longer present;
said step of defining a complex event includes a final negated event; and
said method further comprises upon detection of a final non-negated event in said corresponding complex event, recognizing said complex event if no detection of said negated event occurs before a corresponding object EXITs.