Given a system which detects simple events, one can define a complex event by constructing a list of sub-events. In order to recognize a complex event, the system keeps a record of the sub-events that have occurred thus far and the objects involved in these sub-events. Whenever the first sub-event in a complex event's sequence is recognized, an activation for that complex event is created. The activation contains an indication of the identity of the object involved in the event. The activation also includes an index initialized to one. If a newly detected event matches the next sub-event in any of the currently open complex events, the index for that complex event is incremented. If the index reaches the total number of sub-events in that complex event, the complete complex event is recognized. Thus any desired alarm is generated. Since the complex event that was just recognized may also be a sub-event of another complex event, the activation lists are consulted again to see if the... |
Citations|
| US5666157 | Jan 3, 1995 | Sep 9, 1997 | ARC Incorporated | Abnormality detection and surveillance system | | US5828809 | Oct 1, 1996 | Oct 27, 1998 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus for extracting indexing information from digital video data | | US5969755 | Feb 5, 1997 | Oct 19, 1999 | Texas Instruments Incorporated | Motion based event detection system and method | | US6107918 | Nov 25, 1997 | Aug 22, 2000 | Micron Electronics, Inc. | Method for personal computer-based home surveillance | | US6295367 | Feb 6, 1998 | Sep 25, 2001 | Emtera Corporation | System and method for tracking movement of objects in a scene using correspondence graphs |
Referenced by|
| US6970183 | Jun 14, 2000 | Nov 29, 2005 | E-Watch, Inc. | Multimedia surveillance and monitoring system including network configuration | | US7088846 | Aug 12, 2004 | Aug 8, 2006 | Vidient Systems, Inc. | Video surveillance system that detects predefined behaviors based on predetermined patterns of movement through zones | | US7127083 | Aug 12, 2004 | Oct 24, 2006 | Vidient Systems, Inc. | Video surveillance system with object detection and probability scoring based on object class | | US7131136 | Jul 10, 2002 | Oct 31, 2006 | E-Watch, Inc. | Comprehensive multi-media surveillance and response system for aircraft, operations centers, airports and other commercial transports, centers and terminals | | US7136507 | Aug 12, 2004 | Nov 14, 2006 | Vidient Systems, Inc. | Video surveillance system with rule-based reasoning and multiple-hypothesis scoring | | US7148912 | Aug 12, 2004 | Dec 12, 2006 | Vidient Systems, Inc. | Video surveillance system in which trajectory hypothesis spawning allows for trajectory splitting and/or merging | | US7173526 | Nov 4, 2005 | Feb 6, 2007 | | Apparatus and method of collecting and distributing event data to strategic security personnel and response vehicles | | US7197228 | Aug 28, 1998 | Mar 27, 2007 | | Multifunction remote control system for audio and video recording, capture, transmission and playback of full motion and still images | | US7228429 | Sep 21, 2001 | Jun 5, 2007 | e-Watch | Multimedia network appliances for security and surveillance applications | | US7319479 | Sep 21, 2001 | Jan 15, 2008 | Brickstream Corporation | System and method for multi-camera linking and analysis | | US7343598 | Dec 15, 2003 | Mar 11, 2008 | Microsoft Corporation | Cache-conscious coallocation of hot data streams | | US7359622 | Feb 14, 2005 | Apr 15, 2008 | | Multifunction remote control system for audio and video recording, capture, transmission and playback of full motion and still images | | US7362351 | Dec 5, 2001 | Apr 22, 2008 | Vision IQ | Method, system and device for detecting an object proximate to a water/air type interface | | US7365871 | Jan 3, 2003 | Apr 29, 2008 | | Apparatus for capturing, converting and transmitting a visual image signal via a digital transmission system | | US7386105 | May 27, 2005 | Jun 10, 2008 | Nice Systems Ltd | Method and apparatus for fraud detection | | US7400249 | Feb 14, 2005 | Jul 15, 2008 | | Networked personal security system | | US7400745 | Jan 29, 2007 | Jul 15, 2008 | Brickstream Corporation | Systems and methods for determining if objects are in a queue | | US7428002 | Jun 5, 2002 | Sep 23, 2008 | | Emergency telephone with integrated surveillance system connectivity | | US7428368 | Nov 29, 2005 | Sep 23, 2008 | | Multifunction remote control system for audio and video recording, capture, transmission and playback of full motion and still images | | US7436887 | Feb 6, 2003 | Oct 14, 2008 | Playtex Products, Inc. | Method and apparatus for video frame sequence-based object tracking | | US7495562 | Dec 28, 2006 | Feb 24, 2009 | | Networked personal security system | | US7499571 | Sep 22, 2006 | Mar 3, 2009 | Vidient Systems, Inc | Video surveillance system with rule-based reasoning and multiple-hypothesis scoring | | US7511612 | Sep 15, 2005 | Mar 31, 2009 | | Ground based security surveillance system for aircraft and other commercial vehicles | | US7539357 | Aug 10, 1999 | May 26, 2009 | | Method and apparatus for sending and receiving facsimile transmissions over a non-telephonic transmission system | | US7546173 | Aug 18, 2003 | Jun 9, 2009 | Nice Systems, Ltd. | Apparatus and method for audio content analysis, marking and summing | | US7551075 | Dec 28, 2006 | Jun 23, 2009 | | Ground based security surveillance system for aircraft and other commercial vehicles | | US7561037 | Dec 28, 2006 | Jul 14, 2009 | | Apparatus for and method of collecting and distributing event data to strategic security personnel and response vehicles | | US7573421 | Sep 24, 2002 | Aug 11, 2009 | Nice Systems, Ltd. | System and method for the automatic control of video frame rate | | US7576770 | Feb 11, 2004 | Aug 18, 2009 | | System for a plurality of video cameras disposed on a common network | | US7577246 | Dec 20, 2006 | Aug 18, 2009 | Nice Systems Ltd. | Method and system for automatic quality evaluation | | US7583196 | Jul 28, 2004 | Sep 1, 2009 | Vision IQ | Method and system for detecting a body in a zone located proximate an interface | | US7583815 | Apr 5, 2005 | Sep 1, 2009 | ObjectVideo Inc. | Wide-area site-based video surveillance system | | US7587709 | Oct 24, 2003 | Sep 8, 2009 | Microsoft Corporation | Adaptive instrumentation runtime monitoring and analysis | | US7599475 | May 2, 2007 | Oct 6, 2009 | Nice Systems, Ltd. | Method and apparatus for generic analytics | | US7606425 | Sep 9, 2004 | Oct 20, 2009 | Honeywell International Inc. | Unsupervised learning of events in a video sequence | | US7607119 | Apr 26, 2005 | Oct 20, 2009 | Microsoft Corporation | Variational path profiling | | US7631046 | Oct 26, 2006 | Dec 8, 2009 | Nice Systems, Ltd. | Method and apparatus for lawful interception of web based messaging communication | | US7634334 | Dec 28, 2006 | Dec 15, 2009 | | Record and playback system for aircraft | | US7634662 | Nov 21, 2003 | Dec 15, 2009 | | Method for incorporating facial recognition technology in a multimedia surveillance system | | US7639841 | Dec 19, 2005 | Dec 29, 2009 | Siemens Corporation Siemens VDO Automotive AG | System and method for on-road detection of a vehicle using knowledge fusion | | US7640083 | Nov 21, 2003 | Dec 29, 2009 | | Record and playback system for aircraft | | US7643168 | Dec 28, 2006 | Jan 5, 2010 | | Apparatus for capturing, converting and transmitting a visual image signal via a digital transmission system | | US7683929 | Dec 26, 2002 | Mar 23, 2010 | Nice Systems, Ltd. | System and method for video content analysis-based detection, surveillance and alarm management | | US7698450 | Mar 3, 2005 | Apr 13, 2010 | | Method and apparatus for distributing digitized streaming video over a network | | US7698730 | Mar 16, 2004 | Apr 13, 2010 | Riverbed Technology, Inc. | Service detection | | US7702132 | Jul 14, 2008 | Apr 20, 2010 | Brickstream Corporation | Systems and methods for determining if objects are in a queue | | US7714878 | Aug 9, 2004 | May 11, 2010 | Nice Systems, Ltd. | Apparatus and method for multimedia content based manipulation | | US7716048 | Jan 25, 2006 | May 11, 2010 | Nice Systems, Ltd. | Method and apparatus for segmentation of audio interactions | | US7733371 | Nov 14, 2005 | Jun 8, 2010 | | Digital security multimedia sensor | | US7760908 | Mar 31, 2005 | Jul 20, 2010 | Honeywell International Inc. | Event packaged video sequence | | US7761544 | Mar 6, 2003 | Jul 20, 2010 | Nice Systems, Ltd. | Method and apparatus for internal and external monitoring of a transportation vehicle | | US7768566 | Dec 28, 2006 | Aug 3, 2010 | | Dual-mode camera | | US7770153 | May 20, 2005 | Aug 3, 2010 | Microsoft Corporation | Heap-based bug identification using anomaly detection | | US7770221 | May 18, 2006 | Aug 3, 2010 | Nice Systems, Ltd. | Method and apparatus for combining traffic analysis and monitoring center in lawful interception | | US7801288 | Feb 29, 2008 | Sep 21, 2010 | Nice Systems Ltd. | Method and apparatus for fraud detection | | US7801328 | Mar 31, 2005 | Sep 21, 2010 | Honeywell International Inc. | Methods for defining, detecting, analyzing, indexing and retrieving events using video image processing | | US7822605 | Oct 19, 2006 | Oct 26, 2010 | Nice Systems Ltd. | Method and apparatus for large population speaker identification in telephone interactions | | US7839926 | Apr 21, 2005 | Nov 23, 2010 | | Bandwidth management and control | | US7859396 | Nov 22, 2006 | Dec 28, 2010 | | Multimedia network appliances for security and surveillance applications | | US7868912 | Apr 5, 2005 | Jan 11, 2011 | ObjectVideo, Inc. | Video surveillance system employing video primitives | | US7881537 | Jan 31, 2006 | Feb 1, 2011 | Honeywell International Inc. | Automated activity detection using supervised learning | | US7912877 | May 20, 2005 | Mar 22, 2011 | Microsoft Corporation | Leveraging garbage collection to dynamically infer heap invariants | | US7926043 | Jun 20, 2006 | Apr 12, 2011 | Microsoft Corporation | Data structure path profiling | | US7932923 | Sep 29, 2009 | Apr 26, 2011 | ObjectVideo, Inc. | Video surveillance system employing video primitives | | US7953219 | Jul 18, 2002 | May 31, 2011 | Nice Systems, Ltd. | Method apparatus and system for capturing and analyzing interaction based content | | US7962901 | Apr 17, 2006 | Jun 14, 2011 | Microsoft Corporation | Using dynamic analysis to improve model checking | | US8005675 | Mar 17, 2005 | Aug 23, 2011 | Nice Systems, Ltd. | Apparatus and method for audio analysis | | US8046752 | Nov 15, 2005 | Oct 25, 2011 | Microsoft Corporation | Dynamic prefetching of hot data streams | | US8060364 | Nov 13, 2003 | Nov 15, 2011 | Nice Systems, Ltd. | Apparatus and method for event-driven content analysis | | US8078463 | Nov 23, 2004 | Dec 13, 2011 | Nice Systems, Ltd. | Method and apparatus for speaker spotting | | US8184154 | Feb 27, 2007 | May 22, 2012 | Texas Instruments Incorporated | Video surveillance correlating detected moving objects and RF signals | | US8204884 | May 2, 2011 | Jun 19, 2012 | | Method, apparatus and system for capturing and analyzing interaction based content | | US8207848 | May 18, 2009 | Jun 26, 2012 | Google Inc. | Locking system for shipping container including bolt seal and electronic device with arms for receiving bolt seal | | US8243990 | Dec 31, 2008 | Aug 14, 2012 | Industrial Technology Research Institute | Method for tracking moving object |
Claims1. A computer implemented method for recognizing complex events in a sequence of video images comprising the steps of: - defining at least one complex event as an ordered list of a predetermined number of events;
- recognizing simple events in the sequence of video images;
- upon recognition of a simple event
- determining if said recognized simple event is a first event in said set ordered list of events of a complex event, and if so activating a complex event recognition process for said corresponding complex event and initializing an index for said complex event recognition process to 1;
- determining if said recognized simple event is a next event in said ordered list of events of an activated complex event recognition process, and if so logging said recognized event in said complex event recognition process and incrementing said index of said complex event recognition process; and
- determining if all events in an activated complex event recognition process have been detected by determining if said corresponding index equals said predetermined number of events in said corresponding complex event, and if so recognizing said corresponding complex event.
2. The computer implemented method of claim 1, wherein: - said step of defining at least one complex event comprises defining a complex event including another defined complex event;
- said method further comprises upon recognition of a complex event
- determining if said recognized complex event is a first event in said ordered list of events of another complex event, and if so activating a recognition process for said corresponding complex event and initializing an index for said other complex event recognition process to 1;
- determining if said recognized complex event is a next event in said ordered list of events in an activated complex event recognition process, and if so logging said recognized event in said complex event recognition process and incrementing said index of said complex event recognition process; and
- determining if all events in an activated complex event recognition process have been detected by determining if said corresponding index equals said predetermined number of events in said corresponding complex event, and if so recognizing said corresponding complex event.
3. The computer implemented method of claim 1, wherein: - said step of defining a complex-event comprises defining a complex event includes at least one negated event;
- said method further comprises the steps of upon recognition of an event determining if said recognized event matches a non-negated event corresponding to a next negated event in said ordered list of an activated complex event recognition process,
- if so deactivating said complex event recognition process, and
- if not determining if said recognized event matches a next non-negated event of an activate complex event recognition process, and if so logging said recognized event and all intervening negated events in said complex event recognition process and incrementing said index for said recognized event and for each intervening negated event.
4. The computer implemented process of claim 3, wherein: - said step of defining a complex event includes an initial negated event; and
- said method further comprises determining if said recognized event matches a non-negated event corresponding to said initial negated event of a complex event, and if so disqualifying a corresponding tracked object relative to said corresponding complex event.
5. The computer implemented process of claim 3, wherein: - said step of defining a complex event includes a final negated event; and
- said method further comprises upon detection of a final non-negated event in said corresponding complex event, recognizing said complex event if no detection of said negated event occurs within a predetermined period of time.
6. The computer implemented process of claim 3, wherein: - said simple events include an EXIT event wherein an object previously present in said sequence of video images is no longer present;
- said step of defining a complex event includes a final negated event; and
- said method further comprises upon detection of a final non-negated event in said corresponding complex event, recognizing said complex event if no detection of said negated event occurs before a corresponding object EXITs.
|