|Publication number||US7049965 B2|
|Application number||US 10/677,207|
|Publication date||May 23, 2006|
|Filing date||Oct 2, 2003|
|Priority date||Oct 2, 2003|
|Also published as||US20050073418|
|Publication number||10677207, 677207, US 7049965 B2, US 7049965B2, US-B2-7049965, US7049965 B2, US7049965B2|
|Inventors||Timothy Patrick Kelliher, Jens Rittscher, Peter Henry Tu, Kevin Chean, Harold Woodruff Tomlinson|
|Original Assignee||General Electric Company|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (9), Referenced by (54), Classifications (27), Legal Events (4)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present disclosure generally relates to surveillance systems and methods. In particular, the present disclosure relates to surveillance systems and methods that combine video and radio frequency identification.
Shoplifting prevention and inventory control are becoming more important to many commercial retail stores as way to minimize loses. Surveillance systems and methods are often used to achieve the desired reduction in losses.
Video surveillance systems are a common tool used in the efforts to prevent shoplifting and control inventory. Typical video surveillance systems use one or more cameras to survey an area. This allows a security officer to track a potential shoplifter through a shopping area, which is in the line of sight of the camera. Unfortunately, such video surveillance systems alone have not proven effective at achieving the desired reductions in shoplifting at an acceptable cost.
Radio frequency identification (RFID) systems are also becoming commonplace in the efforts to prevent shoplifting and control inventory. Advantageously, RFID does not require direct contact or line-of-sight scanning as in video surveillance systems. RFID systems incorporate the use of a tag and a scanner. The tag can emit electromagnetic or electrostatic signal in the radio frequency (RF) portion of the electromagnetic spectrum. The tag can then be placed on an object, animal, or person to uniquely identify that item. The scanner can detect the presence or absence of the emitted signal. RFID is sometimes called dedicated short-range communication (DSRC) since the emitted signal can be detected by the scanner within about a one-meter radius. Accordingly, many retail outlets have installed scanners at the points of entry and/or exit and include the tag on a piece of merchandise. In this manner, any merchandise having an active RFID tag will be detected as the item passes the scanner. The retail outlet can selectively deactivate and/or remove the tag of items that are approved to exit the area, such as those purchased by a customer. Unfortunately, such RFID systems alone have also not proven effective at achieving the desired reductions in shoplifting at an acceptable cost.
Accordingly, there is a continuing need for surveillance systems and methods that overcome and/or mitigate one or more of the aforementioned and other deficiencies and deleterious effects of prior systems and methods.
A surveillance system having a video subsystem, a radio frequency identification subsystem, and a processor is provided. The video subsystem detects a video image of a tagged item. The radio frequency identification subsystem detects a position of the tagged item. The processor communicates with the video and radio frequency subsystems to monitor a condition of the tagged item based at least in part on the video image and the position.
A surveillance system having a first loop antenna, a second loop antenna, and a signal processor. The second loop antenna is substantially orthogonal to the first loop antenna. The first and second loop antennas are inductively couplable with a tag through magnetic fields. The signal processor estimates an orientation of said tag based on the magnitude of the inductively coupled modulated signal from the tag as the orientation of the coupling field generated from the first and second loop antennas is scanned through a range of angles.
A surveillance method is also provided. The method includes determining an orientation of an RFID tag, determining a position of the RFID tag; and providing the orientation and the position to a video-processing component.
These and other features, aspects, and advantages of the present disclosure will become better understood with reference to the following description, appended claims, and drawings where:
Referring now to
RFID subsystem 16 includes a number or plurality of scanners 22 at predetermined locations within area 12. Similarly, video subsystem 18 includes a number or plurality of cameras 24 at predetermined locations within area 12. Scanners 22 and cameras 24 are in electrical communication with processor 14 so that surveillance system 10 can integrate data received from the scanners and cameras to provide enhanced surveillance of item 20.
Scanners 22 can detect RFID tagged item 20 when the item is within about a one-meter radius. Cameras 24 can detect RFID tagged item 20 when it is within the field of view of one of the cameras. Advantageously, surveillance system 10 is configured to track tagged item 20 using scanners 22 when with the detection range of the scanners, but using cameras 24 when with the field of view of the cameras. System 10 can automatically switch its surveillance of tagged item 20 between scanners 22 and cameras 24 within area 12 as the item is moved throughout the area. In this manner, system 10 can determine the position of tagged item 20 within a surveillance area 12.
It has been determined that it can be more difficult to discriminate between casual behavior and theft with only position information. Without knowing the orientation of tagged item 20, it can be difficult to recognize desired information about the tagged item when viewed by cameras 24. Accordingly, system 10 is configured to determine the orientation of tagged item 20 within surveillance area 12.
For example, some customer behavior with respect to tagged item 20 cannot be detected under certain conditions of system 10, such as when the tagged item is partially outside the field of cameras 24. It has been found that combining the position and orientation of tagged item 20 from RFID subsystem 16 with the video from video subsystem 18 allows surveillance system 10 to efficiently predict an expected appearance of the tagged item. Here, surveillance system 10 can then compare the expected appearance to an actual video image to detect authorized tampering.
Referring now to
Loop antennas 100 are driven by a z ramp generator 104, a y ramp generator 106, and an x ramp generator 108, respectively, through voltage variable attenuators 110 and amplifiers 112. Antennas 100 receive signals indicative of tagged item 20. These received signals are summed through a splitter 114 and are sent to a receive chain 116.
It should be recognized that scanner 22 is illustrated by way of example having three loop antennas 100. Of course, it is contemplated by the present disclosure for each scanner 22 to have less than three loop antennas 100. For example, it is contemplated for scanner 22 to have two loop antennas 100.
Loop antennas 100 can be any loop antenna, such as the Texas Instruments (TI) Series 6000 Gate Antenna RI-ANT-T01. This TI gate antenna is used with readers having a transmitter frequency of 13.56 MHz and an output impedance of 50 Ohm, such as the TI S6500/6550 Readers.
Tagged item 20, in this example, includes an inductive passive tag capable of being read by loop antennas 100 when the tagged item 20 is within an interrogation zone 102 of scanner 20. Interrogation zone 102 can be about one meter in each direction from loop antennas 100. ISO Standard 15693-2, a communications protocol, defines one method for reading data from inductive passive tags. In this example, tagged item 20 is inductively coupled with magnetic fields through loop antennas 100.
Z ramp generator 104, y ramp generator 106, and x ramp generator 108 control the amplitude of the 13.56 MHz RF antenna excitation waveform for antennas 100 by way of a ramp waveform. For example, ramp generator 104, 106, 108 can be the Agilent Technologies 33220A Function/Arbitrary Waveform Generator. Attenuator 110 is a device for reducing the amplitude of an AC wave without introducing appreciable distortion. Amplifier 112 is an electronic device that increases the voltage, current, and/or power of a signal. Splitter 114 is a device that divides a signal into two or more signals, each carrying a selected frequency range, or reassembles signals from multiple signal sources into a single signal. An example of splitter 114 is Mini-Circuit's power splitter ZSC-2-1.
Receive chain 116 is a signal processing component that includes, for example, a bandpass filter 118, an envelope detector 120, a modulation minimum (null) detector 122, and an angle calculator 124. In some embodiments, there is a receive chain for each loop antenna 100. The resulting orientation calculated by angle calculator 124 is provided to a video processing component 126.
Bandpass filter 118 is an electronic device or circuit that allows signals between two specific frequencies to pass, but that discriminates against signals at other frequencies. An example bandpass filer 118 has a filter passband of 13.98375 MHz±50 KHz. Envelope detector 120 detects the envelope (upper and lower bounds) of the waveform as described in detail below with respect to
Modulation minimum detector 122 finds the point at which the envelope is at a minimum (null). The tag modulation minimum indicates a magnetic field is at substantially right angles to tagged item 20. Angle calculator 124 determines the orientation of tagged item 20. At certain times during the antenna excitation, the magnitude of the tag modulation signal as received by a single antenna can be measured. The measurements for the X, Y, and Z antenna can be used with the orientation of the tagged item to determine the position of the tagged item in the interrogation zone of the antenna.
Video processing component 126 is provided by video subsystem 18 to processor 14. Video subsystem 18 is any video system capable of tracking tagged item 20, such as merchandise, in area 12. In one embodiment, video processing component 126 comprises a tracking mechanism, an object verification mechanism, and a recognition mechanism. The tracking mechanism tracks people and objects. The object verification mechanism verifies tag information with video images. The recognition mechanism recognizes patterns in the video images.
After receiving the orientation from RFID subsystem 16, processor 14 is able to use the orientation of tagged item 20 to compare the video image of the object to the expected appearance of the object at that orientation. As a result, some tampering and shoplifting is detectable.
Processor 14 can communicate with subsystems 16, 18 by known communication methods such as, but not limited to, as Ethernet. Here, video processing component 126 includes software components, such as segmentation routines, temporal association routines, geometric reconstruction routines, RFID object detection, RFID position and orientation detection, object tracking, person tracking, behavior analysis, probabilistic engines, and Bayesian frameworks.
In some embodiments, video processing component 126 activates RFID subsystem 16 when a person is within interrogation zone 102 of scanner 20. If a person is in zone 102 with tagged item 20, the person changes the magnetic coupling between tagged item 20 and loop antennas 100 by virtue of their body being present in the magnetic field. System 10 is configured to detect these changes the magnetic coupling between tagged item 20 and loop antennas 100 by virtue of their body being present in the magnetic field.
A 13.56 MHz clock signal 130, and other clock signals 128 are included in example subsystem 16, which has a frequency of 13.56 MHz. In some embodiments one or more of clock signals 128 is a frame rate clock from video processing component 126.
Referring now to
In some embodiments, the clock signal in first row 300 is a frame rate clock from video processing component 126. In some embodiments, the clock signal is dependent on how long it takes to read tagged item 20.
At the start of the first clock period, x ramp generator 108 is at full power, y ramp generator 106 is at zero, and z ramp generator 104 is at zero. Under these conditions, loop antenna 100 in the x-direction is excited and an x-amplitude tag modulation signal 312 (shown in row six 310) is read from receive chain 116. The x-amplitude of the inductive signal is used to correct for the x, y, and z offset and to get the x-co ordinate of the position (x, y, z) of tagged item 20.
About in the middle of the first clock period, y ramp generator 106 is at full power, x ramp generator 108 is at zero, and z ramp generator 104 is at zero. Under these conditions, loop antenna 100 in the y-direction is excited and a y-amplitude tag modulation signal 314 (shown in row six 310) is read from receive chain 116. The y-amplitude of the inductive signal is used to correct for the x, y, and z offset and to get the y-coordinate of the position (x, y, z) of tagged item 20.
About in the middle of the second clock period, z ramp generator 104 is at full power, x ramp generator 108 is at zero, and y ramp generator 106 is at zero. Under these conditions, loop antenna 100 in the z-direction is excited and a z-amplitude tag modulation signal 316 (shown in row six 310) is read from receive chain 116. The z-amplitude of the inductive signal is used to correct for the x, y, and z offset and to get the z-coordinate of the position (x, y, z) of tagged item 20. In some embodiments, the x-, y-, and z-coordinates are all read within one clock period, or about the time it takes to read tagged item 20.
As shown in row five 308, each angle in the orientation (Φ, α, θ) is calculated at a tag modulation minimum (null) 318 (shown in row six 310). Angle Φ (phi) 200 is calculated, when an x-antenna signal is zero and tag modulation minimum 318 occurs. Angle θ (theta) 206 is calculated, when a z-antenna signal is zero and tag modulation minimum 318 occurs. Angle α (alpha) 210 is calculated, when a y-antenna signal is zero and tag modulation minimum 318 occurs.
For example, suppose video processing component 126 recognizes a person stopping in front of a table displaying tagged item 20. Based on the video information from subsystem 18, RFID subsystem 16 is activated. As the person interacts with the tagged item 20, system 10 tracks hand motions, face motions, RFID information of the object and the like. If the person picks up the item, video subsystem 18 tracks the person within area 12 to establish whether the person placed the item down. For example, video subsystem 18 analyses the tracking of the person and the object, including object recognition and RFID. If the item was not placed anywhere then a strong hypothesis is built based on the interaction that the person still has the item. If so, a real-time alert is produced and a synopsis is provided, including salient video clips. In addition, a history of the person and object tracking is available.
Orientation information provided by RFID subsystem 16 to surveillance system 10 aids in analysis. For example, system 10 can analyze events, such as whether the object was placed in a shopping cart or handled in a secretive fashion using inputs from subsystems 16, 18. Another example is analyzing the appearance of tagged item 20. Here, surveillance system 10 can generate a synthesized appearance of the tagged item at the orientation provided by RFID subsystem 16. Surveillance system 10 can then compare the synthesized appearance with the actual appearance provided by video subsystem 18 to determine whether tagged item has been altered (e.g., authorized tampering).
While the present disclosure has been described with reference to one or more exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the present disclosure. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from the scope thereof. Therefore, it is intended that the present disclosure not be limited to the particular embodiment(s) disclosed as the best mode contemplated, but that the disclosure will include all embodiments falling within the scope of the appended claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5247261||Oct 9, 1991||Sep 21, 1993||The Massachusetts Institute Of Technology||Method and apparatus for electromagnetic non-contact position measurement with respect to one or more axes|
|US5558091||Oct 6, 1993||Sep 24, 1996||Biosense, Inc.||Magnetic determination of position and orientation|
|US5745036||Sep 12, 1996||Apr 28, 1998||Checkpoint Systems, Inc.||Electronic article security system for store which uses intelligent security tags and transaction data|
|US5955971 *||Jul 15, 1998||Sep 21, 1999||Nec Corporation||Window-passage detection system of an airplane|
|US6028626||Jul 22, 1997||Feb 22, 2000||Arc Incorporated||Abnormality detection and surveillance system|
|US6218941||Jul 1, 1998||Apr 17, 2001||International Business Machines Corporation||Method and system for detecting an authorized tamper event|
|US6703935 *||Sep 19, 2002||Mar 9, 2004||Amerasia International Technology, Inc.||Antenna arrangement for RFID smart tags|
|US20020175805 *||Nov 29, 2000||Nov 28, 2002||Ludwig Kipp||Method and system for communicating with and tracking RFID transponders|
|US20040236193 *||Jun 5, 2002||Nov 25, 2004||Yehuda Sharf||Birth monitoring system|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7516888 *||Jun 20, 2005||Apr 14, 2009||Stoplift, Inc.||Method and apparatus for auditing transaction activity in retail and other environments using visual recognition|
|US7631808 *||Mar 29, 2006||Dec 15, 2009||Stoplift, Inc.||Method and apparatus for detecting suspicious activity using video analysis|
|US7859408||Mar 28, 2007||Dec 28, 2010||Round Rock Research, Llc||Methods and systems of determining physical characteristics associated with objects tagged with RFID tags|
|US7880618||Feb 1, 2011||Round Rock Research, Llc||Methods and systems of determining physical characteristics associated with objects tagged with RFID tags|
|US7911346||Mar 22, 2011||Emc Corporation||Inventory control using anonymous transaction tokens|
|US7932814||Oct 4, 2007||Apr 26, 2011||Round Rock Research, Llc||Method and system to determine physical parameters as between a RFID tag and a reader|
|US7944356||May 17, 2011||Round Rock Research, Llc||Method and system to determine physical parameters as between an RFID tag and a reader|
|US8094026||Jan 10, 2012||Robert M Green||Organized retail crime detection security system and method|
|US8102238||Jan 24, 2012||International Business Machines Corporation||Using an RFID device to enhance security by determining whether a person in a secure area is accompanied by an authorized person|
|US8115623||Aug 18, 2011||Feb 14, 2012||Robert M Green||Method and system for hand basket theft detection|
|US8146811||Mar 12, 2008||Apr 3, 2012||Stoplift, Inc.||Cart inspection for suspicious items|
|US8165349||Apr 24, 2012||International Business Machines Corporation||Analyzing repetitive sequential events|
|US8242888||Aug 14, 2012||Keystone Technology Solutions, Llc||Systems and methods to determine motion parameters using RFID tags|
|US8295597 *||Oct 23, 2012||Videomining Corporation||Method and system for segmenting people in a physical space based on automatic behavior analysis|
|US8430312 *||Apr 2, 2012||Apr 30, 2013||Stoplift, Inc.||Cart inspection for suspicious items|
|US8461966||Jun 11, 2013||Micron Technology, Inc.||Systems and methods to determine kinematical parameters using RFID tags|
|US8462212||Jun 11, 2013||Stoplift, Inc.||Correlating detected events with image data|
|US8477033 *||Jun 11, 2007||Jul 2, 2013||Emc Corporation||Inventory control|
|US8570375||Dec 4, 2008||Oct 29, 2013||Stoplift, Inc.||Method and apparatus for random-access review of point of sale transactional video|
|US8633805||Jun 26, 2012||Jan 21, 2014||Micron Technology, Inc.||Systems and methods to determine motion parameters using RFID tags|
|US8750156||Nov 21, 2013||Jun 10, 2014||DGS Global Systems, Inc.||Systems, methods, and devices for electronic spectrum management for identifying open space|
|US8777224||Nov 9, 2007||Jul 15, 2014||Igt||System and methods for dealing a video card|
|US8780968||Nov 18, 2013||Jul 15, 2014||DGS Global Systems, Inc.||Systems, methods, and devices for electronic spectrum management|
|US8787836||Nov 22, 2013||Jul 22, 2014||DGS Global Systems, Inc.||Systems, methods, and devices having databases and automated reports for electronic spectrum management|
|US8798548||Nov 21, 2013||Aug 5, 2014||DGS Global Systems, Inc.||Systems, methods, and devices having databases for electronic spectrum management|
|US8805291||Nov 18, 2013||Aug 12, 2014||DGS Global Systems, Inc.||Systems, methods, and devices for electronic spectrum management|
|US8805292||Nov 21, 2013||Aug 12, 2014||DGS Global Systems, Inc.||Systems, methods, and devices for electronic spectrum management for identifying signal-emitting devices|
|US8824536||Nov 18, 2013||Sep 2, 2014||DGS Global Systems, Inc.||Systems, methods, and devices for electronic spectrum management|
|US8830062||Jun 5, 2008||Sep 9, 2014||Micron Technology, Inc.||Systems and methods to use radar in RFID systems|
|US8847761||Mar 31, 2008||Sep 30, 2014||Emc Corporation||Anonymous transaction tokens|
|US9013308 *||Nov 13, 2012||Apr 21, 2015||Apple Inc.||Three dimensional RF signatures|
|US9030301||May 9, 2013||May 12, 2015||Micron Technology, Inc.||Systems and methods to determine kinematical parameters using RFID tags|
|US9068836 *||Oct 18, 2007||Jun 30, 2015||Carlos Arteaga||Real-time location information system using multiple positioning technologies|
|US9078162||Jun 7, 2013||Jul 7, 2015||DGS Global Systems, Inc.||Systems, methods, and devices for electronic spectrum management|
|US9288683||Jun 7, 2013||Mar 15, 2016||DGS Global Systems, Inc.||Systems, methods, and devices for electronic spectrum management|
|US20060243798 *||Mar 29, 2006||Nov 2, 2006||Malay Kundu||Method and apparatus for detecting suspicious activity using video analysis|
|US20070057049 *||Mar 29, 2006||Mar 15, 2007||Malay Kundu||Method and apparatus for detecting suspicious activity using video analysis|
|US20080001753 *||Jun 11, 2007||Jan 3, 2008||Claudatos Christopher H||Inventory control|
|US20080226129 *||Mar 12, 2008||Sep 18, 2008||Malay Kundu||Cart Inspection for Suspicious Items|
|US20080238685 *||Mar 28, 2007||Oct 2, 2008||Micron Technology, Inc.||Methods and systems of determining physical characteristics associated with objects tagged with rfid tags|
|US20080238686 *||Mar 28, 2007||Oct 2, 2008||Micron Technology, Inc.||Methods and systems of determining physical characteristics associated with objects tagged with rfid tags|
|US20080280560 *||May 9, 2007||Nov 13, 2008||Micron Technology, Inc.||Method and system of placing a rfid tag in a continuous transmission mode|
|US20090091428 *||Mar 7, 2008||Apr 9, 2009||Keystone Technology Solutions, Llc||Method and System to Determine Physical Parameters as Between an RFID Tag and a Reader|
|US20090091454 *||Oct 4, 2007||Apr 9, 2009||Micron Technology, Inc.||Method and System to Determine Physical Parameters as Between A RFID Tag and a Reader|
|US20090105950 *||Oct 18, 2007||Apr 23, 2009||Ianywhere Solutions, Inc.||Real-Time Location Information System Using Multiple Positioning Technologies|
|US20090124321 *||Nov 9, 2007||May 14, 2009||Gadda Christian E||System and methods for dealing a video card|
|US20090295534 *||Dec 3, 2009||International Business Machines Corporation||Using an RFID Device to Enhance Security by Determining Whether a Person in a Secure Area is Accompanied by an Authorized Person|
|US20090303004 *||Jun 5, 2008||Dec 10, 2009||Keystone Technology Solutions, Llc||Systems and Methods to Determine Motion Parameters using RFID Tags|
|US20090303005 *||Jun 5, 2008||Dec 10, 2009||Keystone Technology Solutions, Llc||Systems and Methods to Determine Kinematical Parameters using RFID Tags|
|US20100030685 *||Feb 4, 2010||Bobbitt Russell P||Transaction analysis|
|US20100114623 *||Oct 31, 2008||May 6, 2010||International Business Machines Corporation||Using detailed process information at a point of sale|
|US20100135528 *||Nov 29, 2008||Jun 3, 2010||International Business Machines Corporation||Analyzing repetitive sequential events|
|US20120188377 *||Jul 26, 2012||Malay Kundu||Cart inspection for suspicious items|
|US20130088355 *||Nov 13, 2012||Apr 11, 2013||Apple, Inc.||Three dimensional rf signatures|
|U.S. Classification||340/572.4, 340/541, 342/127, 340/10.31, 342/125, 340/572.1, 340/572.7, 340/539.25, 342/126|
|International Classification||G08B13/24, G08B13/14|
|Cooperative Classification||G08B13/2471, G08B13/19697, G08B13/2462, G08B13/19608, G08B13/19641, G08B13/248, G08B13/19634, G08B13/2417|
|European Classification||G08B13/196Y, G08B13/196L1, G08B13/24B7D, G08B13/196E, G08B13/24B7A1, G08B13/196A3, G08B13/24B1G1, G08B13/24B5T|
|Oct 2, 2003||AS||Assignment|
Owner name: GENERAL ELECTRIC COMPANY, NEW YORK
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KELLIHER, TIMOTHY PATRICK;RITTSCHER, JENS;TU, PETER HENRY;AND OTHERS;REEL/FRAME:014579/0750;SIGNING DATES FROM 20030929 TO 20031002
|Nov 23, 2009||FPAY||Fee payment|
Year of fee payment: 4
|Jan 29, 2010||AS||Assignment|
Owner name: GE SECURITY, INC.,FLORIDA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GENERAL ELECTRIC COMPANY;REEL/FRAME:023961/0646
Effective date: 20100122
Owner name: GE SECURITY, INC., FLORIDA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GENERAL ELECTRIC COMPANY;REEL/FRAME:023961/0646
Effective date: 20100122
|Oct 23, 2013||FPAY||Fee payment|
Year of fee payment: 8