An image-based information retrieval system is disclosed that includes a mobile telephone and a remote server. The mobile telephone has a built-in camera and a communication link for transmitting an image from the built-in camera to the remote server. The remote server has an optical character recognition engine for generating a first confidence value based on an image from the mobile telephone, an object recognition engine for generating a second confidence value based on an image from the mobile telephone, a face recognition engine for generating a third confidence value based on an image from the mobile telephone, and an integrator module for receiving the first, second, and third confidence values and generating a recognition output. |
Citations|
| US5884247 | Oct 31, 1996 | Mar 16, 1999 | Dialect Corporation | Method and apparatus for automated language translation | | US6023241 | Nov 13, 1998 | Feb 8, 2000 | Intel Corporation | Digital multimedia navigation player/recorder | | US6148105 | Apr 22, 1999 | Nov 14, 2000 | Hitachi, Ltd. | Character recognizing and translating system and voice recognizing and translating system | | US6208626 | Dec 24, 1998 | Mar 27, 2001 | | Real-time satellite communication system using separate control and data transmission paths | | US6272231 | Nov 6, 1998 | Aug 7, 2001 | Eyematic Interfaces, Inc. | Wavelet-based facial motion capture for avatar animation | | US6301370 | Dec 4, 1998 | Oct 9, 2001 | Eyematic Interfaces, Inc. | Face recognition from video images | | US6470264 | May 19, 2000 | Oct 22, 2002 | | Portable information-providing apparatus | | US6504571 | May 18, 1998 | Jan 7, 2003 | International Business Machines Corporation | System and methods for querying digital image archives using recorded parameters | | US20010032070 | Dec 28, 2000 | | | Apparatus and method for translating visual text | | US20020101568 | Jan 30, 2002 | | | Interactive data view and command system | | US20020184203 | Jun 13, 2002 | | LTU Technologies | Process for electronically marketing goods or services on networks of the internet type | | US20020187774 | May 14, 2002 | | | Product order method and system | | US20030044608 | Feb 26, 2002 | | FUJI XEROX CO., LTD. | Nanowire, method for producing the nanowire, nanonetwork using the nanowires, method for producing the nanonetwork, carbon structure using the nanowire, and electronic device using the nanowire | | US20030164819 | Mar 4, 2002 | | | Portable object identification and translation system | | US20030198368 | Apr 22, 2003 | | Samsung Electronics Co., Ltd. | Method for verifying users and updating database, and face verification system using the same | | US20040004616 | Jul 3, 2003 | | | Mobile equipment with three dimensional display function | | US20050041862 | Nov 6, 2003 | | | MOBILE PHONE SYSTEM WITH A CARD CHARACTER RECOGNITION FUNCTION | | US20050185060 | Feb 20, 2004 | | | Image base inquiry system for search engines for mobile telephones with integrated camera | | US20060012677 | May 13, 2005 | | | Image-based search engine for mobile phones with camera | | US20060026202 | Oct 22, 2003 | | | Mobile resemblance estimation |
Referenced by|
| US7907940 | Apr 30, 2010 | Mar 15, 2011 | Jumptap, Inc. | Presentation of sponsored content based on mobile transaction event | | US7970389 | Apr 16, 2010 | Jun 28, 2011 | JumpTap, Inc. | Presentation of sponsored content based on mobile transaction event | | US8027879 | Oct 30, 2007 | Sep 27, 2011 | JumpTap, Inc. | Exclusivity bidding for mobile sponsored content | | US8041717 | Jul 30, 2010 | Oct 18, 2011 | JumpTap, Inc. | Mobile advertisement syndication | | US8050675 | Sep 24, 2010 | Nov 1, 2011 | JumpTap, Inc. | Managing sponsored content based on usage history | | US8051169 | Apr 13, 2007 | Nov 1, 2011 | Digimarc Corporation | Methods and systems useful in linking from objects to remote resources | | US8098894 | Jun 20, 2008 | Jan 17, 2012 | YAHOO! Inc. | Mobile imaging device as navigator | | US8099434 | Apr 29, 2010 | Jan 17, 2012 | Jumptap, Inc. | Presenting sponsored content on a mobile communication facility | | US8103545 | Nov 5, 2005 | Jan 24, 2012 | Jumptap, Inc. | Managing payment for sponsored content presented to mobile communication facilities | | US8121618 | Feb 24, 2010 | Feb 21, 2012 | Digimarc Corporation | Intuitive computing methods and systems | | US8131271 | Oct 30, 2007 | Mar 6, 2012 | Jumptap, Inc. | Categorization of a mobile user profile based on browse behavior | | US8156128 | Jun 12, 2009 | Apr 10, 2012 | Jumptap, Inc. | Contextual mobile content placement on a mobile communication facility | | US8175585 | Sep 18, 2011 | May 8, 2012 | Jumptap, Inc. | System for targeting advertising content to a plurality of mobile communication facilities | | US8180332 | Sep 18, 2011 | May 15, 2012 | Jumptap, Inc. | System for targeting advertising content to a plurality of mobile communication facilities | | US8189964 | Dec 7, 2009 | May 29, 2012 | Google Inc. | Matching an approximately located query image against a reference image set | | US8195133 | Oct 30, 2007 | Jun 5, 2012 | Jumptap, Inc. | Mobile dynamic advertisement creation and placement | | US8195513 | Nov 12, 2011 | Jun 5, 2012 | Jumptap, Inc. | Managing payment for sponsored content presented to mobile communication facilities | | US8200205 | Jul 14, 2011 | Jun 12, 2012 | Jumptap, Inc. | Interaction analysis and prioritzation of mobile content | | US8209344 | Jul 19, 2010 | Jun 26, 2012 | Jumptap, Inc. | Embedding sponsored content in mobile applications | | US8229914 | May 8, 2006 | Jul 24, 2012 | Jumptap, Inc. | Mobile content spidering and compatibility determination | | US8238671 | Dec 7, 2009 | Aug 7, 2012 | Google Inc. | Scene classification for place recognition | | US8238888 | Mar 23, 2011 | Aug 7, 2012 | Jumptap, Inc. | Methods and systems for mobile coupon placement |
Claims1. A method for updating object representations in an image database, the method comprising: - receiving an image of an object from a client device associated with a user, the receiving of the image constituting a first user query for information related to the object;
- recognizing the object using an image recognition engine and an image database, the image database storing object identifiers in association with objects;
- associating, from the image database, an object identifier with the recognized object;
- transmitting to the client device associated with the user, information related to the recognized object;
- receiving an indication of whether the user accepted the transmitted information as being relevant to the object; and
- responsive to a computed confidence measure of the recognition of the object exceeding a given threshold, storing the image of the object in the image database in association with the object identifier;
- wherein the confidence measure is computed at least in part based on the indication that the user accepted the transmitted information as being relevant to the object.
2. The method of claim 1, wherein the confidence measure is computed at least based in part on an amount of time that the client device displays the transmitted information. 3. The method of claim 1, wherein the confidence measure is computed at least based in part on whether a second user query for information related to the object is received within a predetermined amount of time of the first user query. 4. The method of claim 1, further comprising identifying a web page address related to the recognized object and storing the web page address in association with the object identifier. 5. The method of claim 1, further comprising - identifying an audio stream related to the recognized object and storing the audio stream in association with the object identifier.
6. The method of claim 1, wherein the recognized object comprises text, the method further comprising translating the text and storing the translated text in association with the object identifier. 7. The method of claim 1, wherein recognizing the object comprises at least one of: performing optical character recognition, performing object recognition, and performing facial recognition. 8. The method of claim 7, wherein performing object recognition comprises performing rigid texture object recognition. 9. The method of claim 7, wherein performing object recognition comprises performing articulate object recognition. 10. The method of claim 1, wherein the image of the object is obtained using a built-in camera of the client device. 11. A recognition computer server system for updating object representations in an image database, the recognition computer server system adapted to perform actions comprising: - receiving an image of an object from a client device associated with a user, the receiving of the image constituting a first user query for information related to the object;
- recognizing the object using an image recognition engine and an image database, the image database storing object identifiers in association with objects;
- associating, from the image database, an object identifier with the recognized object;
- transmitting to the client device associated with the user, information related to the recognized object;
- receiving an indication of whether the user accepted the transmitted information as being relevant to the object; and
- responsive to a computed confidence measure of the recognition of the object exceeding a given threshold, storing the image of the object in the image database in association with the object identifier;
- wherein the confidence measure is computed at least in part based on the indication that the user accepted the transmitted information as being relevant to the object.
12. The recognition computer server system of claim 11, wherein the confidence measure is computed at least based in part on an amount of time that the client device displays the transmitted information. 13. The recognition computer server system of claim 11, wherein the confidence measure is computed at least based in part on whether a second user query for information related to the object is received within a predetermined amount of time of the first user query. 14. The recognition computer server system of claim 11, the actions of the recognition computer server system further comprising identifying a web page address related to the recognized object and storing the web page address in association with the object identifier. 15. The recognition computer server system of claim 11, the actions of the recognition computer server system further comprising identifying an audio stream related to the recognized object and storing the audio stream in association with the object identifier. 16. The recognition computer server system of claim 11, wherein the recognized object comprises text, the method further comprising translating the text and storing the translated text in association with the object identifier. 17. The recognition computer server system of claim 11, wherein recognizing the object comprises at least one of: performing optical character recognition, performing object recognition, and performing facial recognition. 18. The recognition computer server system of claim 17, wherein performing object recognition comprises performing rigid texture object recognition. 19. The recognition computer server system of claim 17, wherein performing object recognition comprises performing articulate object recognition. 20. The recognition computer server system of claim 11, wherein the image of the object is obtained using a built-in camera of the client device. |