|Publication number||US8150216 B2|
|Application number||US 12/584,894|
|Publication date||Apr 3, 2012|
|Filing date||Sep 14, 2009|
|Priority date||May 5, 2004|
|Also published as||US7590310, US8903199, US8908996, US8908997, US20050271304, US20100082597, US20120134583, US20120140990, US20140270508, US20150104104|
|Publication number||12584894, 584894, US 8150216 B2, US 8150216B2, US-B2-8150216, US8150216 B2, US8150216B2|
|Inventors||Jamie E. Retterath, Robert A. Laumeyer|
|Original Assignee||Google Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (188), Non-Patent Citations (38), Referenced by (12), Classifications (21), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is a continuation of application Ser. No. 11/122,969 filed May 5, 2005, now U.S. Pat. No. 7,590,310, which claims the benefit of U.S. Provisional Application No. 60/568,412 filed May 5, 2004, each of which is hereby fully incorporated herein by reference.
This invention generally relates to query processing or search techniques for databases or files containing visual information or images. More specifically, the present invention is directed to an automated and extensible system for the analysis and retrieval of images based on region-of-interest (ROI) analysis of one or more true objects depicted by an image.
The proliferation of digital images and video has been a direct result of more efficient methods for gathering digital imagery. From recording devices such as digital cameras and digital video recorders, to digitizers for CCTV and broadcast media, to 2-D home and office scanners for producing digitized versions of photographs, the myriad digital recording methods has created an explosion of digital media.
For purposes of the present invention, the term “image” will signify a two-dimensional digital representation of a scene, typically created and digitized by an image capture device, signifying a standalone event or a momentary representation of a sequence of events. The term “object” will refer to a uniquely identifiable physical thing or item at least partially depicted in an image. The term “content” will refer to the subject matter of an image.
Along with the countless forms of digital imagery, many different methods have appeared that attempt to organize and structure the digital repositories in such a way that permit retrieval of images in a structured or semi-structured fashion. Unfortunately, providing some structure to the imagery requires conformance to a certain format, necessitates an input process that structures the information, or requires human interaction to annotate the imagery for later retrieval. The most popular examples of image search engines, such as Alta Vista and Google, for example, rely exclusively on annotations and/or file names associated with the images as the only basis for determining whether that image contains desired visual information.
Some image search systems have attempted to overcome the “input requirements” of an annotated image repository by automating the extraction of image features for objects depicted within the data set of the image that results in significantly-reduced representations of the contents of the image repository, thereby decreasing the search time for desired images. U.S. Pat. No. 6,438,130, for example, describes a multi-level filtering technique for reducing the data for content-based image searches. U.S. Pat. No. 6,271,840 discusses a browser interface for displaying reduced versions of images created by an integrated page renderer.
Several techniques exist for the indexing, categorization, and description of images that will be retrieved via a multitude of searching algorithms. Typical image indexing techniques utilize basic attributes like color, shape and texture to describe images or regions of interest within those images.
U.S. Pat. No. 6,778,697 discloses a color image processing method for retrieving a color feature descriptor for describing color features of an image. The color image processing method includes obtaining color vectors of an input image, classifying the color vectors to obtain dominant colors of the input image and their ratios, and representing the dominant colors and their ratios as a color feature descriptor of the input image to allow fast search and retrieval. U.S. Pat. No. 6,411,953 provides a perceptually-based system for pattern retrieval and matching which uses a predetermined vocabulary comprising one or more dimensions to extract color and texture information from an image selected by a user. The system then generates a distance measure characterizing the relationship of the selected image to another image stored in a database, by applying a grammar, comprising a set of predetermined rules, to the color and texture information extracted from the selected image and corresponding color and texture information associated with the stored image.
U.S. Pat. No. 5,802,361 discloses an analysis and retrieval system that uses image attributes to retrieve selected images from a database. The patent provides a fundamental description of the process required to search large image archives for desired images.
U.S. Pat. Nos. 5,893,095 and 5,911,139 describe a system for content-based search and retrieval that utilizes primitives to operate on visual objects. The fundamental approach taught in this patent is the comparison of visual objects and the ability to identify similarities in visual perception of the images. Another approach that utilizes similarity of images is described in U.S. Pat. Nos. 6,463,432 and 6,226,636 ('636). The '636 patent discloses a system that builds a database which stores data corresponding to a plurality of images by dividing each image into several regions. Then, the system calculates a histogram of each region. The database may then be used to determine images which are similar to a query image. U.S. Pat. No. 6,502,105 teaches a system that builds a database of image-related data by inputting a plurality of images, and for each image: dividing the image into a plurality of regions and generating a graph based on the regions, and storing data for the graph in the database. The database may then be used to determine whether a query image is similar to one or more of the plurality of images by dividing the query image into regions and generating a graph based on the regions, and comparing the generated graph to other graphs in the database that correspond to the plurality of images. U.S. Pat. No. 6,240,424 teaches a method and apparatus for classifying and querying a database of images, in which the images in the database are classified using primary objects as a clustering center.
U.S. Pat. No. 5,983,237 discloses an image analysis system consisting of a visual dictionary, a visual query processing method, an image database, a visual query processing system, and the creation of feature vectors. This system discusses the processing of imagery as an integral part of storing the image in the database, whereby both annotated text and feature vectors can be associated with the image. The subsequent searching of images can occur via text searches based on the annotation and on visual searches based on the feature vectors of that image.
U.S. Pat. No. 6,084,595 discloses a method of utilizing indexed retrieval to improve computational efficiency for searching large databases of objects such as images. The premise of the approach is to negate the requirement of a query engine to search all vectors for an image within a feature vector database.
U.S. Pat. No. 6,389,417 discloses a more advanced approach to image retrieval based on determining characteristics of images within regions, not for an entire image. The system is limited, however, to using query regions based on an image supplied by the user, thus limiting the invention to finding images that contain regions similar to a target region within a provided image.
U.S. Pat. No. 6,566,710 reveals a graphical search technique based on joint histograms. The invention discloses usable methods for achieving matches and/or near-matches of images based on similar joint histograms.
U.S. Pat. No. 6,574,378 describes a search and retrieval system based on “visual keywords derived using a learning technique from a plurality of visual tokens extracted across a predetermined number of visual elements.” As with the other systems and methods described above, this patent utilizes the analysis of the image and its contents, with no mention of understanding the various elements or objects that comprise the components within the scene.
U.S. Pat. No. 6,574,616 attempts to improve on image understanding of the preferences of a given user by using probability functions. The system suffers from the requirement that images must be iteratively selected by a user, with more images presented based on the iterative selections and the probability functions of the images.
U.S. Pat. No. 6,584,221 describes advances in image search and retrieval by separating images into regions of interest and making searches based on color and texture. While image color and texture of a region of interest are used in this patent, no attempt is made to determine the physical attributes of the objects depicted in the images.
U.S. Pat. No. 6,611,628 reveals an image feature-coding scheme that assigns a color and a relative area to regions within an image frame. This approach utilizes image and frame color for the feature coding, with no attempts made to interpret the actual color of the objects depicted in the images.
U.S. Pat. Nos. 5,579,471, 5,647,058, and 6,389,424 describe various techniques for creating high dimensional index structures that are used for content-based image retrieval. U.S. Pat. No. 5,647,058 discloses a high dimensional indexing method which takes a set of objects that can be viewed as N-dimensional data vectors and builds an indexed set of truncated transformed vectors which treats the objects like k-dimensional points. The set of truncated transformed vectors can be used to conduct a preliminary similarity search using the previously created index to retrieve the qualifying records.
U.S. Pat. No. 6,563,959 describes a perceptual similarity image retrieval system in which images are subdivided into spots with similar characteristics and each image is represented by a group of spot descriptors.
Techniques have been described for creating a so-called blobworld or regional representation of an image as part of a region-based image querying. See, Carson et al., “Region Base Image Querying,” Proc. of IEEE CVPR Workshop on Content-Based Access of Images and Video Libraries, 1997, Lui et al., “Scalable Object-Based Image Retrieval,” .pdf paper, Ozer et al., “A Graph Based Object Description for Information Retrieval in Digital Image and Video Libraries,” .pdf paper, Fan et al., “Automatic Model-Based Semantic Object Extraction Algorithm,” IEEE Trans. on Circuits and Systems for Video Technology, Vol. 11, No. 10, October 2001, pp. 1073, and Ardizzoni, et al., “Windsurf: Region-based image retrieval using wavelets,” Proc. of the 1st Int'l Workshop on Similarity Search, September 1999, pp. 167-173. These references describe different techniques for the creation, segmentation or hierarchical arrangement of representations of an image using a region-based, blobworld representation of that image.
Large image repositories will undoubtedly contain imagery from a multitude of input sources. Any image search and retrieval mechanism must be specifically designed to deal with the wide variation of attribute types that will result from the myriad input sources. For a repository as complex and diverse as the Internet, a flexible image search and retrieval engine should be able to interpret information from the original image independent of the type of input device or digitizer that was used to create the image.
Computer vision and image processing techniques alone are not adequate for robust image search and retrieval. In addition, implementing image retrieval on a large scale requires automated techniques for analyzing, tagging and managing the visual assets. What is needed for large-scale image search and retrieval that is an automated system that can operate on unstructured data. This automated system must utilize techniques beyond traditional image processing in order to increase the hit rate of image retrieval that demanding users will require. Furthermore, the system must be capable of dealing with a constantly changing target image set wherein the system has little or no control over the content placed within the image search space.
The present invention is an automated and extensible system for the analysis and retrieval of images based on region-of-interest (ROI) analysis of one or more true objects depicted by an image. The system uses a Regions Of Interest (ROI) database that is a relational or analytical database containing searchable vectors that represent the images stored in a repository. Entries in the ROI database are created by an image locator and ROI classifier that work in tandem to locate images within the repository and extract the relevant information that will be stored in the ROI database. Unlike existing region-of-interest search systems, the ROI classifier analyzes objects in an image to arrive at the actual features of the true object, instead of merely describing the features of the image of that object. Graphical searches are performed by the collaborative workings of an image retrieval module, an image search requestor and an ROI query module. The image search requestor is an abstraction layer that translates user or agent search requests into the language understood by the ROI query.
The ROI analysis of the present invention focuses on the actual features of an object, which are referred to for purposes of this invention as the “true object” or “physical object,” instead of the features as represented by the content of an image. For example, if an image of an object, such as a car, is shot at sunset, a person understands that the color features of that image will be distorted from the actual color of the true object. A red car may appear as a purple car or an orange car in the image, but the true physical object is still a red car. The ROI analysis preferably utilizes a combination of techniques to arrive at the features of the true object. In the case of color, for example, a fuzzy logic color filter technique is utilized to deduce the true color as a true or “physical” vector of a given object represented by a region of interest. For purposes of the present invention, the actual qualities or characteristics of an object derived by the ROI analysis will be referred to as the “physical feature vectors” associated with that true physical object.
Regions of interest representing an object in an image can be described in a multitude of types and formats. Examples of region of interest descriptions include, but are not limited to bounding boxes, polygons, and masks. Entries in the ROI database have an ROI type field that contains the type of the region of interest, and an ROI descriptor field that contains the description of the region of interest that is of the format indicated by the type. Since the number of bytes needed to describe a region of interest is quite diverse, the ROI descriptor field can be a small as a few bytes and can be as large as several kilobytes. Other fields in the ROI data structure for each entry in the ROI database are the physical color vector, physical shape vector and physical texture vector. Other physical vectors are also allowed within the data structure, including physical size vector, physical location vector and physical content vector.
Images will typically contain multiple regions of interest. The invention described herein allows search and retrieval at the region of interest level, not at the image level. This allows specific searches to be performed for desired objects without regard to other items that may appear in the imagery. This approach does not preclude searches for images that contain multiple objects. It merely confines the search criteria to the physical object level, leaving multi-object searches to a hierarchical layer above the object search layer.
The image locator preferably contains automated routines that continually search the repository for the existence of new imagery. Software utilized for searching can be web crawlers, spiders, or any other type of intelligent agent that can continuously search an unstructured network for the existence of new or updated files.
Any graphical search request will be initiated either by an intelligent agent or by a user. The user search request can originate from inside or outside the network on which the search will take place.
The systems described in the prior art disclosures have gone to great lengths to categorize, classify, locate and retrieve imagery based on image characteristics. Image attributes like texture, shape and color have proven to be fairly reliable in retrieving images based on selections of combinations of these attributes. The main drawback, however, has been the focus on utilizing image characteristics for search criteria.
As an example, assume an image exists for a red car. As long as the angle of the camera with respect to the car allows the image to show a strong shape component for a car, a shape search for car will yield reasonable results. Furthermore, if the image were taken on a sunny day the color in the image will contain a strong red component. If, however, the image were taken with the car sitting under a tree, the shape characteristic would be impacted by the outline of the tree, and the bright red finish of the car would be duller, and perhaps not even red. If the image's color space were RGB, the red component could possibly be the weakest of the three components.
In this example, a traditional search for “red car” would probably not yield a hit on this object because the shape of the object was compromised and because the color of the object was not well represented in the image. In contrast, the present invention represents searchable regions of interest not by their image attributes, but by their true physical object attributes.
Oftentimes the process of capturing imagery will add noise (which can impact the object's texture and/or shape) or will dramatically change the representative color of the imaged object. Imprecise or ill-controlled image capture techniques should not spoil the quality of the search and retrieval system. The present invention overcomes these problems by providing for an image search and retrieval system based on the actual characteristics of the physical object depicted by the image, not based on the characteristics contained within the image.
Determining physical vectors or attributes of true physical objects is a compute-intensive operation. These computations, however, are part of the image categorization phase, not the retrieval phase. The system described herein is designed to retrieve information very rapidly by utilizing simple searches for object attributes that have been created as part of the previously completed image categorization phase.
Large repositories like corporate intranets and the Internet have a multitude of sources for new and updated information. The system described herein utilizes a crawler, spider or other intelligent agent to constantly update the image categorization process of search information for all imagery. A limitation of the invention, at least with today's technology, is the image search space is confined to those images that have been processed by the image categorization classifier. A repository the size of the Internet might not be fully searchable until the classifier had been given sufficient time to locate and analyze most of the images in the available search space, which could require months of operation on a large farm of computers before a critical mass of image analysis had been performed to allow reasonably representative searches. It will be noted that the inputs to the classifier, however, can be directed to concentrate on certain pockets of information within the image domain to achieve the earliest functionality for the search and retrieval system.
Both the classification and the retrieval functionality of this invention are designed to operate on single computers, on multiple computers, or on a distributed network of computers. The system diagrams show a single Region Of Interest (ROI) database, but the system works just as well with multiple databases populated at several points throughout the network. If multiple ROI databases are utilized, there is no strict requirement that any synchronization exists between the different versions. It is possible, however, to have a completely separate or a fully integrated software program that searches the other ROI databases for new or updated ROI entries. These entries could then be replicated into the local ROI database.
Upon achieving the proper description of the image color space 15, the color filter transform 20 will be used to determine proper representation of the color 35 in the physical object color space 30. The physical object color space 30 is a 3-D volume that describes the range of colors for physical objects. Within the physical object color space 30, there is a multitude of overlapping volumes 40 that represent a portion of the physical object color space 30. These overlapping volumes 40 are each represented by fields within the physical color vector 50.
The fields in the physical color vector 50 may be single-bit or multi-bit fields. When multi-bit fields are used, the available quantum levels can represent: 1) the probability that the physical object is the designated color 40, 2) the percentage of the designated color volume 40 contained within the ROI physical color representation 35, 3) the distance from the center of mass of the ROI physical color representation 35 to the center of mass of the designated color volume 40, or some other measure that signifies the relative strength that the representative color 40 contributes to the physical color of the original true object.
The fields in the physical shape vector 90 may be single-bit or multi-bit fields. When multi-bit fields are used, the available quantum levels can represent the probability that the physical object is the designated shape, that it contains feature aspects of the designated shape, or that some other measure that signifies the relative strength that the representative shape contributes to the physical shape of the original object.
The physical object shape space 80 may or may not be expressed in the same coordinate system as the image shape space 60. The selection of coordinate system for the physical object shape space 80 will depend on the image type, the scene descriptors, the context type (single image, series of still images, digital video, etc.), etc.
The fields in the physical texture vector 130 may be single-bit or multi-bit fields. When multi-bit fields are used, the available quantum levels can represent the probability that the physical object is the designated texture, is in a similar class of textures, or some other measure that signifies the relative strength that the representative texture contributes to the physical texture of the original object.
Preferably, the physical texture vector will contain more than just texture representations. For example, adjectives like shiny, dull, metallic, transparent, translucent, and reflective can be represented by fields within the physical texture vector.
Entries in the ROI database 220 are created by the image locator 160 and the region of interest (ROI) classifier 170. These two modules work in tandem to locate images within the repository 140 via an interface 150 and extract the relevant information that will be stored in the ROI database 220. The image locator 160 contains automated routines that continually search the repository 140 for the existence of new imagery. Software utilized for searching can be web crawlers, spiders, or any other type of intelligent agent that can continuously search an unstructured network 140 for the existence of new or updated files.
The ROI classifier 170 takes newly located images and splits them into regions of interest using techniques including, but not limited to segmentation, blob analysis, high frequency filter and Fourier transforms. Most images will contain multiple regions of interest. Each region will be analyzed separately and will receive its own data structure representation within the ROI database 220.
Graphical searches are performed by the collaborative workings of image retrieval 190, the image search requestor 200 and the region of interest (ROI) query 210. Any graphical search request will be initiated either by an intelligent agent or by a user. The user search request can originate from inside or outside the network on which the search will take place. The image search requestor is an abstraction layer that translates user or agent search requests into the language understood by the ROI query 210.
As an example, assume that a graphical search user wishes to search for an “Irish Setter.” Since these are terms not utilized by the ROI query 210, the image search requestor 200 may format the search request as color=reddish-brown, texture=furry, and shape=dog. Upon submitting this search request, the ROI query 210 will search the ROI database 220 for a list of the files most likely to contain the desired object. Upon determining this list, the ROI query 210 will tell the image retrieval 190 to retrieve the files from the repository 140. The resulting files are retrieved via the interface 180 and returned to the image search requestor 200, which will format them for the user.
Another embodiment of the present invention eliminates the need for the image retrieval 190 and the associated interface to the repository 180. In this embodiment, the image search requestor 200 would simply pass the image file names to the user and allow the higher-level functionality to retrieve the desired files.
Another preferred embodiment includes multiple ROI databases 220. It is not necessary for all copies of the ROI database 220 to be in tight synchronization. Since the ROI databases 220 are continuously being modified by the ROI classifier 170, the constraint of requiring synchronized ROI databases 220 places unnecessary burdens on the system.
The file name field 235 contains the name of the file that contains the region of interest, and the file location field 240 stores the files location within the network. With a fluid medium like the Internet, graphical files can be changed frequently without any changes to the file name or location. When an ROI data structure is initially created, the cache key is created for the image that is a semi-unique qualifier for the image. The cache key field 245 retains the cache key for later use by the image locator 160 and the ROI query 210. Every time a file is retrieved from the repository the cache key is recomputed and compared to the cache key field 245 in the ROI data structure 230. A mismatch in the two values signifies that the image has been changed since the ROI classifier 170 operated on the original image.
Regions of interest can be described in a multitude of types and formats. Examples of region of interest descriptions include, but are not limited to bounding boxes, polygons, and masks. The ROI type field 250 contains the type of the region of interest, and the ROI descriptor field 255 contains the description of the region of interest that is of the format indicated by the type 250. Since the number of bytes needed to describe a region of interest is quite diverse, the ROI descriptor field 255 can be a small as a few bytes and can be as large as several kilobytes.
The next three fields in the ROI data structure 230 are the previously discussed fields of the physical color vector 50, physical shape vector 90 and physical texture vector 130. Other physical vectors are allowed within the data structure 230. Some possible options are the physical size vector 260, physical location vector 265 and the physical content vector 270.
In a preferred embodiment of the present invention, the ROI query 210 would be able to alter its search capabilities based on the versions contained with the vectors 50, 90, 130 in the ROI data structure. For example, more advanced versions of a shape vector may include higher-level descriptions of shapes, thus allowing more thorough searches based on higher-level shape constructs.
A preferred embodiment of the present invention allows the ROI classifier 170 to utilize vector versions 285, 305, 325 to compare with the latest versions of each of the vectors 50, 90, 130. When an image is retrieved by the image locator 160 that is already in the ROI database 220, the ROI classifier 170 would check the version numbers of all of the vectors and, when a type of vector is not constructed with the latest version, the ROI classifier 170 would reprocess all of the regions of interest for that image and update the vectors 50, 90, 130 to the latest versions.
As shown in
In this embodiment, a Fuzzy Logic Color Filter (FLCF) is applied to objects for all known or suspected color sets (combinations). The FLCF can be applied to different color sets in parallel. The FLCF can be accomplished with multiple processes on one CPU, or can be implemented within multiple CPUs within one box, or can be effected by a distributed processing approach.
The complete disclosures of the patents, patent applications and publications cited herein are incorporated by reference in their entirety as if each were individually incorporated. Various modifications and alterations to this invention will become apparent to those skilled in the art without departing from the scope and spirit of this invention. It should be understood that this invention is not intended to be unduly limited by the illustrative embodiments and examples set forth herein and that such examples and embodiments are presented by way of example only with the scope of the invention intended to be limited only by the claims set forth herein.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US3636250||Feb 26, 1964||Jan 18, 1972||Andrew V Haeff||Apparatus for scanning and reproducing a three-dimensional representation of an object|
|US4348652||Sep 29, 1980||Sep 7, 1982||Robert R. Barnes||Driver alert system|
|US4373819||Dec 22, 1980||Feb 15, 1983||Stefano Pallotta||Retroreflectometer giving direct reading|
|US4491923||Feb 9, 1982||Jan 1, 1985||Minnesota Mining And Manufacturing Company||Method and apparatus for extracting information from an object that is within a large interrogation zone|
|US4721389||Apr 18, 1985||Jan 26, 1988||Potters Industries, Inc.||Method and apparatus for measuring retroreflectivity of a reflective layer on a surface|
|US4726134||Nov 21, 1986||Feb 23, 1988||Minnesota Mining And Manufacturing Company||Roadway sign|
|US4920385||Apr 6, 1989||Apr 24, 1990||Diffracto Ltd.||Panel surface flaw inspection|
|US4937950||Jul 17, 1989||Jul 3, 1990||Wilfred Farnworth||Spacing and holding device for horizontal siding|
|US5050327||Nov 17, 1989||Sep 24, 1991||Minnesota Mining And Manufacturing Company||Retroreflective sign having improved legibility|
|US5115398||Jul 29, 1991||May 19, 1992||U.S. Philips Corp.||Method of displaying navigation data for a vehicle in an image of the vehicle environment, a navigation system for performing the method, and a vehicle comprising a navigation system|
|US5164785||Feb 8, 1991||Nov 17, 1992||Hopkins Manufacturing Corporation||Headlight aiming apparatus and display|
|US5353392||Apr 10, 1991||Oct 4, 1994||Multi Media Techniques||Method and device for modifying a zone in successive images|
|US5373357||Oct 21, 1992||Dec 13, 1994||Hopkins Manufacturing Corporation||Headlight aiming method using pattern framing|
|US5392365||Dec 23, 1991||Feb 21, 1995||Eastman Kodak Company||Apparatus for detecting text edges in digital image processing|
|US5448484||Nov 3, 1992||Sep 5, 1995||Bullock; Darcy M.||Neural network-based vehicle detection system and method|
|US5465115||Jan 10, 1995||Nov 7, 1995||Rct Systems, Inc.||Video traffic monitor for retail establishments and the like|
|US5465308||Aug 25, 1993||Nov 7, 1995||Datron/Transoc, Inc.||Pattern recognition system|
|US5491517||Jan 19, 1995||Feb 13, 1996||Scitex America Corporation||System for implanting an image into a video stream|
|US5530549||May 24, 1994||Jun 25, 1996||Spatialmetrix Corp.||Probing retroreflector and methods of measuring surfaces therewith|
|US5533388||Feb 21, 1995||Jul 9, 1996||Kabushiki Kaisha Meidensha||Apparatus for electromagnetically isolating an automobile|
|US5579471||Mar 23, 1994||Nov 26, 1996||International Business Machines Corporation||Image query system and method|
|US5613055||Jul 7, 1993||Mar 18, 1997||Sumitomo Electric Industries, Ltd.||Method of and apparatus for producing an animation having a series of road drawings to be watched from a driver's seat of a vehicle|
|US5627549||Jan 16, 1996||May 6, 1997||Seiko Communications Holding N.V.||Dual channel advertising referencing vehicle location|
|US5627915||Jan 31, 1995||May 6, 1997||Princeton Video Image, Inc.||Pattern recognition system employing unlike templates to detect objects having distinctive features in a video field|
|US5633944||Apr 19, 1995||May 27, 1997||Automobiles Peugeot||Method and apparatus for automatic optical recognition of road signs|
|US5633946||Nov 3, 1995||May 27, 1997||Geospan Corporation||Method and apparatus for collecting and processing visual and spatial position information from a moving platform|
|US5647058||Feb 28, 1996||Jul 8, 1997||International Business Machines Corporation||Method for high-dimensionality indexing in a multi-media database|
|US5649032||Nov 14, 1994||Jul 15, 1997||David Sarnoff Research Center, Inc.||System for automatically aligning images to form a mosaic image|
|US5664948||Oct 11, 1994||Sep 9, 1997||Seiko Communications Holding N.V.||Delivery of data including preloaded advertising data|
|US5696503||Jul 23, 1993||Dec 9, 1997||Condition Monitoring Systems, Inc.||Wide area traffic surveillance using a multisensor tracking system|
|US5699444||Mar 31, 1995||Dec 16, 1997||Synthonics Incorporated||Methods and apparatus for using image data to determine camera location and orientation|
|US5740274||Aug 29, 1994||Apr 14, 1998||Fuji Photo Film Co., Ltd.||Method for recognizing object images and learning method for neural networks|
|US5757878||Aug 16, 1996||May 26, 1998||Analogic Corporation||Detector arrangement for x-ray tomography system|
|US5764411||Aug 9, 1993||Jun 9, 1998||Thorn Emi Plc||Apparatus for displaying an image|
|US5790691||Mar 12, 1996||Aug 4, 1998||The Regents Of The University Of Colorado||Method and apparatus for robust shape detection using a hit/miss transform|
|US5793034||Sep 13, 1996||Aug 11, 1998||Daedalus Enterprises, Inc.||Target detection system utilizing multiple optical criteria|
|US5802361||Sep 30, 1994||Sep 1, 1998||Apple Computer, Inc.||Method and system for searching graphic images and videos|
|US5818640||Jul 27, 1995||Oct 6, 1998||Minnesota Mining And Manufacturing Company||Sign illumination system and method|
|US5844699||Jun 7, 1995||Dec 1, 1998||Canon Kabushiki Kaisha||Color image processing apparatus|
|US5852823||Oct 16, 1996||Dec 22, 1998||Microsoft||Image classification and retrieval system using a query-by-example paradigm|
|US5864630||Nov 20, 1996||Jan 26, 1999||At&T Corp||Multi-modal method for locating objects in images|
|US5892847||Apr 22, 1996||Apr 6, 1999||Johnson-Grace||Method and apparatus for compressing images|
|US5893095||Mar 28, 1997||Apr 6, 1999||Virage, Inc.||Similarity engine for content-based retrieval of images|
|US5911139||Mar 28, 1997||Jun 8, 1999||Virage, Inc.||Visual image database search engine which allows for different schema|
|US5915032||Aug 6, 1991||Jun 22, 1999||Minnesota Mining And Manufacturing Company||Method and apparatus for machine reading of retroreflective vehicle identification articles|
|US5938319||Jun 17, 1996||Aug 17, 1999||Robert Bosch Gmbh||Vehicle headlight with adjusting means for different traffic conditions|
|US5941944 *||Mar 3, 1997||Aug 24, 1999||Microsoft Corporation||Method for providing a substitute for a requested inaccessible object by identifying substantially similar objects using weights corresponding to object features|
|US5949914||Mar 17, 1997||Sep 7, 1999||Space Imaging Lp||Enhancing the resolution of multi-spectral image data with panchromatic image data using super resolution pan-sharpening|
|US5950190||May 13, 1997||Sep 7, 1999||Aptek, Inc.||Dynamic, self-modifying graphical user interface for relational database applications|
|US5974521||Mar 31, 1998||Oct 26, 1999||Neomagic Israel Ltd.||Apparatus and method for signal processing|
|US5982298||Nov 14, 1996||Nov 9, 1999||Microsoft Corporation||Interactive traffic display and trip planner|
|US5983237||Aug 21, 1997||Nov 9, 1999||Virage, Inc.||Visual dictionary|
|US5991085||Jul 12, 1996||Nov 23, 1999||I-O Display Systems Llc||Head-mounted personal visual display apparatus with image generator and holder|
|US6011515||Oct 6, 1997||Jan 4, 2000||The Johns Hopkins University||System for measuring average speed and traffic volume on a roadway|
|US6018697||Dec 11, 1996||Jan 25, 2000||Aisin Aw Co., Ltd.||Navigation system for vehicles|
|US6023967||Oct 10, 1997||Feb 15, 2000||Kumho Tire Co., Ltd.||Method for measuring tire wear using intensity of reflected light|
|US6036322||Dec 1, 1997||Mar 14, 2000||Reflexite Corporation||Multi-orientation retroreflective structure|
|US6048069||Mar 9, 1999||Apr 11, 2000||3M Innovative Properties Company||Wide angle image-displaying sheet and system|
|US6064768||Jun 30, 1997||May 16, 2000||Wisconsin Alumni Research Foundation||Multiscale feature detector using filter banks|
|US6084595||Feb 24, 1998||Jul 4, 2000||Virage, Inc.||Indexing method for image search engine|
|US6123263||Jan 29, 1998||Sep 26, 2000||Meta Holdings Corporation||Hand held dataform reader having strobing ultraviolet light illumination assembly for reading fluorescent dataforms|
|US6134819||Feb 11, 1998||Oct 24, 2000||Mcclain; Ronald G.||Portable illuminated traffic signal device|
|US6141433||Dec 24, 1997||Oct 31, 2000||Ncr Corporation||System and method for segmenting image regions from a scene likely to represent particular objects in the scene|
|US6141434||Feb 6, 1998||Oct 31, 2000||Christian; Andrew Dean||Technique for processing images|
|US6142871||Jul 28, 1997||Nov 7, 2000||Konami Co., Ltd.||Apparatus, method and recorded programmed medium for simulating driving using mirrors displayed in a game space|
|US6166813||Apr 16, 1998||Dec 26, 2000||Georgia Tech Research Corporation||Retroreflectometer and method for measuring retroreflectivity of materials|
|US6173231||Jan 31, 2000||Jan 9, 2001||Navigation Technologies Corp.||Method and system for collecting data concerning thermal properties of roads for a geographic database and use thereof in a vehicle safety system|
|US6182011||Aug 10, 1998||Jan 30, 2001||The United States Of America As Represented By The Administrator Of National Aeronautics And Space Administration||Method and apparatus for determining position using global positioning satellites|
|US6199014||Dec 23, 1997||Mar 6, 2001||Walker Digital, Llc||System for providing driving directions with visual cues|
|US6208386||Sep 9, 1996||Mar 27, 2001||Orad Hi-Tec Systems Limited||Method and apparatus for automatic electronic replacement of billboards in a video image|
|US6212480||Jan 25, 2000||Apr 3, 2001||Laser Technology, Inc.||Apparatus and method for determining precision reflectivity of highway signs and other reflective objects utilizing an optical range finder instrument|
|US6226636||Nov 20, 1998||May 1, 2001||Philips Electronics North America Corp.||System for retrieving images using a database|
|US6240424||Apr 22, 1998||May 29, 2001||Nbc Usa, Inc.||Method and system for similarity-based image classification|
|US6240664||Nov 2, 1998||Jun 5, 2001||Airlite Unlimited||Illuminated sign and sign plate therefor|
|US6252632||Jan 17, 1997||Jun 26, 2001||Fox Sports Productions, Inc.||System for enhancing a video presentation|
|US6253477||Sep 23, 1998||Jul 3, 2001||Hallmark Technologies, Inc.||Retro-reflective sign|
|US6266442||Oct 23, 1998||Jul 24, 2001||Facet Technology Corp.||Method and apparatus for identifying objects depicted in a videostream|
|US6271840||Sep 24, 1998||Aug 7, 2001||James Lee Finseth||Graphical search engine visual index|
|US6292227||Sep 9, 1996||Sep 18, 2001||Orad Hi-Tec Systems Limited||Method and apparatus for automatic electronic replacement of billboards in a video image|
|US6317058||Sep 15, 1999||Nov 13, 2001||Jerome H. Lemelson||Intelligent traffic control and warning system and method|
|US6321158||Aug 31, 1998||Nov 20, 2001||Delorme Publishing Company||Integrated routing/mapping information|
|US6351710||Sep 28, 2000||Feb 26, 2002||Michael F. Mays||Method and system for visual addressing|
|US6363161||Jun 18, 2001||Mar 26, 2002||Facet Technology Corp.||System for automatically generating database of objects of interest by analysis of images recorded by moving vehicle|
|US6381362||Apr 8, 1999||Apr 30, 2002||Tata America International Corporation||Method and apparatus for including virtual ads in video presentations|
|US6382126||Sep 28, 2000||May 7, 2002||Craig Alan Findley||Reflective and adhesive patch|
|US6383871||Aug 31, 1999||May 7, 2002||Micron Technology, Inc.||Method of forming multiple oxide thicknesses for merged memory and logic applications|
|US6384471||May 4, 2000||May 7, 2002||Bull S.A.||Pbga package with integrated ball grid|
|US6384871||Nov 1, 2000||May 7, 2002||Orad Hi-Tec Systems Limited||Method and apparatus for automatic electronic replacement of billboards in a video image|
|US6389417||Jun 29, 1999||May 14, 2002||Samsung Electronics Co., Ltd.||Method and apparatus for searching a digital image|
|US6389424||Oct 28, 1999||May 14, 2002||Electronics And Telecommunications Research Institute||Insertion method in a high-dimensional index structure for content-based image retrieval|
|US6405132||Oct 4, 2000||Jun 11, 2002||Intelligent Technologies International, Inc.||Accident avoidance system|
|US6407674||Mar 5, 1998||Jun 18, 2002||Philip Anthony Gallagher||Reflectivity measuring apparatus and method|
|US6411953||Jan 25, 1999||Jun 25, 2002||Lucent Technologies Inc.||Retrieval and matching of color patterns based on a predetermined vocabulary and grammar|
|US6424914||Dec 26, 2000||Jul 23, 2002||American Gnc Corporation||Fully-coupled vehicle positioning method and system thereof|
|US6430499||Mar 12, 2002||Aug 6, 2002||Matsushita Electric Industrial Co., Ltd.||Information terminal and cartographic information providing system|
|US6438130||Jun 28, 2001||Aug 20, 2002||Mellanox Technologies Ltd.||Forwarding database cache|
|US6438561||Nov 19, 1998||Aug 20, 2002||Navigation Technologies Corp.||Method and system for using real-time traffic broadcasts with navigation systems|
|US6449384||Jan 29, 2001||Sep 10, 2002||Facet Technology Corp.||Method and apparatus for rapidly determining whether a digitized image frame contains an object of interest|
|US6453056||Mar 20, 2001||Sep 17, 2002||Facet Technology Corporation||Method and apparatus for generating a database of road sign images and positions|
|US6463432||Aug 2, 1999||Oct 8, 2002||Minolta Co., Ltd.||Apparatus for and method of retrieving images|
|US6470267||Sep 20, 2000||Oct 22, 2002||Pioneer Corporation, Increment P Corporation||Man navigation system|
|US6476910||Aug 29, 2000||Nov 5, 2002||The Regents Of The University Of California||Light scattering apparatus and method for determining radiation exposure to plastic detectors|
|US6477460||Feb 27, 2001||Nov 5, 2002||Metro One Telecommunications, Inc.||Process and system for the annotation of machine-generated directions with easily recognized landmarks and other relevant information|
|US6498982||Jul 10, 2001||Dec 24, 2002||Mapquest. Com, Inc.||Methods and apparatus for displaying a travel route and/or generating a list of places of interest located near the travel route|
|US6502105||Jan 15, 1999||Dec 31, 2002||Koninklijke Philips Electronics N.V.||Region-based image archiving and retrieving system|
|US6507441||Oct 16, 2000||Jan 14, 2003||Optid, Optical Identification Technologies Ltd.||Directed reflectors and systems utilizing same|
|US6512857||May 8, 1998||Jan 28, 2003||Sarnoff Corporation||Method and apparatus for performing geo-spatial registration|
|US6514597||Feb 4, 2000||Feb 4, 2003||#M Innovative Properties Company||Embossed oriented polymer films|
|US6525768||Oct 21, 1999||Feb 25, 2003||American Calcar, Inc.||Positional camera and GPS data interchange device|
|US6525780||Dec 18, 1998||Feb 25, 2003||Symah Vision, Sa||“Midlink” virtual insertion system|
|US6526352||Jul 19, 2001||Feb 25, 2003||Intelligent Technologies International, Inc.||Method and arrangement for mapping a road|
|US6538751||Feb 16, 2001||Mar 25, 2003||Fuji Photo Film Co., Ltd.||Image capturing apparatus and distance measuring method|
|US6542822||Jul 24, 2001||Apr 1, 2003||Trimble Navigation Limited||Directed user-based dynamic advertising|
|US6558021||Aug 10, 2001||May 6, 2003||Leotek Electronics Corporation||Light emitting diode modules for illuminated signs|
|US6563959||Jul 30, 1999||May 13, 2003||Pixlogic Llc||Perceptual similarity image retrieval method|
|US6566710||Aug 29, 2001||May 20, 2003||National Semiconductor Corporation||Power MOSFET cell with a crossed bar shaped body contact area|
|US6567103||Aug 2, 2000||May 20, 2003||Verity, Inc.||Graphical search results system and method|
|US6567551||Apr 26, 1999||May 20, 2003||Canon Kabushiki Kaisha||Image search apparatus and method, and computer readable memory|
|US6574378||Jan 22, 1999||Jun 3, 2003||Kent Ridge Digital Labs||Method and apparatus for indexing and retrieving images using visual keywords|
|US6574616||Apr 7, 2000||Jun 3, 2003||Index Stock Imagery, Inc.||Stochastic visually based image query and retrieval system|
|US6575378||Jan 30, 2002||Jun 10, 2003||Denso Corporation||Solar radiation detection unit and vehicle air conditioner using the same|
|US6584221||Aug 30, 1999||Jun 24, 2003||Mitsubishi Electric Research Laboratories, Inc.||Method for image retrieval with multiple regions of interest|
|US6594931||Aug 21, 2000||Jul 22, 2003||Jeffrey C. Barton||Fluorescent illuminated sign assembly|
|US6604398||Jan 18, 2000||Aug 12, 2003||Sms Demag Ag||Working method and installation for the flexible and economical pickling and cold-rolling of metal strips|
|US6611628||Nov 17, 2000||Aug 26, 2003||Mitsubishi Denki Kabushiki Kaisha||Method of image feature coding and method of image search|
|US6625315||Sep 16, 2002||Sep 23, 2003||Facet Technology Corp.||Method and apparatus for identifying objects depicted in a videostream|
|US6653990||Mar 8, 1999||Nov 25, 2003||Societe Rasterland S.A.||System for displaying realistic virtual three-dimensional images in real time|
|US6671615||May 2, 2000||Dec 30, 2003||Navigation Technologies Corp.||Navigation system with sign assistance|
|US6674878||Jul 30, 2001||Jan 6, 2004||Facet Technology Corp.||System for automated determination of retroreflectivity of road signs and other reflective objects|
|US6678591||Jan 29, 2002||Jan 13, 2004||Mazda Motor Corporation||Server for remote vehicle troubleshooting and the like|
|US6714860||Nov 27, 2000||Mar 30, 2004||Robert Bosch Gmbh||Navigation device with means for displaying advertisements|
|US6772062||Jan 8, 2002||Aug 3, 2004||The Regents Of The University Of California||Intelligent ultra high speed distributed sensing system and method for sensing roadway markers for intelligent vehicle guidance and control|
|US6774908||Aug 31, 2001||Aug 10, 2004||Creative Frontier Inc.||System and method for tracking an object in a video and linking information thereto|
|US6778224||Jun 25, 2001||Aug 17, 2004||Koninklijke Philips Electronics N.V.||Adaptive overlay element placement in video|
|US6778697||Feb 4, 2000||Aug 17, 2004||Samsung Electronics Co., Ltd.||Color image processing method and apparatus thereof|
|US6791492||Oct 24, 2002||Sep 14, 2004||Mitsui & Co., Ltd.||GPS positioning system|
|US6820277||Apr 20, 2000||Nov 16, 2004||Expanse Networks, Inc.||Advertising management system for digital video streams|
|US6836724||Nov 3, 2003||Dec 28, 2004||Navteq North America, Llc||Navigation system with sign assistance|
|US6885767||May 11, 2001||Apr 26, 2005||Paul Howell||Silhouetting apparatus and method|
|US6888622||Feb 3, 2003||May 3, 2005||Nissan Motor Co., Ltd.||Method for determining object type of reflective object on track|
|US6891960||Aug 10, 2001||May 10, 2005||Facet Technology||System for road sign sheeting classification|
|US7031983||Jul 16, 2002||Apr 18, 2006||Navteq North America, Llc||Method and system for using real-time traffic broadcasts with navigation systems|
|US7039630||Jul 24, 2001||May 2, 2006||Nec Corporation||Information search/presentation system|
|US7043057||Dec 15, 2003||May 9, 2006||Facet Technology, Corporation||System for automated determination of retroreflectivity of road signs and other reflective objects|
|US7082426 *||Sep 11, 2003||Jul 25, 2006||Cnet Networks, Inc.||Content aggregation method and apparatus for an on-line product catalog|
|US7092548||Aug 5, 2003||Aug 15, 2006||Facet Technology Corporation||Method and apparatus for identifying objects depicted in a videostream|
|US7173707||May 3, 2006||Feb 6, 2007||Facet Technology Corporation||System for automated determination of retroreflectivity of road signs and other reflective objects|
|US7272498||Sep 30, 2004||Sep 18, 2007||Scenera Technologies, Llc||Method for incorporating images with a user perspective in navigation|
|US7409110 *||Jun 28, 2005||Aug 5, 2008||Seiko Epson Corporation||Image retrieval system, image retrieval program and storage medium and image retrieval method|
|US7411681||Feb 5, 2007||Aug 12, 2008||Facet Technology Corporation||System for automated determination of retroreflectivity of road signs and other reflective objects|
|US7440003||Feb 6, 2003||Oct 21, 2008||Nec Corporation||Folding portable information terminal with fold-position-based recording control|
|US7444003||Jul 13, 2006||Oct 28, 2008||Facet Technology Corporation||Method and apparatus for identifying objects depicted in a videostream|
|US7515736||Feb 11, 2005||Apr 7, 2009||Facet Technology, Corp.||System for road sign sheeting classification|
|US7590310||May 5, 2005||Sep 15, 2009||Facet Technology Corp.||Methods and apparatus for automated true object-based image analysis and retrieval|
|US20010021011||Feb 16, 2001||Sep 13, 2001||Shuji Ono||Image capturing apparatus and distance measuring method|
|US20010036293||Jun 18, 2001||Nov 1, 2001||Facet Technology Corporation||System for automatically generating database of objects of interest by analysis of images recorded by moving vehicle|
|US20010043717||Jan 29, 2001||Nov 22, 2001||Facet Technology Corporation||Method and apparatus for rapidly determining whether a digitized image frame contains an object of interest|
|US20010043718||Mar 20, 2001||Nov 22, 2001||Facet Technology Corporation||Method and apparatus for generating a database of road sign images and positions|
|US20010045034||Jul 13, 2001||Nov 29, 2001||Lance Mueller||Illuminated sign and method for design|
|US20020044278||Jul 13, 2001||Apr 18, 2002||Tuan Le||Apparatus for optical inspection of a working surface having a dynamic reflective spatial attenuator|
|US20020063638||Mar 5, 1998||May 30, 2002||Philip Anthony Gallagher||Reflectivity measuring apparatus and method|
|US20020090492||Apr 2, 1999||Jul 11, 2002||Dale H. Haunschild||Retroreflective article|
|US20020106109||Aug 10, 2001||Aug 8, 2002||Retterath James E.||System for road sign sheeting classification|
|US20020163942||Dec 20, 2001||Nov 7, 2002||Applied Optoelectronics, Inc.||Multiple reflectivity band reflector for laser wavelength monitoring|
|US20020186865||Jul 30, 2001||Dec 12, 2002||Facet Technology, Inc.||System for automated determination of retroreflectivity of road signs and other reflective objects|
|US20030004644||Aug 19, 2002||Jan 2, 2003||Telanon, Inc.||Methods and apparatus for stationary object detection|
|US20030006911||Dec 21, 2001||Jan 9, 2003||The Cadre Group Inc.||Interactive advertising system and method|
|US20030016869||Sep 16, 2002||Jan 23, 2003||Facet Technology Corp.||Method and apparatus for identifying objects depicted in a videostream|
|US20030090415||Oct 24, 2002||May 15, 2003||Mitsui & Co., Ltd.||GPS positioning system|
|US20030174054||Feb 3, 2003||Sep 18, 2003||Nissan Motor Co., Ltd.||Method for determining object type of reflective object on track|
|US20040062442||Aug 5, 2003||Apr 1, 2004||Facet Technology Corp.||Method and apparatus for identifying objects depicted in a videostream|
|US20040078813||Aug 5, 2003||Apr 22, 2004||Sony Corporation||Electronic guide system, contents server for electronic guide system, portable electronic guide device, and information processing method for electronic guide system|
|US20040127614||Oct 15, 2003||Jul 1, 2004||Peijun Jiang||Polyolefin adhesive compositions and articles made therefrom|
|US20040156531||Dec 15, 2003||Aug 12, 2004||Facet Technology Corporation||System for automated determination of retroreflectivity of road signs and other reflective objects|
|US20040218910||Jul 23, 2003||Nov 4, 2004||Chang Nelson L.||Enabling a three-dimensional simulation of a trip through a region|
|US20050021472||Jun 8, 2004||Jan 27, 2005||David Gettman||Transactions in virtual property|
|US20050249378||Feb 11, 2005||Nov 10, 2005||Facet Technology, Inc.||System for road sign sheeting classification|
|US20050271304||May 5, 2005||Dec 8, 2005||Retterath Jamie E||Methods and apparatus for automated true object-based image analysis and retrieval|
|US20060262312||May 3, 2006||Nov 23, 2006||Facet Technology Corp.||System for automated determination of retroreflectivity of road signs and other reflective objects|
|US20070043707 *||Oct 17, 2005||Feb 22, 2007||Parashuram Kulkarni||Unsupervised learning tool for feature correction|
|US20070081744 *||Oct 3, 2006||Apr 12, 2007||Gokturk Salih B||System and method for use of images with recognition analysis|
|US20070154067||Jul 13, 2006||Jul 5, 2007||Facet Technology Corp.||Method and apparatus for identifying objects depicted in a videostream|
|US20070216904||Feb 5, 2007||Sep 20, 2007||Facet Technology Corporation||System for automated determination of retroreflectivity of road signs and other reflective objects|
|US20090252376||Apr 7, 2009||Oct 8, 2009||Retterath Jamie E||System for Road Sign Sheeting Classification|
|WO1997009823A1||Sep 9, 1996||Mar 13, 1997||Orad Hi-Tec Systems Limited||Method and apparatus for automatic electronic replacement of billboards in a video image|
|WO1998024242A1||Nov 25, 1997||Jun 4, 1998||Princeton Video Image, Inc.||Image insertion in video streams using a combination of physical sensors and pattern recognition|
|WO1999039160A1||Jan 28, 1999||Aug 5, 1999||Joyce Russ Advertising Pty. Ltd.||Navigational apparatus using visual description of the route|
|WO1999054848A1||Apr 9, 1999||Oct 28, 1999||Magellan Dis, Inc.||3-dimensional intersection display for vehicle navigation system|
|1||Application and File History for U.S. Appl. No. 09/918,375, filed Jul. 30, 2001, now U.S. Patent No. 6,674,878, Inventors Retterath et al.|
|2||Application and File History for U.S. Appl. No. 09/928,218, filed Aug. 10, 2001, now U.S. Patent No. 6,891,960, Inventors Retterath et al.|
|3||Application and File History for U.S. Appl. No. 10/736,454, filed Dec. 15, 2003, now U.S. Patent No. 7,043,057, Inventors Retterath et al.|
|4||Application and File History for U.S. Appl. No. 11/056,926, filed Feb. 11, 2005, now U.S. Patent No. 7,515,736, Inventors Retterath et al.|
|5||Application and File History for U.S. Appl. No. 11/122,969, filed May 5, 2005. Inventors: Jamie E. Retterath et al.|
|6||Application and File History for U.S. Appl. No. 11/381,503, filed May 3, 2006, now U.S. Patent No. 7,173,707, Inventors Retterath et al.|
|7||Application and File History for U.S. Appl. No. 11/702,421, filed Feb. 5, 2007, now U.S. Patent No. 7,411,681, Inventors Retterath et al.|
|8||Application and File History for U.S. Appl. No. 12/419,843, filed Apr. 7, 2009, Inventors Retterath et al.|
|9||Ardizzoni et al., "Windsurf: Region Based Image Retrieval Using Wavelets," Proc. of the 1st Int'l Workshop on Similarity Search, Sep. 1999, pp. 167-173, USA.|
|10||Baker et al., "Representation of Uncertainty in Spatial Target Tracking," DERA Farnborough, United Kingdom, 4 pgs., Aug. 1998.|
|11||Carson et al., "Region Base Image Querying," Proc. of IEEE CUPR Workshop on Content-Based Access of Images and Video Libraries, 1997, USA.|
|12||Celentano, "Feature Integration and Relevance Feedback Analysis in Image Similarity Evaluation," Journal of Electronic Imaging, Apr. 1998, pp. 308-317, vol. 7(2), USA.|
|13||Crowley et al., "Multi-Modal Tracking of Faces for Video Communications," GRAVIR-IMAG, I.N.P. Grenoble, Grenoble, France, IEEE, pp. 640-645, 1997.|
|14||Escalera et al., "Road Traffic Sign Detection and Classification," IEEE Transactions on Industrial Electronics, vol. 44, No. 6, pp. 848-859, Dec. 1997.|
|15||Estable et al., "A Real-Time Traffic Sign Recognition Systems," Daimler-Benz Research Center, Proceedings of the Intelligent Vehicles '94 Symposium, Paris, France, pp. 213-218, Oct. 1994.|
|16||Estevez, "Auto-Associative Segmentation for Real-Time Object Recognition in Realistic Outdoor Images," Journal of Electronic Imaging, Apr. 1998, pp. 378-385, vol. 7(2), Apr. 1998, USA.|
|17||Evgenious et al., "A Trainable Pedestrian Detection System," Center for Biological and Computational Learning and Artificial Intelligence Laboratory, MIT IEEE International Conference on Intelligent Vehicles, pp. 241-246, 1998.|
|18||Fan et al., "Automatic Model-Based Semantic Object Extraction Algorithm," IEEE Trans on Circuits and Systems for Video Technology, vol. 11, No. 10, Oct. 2001, pp. 1073, USA.|
|19||Ghica et al., "Recognition of Traffic Signs by Artificial Neural Network," Dept. of Computer Science Memorial University of Newfoundland, IEEE, pp. 1444-1449, Mar. 1995.|
|20||Hak et al., "Dominant Color Transform and Circular Pattern Vector for Traffic Sign Detection and Recognition," IEICE Transaction Fundamentals, vol. E81-A, No. 6, pp. 1128-1135, Jun. 1998.|
|21||Hjaltason et al., "Using Spatial Sorting and Ranking in Model Based Object Recognition," Computer Science Dept. University of Maryland, College Park, Maryland, 3 pgs., Aug. 1998.|
|22||Internet Printout: The Chamfer System, 4 pgs., c. approximately 1999.|
|23||Internet Printout: The Road Sign Recognition System-Rs2, Faculty of Transportation Sciences, Prague, Czech Republic, 7 pgs., c. approximately 1999.|
|24||Internet Printout: The Road Sign Recognition System—Rs2, Faculty of Transportation Sciences, Prague, Czech Republic, 7 pgs., c. approximately 1999.|
|25||J Patrick Bixler, Department of Computer Science, Virginia Tech, Blacksburg, Virginia; David P. Miller Artificial Intelligence Group, Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California, "Extracting text from real-world scenes", Article, 8 pp., 1988.|
|26||Kalinke et al., "A Texture-based Object detection and an Adaptive Model-Based Classification," Institute Für Neuroinformatik, Bochum, Germany, IEEE International Conference on Intelligent Vehicles, pp. 143-148, 1998.|
|27||Liu et al., "Using Centroid Covariance in Target Recognition," Dept. of Electrical Engineering, University of Washington, Seattle, Washington, 4 pgs, prior to May 2004.|
|28||Logemann, "Realtime Traffic Sign Recognition (TSR)," Ed., Univeritat Koblenz-Landau, 3 pgs., Nov. 1997.|
|29||Lui et al., "Scalable Object-Based Image Retrieval," a pdf paper, prior to May 2004, USA.|
|30||Luo et al., "Landmark Recognition Using Projection Learning for Mobile Robot Navigation," Center for Robotics and Intelligent Machines, IEEE World Congress on Computational Intelligence, vol. IV, pp. 2703-2708, Jun. 1994.|
|31||Mandal, "Illumination Invariant Image Indexing Using Moments and Wavelets," Journal of Electronic Imaging, Apr. 1998, pp. 282-293, vol. 7, (2), USA.|
|32||Moss et al., "Registering Multiple Cartographic Models with the Hierarchical Mixture of Experts Algorithm," Dept. of Computer Science, University of New York, IEEE, pp. 909-914, 1997.|
|33||Nwagboso et al., "Surveillance Systems for Terrestrial Transport Safety and Improved User Information Capability," Bolton Institute, Bolton, United Kingdom, Dept. of Biophysical & Electric Engineering, Genova, Italy, Vigitec, Brussels, Belgium, pp. 1-7, prior to May 2004.|
|34||Ozer et al., "A Graph Based Object Description for Information Retrieval in Digital Image and Video Libraries," a pdf paper, 1999, USA.|
|35||Risack et al., "Robust Lane Recognition Embedded in a Real-Time Driver Assistance System," Fraunhofer-Institut Für Informations, Karlsruhe, Germany IEEE International Conference on Intelligent Cehicles, pp. 35-40, 1998.|
|36||Schütz et al., "Multi-Feature Matching Algorithm for Free-Form 3D Surface Registration," Institute for Microtechnology, Neuchatel, Switzerland, 3 pgs., Aug. 1998.|
|37||Yanai et al., "An Architecture of Object Recognition System for Various Images Based on Multi-Agent," Dept. of Computer Science, University of Electro-Communications, Tokyo, Japan, and Dept. of Mathematical Engineering and Information Physics, University of Tokyo, Tokyo, Japan, 4 pgs., prior to May 2004.|
|38||You et al., "Real-Time Object Recognition: Hierarchical Image Matching in a Parallel Virtual Machine Environment," School of Computing and Information Technology, Griffith University, Brisbane, Australia, Dept. of Computer Engineering, University of Nebraska, Lincoln, Nebraska, 3 pgs., Aug. 1998.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8538680 *||Jun 13, 2011||Sep 17, 2013||Institute For Information Industry||Real-time navigation electronic device and method based on determining current traffic rule information, and corresponding computer readable storage medium for storing program thereof|
|US8860944||Sep 12, 2013||Oct 14, 2014||Facet Technology Corp.||System and assessment of reflective objects along a roadway|
|US8903199||Feb 6, 2012||Dec 2, 2014||Google Inc.||Methods and apparatus for automated true object-based image analysis and retrieval|
|US8908996 *||Jan 31, 2012||Dec 9, 2014||Google Inc.||Methods and apparatus for automated true object-based image analysis and retrieval|
|US8908997||May 29, 2014||Dec 9, 2014||Google Inc.||Methods and apparatus for automated true object-based image analysis and retrieval|
|US9037600 *||Jan 28, 2011||May 19, 2015||Yahoo! Inc.||Any-image labeling engine|
|US9218364||Jan 28, 2011||Dec 22, 2015||Yahoo! Inc.||Monitoring an any-image labeling engine|
|US9335255||Oct 13, 2014||May 10, 2016||Facet Technology Corp.||System and assessment of reflective objects along a roadway|
|US20120134583 *||May 31, 2012||Google Inc.||Methods and apparatus for automated true object-based image analysis and retrieval|
|US20120265435 *||Jun 13, 2011||Oct 18, 2012||Institute For Information Industry||Real-time navigation electronic device and method based on determining current traffic rule information, and corresponding computer readable storage medium for storing program thereof|
|USD739859||Jun 6, 2012||Sep 29, 2015||Apple Inc.||Display screen or portion thereof with graphical user interface|
|USD740832 *||Jun 6, 2012||Oct 13, 2015||Apple Inc.||Display screen or portion thereof with a graphical user interface|
|U.S. Classification||382/305, 707/709, 707/E17.108, 707/E17.03, 382/100|
|International Classification||G06F17/30, G06K9/32, G06K9/46, G06F7/00, G06K9/00|
|Cooperative Classification||Y10S707/99933, G06F17/30256, G06K9/46, G06F17/30262, G06F17/30259, G06F17/3025, G06K9/3233, G06K9/4652, G06K2009/4666, G06K9/00536|
|Aug 8, 2011||AS||Assignment|
Owner name: GOOGLE INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:FACET TECHNOLOGY CORP.;REEL/FRAME:026731/0823
Effective date: 20110621
|Jul 15, 2014||CC||Certificate of correction|
|Oct 5, 2015||FPAY||Fee payment|
Year of fee payment: 4