WO2006007485A2 - Application for united states letters patent - Google Patents

Application for united states letters patent Download PDF

Info

Publication number
WO2006007485A2
WO2006007485A2 PCT/US2005/022617 US2005022617W WO2006007485A2 WO 2006007485 A2 WO2006007485 A2 WO 2006007485A2 US 2005022617 W US2005022617 W US 2005022617W WO 2006007485 A2 WO2006007485 A2 WO 2006007485A2
Authority
WO
WIPO (PCT)
Prior art keywords
intensity
region
border
dithered
intensity level
Prior art date
Application number
PCT/US2005/022617
Other languages
French (fr)
Other versions
WO2006007485A3 (en
Inventor
David L. Weigand
Original Assignee
Exphand, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Exphand, Inc. filed Critical Exphand, Inc.
Publication of WO2006007485A2 publication Critical patent/WO2006007485A2/en
Publication of WO2006007485A3 publication Critical patent/WO2006007485A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/40Picture signal circuits
    • H04N1/40081Soft dot halftoning, i.e. producing halftone dots with gradual edges

Definitions

  • the present invention in one implementation, relates to a method that facilitates the capture of transitions between two differently colored regions by a digital image capture device.
  • the present invention has application to an interactive electronic commerce system that utilizes dithered code blocks to communicate message codes to users viewing television and other video images.
  • the message code facilitates financial transactions and data interchange, as well as dissemination, management and exchange of commercial information, for example, advertising, promotions, sales announcements, and the like.
  • the message carriers and message codes are disseminated through advertising and content from analog and digital television broadcasts, television receivers, the Internet, wireless signals, satellite broadcasts, and print media.
  • the message carrier is typically captured by a Universal Digital Assistant (UDA), such as a personal digital assistant (PDA), using a digital camera.
  • UDA Universal Digital Assistant
  • PDA personal digital assistant
  • the UDA captures a digital image of a television screen displaying the message carrier, and software methods are then used to analyze the digital picture and resolve the message code contained in the message carrier.
  • the message carrier can be an image composed of blocks representing data that is shown on a television screen (e.g., as shown in Figure 1). These blocks can vary in color and/or intensity to convey the data.
  • this block-based encoding scheme can use low-intensity blocks to represent one bit value and high-intensity blocks to represent another bit value.
  • a digital camera uses an image sensor, such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor, to capture a digital image.
  • CCD charge-coupled device
  • CMOS complementary metal-oxide semiconductor
  • the image sensor is generally composed of thousands of photosites that receive light passing through the lens of the digital camera. This light is converted into electrical charges by the photosites and there is a limit to how much charge each photosite can store. If there is too much charge for one photosite, the charge will overflow to neighboring photosites causing a two-dimensional inter-symbol interference (ISI) effect which is called image blooming or streaking.
  • ISI inter-symbol interference
  • this blooming effect can obscure or blur the transitions between the low-intensity bits and the high-intensity bits in the block-based encoding scheme discussed above (e.g., as shown in Figure 2), rendering it more difficult, and in some instances impossible, to detect these transitions and decode the message carrier.
  • intersymbol interference refers to distortion of a received signal, that distortion being manifested in the temporal spreading and consequent overlap of portions of the signal to the degree that the receiver cannot reliably distinguish between changes of state, for example, between individual signal elements. Similar to blooming effects, ISI can blur transitions between low-intensity bits and high-intensity bits,.
  • the invention provides methods for encoding transitions between low- intensity regions and high-intensity regions such that the transitions are more readily detectable in a digital image captured by a digital image capture device.
  • the encoding applied to the transitions act to retard and minimize blooming effects and ISI that can occur in the digital image, especially when the transitions are captured from a television, computer display screen or other transmissive media.
  • the dithering method described herein can be used in connection with a graphical message carrier transmitted in television or other video signals.
  • a block-based message carrier containing at least one message code is transmitted with a video or television signal.
  • the message carrier includes transitions between low-intensity regions and high-intensity regions that represent data.
  • the message carrier is shown on a display screen and subsequently captured by taking a digital image of the display screen; the digital image is then decoded to identify the at least one message code.
  • the high-intensity regions of the message carrier include dithered encoded regions to counteract blooming effects and ISI in the digital image.
  • the dithered regions border at least the transitions in the high intensity regions and have a lower intensity level than the high-intensity regions.
  • Photosites associated with the dithered regions are therefore no longer at the highest intensity level and can absorb overflowing charge from the bordering high intensity regions. This substantially prevents the charge at these photosites from overflowing into low-intensity regions of the message carrier captured by the digital imaging device, thereby minimizing corruption of the transitions between high-intensity regions and low-intensity regions.
  • Figure 1 illustrates a screen display including one implementation of a message carrier.
  • Figure 2 illustrates the blooming effects associated with a message carrier displayed by a television or other display device.
  • Figure 3A illustrates one implementation of a dithered white block adjacent to a black block, as well as its corresponding intensity profile.
  • Figure 3B illustrates one implementation of a partially dithered white block adjacent to a black block.
  • Figure 4 illustrates another implementation of a dithered white block adjacent to a black block.
  • Figure 5 illustrates yet another implementation of a dithered white block adjacent to a black block.
  • Figure 6 illustrates another implementation of a dithered white block adjacent to a black block.
  • Figure 7 illustrates two message carriers using different implementations of the dithered white blocks.
  • Figure 8 illustrates a vertical strip message carrier that runs along an edge of a display screen.
  • Figure 9 sets forth a computer network environment including an interactive electronic commerce system.
  • Figure 10 is a block diagram illustrating the functionality associated with a universal digital assistant.
  • Figure 1 illustrates a message carrier 100 that presents at least one message code on a television or other display screen 96.
  • Message carrier 100 utilizes a block- based encoding scheme resembling an overt checkerboard-like design.
  • the normal television image occupies most of display screen 96 while message carrier 100 is generally displayed in one of the corners or on the side of display screen 96.
  • Users can be informed, through advertising and the like, that when a message carrier 100 appears on a television screen, they should capture a digital image of message carrier 100 if they desire further information or promotions associated with a product or service shown.
  • the display of message carrier 100 can act as a signal to a user that they can use a digital image capture device, such as a digital camera built into a mobile telephone, to capture a digital image of display screen 96 with message carrier 100. Users can also be informed that message carrier 100 includes a message code that can be transmitted to a remote system and the user will then receive information or a promotional message related to the product or service.
  • a digital image capture device such as a digital camera built into a mobile telephone
  • the block-based encoding scheme of message carrier 100 uses a series of two- dimensional blocks.
  • the blocks use at least two intensity levels and the intensity levels of the blocks achieves a binary encoding scheme.
  • a digital image of message carrier 100 can be captured and processed to read the pattern of blocks using either the intensity level of the blocks or the transitions between blocks. Reading the pattern of blocks yields the message code.
  • message carrier 100 of Figure 1 uses low-intensity regions 102 (e.g., dark-colored blocks such as black) and high-intensity regions 104 (e.g., light-colored blocks such as white) to represent bits that form the message code.
  • low-intensity regions 102 can represent one bit value (e.g., a value of "0") and high-intensity regions 104 can represent another bit value (e.g., a value of "1").
  • the use of low-intensity regions 102 and high-intensity regions 104 provides an acute distinction between the two data values, thereby improving the reliability of data detection by any device that is configured to extract the message code from message carrier 100.
  • transitions between regions can be used to represent data.
  • a transition from high to low-intensity or from low to high-intensity can represent a "1" bit, while the absence of a transition (i.e., two adjacent blocks of the same intensity level) can represent a "0" bit.
  • both low-intensity regions 102 and high-intensity regions 104 are squares or blocks. In other implementations, however, the regions can be any shape possible, including but not limited to circles, ovals, rectangles, triangles, pentagons, and other polygons.
  • the data can be represented by a cell pair.
  • this transition can represent one bit value (e.g., a value of "0") and if the cell pair is a high intensity cell adjacent to a low intensity cell this transition can represent another bit value (e.g., a value of "1").
  • colors on a television or computer display can be characterized by the intensity level (0 to 255) of each of the primary color components (i.e., red, blue, and green) used to form that color.
  • the low-intensity color black is represented as 0 red, 0 blue, and 0 green
  • the high-intensity color white is represented as 255 red, 255 blue, and 255 green. Because all three components of the color white are at the highest intensity level available, white regions 104 used in message carrier 100 are particularly susceptible to blooming effects and ISI when captured in a digital image.
  • Figure 2 illustrates the blooming effects associated with a message carrier 100 displayed by a television or other display device.
  • Light transmitted by high-intensity regions 104 of message carrier 100 can overwhelm photosites of an image sensor in a digital image capture device.
  • the photosites convert the light into electrical charges, they reach their maximum capacity and the excess charge overflows into neighboring photosites. As these neighboring photosites reach their maximum capacities, the charge continues to overflow until photosites are reached that can absorb this excess charge.
  • the photosites corresponding to low-intensity regions 102 of message carrier 100 generally absorb this excess charge.
  • Implementations of the invention employ modifications to message carrier 100 to counteract the blooming effects and ISI.
  • blooming effects occur when charge overflows from photosites corresponding to high-intensity regions 104. Therefore, in accordance with the invention, the intensity level of certain portions of each high-intensity region 104 is reduced to create a buffer for the overflowing charge. These areas with lowered intensity levels are referred to herein as dithered regions. Photosites capturing the dithered regions are now at a lower intensity level and can absorb some of the overflowing charge. This substantially prevents the charge from overflowing into neighboring low-intensity regions 102 of message carrier 100.
  • the dithered regions absorb the excess charge, their intensity levels will often rise to match the intensity level of high-intensity regions 104; therefore, these dithered regions will appear substantially similar to high-intensity regions 104 in the captured digital image that is processed as described below.
  • Figure 3A illustrates one implementation of the invention where a high- intensity region of message carrier 100 is modified to minimize the blooming effects.
  • Figure 3A illustrates a high-intensity region 300 adjacent to low-intensity region 102, both of which can be used in message carrier 100.
  • Figure 3 A also shows an overall intensity profile 302, measured from 0 to 255, for the two regions along a cross- section A - A'. According to this profile 302, an intensity level of 255 represents the color white, an intensity level of 0 represents the color black, and intensity levels between 0 and 255 represent various shades of gray.
  • the intensity levels for each of the individual color components i.e., red, blue, and green
  • This dithering methodology can also be applied to message carriers that only use one of the primary color components.
  • high-intensity region 300 includes a middle region 304 at a high-intensity level (e.g., 255 for white) and includes at least one dithered region 306 at a lower intensity level to minimize the blooming effects.
  • the shape of dithered region 306 generally corresponds to the shape or perimeter of high-intensity region 300.
  • the shape of dithered region 306 is square as well.
  • the shape of dithered region 306 generally follows the transition between high-intensity region 300 and low-intensity region 102.
  • dithered region 306 is shown in profile 302 as having an intensity level slightly below 255.
  • the intensity level of dithered region 306 can be any value that is less than the intensity value of middle region 304 (e.g., 255) and higher than 0.
  • the width and intensity level of dithered region 306 can be adjusted based on how strong the blooming effects are for a given combination of display screen type and digital image capture device.
  • the width and numbers of the encoded dithered region increase as the ISI levels increase.
  • the width of dithered region 306 has a lower limit that is based on the pixel size of the display screen. Dithered region 306 appears as a shade of gray when displayed on a television or computer display.
  • low-intensity region 102 has a single intensity level (e.g., 0 for black) across its entire cross-section. This is also shown in profile 302. In most implementations of the invention, only the high-intensity regions of message carrier 100 are modified with dithered regions because the low-intensity regions generally do not cause blooming effects to occur.
  • high-intensity region 300 includes a second dithered region 308 that can surround dithered region 306 with an even lower intensity level.
  • the shape of dithered region 308 can generally correspond to the shape of high-intensity region 300 or dithered region 306.
  • the intensity level of dithered region 308 can be any value that is less than the intensity value of middle region 304 (e.g., 255) and higher than 0.
  • the intensity level of dithered region 308 is slightly lower than the intensity level of dithered region 306, as shown in intensity profile 302.
  • the intensity level of dithered region 308 can be lower than the intensity value of middle region 304 but higher than the intensity level of dithered region 306. Dithered region 308 also appears as a shade of gray when displayed on a television or computer display. In further implementations of the invention, any number of dithered regions with varying intensity levels can be added to high- intensity region 300.
  • Dithered regions 306 and 308 therefore substantially prevent the blooming effects from expanding beyond the edges of high-intensity region 300.
  • dithered regions 306 and 308 tend to appear substantially white because their intensity levels will increase to at or around 255 as they absorb the overflowing charge.
  • Low-intensity regions 102 will appear substantially as solid black squares since the blooming effects will be minimized by dithered regions 306 and 308.
  • message carrier 100 will generally appear as a series of substantially solid white blocks and solid black blocks.
  • FIG. 4 illustrates another implementation of a dithered high-intensity region 400 adjacent to low-intensity region 102.
  • high- intensity region 400 includes a middle region 402 that is at a high-intensity level (i.e., 255 for white).
  • High-intensity region 400 also includes dithered regions to minimize blooming effects and ISI.
  • One dithered region is a circular dithered region 404 that is at a lower intensity level than middle region 402.
  • the shape of the dithered region does not correspond to the shape of high-intensity region 400.
  • the intensity level of circular dithered region 404 can be any value that is less than the intensity level of middle region 402 (e.g., 255) and higher than 0. In one implementation, the intensity level of circular dithered region 404 is slightly less than the intensity level of middle region 402.
  • High-intensity region 400 also includes a second dithered region 406 that surrounds circular dithered region 404 and has an even lower intensity level.
  • the intensity level of second dithered region 406 can be any value that is less than the intensity level of middle region 402 and higher than 0. In the implementation shown, the intensity level of second dithered region 406 is slightly lower than the intensity level of circular dithered region 404. In other implementations, the intensity level of second dithered region 406 can be lower than the intensity level of middle region 402 but higher than the intensity level of the circular dithered region 404.
  • the width and intensity level of circular dithered region 404 and second dithered region 406 can be adjusted based on how strong the blooming effects are for a given combination of display screen type and digital camera. Both circular dithered region 404 and second dithered region 406 appear as shades of gray on a television or computer display screen. In further implementations of the invention, any number of circular dithered regions with varying intensity levels can be added to high-intensity region 400.
  • Figure 5 illustrates yet another implementation of the invention with a dithered high-intensity region 500 adjacent to low-intensity region 102.
  • high-intensity region 500 has a middle region 502 that is at a high intensity level (i.e., 255 for white) and includes a series of dithered regions 504 that are spaced apart. The spacing allows portions of high-intensity region 500 to appear between dithered regions 504, as shown at areas 500A.
  • Dithered regions 504 can have an intensity level that is lower than the intensity level of middle region 502 (e.g., 255) and higher than 0. In one implementation, the intensity level of dithered regions 504 is slightly less than the intensity level of middle region 502. The use of spaced apart dithered regions 504 in this implementation provides a technique by which the blooming effects can be controlled or mitigated over a larger area.
  • dithered regions 504 can be used to retard the build-up of overflowing charge, thereby preventing the edges of high- intensity region 500 from being overwhelmed.
  • the areas 500A are included to ensure that dithered regions 504 do not cause the intensity level of the outer areas of high- intensity region 500 to fall substantially below the intensity level of middle region 502.
  • dithered regions 504 can be used in conjunction with dithered regions 306 and 308, as shown in Figure 3A, to counteract blooming effects.
  • the intensity levels of the dithered regions 504 are not equal. Again, the width and intensity level of dithered regions 504 can be adjusted based on how strong the blooming effects are for a given combination of display screen type and digital camera.
  • Figure 6 illustrates a variation of the implementation shown in Figure 5.
  • Figure 6 illustrates a dithered high-intensity region 600 adjacent to low-intensity region 102.
  • High-intensity region 600 includes a middle region 602 that is at a high intensity level (i.e., 255 for white) and includes a series of spaced apart circular dithered regions 604 surrounding middle region 602.
  • High-intensity region 600 can be seen in between circular dithered regions 604 at areas 606.
  • circular dithered regions 604 can have an intensity level that is lower than the intensity level of middle region 602 and higher than 0.
  • the intensity level of circular dithered regions 604 is slightly less than the intensity level of middle region 602.
  • circular dithered regions 604 provide a technique by which the blooming effects can be controlled or mitigated over a larger area.
  • circular dithered regions 604 can be used in conjunction with dithered regions 404 and 406 as shown in Figure 4.
  • the intensity levels of circular dithered regions 604 are not equal.
  • the width and intensity level of circular dithered regions 604 can be adjusted based on how strong the blooming effects are for a given combination of display screen type and digital camera.
  • the dithered regions can be applied only to sides of high-intensity region 104 where a transition occurs between high-intensity region 104 and low-intensity region 102.
  • a transition occurs only on one lateral side of each of the high intensity regions. Therefore, in this implementation, there is only one transition that requires a dithered region. This is shown in Figure 3B.
  • the dithered regions can be applied anywhere within the high-intensity regions that they are needed to minimize blooming effects and ISI.
  • dithered regions can be applied anywhere on the television or computer display screen 96 where transitions occur between high-intensity regions and low-intensity region anywhere. This allows users to capture digital images of the actual television images themselves without the images being corrupted due to blooming effects and ISI.
  • the television images can be captured in addition to, or in lieu, of message carriers 100.
  • Figure 7 shows a display screen 96 displaying two different message carriers 100.
  • Message carrier IOOA utilizes the dithered high-intensity regions 300 shown in Figure 3A.
  • Message carrier IOOB implements the dithered high-intensity regions 400 shown in Figure 4.
  • Figure 7 illustrates how message carriers IOOA and IOOB appear to a viewer when displayed on a television or computer display screen 96. In a captured digital image, however, both message carriers IOOA and IOOB will more closely resemble message carrier 100 of Figure 1 due to blooming effects from the middle regions of the high-intensity regions being absorbed by the dithered regions.
  • message carrier 100 can pair high- intensity regions that are white with low-intensity regions that are black. In other implementations, message carrier 100 can pair high-intensity regions that are white with low-intensity regions that are colors other than black. For example, a color combination of white and green can be used to implement the binary encoding scheme used by message carrier 100. In such an implementation, the intensity level of the green blocks will be 0 red, 0 blue, and 255 green, while the white blocks will still be 255 red, 255 blue, and 255 green. The intensity level of the dithered regions can be manipulated by lowering the intensity levels of only the red and blue components while leaving the green component intact.
  • the dithered regions will therefore appear as shades of green rather then shades of gray.
  • Other colors including but not limited to black, brown, blue, green, purple, and red, can also be used. Dark colors tend to work better than lighter colors. Similar techniques can be used for the other colors used in message carriers 100,
  • the dithered regions can be implemented for each and every high-intensity region 104 used in message carrier 100. In other implementations, the dithered regions can be implemented only in areas where a transition occurs between high-intensity region 104 and low-intensity region 102.
  • the message carrier can be a vertical strip message carrier 800 that runs along an edge of display screen 96.
  • message carrier 800 can be oriented along the left side of display screen 96; however, message carrier 800 can extend along any edge of screen 96, including the top, right, or bottom edges.
  • Message carrier 800 can include dithered regions 802 on high-intensity regions to counteract any blooming effects and ISI that can occur.
  • the implementations shown above use block-shaped high-intensity regions and low-intensity regions
  • further implementations of the message carrier can use regions that are any other shape.
  • the message carrier can use circular high and low-intensity regions.
  • the message carrier can be formed using an array of high-intensity circles or another shape on a low-intensity background.
  • the message carrier can be formed using low-intensity circles or another shape on a high-intensity background.
  • message carriers 100 using dithered high-intensity regions can be used in print media and the like.
  • print media that uses high-gloss paper or highly reflective paper can benefit from the use of implementations of the invention.
  • the highly reflective surface of glossy paper can reflect too much light back to the photosites of a digital camera, particularly if a flash is used when capturing the digital image.
  • the use of dithered high-intensity regions can counteract the subsequent blooming effects that can occur.
  • the message carriers of the invention can be used in electronic commerce systems to transmit data to users.
  • One such electronic commerce system is described in U.S. Patent Application No.10/618,445 entitled "Interactive Electronic Commerce and Message Interchange System," which is incorporated by reference.
  • Figure 9 illustrates an electronic commerce system 10 utilizing a data encoding and decoding scheme according to one implementation of the invention.
  • electronic commerce system 10 includes one or more universal digital assistants (UDA) 90 that can receive data from a media source, for example, a display device 95 having display screen 96.
  • the display device 95 can be a television or a computer monitor.
  • Electronic commerce system 10 also includes a UDA application site 50 that interacts with UDA 90 to facilitate financial transactions and carry out other processes as described herein.
  • UDA 90 interacts with UDA application site 50 through a telecommunications network 20 and a computer network 40.
  • Electronic commerce system 10 can also include a merchant system 60, an advertising system 62, at least one network access device such as client computer 64, at least one transaction processing network 70 (e.g., an Automated Clearing House (ACH) network, credit card network, etc.), and one or more financial institutions 72.
  • ACH Automated Clearing House
  • Telecommunications network 20 can be a wireless telephone network (as shown in Figure 9), a land-based telephone network, and/or a combination thereof.
  • Telecommunications network 20 can be any suitable digital or analog wireless network, such as a Time Division Multiple Access (TDMA) network, a Global System for Mobile communication (GSM) network, or a Code-Division Multiple Access (CDMA) network.
  • TDMA Time Division Multiple Access
  • GSM Global System for Mobile communication
  • CDMA Code-Division Multiple Access
  • Telecommunications network 20 enables transmission of voice and other data to mobile wireless devices, such as UDA 90. Telecommunications network 20 therefore enables communication between UDA 90 and other systems operably connected thereto.
  • telecommunications network 20 includes a Wireless Application Protocol (WAP) gateway 22, which provides a set of communication protocols enabling wireless devices, such as UDA 90, to access the Internet or similar computer network 40.
  • WAP Wireless Application Protocol
  • WAP gateway 22 is operative to establish a connection (e.g., a Wireless Session Protocol (WSP) connection) with UDA 90, to receive requests designating an application server or other resource on computer network 40 from UDA 90, to translate the request into a hypertext transfer protocol (HTTP) or other suitable request to the appropriate application server, to receive a response from the application server, and to translate and transmit the response to UDA 90.
  • WSP Wireless Session Protocol
  • HTTP hypertext transfer protocol
  • the transmission of data between UDA 90 and UDA application site 50 can occur over a Short Message Service (SMS) gateway
  • SMS Short Message Service
  • the client functionality of UDA 90 is operative to configure SMS messages into a predefined format recognized by UDA appli cation site 50.
  • Computer network 40 can be any suitable computer network, including but not limited to an Internet Protocol (IP) network, such as the Internet.
  • IP Internet Protocol
  • Computer network 40 can be an open, wide-area network (WAN), an electronic network, an optical network, a wireless network, and/or any combination thereof.
  • Computer network 40 can be a packet-based communications environment, employing Transmission Control Protocol/Internet Protocol (TCP/IP) standards and having a plurality of interconnected digital packet transmission stations operative to route data between TCP/IP end systems.
  • TCP/IP Transmission Control Protocol/Internet Protocol
  • the invention however, has applications in computer network environments employing any suitable transport layer and network layer protocols.
  • UDA 90 in one implementation, is a wireless device that provides computing and information storage and retrieval capabilities for applications installed therein. UDA 90 is operative to communicate with remote devices over telecommunications network 20. In one implementation, UDA 90 includes hardware and/or software operative to acquire message codes using one or more code capture technologies and then communicate such codes to UDA application site 50. UDA application site 50 is operative to return at least one promotional message to UDA 90 for display or other action to the user.
  • the functionality associated with UDA 90 can be integrated into a variety of wireless device platforms, such as cell phones, wireless personal digital assistants (PDA), universal remote controllers, television remote controllers, pagers, wearable computing devices, and the like.
  • UDA 90 can further include short-range wireless functionality for wireless communication of data with a second device, such as a point-of-sale (POS) terminal or any other suitably configured device.
  • POS point-of-sale
  • This functionality can be implemented using, for example, infrared signals (e.g., IrDA), radio signals (e.g., 802.11 technologies (Wi- Fi) or Bluetooth), short burst Ethernet, wireless network signals (e.g., GSM, TDMA, or CDMA), and message services such as SMS and Multimedia Message Service (MMS).
  • IrDA infrared signals
  • radio signals e.g., 802.11 technologies (Wi- Fi) or Bluetooth
  • short burst Ethernet e.g., wireless network signals (e.g., GSM, TDMA, or CDMA)
  • MMS Multimedia Message Service
  • Such functionality allows UDA 90, for example, to transmit credit card account information and other information directly to a POS terminal equipped with complementary wireless communication functionality
  • UDA 90 in one implementation, is also operative to receive data from telecommunications network 20 (or computer network 40) and transmit data to telecommunications network 20 (or computer network 40) for routing to appropriate devices.
  • UDA 90 in one implementation, is an Internet-enabled device capable of receiving data from remote servers and displaying data on a user interface screen.
  • UDA 90 is a WAP-enabled device, such as a WAP mobile phone, that includes a WAP client (e.g., a Wireless Application Environment (WAE) user agent, such as a WAP browser, and a Wireless Telephony Application (WTA) user agent).
  • WAP client e.g., a Wireless Application Environment (WAE) user agent, such as a WAP browser, and a Wireless Telephony Application (WTA) user agent.
  • WTA Wireless Telephony Application
  • UDA 90 can be a wireless PDA including HTML compliant or HTML-supported browser functionality, such as Pocket PC including Pocket Internet Explorer® (PIE), which is a mobile version of Microsoft's Internet Explorer®, including limited Javascript support and the ability to display HTML and Macromedia Flash files (assuming the Flash plug-in is installed).
  • PIE Pocket Internet Explorer®
  • FIG 10 is a block diagram illustrating the functionality of UDA 90 according to one implementation of the invention.
  • UDA 90 includes an image capture device 1002, a display device 1004, an antenna 1006, a client application control logic 1008, and a wireless communication interface 1010.
  • Image capture device 1002 is operative to capture digital images of message carriers 100 with dithered high-intensity regions and message codes transmitted in visual media.
  • image capture device 1002 can comprise a digital camera allowing for the capture of images containing message codes.
  • the digital camera can include an image sensor (not shown), such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor, to capture a digital image.
  • the image sensor is generally composed of thousands of photosites that receive light passing through the lens of the digital camera. This light is converted into electrical charges by the photosites and, as explained above, there is a limit to how much charge each photosite can store.
  • Display device 1004 is a liquid-crystal or other display device. In one implementation, display device is a touch-sensitive liquid crystal display device.
  • Client application control logic 1008 executes the client-side functionality described herein, such as processing captured digital images of message carrier 100 to acquire message codes and transmitting the message codes through wireless communication interface 1010 to UDA application site 50. Because message carriers 100 use dithered high-intensity regions, blooming effects and ISI should not corrupt low-intensity regions 102 of the digital image that is processed.
  • the image processing functionality of client application control logic 1008 can process a digital image by reading intensity values of the pixels that form the digital image, and then analyzing those intensity values to identify patterns or other behavior. Generally, the pixels are read by scanning vertical columns of pixels in the digital image, known as vertical scan lines. The pixel values can also be read by scanning horizontal rows of pixels, known as horizontal scan lines.
  • Wireless communication interface 1010 is operative to establish and maintain a connection to telecommunications network 20 using antenna 1006 to enable communication of data between UDA 90 and remote devices.
  • client application control logic 1008 can be implemented into a mobile telephone with an embedded runtime environment such as Java.
  • the runtime environment can execute a script or byte code program to perform the functions described herein and use the wireless communication system of the phone to transmit and receive data from remote devices.
  • image capture device 1002 can also be integrated into a mobile telephone.
  • image capture devices 1002 There are already numerous mobile telephones on the market that include image capture devices 1002. Some examples include, but are not limited to, the Sony Ericsson® T610, the Samsung® A600, the Sanyo® VM4500, the Motorola® V300, and the Nokia® 3650.
  • UDA application site 50 is connected to computer network 40 to communicate with end systems and other nodes operably connected thereto, such as client computer 64.
  • UDA application site 50 interacts with UDA 90 through computer network 40, WAP gateway 22, and wireless telecommunications network 20 to facilitate data interchanges, financial transactions, and other operations as described herein.
  • UDA application site 50 can be configured to support a variety of wireless devices, including IMode phones, and mobile wireless devices including Binary Runtime Environment for Wireless (BREW) or Java 2 Platform Micro Edition (J2ME) technologies.
  • UDA application site 50 includes at least one application server 52, a user account database 54, and a promotional message database 56.
  • Application server 52 includes functionality implementing the process flows described herein.
  • User account database 54 stores information related to at least one user account. Such user account information includes a user account identification, name, address, e-mail address, as well as system settings, preferences, user profile information (e.g., psychographic information, demographic information, and behavioral information) obtained by monitoring a user's use of the system or by direct interaction with the user through questionnaires and the like. User account database 54 can also store financial account information for at least one user to facilitate financial transactions using UDA 90.
  • Promotional message database 56 stores promotional messages, such as advertisements, coupons, and other commercial offers, in association with one or more message codes.
  • advertising system 62 transmits promotional messages in electronic form to promotional message database 56 for storage in association with message codes.
  • application server 52 can receive a message code from UDA 90 and return one or more promotional messages associated with that message code, as discussed herein.
  • UDA 90 operates in connection with UDA application site 50 to enable users to receive and review advertising, promotional, informational and other messages, as well as conduct financial transactions and other operations.
  • UDA 90 acquires message codes (e.g., from television advertisements or print advertisements) and transmits them to UDA application site 50, which pushes an advertising or promotional message back to UDA 90 in return.
  • Advertising and promotional messages can take a variety of forms, including but not limited to electronic coupons, product pricing, descriptive data, loyalty offerings, and other similar items.
  • a user can elect to save the advertising or promotional message for later use or inspection by storing it locally on UDA 90 or remotely on UDA application site 50 in association with his or her user account.
  • users may access UDA application site 50 through network access devices over computer network 40 to manage their respective user accounts through network access devices connected to computer network 40.
  • the network access device is a browser executed on a client computer 64, or another browser executed on a network computer (not shown).
  • page-based interfaces for accessing UDA application site 50 are transmitted to client computer 64 having a browser and a connection to computer network 40.
  • Client computer 64 can include any computer, special-purpose computing device, or any other suitable device having the required functionality.
  • client computer 64 includes at least one processor, a data storage system (including volatile and non-volatile memory), at least one input device (e.g., a keyboard), and at least one output device (e.g., a display).
  • client computer 64 is connected to computer network 40 through a modem connection (e.g., dial-up, DSL, or cable) or through a network line. Such connections can be wireless as well.
  • modem connection e.g., dial-up, DSL, or cable
  • Such connections can be wireless as well.
  • any suitable device, application, or client software for receiving, displaying, and transmitting data over a computer network can be used.
  • the browser implemented on client computer 64 supports the SSL ("Secure Sockets Layer”) protocol, the S-HTTP or HTTPS ("Secure HTTP”) protocol, or any other similar protocol for transmitting confidential or private information over an open computer network.
  • SSL Secure Sockets Layer
  • S-HTTP S-HTTP
  • HTTPS HTTPS
  • Users are individuals or other legal entities having the capacity to possess financial accounts, such as corporations, partnerships, non-profit organizations, trusts, and the like.
  • Client computer 64 is a TCP/IP end system operably connected to computer network 40 via any suitable means, such as through an Internet Services Provider (ISP) and the like.
  • Client computer 64 can be any suitable internet-enabled computing device, including but not limited to a desktop computer, a laptop computer, a mobile telephone, a personal digital assistant (PDA), or UDA 90 having wireless or wired access to computer network 40.
  • the wireless or wired access can be through a router, a wireless router executing the 802.11 wireless protocol, Mobitex, DataTAC, General Packet Radio Service (GPRS), or any other packet- switched wireless network.
  • client computer 64 includes internet browsing software for receiving, displaying, and transmitting data over a computer network.
  • Payment transaction processing network 70 supports a variety of non-cash payment mechanisms, including but not limited to purchases made by credit card, debit card, and Automated Clearing House (ACH) transactions.
  • payment transaction processing network 70 can be a credit card or debit card transaction processing network, such as Visa®, MasterCard®, Discover®, or American Express®.
  • payment transaction processing network 70 enables users, using UDA 90, to provide non-cash methods of payment to a merchant system 60 or at a POS terminal.
  • Financial institution 72 is where users maintain financial accounts, such as checking accounts, savings accounts, money market accounts, credit and/or debit card accounts, and other financial accounts.
  • a user can have one or more bank accounts at each of one or more financial institutions.
  • a user can have a checking account and a credit and debit card account at one financial institution and still have a credit account with a different financial institution.
  • UDA 90 is operative to detect and acquire message codes in message carriers 100 using one or more code capture technologies.
  • UDA 90 can be configured to acquire message codes from broadcast media, print media, or wireless transmissions.
  • UDA 90 is operative to acquire message codes in message carrier 100 which is embedded or otherwise transmitted in connection with a television broadcast signal and displayed on a display device 95, such as a television or computer display.
  • UDA 90 can include a code capture technology whereby a user can aim UDA 90 at display device 95 and capture a digital image of message carrier 100 being shown on display screen 96. The captured digital image or a portion thereof is then processed to locate message carrier 100 and identify a message code that is then acquired.
  • UDA 90 can use image capture device 1002 to capture a digital image and then use the image processing functionality of client application control logic 1008 to locate and acquire a message code in the captured image.
  • UDA 90 can emit a beam of light (e.g., a Class II laser pointer) that can assist a user to aim the image capture device of UDA 90 at screen 96. This is useful in situations where the user cannot see what the image capture device is pointed at.
  • the view through the image capture device can be shown to the user on display screen 1004 of UDA 90, thereby eliminating the need for a laser pointer.
  • UDA 90 can process the digital image to retrieve a message code contained in message carrier 100.
  • the message code can contain any type of data and can identify any of a number of items.
  • the message code can identify the television station to which display device 95 is tuned (e.g., station call letters or a station identifier).
  • an advertisement or promotional message can then be identified using the station identifier and the time the station identifier was captured.
  • UDA application site 50 can scan this information against a station log to resolve the advertising or other content the user was watching. UDA application site 50 can then return an advertising or promotional message associated with the broadcast advertising or other content.
  • the image processing functionality can reside on a remote server and the captured image can be transmitted to this remote server for processing.
  • Such an implementation moves the burden of processing the digital image to a remote server that will typically have much greater processing power than UDA 90. The time required to process the captured image will then be reduced.
  • UDA 90 can transmit the captured image to the remote server using any of the wireless technologies described herein, for instance, the captured image can be transmitted in an MMS message.
  • One or more features of message carrier 100 can be preset or constant to allow client application control logic 1008 of UDA 90 to locate message carrier 100 in a captured image and then locate the message code within the message carrier.
  • client application control logic 1008 reads and analyzes the intensity values of pixels in the captured image (e.g., using vertical scan lines or horizontal scan lines)
  • these preset or constant features can be detected and message carrier 100 can then be identified. From there, the message code can then be identified and acquired.
  • message carrier 100 can have one or more borders 106 (shown in Figure 1) that remain constant as the message code changes.
  • the one or more borders 106 can be one continuous color such as black.
  • a two-sided border 106 can be used around message carrier 100 to aid in its identification, where two-sided border 106 is one continuous color.
  • each message carrier 100 includes top and left border regions 106 that are colored black or any other suitable color. Accordingly, the checkerboard region is bordered on two sides by a black color, and by the television image on the third and fourth sides.
  • the image processing functionality of client application control logic 1008 can use two-sided border 106 to locate message carrier 100 and can then filter out two-side border 106 to extract the message code itself.
  • UDA 90 executes a process to decode the message code that includes capturing an image of the display screen including message carrier 100, analyzing vertical scan lines through the captured image to identify scan lines that intersect message carrier 100, locating and aligning an edge of message carrier 100, optionally validating one or more features surrounding the transitions, and reading the data bits in the message code.
  • UDA 90 includes image capture device 1002 to capture a digital image of display screen 96 containing message carrier 100.
  • the user aims image capture device 1002 at a television that is displaying message carrier 100, then the user triggers the capture functionality to capture a digital image.
  • the image can be pre-processed to indicate which pixels are white, which pixels are black, and which pixels are neither (i.e., a middle tone), thus reducing each pixel to a two-bit value (e.g., 11 for a white pixel, 10 for a black pixel, and 00 for a middle tone pixel).
  • the middle tone should generally represent much of the television image and any surrounding items in the camera image.
  • the image processing functionality located message carrier 100 by analyzing vertical scan lines to find solid border 106. In this implementation, the image processing functionality is looking for a large range of pixels that are all the same color.
  • border 106 can be made wider or taller than the blocks that form the message code.
  • Using an over-sized border improves the likelihood of a vertical scan line passing through border 106 in its entirety when the image processing functionality is trying to locate message carrier 100. Since the captured image may not be properly aligned, depending on how the user was holding the image capture device 1002 when the image was captured, the vertical scan lines may be at an angle relative to message carrier 100 and border 106.
  • the use of an over-sized border 106 increases the range of allowable angles for the vertical scan lines relative to border 106 where a vertical scan line can still pass through an entire border 106.
  • UDA 90 can transmit the message code to UDA application site 50 which resolves the message code and pushes an electronic commerce and/or data message back to UDA 90.
  • application server 52 receives a request including the message code and accesses promotional message database 56 to retrieve a promotional message associated with the message code. If a promotional message is located, application server 52 pushes the promotional message to UDA 90.
  • Application server 52 can also carry out other functions such as tracking the user's interests and behavior. For example, application server 52 can generate a log, stored in association with the user's account, which records the message codes received from the user and the promotional messages sent in response.
  • application server 52 can use the log to track system usage by the user to enable a fee-based service for advertisers. For example, an advertiser can be charged a fee for each promotional message pushed to a user's UDA 90.
  • UDA application site 50 in one implementation, can be configured to transmit usage or activity reports to advertising system 62 detailing the activity associated with promotional messages and/or user purchasing behavior.

Abstract

Methods for encoding transitions between low-intensity regions (102) and high-intensity regions (104) such that the transitions are more readily detectable in a digital image captured by a digital image capture device. The encoding applied to the transitions act to retard and minimize blooming effects and ISI that can occur in the digital image, especially when the transitions are captured from a television, computer display screen (96) or other transmissive media.

Description

APPLICATION FOR UNITED STATES LETTERS PATENT
Dithered Encoding and Decoding Information Transference System and Method
FIELD OF THE INVENTION
[0001] The present invention, in one implementation, relates to a method that facilitates the capture of transitions between two differently colored regions by a digital image capture device. In one implementation, the present invention has application to an interactive electronic commerce system that utilizes dithered code blocks to communicate message codes to users viewing television and other video images.
BACKGROUND OF THE INVENTION
[0002] Technologies have recently been developed that enable information to be transmitted to electronic devices by means such as television and print media. For example, U.S. Patent Application No. 10/673,047, filed on September 26, 2003 and entitled "Block-Based Encoding and Decoding Information Transference System and Method," and U.S. Patent Application No. 10/618,445, filed on July 11, 2003 and entitled "Interactive Electronic Commerce and Message Interchange System," both of which are hereby incorporated by reference, disclose methods and systems that enable the transmission of a message carrier within a stream of video and television data. This message carrier is subsequently captured by an electronic device and decoded to identify a message code. The message code facilitates financial transactions and data interchange, as well as dissemination, management and exchange of commercial information, for example, advertising, promotions, sales announcements, and the like. The message carriers and message codes are disseminated through advertising and content from analog and digital television broadcasts, television receivers, the Internet, wireless signals, satellite broadcasts, and print media.
[0003] In the above referenced patent applications, the message carrier is typically captured by a Universal Digital Assistant (UDA), such as a personal digital assistant (PDA), using a digital camera. The UDA captures a digital image of a television screen displaying the message carrier, and software methods are then used to analyze the digital picture and resolve the message code contained in the message carrier. As disclosed in U.S. Patent Application No. 10/673,047, the message carrier can be an image composed of blocks representing data that is shown on a television screen (e.g., as shown in Figure 1). These blocks can vary in color and/or intensity to convey the data. For instance, this block-based encoding scheme can use low-intensity blocks to represent one bit value and high-intensity blocks to represent another bit value.
[0004] Unfortunately, complications can arise when capturing a digital image of a message carrier that is being shown on a television or computer display. A digital camera uses an image sensor, such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor, to capture a digital image. The image sensor is generally composed of thousands of photosites that receive light passing through the lens of the digital camera. This light is converted into electrical charges by the photosites and there is a limit to how much charge each photosite can store. If there is too much charge for one photosite, the charge will overflow to neighboring photosites causing a two-dimensional inter-symbol interference (ISI) effect which is called image blooming or streaking.
[0005] Conventional digital camera images generally do not suffer from blooming effects because the light captured by the photosites is simply ambient light reflected by subjects within the image. Unlike conventional images, a digital image of a television or computer display is more susceptible to blooming or inter-symbol interference effects because these displays are transmitters of light, and do not simply reflect ambient light. The large amount of light transmitted by a television or computer display, especially from white or high-intensity regions on the display, can easily overload one or more photosites of an image sensor, thereby inducing the blooming effects. If such blooming effects occur within a message carrier, the integrity of the data being conveyed can be corrupted or destroyed. Specifically, this blooming effect can obscure or blur the transitions between the low-intensity bits and the high-intensity bits in the block-based encoding scheme discussed above (e.g., as shown in Figure 2), rendering it more difficult, and in some instances impossible, to detect these transitions and decode the message carrier.
[0006] Another complication we have mentioned that can arise when capturing a digital image from a television or computer display is called intersymbol interference (ISI). In a digital transmission system, intersymbol interference refers to distortion of a received signal, that distortion being manifested in the temporal spreading and consequent overlap of portions of the signal to the degree that the receiver cannot reliably distinguish between changes of state, for example, between individual signal elements. Similar to blooming effects, ISI can blur transitions between low-intensity bits and high-intensity bits,.
[0007] Accordingly, methods are needed to minimize the effect of blooming and ISI when capturing digital images of message carriers being shown on television or computer displays.
SUMMARY OF THE INVENTION
[0008] The invention provides methods for encoding transitions between low- intensity regions and high-intensity regions such that the transitions are more readily detectable in a digital image captured by a digital image capture device. The encoding applied to the transitions act to retard and minimize blooming effects and ISI that can occur in the digital image, especially when the transitions are captured from a television, computer display screen or other transmissive media.
[0009] The dithering method described herein can be used in connection with a graphical message carrier transmitted in television or other video signals. In one implementation, a block-based message carrier containing at least one message code is transmitted with a video or television signal. The message carrier includes transitions between low-intensity regions and high-intensity regions that represent data. The message carrier is shown on a display screen and subsequently captured by taking a digital image of the display screen; the digital image is then decoded to identify the at least one message code. The high-intensity regions of the message carrier include dithered encoded regions to counteract blooming effects and ISI in the digital image. The dithered regions border at least the transitions in the high intensity regions and have a lower intensity level than the high-intensity regions. Photosites associated with the dithered regions are therefore no longer at the highest intensity level and can absorb overflowing charge from the bordering high intensity regions. This substantially prevents the charge at these photosites from overflowing into low-intensity regions of the message carrier captured by the digital imaging device, thereby minimizing corruption of the transitions between high-intensity regions and low-intensity regions. DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 illustrates a screen display including one implementation of a message carrier.
[0011] Figure 2 illustrates the blooming effects associated with a message carrier displayed by a television or other display device.
[0012] Figure 3A illustrates one implementation of a dithered white block adjacent to a black block, as well as its corresponding intensity profile.
[0013] Figure 3B illustrates one implementation of a partially dithered white block adjacent to a black block.
[0014] Figure 4 illustrates another implementation of a dithered white block adjacent to a black block.
[0015] Figure 5 illustrates yet another implementation of a dithered white block adjacent to a black block.
[0016] Figure 6 illustrates another implementation of a dithered white block adjacent to a black block.
[0017] Figure 7 illustrates two message carriers using different implementations of the dithered white blocks.
[0018] Figure 8 illustrates a vertical strip message carrier that runs along an edge of a display screen.
[0019] Figure 9 sets forth a computer network environment including an interactive electronic commerce system.
[0020] Figure 10 is a block diagram illustrating the functionality associated with a universal digital assistant.
DETAILED DESCRIPTION
[0021] Figure 1 illustrates a message carrier 100 that presents at least one message code on a television or other display screen 96. Message carrier 100 utilizes a block- based encoding scheme resembling an overt checkerboard-like design. The normal television image occupies most of display screen 96 while message carrier 100 is generally displayed in one of the corners or on the side of display screen 96. Users can be informed, through advertising and the like, that when a message carrier 100 appears on a television screen, they should capture a digital image of message carrier 100 if they desire further information or promotions associated with a product or service shown. The display of message carrier 100 can act as a signal to a user that they can use a digital image capture device, such as a digital camera built into a mobile telephone, to capture a digital image of display screen 96 with message carrier 100. Users can also be informed that message carrier 100 includes a message code that can be transmitted to a remote system and the user will then receive information or a promotional message related to the product or service.
[0022] The block-based encoding scheme of message carrier 100 uses a series of two- dimensional blocks. The blocks use at least two intensity levels and the intensity levels of the blocks achieves a binary encoding scheme. In such an implementation, a digital image of message carrier 100 can be captured and processed to read the pattern of blocks using either the intensity level of the blocks or the transitions between blocks. Reading the pattern of blocks yields the message code.
[0023] In one implementation, message carrier 100 of Figure 1 uses low-intensity regions 102 (e.g., dark-colored blocks such as black) and high-intensity regions 104 (e.g., light-colored blocks such as white) to represent bits that form the message code. For instance, low-intensity regions 102 can represent one bit value (e.g., a value of "0") and high-intensity regions 104 can represent another bit value (e.g., a value of "1"). The use of low-intensity regions 102 and high-intensity regions 104 provides an acute distinction between the two data values, thereby improving the reliability of data detection by any device that is configured to extract the message code from message carrier 100. Alternately, transitions between regions can be used to represent data. For example, a transition from high to low-intensity or from low to high-intensity can represent a "1" bit, while the absence of a transition (i.e., two adjacent blocks of the same intensity level) can represent a "0" bit. In the implementation shown, both low-intensity regions 102 and high-intensity regions 104 are squares or blocks. In other implementations, however, the regions can be any shape possible, including but not limited to circles, ovals, rectangles, triangles, pentagons, and other polygons. In addition, the data can be represented by a cell pair. Specifically if the cell pair is a low intensity cell adjacent to a high intensity cell this transition can represent one bit value (e.g., a value of "0") and if the cell pair is a high intensity cell adjacent to a low intensity cell this transition can represent another bit value (e.g., a value of "1").
[0024] As is known to one of ordinary skill in the art, colors on a television or computer display can be characterized by the intensity level (0 to 255) of each of the primary color components (i.e., red, blue, and green) used to form that color. For instance, the low-intensity color black is represented as 0 red, 0 blue, and 0 green, while the high-intensity color white is represented as 255 red, 255 blue, and 255 green. Because all three components of the color white are at the highest intensity level available, white regions 104 used in message carrier 100 are particularly susceptible to blooming effects and ISI when captured in a digital image.
[0025] Figure 2 illustrates the blooming effects associated with a message carrier 100 displayed by a television or other display device. Light transmitted by high-intensity regions 104 of message carrier 100 can overwhelm photosites of an image sensor in a digital image capture device. As the photosites convert the light into electrical charges, they reach their maximum capacity and the excess charge overflows into neighboring photosites. As these neighboring photosites reach their maximum capacities, the charge continues to overflow until photosites are reached that can absorb this excess charge. For a digital image of message carrier 100, the photosites corresponding to low-intensity regions 102 of message carrier 100 generally absorb this excess charge. As the photosites for low-intensity regions 102 absorb the overflowing charge, high-intensity regions 104 appear to "bloom" and overtake portions of low-intensity regions 102. The digital image is then altered and several low-intensity regions 102, as well as transitions between regions, become corrupted by the overflowing charge. The resolution needed to decode message carrier 100 is often lost.
[0026] Implementations of the invention employ modifications to message carrier 100 to counteract the blooming effects and ISI. As described above, blooming effects occur when charge overflows from photosites corresponding to high-intensity regions 104. Therefore, in accordance with the invention, the intensity level of certain portions of each high-intensity region 104 is reduced to create a buffer for the overflowing charge. These areas with lowered intensity levels are referred to herein as dithered regions. Photosites capturing the dithered regions are now at a lower intensity level and can absorb some of the overflowing charge. This substantially prevents the charge from overflowing into neighboring low-intensity regions 102 of message carrier 100. Furthermore, as the dithered regions absorb the excess charge, their intensity levels will often rise to match the intensity level of high-intensity regions 104; therefore, these dithered regions will appear substantially similar to high-intensity regions 104 in the captured digital image that is processed as described below.
[0027] Figure 3A illustrates one implementation of the invention where a high- intensity region of message carrier 100 is modified to minimize the blooming effects. Figure 3A illustrates a high-intensity region 300 adjacent to low-intensity region 102, both of which can be used in message carrier 100. Figure 3 A also shows an overall intensity profile 302, measured from 0 to 255, for the two regions along a cross- section A - A'. According to this profile 302, an intensity level of 255 represents the color white, an intensity level of 0 represents the color black, and intensity levels between 0 and 255 represent various shades of gray. Thus, in this implementation, the intensity levels for each of the individual color components (i.e., red, blue, and green) are changed evenly so their values remain equal. This dithering methodology, however, can also be applied to message carriers that only use one of the primary color components.
[0028] As shown in profile 302, high-intensity region 300 includes a middle region 304 at a high-intensity level (e.g., 255 for white) and includes at least one dithered region 306 at a lower intensity level to minimize the blooming effects. In some implementations, the shape of dithered region 306 generally corresponds to the shape or perimeter of high-intensity region 300. Here, since the shape of high-intensity region 300 is square, the shape of dithered region 306 is square as well. In another implementation shown in Figure 3B, the shape of dithered region 306 generally follows the transition between high-intensity region 300 and low-intensity region 102. Returning the Figure 3A, dithered region 306 is shown in profile 302 as having an intensity level slightly below 255. According to the invention, the intensity level of dithered region 306 can be any value that is less than the intensity value of middle region 304 (e.g., 255) and higher than 0. The width and intensity level of dithered region 306 can be adjusted based on how strong the blooming effects are for a given combination of display screen type and digital image capture device. The width and numbers of the encoded dithered region increase as the ISI levels increase. The width of dithered region 306 has a lower limit that is based on the pixel size of the display screen. Dithered region 306 appears as a shade of gray when displayed on a television or computer display.
[0029] Unlike high-intensity region 300, low-intensity region 102 has a single intensity level (e.g., 0 for black) across its entire cross-section. This is also shown in profile 302. In most implementations of the invention, only the high-intensity regions of message carrier 100 are modified with dithered regions because the low-intensity regions generally do not cause blooming effects to occur.
[0030] In another implementation of the invention, high-intensity region 300 includes a second dithered region 308 that can surround dithered region 306 with an even lower intensity level. The shape of dithered region 308 can generally correspond to the shape of high-intensity region 300 or dithered region 306. In implementations of the invention, the intensity level of dithered region 308 can be any value that is less than the intensity value of middle region 304 (e.g., 255) and higher than 0. In the implementation shown, the intensity level of dithered region 308 is slightly lower than the intensity level of dithered region 306, as shown in intensity profile 302. In other implementations, the intensity level of dithered region 308 can be lower than the intensity value of middle region 304 but higher than the intensity level of dithered region 306. Dithered region 308 also appears as a shade of gray when displayed on a television or computer display. In further implementations of the invention, any number of dithered regions with varying intensity levels can be added to high- intensity region 300.
[0031] In the implementation shown in Figure 3A, when a digital image of high- intensity region 300 and low-intensity region 102 is captured, blooming effects will still occur to photosites capturing middle region 304. The photosites capturing dithered region 306 and dithered region 308, however, will be at a lower intensity level (i.e., less than 255) and will therefore have the capacity to absorb a substantial amount of the charge overflowing from photosites capturing middle region 304. This prevents a substantial amount of the charge from overflowing into photosites capturing low-intensity regions 102, thereby minimizing the corruption of the transition between high-intensity region 300 and low-intensity region 102 in the captured digital image. Dithered regions 306 and 308 therefore substantially prevent the blooming effects from expanding beyond the edges of high-intensity region 300. [0032] In a captured digital image of the implementation shown, dithered regions 306 and 308 tend to appear substantially white because their intensity levels will increase to at or around 255 as they absorb the overflowing charge. Low-intensity regions 102 will appear substantially as solid black squares since the blooming effects will be minimized by dithered regions 306 and 308. Thus, in a captured digital image, message carrier 100 will generally appear as a series of substantially solid white blocks and solid black blocks.
[0033] Figure 4 illustrates another implementation of a dithered high-intensity region 400 adjacent to low-intensity region 102. In this implementation, high- intensity region 400 includes a middle region 402 that is at a high-intensity level (i.e., 255 for white). High-intensity region 400 also includes dithered regions to minimize blooming effects and ISI. One dithered region is a circular dithered region 404 that is at a lower intensity level than middle region 402. In this implementation, the shape of the dithered region does not correspond to the shape of high-intensity region 400. The intensity level of circular dithered region 404 can be any value that is less than the intensity level of middle region 402 (e.g., 255) and higher than 0. In one implementation, the intensity level of circular dithered region 404 is slightly less than the intensity level of middle region 402.
[0034] High-intensity region 400 also includes a second dithered region 406 that surrounds circular dithered region 404 and has an even lower intensity level. The intensity level of second dithered region 406 can be any value that is less than the intensity level of middle region 402 and higher than 0. In the implementation shown, the intensity level of second dithered region 406 is slightly lower than the intensity level of circular dithered region 404. In other implementations, the intensity level of second dithered region 406 can be lower than the intensity level of middle region 402 but higher than the intensity level of the circular dithered region 404. As described above, the width and intensity level of circular dithered region 404 and second dithered region 406 can be adjusted based on how strong the blooming effects are for a given combination of display screen type and digital camera. Both circular dithered region 404 and second dithered region 406 appear as shades of gray on a television or computer display screen. In further implementations of the invention, any number of circular dithered regions with varying intensity levels can be added to high-intensity region 400. [0035] Figure 5 illustrates yet another implementation of the invention with a dithered high-intensity region 500 adjacent to low-intensity region 102. In this implementation, high-intensity region 500 has a middle region 502 that is at a high intensity level (i.e., 255 for white) and includes a series of dithered regions 504 that are spaced apart. The spacing allows portions of high-intensity region 500 to appear between dithered regions 504, as shown at areas 500A. Dithered regions 504 can have an intensity level that is lower than the intensity level of middle region 502 (e.g., 255) and higher than 0. In one implementation, the intensity level of dithered regions 504 is slightly less than the intensity level of middle region 502. The use of spaced apart dithered regions 504 in this implementation provides a technique by which the blooming effects can be controlled or mitigated over a larger area. For example, in a particularly large high-intensity region 500, dithered regions 504 can be used to retard the build-up of overflowing charge, thereby preventing the edges of high- intensity region 500 from being overwhelmed. The areas 500A are included to ensure that dithered regions 504 do not cause the intensity level of the outer areas of high- intensity region 500 to fall substantially below the intensity level of middle region 502. In an implementation, dithered regions 504 can be used in conjunction with dithered regions 306 and 308, as shown in Figure 3A, to counteract blooming effects. In another implementation, the intensity levels of the dithered regions 504 are not equal. Again, the width and intensity level of dithered regions 504 can be adjusted based on how strong the blooming effects are for a given combination of display screen type and digital camera.
[0036] Figure 6 illustrates a variation of the implementation shown in Figure 5. Figure 6 illustrates a dithered high-intensity region 600 adjacent to low-intensity region 102. High-intensity region 600 includes a middle region 602 that is at a high intensity level (i.e., 255 for white) and includes a series of spaced apart circular dithered regions 604 surrounding middle region 602. High-intensity region 600 can be seen in between circular dithered regions 604 at areas 606. As in the implementation of Figure 5, circular dithered regions 604 can have an intensity level that is lower than the intensity level of middle region 602 and higher than 0. In one implementation, the intensity level of circular dithered regions 604 is slightly less than the intensity level of middle region 602. As described above, the use of a series of circular dithered regions 604 provides a technique by which the blooming effects can be controlled or mitigated over a larger area. In an implementation, circular dithered regions 604 can be used in conjunction with dithered regions 404 and 406 as shown in Figure 4. In another implementation, the intensity levels of circular dithered regions 604 are not equal. As before, the width and intensity level of circular dithered regions 604 can be adjusted based on how strong the blooming effects are for a given combination of display screen type and digital camera.
[0037] In an alternative implementation of the invention, the dithered regions can be applied only to sides of high-intensity region 104 where a transition occurs between high-intensity region 104 and low-intensity region 102. For example, in Figures 3 to 6, a transition occurs only on one lateral side of each of the high intensity regions. Therefore, in this implementation, there is only one transition that requires a dithered region. This is shown in Figure 3B. In other implementations, the dithered regions can be applied anywhere within the high-intensity regions that they are needed to minimize blooming effects and ISI. In further implementations, dithered regions can be applied anywhere on the television or computer display screen 96 where transitions occur between high-intensity regions and low-intensity region anywhere. This allows users to capture digital images of the actual television images themselves without the images being corrupted due to blooming effects and ISI. The television images can be captured in addition to, or in lieu, of message carriers 100.
[0038] Figure 7 shows a display screen 96 displaying two different message carriers 100. Message carrier IOOA utilizes the dithered high-intensity regions 300 shown in Figure 3A. Message carrier IOOB implements the dithered high-intensity regions 400 shown in Figure 4. Figure 7 illustrates how message carriers IOOA and IOOB appear to a viewer when displayed on a television or computer display screen 96. In a captured digital image, however, both message carriers IOOA and IOOB will more closely resemble message carrier 100 of Figure 1 due to blooming effects from the middle regions of the high-intensity regions being absorbed by the dithered regions.
[0039] In implementations of the invention, message carrier 100 can pair high- intensity regions that are white with low-intensity regions that are black. In other implementations, message carrier 100 can pair high-intensity regions that are white with low-intensity regions that are colors other than black. For example, a color combination of white and green can be used to implement the binary encoding scheme used by message carrier 100. In such an implementation, the intensity level of the green blocks will be 0 red, 0 blue, and 255 green, while the white blocks will still be 255 red, 255 blue, and 255 green. The intensity level of the dithered regions can be manipulated by lowering the intensity levels of only the red and blue components while leaving the green component intact. The dithered regions will therefore appear as shades of green rather then shades of gray. Other colors, including but not limited to black, brown, blue, green, purple, and red, can also be used. Dark colors tend to work better than lighter colors. Similar techniques can be used for the other colors used in message carriers 100, In some implementations of the invention, the dithered regions can be implemented for each and every high-intensity region 104 used in message carrier 100. In other implementations, the dithered regions can be implemented only in areas where a transition occurs between high-intensity region 104 and low-intensity region 102.
[0040] Other implementations of the message carrier are also possible. For example, in one implementation shown in Figure 8, the message carrier can be a vertical strip message carrier 800 that runs along an edge of display screen 96. As Figure 8 illustrates, message carrier 800 can be oriented along the left side of display screen 96; however, message carrier 800 can extend along any edge of screen 96, including the top, right, or bottom edges. Message carrier 800 can include dithered regions 802 on high-intensity regions to counteract any blooming effects and ISI that can occur.
[0041] Although the implementations shown above use block-shaped high-intensity regions and low-intensity regions, further implementations of the message carrier can use regions that are any other shape. For instance, in one implementation, the message carrier can use circular high and low-intensity regions. In another implementation, the message carrier can be formed using an array of high-intensity circles or another shape on a low-intensity background. In yet another implementation, the message carrier can be formed using low-intensity circles or another shape on a high-intensity background.
[0042] In another implementation of the invention, message carriers 100 using dithered high-intensity regions can be used in print media and the like. In particular, print media that uses high-gloss paper or highly reflective paper can benefit from the use of implementations of the invention. The highly reflective surface of glossy paper can reflect too much light back to the photosites of a digital camera, particularly if a flash is used when capturing the digital image. The use of dithered high-intensity regions can counteract the subsequent blooming effects that can occur.
[0043] The message carriers of the invention can be used in electronic commerce systems to transmit data to users. One such electronic commerce system is described in U.S. Patent Application No.10/618,445 entitled "Interactive Electronic Commerce and Message Interchange System," which is incorporated by reference. Figure 9 illustrates an electronic commerce system 10 utilizing a data encoding and decoding scheme according to one implementation of the invention. As Figure 9 illustrates, electronic commerce system 10 includes one or more universal digital assistants (UDA) 90 that can receive data from a media source, for example, a display device 95 having display screen 96. In implementations of the invention, the display device 95 can be a television or a computer monitor. Electronic commerce system 10 also includes a UDA application site 50 that interacts with UDA 90 to facilitate financial transactions and carry out other processes as described herein. UDA 90 interacts with UDA application site 50 through a telecommunications network 20 and a computer network 40. Electronic commerce system 10 can also include a merchant system 60, an advertising system 62, at least one network access device such as client computer 64, at least one transaction processing network 70 (e.g., an Automated Clearing House (ACH) network, credit card network, etc.), and one or more financial institutions 72.
[0044] Telecommunications network 20 can be a wireless telephone network (as shown in Figure 9), a land-based telephone network, and/or a combination thereof. Telecommunications network 20 can be any suitable digital or analog wireless network, such as a Time Division Multiple Access (TDMA) network, a Global System for Mobile communication (GSM) network, or a Code-Division Multiple Access (CDMA) network.
[0045] Telecommunications network 20 enables transmission of voice and other data to mobile wireless devices, such as UDA 90. Telecommunications network 20 therefore enables communication between UDA 90 and other systems operably connected thereto. In one implementation, telecommunications network 20 includes a Wireless Application Protocol (WAP) gateway 22, which provides a set of communication protocols enabling wireless devices, such as UDA 90, to access the Internet or similar computer network 40. WAP gateway 22 is operative to establish a connection (e.g., a Wireless Session Protocol (WSP) connection) with UDA 90, to receive requests designating an application server or other resource on computer network 40 from UDA 90, to translate the request into a hypertext transfer protocol (HTTP) or other suitable request to the appropriate application server, to receive a response from the application server, and to translate and transmit the response to UDA 90. Alternatively, the transmission of data between UDA 90 and UDA application site 50 can occur over a Short Message Service (SMS) gateway, In one implementation, the client functionality of UDA 90 is operative to configure SMS messages into a predefined format recognized by UDA appli cation site 50.
[0046] Computer network 40 can be any suitable computer network, including but not limited to an Internet Protocol (IP) network, such as the Internet. Computer network 40 can be an open, wide-area network (WAN), an electronic network, an optical network, a wireless network, and/or any combination thereof. Computer network 40 can be a packet-based communications environment, employing Transmission Control Protocol/Internet Protocol (TCP/IP) standards and having a plurality of interconnected digital packet transmission stations operative to route data between TCP/IP end systems. The invention, however, has applications in computer network environments employing any suitable transport layer and network layer protocols.
[0047] UDA 90, in one implementation, is a wireless device that provides computing and information storage and retrieval capabilities for applications installed therein. UDA 90 is operative to communicate with remote devices over telecommunications network 20. In one implementation, UDA 90 includes hardware and/or software operative to acquire message codes using one or more code capture technologies and then communicate such codes to UDA application site 50. UDA application site 50 is operative to return at least one promotional message to UDA 90 for display or other action to the user. The functionality associated with UDA 90 can be integrated into a variety of wireless device platforms, such as cell phones, wireless personal digital assistants (PDA), universal remote controllers, television remote controllers, pagers, wearable computing devices, and the like. In one implementation, the functionality of UDA 90 is incorporated into a television or set-top television receiver remote control unit. [0048] UDA 90 can further include short-range wireless functionality for wireless communication of data with a second device, such as a point-of-sale (POS) terminal or any other suitably configured device. This functionality can be implemented using, for example, infrared signals (e.g., IrDA), radio signals (e.g., 802.11 technologies (Wi- Fi) or Bluetooth), short burst Ethernet, wireless network signals (e.g., GSM, TDMA, or CDMA), and message services such as SMS and Multimedia Message Service (MMS). Such functionality allows UDA 90, for example, to transmit credit card account information and other information directly to a POS terminal equipped with complementary wireless communication functionality. :
[0049] UDA 90, in one implementation, is also operative to receive data from telecommunications network 20 (or computer network 40) and transmit data to telecommunications network 20 (or computer network 40) for routing to appropriate devices. UDA 90, in one implementation, is an Internet-enabled device capable of receiving data from remote servers and displaying data on a user interface screen. In one implementation, UDA 90 is a WAP-enabled device, such as a WAP mobile phone, that includes a WAP client (e.g., a Wireless Application Environment (WAE) user agent, such as a WAP browser, and a Wireless Telephony Application (WTA) user agent). In another implementation, UDA 90 can be a wireless PDA including HTML compliant or HTML-supported browser functionality, such as Pocket PC including Pocket Internet Explorer® (PIE), which is a mobile version of Microsoft's Internet Explorer®, including limited Javascript support and the ability to display HTML and Macromedia Flash files (assuming the Flash plug-in is installed).
[0050] Figure 10 is a block diagram illustrating the functionality of UDA 90 according to one implementation of the invention. As Figure 10 illustrates, UDA 90 includes an image capture device 1002, a display device 1004, an antenna 1006, a client application control logic 1008, and a wireless communication interface 1010.
[0051] Image capture device 1002 is operative to capture digital images of message carriers 100 with dithered high-intensity regions and message codes transmitted in visual media. In one implementation, image capture device 1002 can comprise a digital camera allowing for the capture of images containing message codes. The digital camera can include an image sensor (not shown), such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor, to capture a digital image. The image sensor is generally composed of thousands of photosites that receive light passing through the lens of the digital camera. This light is converted into electrical charges by the photosites and, as explained above, there is a limit to how much charge each photosite can store. Having dithered regions on the high-intensity regions of message carrier 100 therefore minimizes the drawbacks of the photosites. Display device 1004 is a liquid-crystal or other display device. In one implementation, display device is a touch-sensitive liquid crystal display device.
[0052] Client application control logic 1008 executes the client-side functionality described herein, such as processing captured digital images of message carrier 100 to acquire message codes and transmitting the message codes through wireless communication interface 1010 to UDA application site 50. Because message carriers 100 use dithered high-intensity regions, blooming effects and ISI should not corrupt low-intensity regions 102 of the digital image that is processed. The image processing functionality of client application control logic 1008 can process a digital image by reading intensity values of the pixels that form the digital image, and then analyzing those intensity values to identify patterns or other behavior. Generally, the pixels are read by scanning vertical columns of pixels in the digital image, known as vertical scan lines. The pixel values can also be read by scanning horizontal rows of pixels, known as horizontal scan lines. Wireless communication interface 1010 is operative to establish and maintain a connection to telecommunications network 20 using antenna 1006 to enable communication of data between UDA 90 and remote devices.
[0053] As one skilled in the art will recognize, the functionality described above can be integrated into existing wireless device platforms. For example, client application control logic 1008 can be implemented into a mobile telephone with an embedded runtime environment such as Java. The runtime environment can execute a script or byte code program to perform the functions described herein and use the wireless communication system of the phone to transmit and receive data from remote devices. Furthermore, image capture device 1002 can also be integrated into a mobile telephone. There are already numerous mobile telephones on the market that include image capture devices 1002. Some examples include, but are not limited to, the Sony Ericsson® T610, the Samsung® A600, the Sanyo® VM4500, the Motorola® V300, and the Nokia® 3650. [0054] UDA application site 50 is connected to computer network 40 to communicate with end systems and other nodes operably connected thereto, such as client computer 64. UDA application site 50 interacts with UDA 90 through computer network 40, WAP gateway 22, and wireless telecommunications network 20 to facilitate data interchanges, financial transactions, and other operations as described herein. UDA application site 50 can be configured to support a variety of wireless devices, including IMode phones, and mobile wireless devices including Binary Runtime Environment for Wireless (BREW) or Java 2 Platform Micro Edition (J2ME) technologies. In one implementation, UDA application site 50 includes at least one application server 52, a user account database 54, and a promotional message database 56.
[0055] Application server 52 includes functionality implementing the process flows described herein. User account database 54 stores information related to at least one user account. Such user account information includes a user account identification, name, address, e-mail address, as well as system settings, preferences, user profile information (e.g., psychographic information, demographic information, and behavioral information) obtained by monitoring a user's use of the system or by direct interaction with the user through questionnaires and the like. User account database 54 can also store financial account information for at least one user to facilitate financial transactions using UDA 90.
[0056] Promotional message database 56 stores promotional messages, such as advertisements, coupons, and other commercial offers, in association with one or more message codes. In one implementation, advertising system 62 transmits promotional messages in electronic form to promotional message database 56 for storage in association with message codes. In another implementation, application server 52 can receive a message code from UDA 90 and return one or more promotional messages associated with that message code, as discussed herein.
[0057] UDA 90 operates in connection with UDA application site 50 to enable users to receive and review advertising, promotional, informational and other messages, as well as conduct financial transactions and other operations. According to an implementation of the invention, UDA 90 acquires message codes (e.g., from television advertisements or print advertisements) and transmits them to UDA application site 50, which pushes an advertising or promotional message back to UDA 90 in return. Advertising and promotional messages can take a variety of forms, including but not limited to electronic coupons, product pricing, descriptive data, loyalty offerings, and other similar items. According to one implementation, a user can elect to save the advertising or promotional message for later use or inspection by storing it locally on UDA 90 or remotely on UDA application site 50 in association with his or her user account.
[0058] To facilitate initiation and maintenance of user accounts, users may access UDA application site 50 through network access devices over computer network 40 to manage their respective user accounts through network access devices connected to computer network 40. In one implementation, the network access device is a browser executed on a client computer 64, or another browser executed on a network computer (not shown). In one implementation, page-based interfaces for accessing UDA application site 50 are transmitted to client computer 64 having a browser and a connection to computer network 40.
[0059] Client computer 64 can include any computer, special-purpose computing device, or any other suitable device having the required functionality. In one implementation, client computer 64 includes at least one processor, a data storage system (including volatile and non-volatile memory), at least one input device (e.g., a keyboard), and at least one output device (e.g., a display). In one implementation, client computer 64 is connected to computer network 40 through a modem connection (e.g., dial-up, DSL, or cable) or through a network line. Such connections can be wireless as well. In addition, although implementations of the system are described as working in conjunction with a browser, any suitable device, application, or client software for receiving, displaying, and transmitting data over a computer network can be used. In one implementation, the browser implemented on client computer 64 supports the SSL ("Secure Sockets Layer") protocol, the S-HTTP or HTTPS ("Secure HTTP") protocol, or any other similar protocol for transmitting confidential or private information over an open computer network. Users are individuals or other legal entities having the capacity to possess financial accounts, such as corporations, partnerships, non-profit organizations, trusts, and the like.
[0060] Client computer 64 is a TCP/IP end system operably connected to computer network 40 via any suitable means, such as through an Internet Services Provider (ISP) and the like. Client computer 64 can be any suitable internet-enabled computing device, including but not limited to a desktop computer, a laptop computer, a mobile telephone, a personal digital assistant (PDA), or UDA 90 having wireless or wired access to computer network 40. The wireless or wired access can be through a router, a wireless router executing the 802.11 wireless protocol, Mobitex, DataTAC, General Packet Radio Service (GPRS), or any other packet- switched wireless network. In one implementation, client computer 64 includes internet browsing software for receiving, displaying, and transmitting data over a computer network.
[0061] Payment transaction processing network 70 supports a variety of non-cash payment mechanisms, including but not limited to purchases made by credit card, debit card, and Automated Clearing House (ACH) transactions. In one implementation, payment transaction processing network 70 can be a credit card or debit card transaction processing network, such as Visa®, MasterCard®, Discover®, or American Express®. In one implementation, payment transaction processing network 70 enables users, using UDA 90, to provide non-cash methods of payment to a merchant system 60 or at a POS terminal.
[0062] Financial institution 72 is where users maintain financial accounts, such as checking accounts, savings accounts, money market accounts, credit and/or debit card accounts, and other financial accounts. A user can have one or more bank accounts at each of one or more financial institutions. For example, a user can have a checking account and a credit and debit card account at one financial institution and still have a credit account with a different financial institution.
[0063] As discussed above, UDA 90 is operative to detect and acquire message codes in message carriers 100 using one or more code capture technologies. For example, UDA 90 can be configured to acquire message codes from broadcast media, print media, or wireless transmissions. In one implementation, UDA 90 is operative to acquire message codes in message carrier 100 which is embedded or otherwise transmitted in connection with a television broadcast signal and displayed on a display device 95, such as a television or computer display. For example, UDA 90 can include a code capture technology whereby a user can aim UDA 90 at display device 95 and capture a digital image of message carrier 100 being shown on display screen 96. The captured digital image or a portion thereof is then processed to locate message carrier 100 and identify a message code that is then acquired. For example, UDA 90 can use image capture device 1002 to capture a digital image and then use the image processing functionality of client application control logic 1008 to locate and acquire a message code in the captured image. In one implementation of the invention, UDA 90 can emit a beam of light (e.g., a Class II laser pointer) that can assist a user to aim the image capture device of UDA 90 at screen 96. This is useful in situations where the user cannot see what the image capture device is pointed at. In another implementation, the view through the image capture device can be shown to the user on display screen 1004 of UDA 90, thereby eliminating the need for a laser pointer.
[0064] After a digital image of message carrier 100 is captured, UDA 90 can process the digital image to retrieve a message code contained in message carrier 100. The message code can contain any type of data and can identify any of a number of items. For example, in one implementation the message code can identify the television station to which display device 95 is tuned (e.g., station call letters or a station identifier). In such an implementation, an advertisement or promotional message can then be identified using the station identifier and the time the station identifier was captured. In one implementation, UDA application site 50 can scan this information against a station log to resolve the advertising or other content the user was watching. UDA application site 50 can then return an advertising or promotional message associated with the broadcast advertising or other content.
[0065] In another implementation, the image processing functionality can reside on a remote server and the captured image can be transmitted to this remote server for processing. Such an implementation moves the burden of processing the digital image to a remote server that will typically have much greater processing power than UDA 90. The time required to process the captured image will then be reduced. UDA 90 can transmit the captured image to the remote server using any of the wireless technologies described herein, for instance, the captured image can be transmitted in an MMS message.
[0066] One or more features of message carrier 100 can be preset or constant to allow client application control logic 1008 of UDA 90 to locate message carrier 100 in a captured image and then locate the message code within the message carrier. When the image processing functionality of client application control logic 1008 reads and analyzes the intensity values of pixels in the captured image (e.g., using vertical scan lines or horizontal scan lines), these preset or constant features can be detected and message carrier 100 can then be identified. From there, the message code can then be identified and acquired.
[0067] In one implementation, message carrier 100 can have one or more borders 106 (shown in Figure 1) that remain constant as the message code changes. For example, the one or more borders 106 can be one continuous color such as black. This can aid the image processing functionality of client application control logic 1008 in locating message carrier 100. In one implementation, a two-sided border 106 can be used around message carrier 100 to aid in its identification, where two-sided border 106 is one continuous color. As shown in Figures 1 and 7, each message carrier 100 includes top and left border regions 106 that are colored black or any other suitable color. Accordingly, the checkerboard region is bordered on two sides by a black color, and by the television image on the third and fourth sides. The image processing functionality of client application control logic 1008 can use two-sided border 106 to locate message carrier 100 and can then filter out two-side border 106 to extract the message code itself.
[0068] In one implementation, UDA 90 executes a process to decode the message code that includes capturing an image of the display screen including message carrier 100, analyzing vertical scan lines through the captured image to identify scan lines that intersect message carrier 100, locating and aligning an edge of message carrier 100, optionally validating one or more features surrounding the transitions, and reading the data bits in the message code.
[0069] As discussed above, UDA 90 includes image capture device 1002 to capture a digital image of display screen 96 containing message carrier 100. The user aims image capture device 1002 at a television that is displaying message carrier 100, then the user triggers the capture functionality to capture a digital image. The image can be pre-processed to indicate which pixels are white, which pixels are black, and which pixels are neither (i.e., a middle tone), thus reducing each pixel to a two-bit value (e.g., 11 for a white pixel, 10 for a black pixel, and 00 for a middle tone pixel). The middle tone should generally represent much of the television image and any surrounding items in the camera image. In one implementation, the image processing functionality located message carrier 100 by analyzing vertical scan lines to find solid border 106. In this implementation, the image processing functionality is looking for a large range of pixels that are all the same color.
[0070] In an implementation, border 106 can be made wider or taller than the blocks that form the message code. Using an over-sized border improves the likelihood of a vertical scan line passing through border 106 in its entirety when the image processing functionality is trying to locate message carrier 100. Since the captured image may not be properly aligned, depending on how the user was holding the image capture device 1002 when the image was captured, the vertical scan lines may be at an angle relative to message carrier 100 and border 106. The use of an over-sized border 106 increases the range of allowable angles for the vertical scan lines relative to border 106 where a vertical scan line can still pass through an entire border 106.
[0071] After a message code is acquired, UDA 90 can transmit the message code to UDA application site 50 which resolves the message code and pushes an electronic commerce and/or data message back to UDA 90. In one implementation, application server 52 receives a request including the message code and accesses promotional message database 56 to retrieve a promotional message associated with the message code. If a promotional message is located, application server 52 pushes the promotional message to UDA 90. Application server 52 can also carry out other functions such as tracking the user's interests and behavior. For example, application server 52 can generate a log, stored in association with the user's account, which records the message codes received from the user and the promotional messages sent in response. In one implementation, application server 52 can use the log to track system usage by the user to enable a fee-based service for advertisers. For example, an advertiser can be charged a fee for each promotional message pushed to a user's UDA 90. In addition, UDA application site 50, in one implementation, can be configured to transmit usage or activity reports to advertising system 62 detailing the activity associated with promotional messages and/or user purchasing behavior.
[0072] Lastly, the invention has been described with reference to specific implementations. Other implementations of the invention will be apparent to those of ordinary skill in the art. It is, therefore, intended that the scope of the invention not be limited to the implementations described above.

Claims

1. A method of displaying a message carrier containing a message code on a display screen comprising: transmitting a first portion of a signal that defines an image of a high-intensity region, wherein the high-intensity region represents a first data bit value; transmitting a second portion of the signal that defines an image of a dithered region, wherein the dithered region is located adjacent to the high-intensity region; and transmitting a third portion of the signal that defines an image of a low- intensity region, wherein the low-intensity region represents a second data bit value and is located adjacent to the dithered region.
2. The method of claim 1, wherein the display screen comprises a television screen or a computer monitor screen.
3. The method of claim 1, wherein the high-intensity region comprises a white block.
4. The method of claim 1, wherein the low-intensity region comprises a block having a color chosen from the group consisting of black, brown, blue, green, purple, and red.
5. The method of claim 1, wherein the high-intensity region has a first intensity level.
6. The method of claim 5, wherein the first intensity level is around 255.
7. The method of claim 5, wherein the dithered region comprises at least one border having a second intensity level that is lower than the first intensity level.
8. The method of claim 7, wherein the dithered region comprises a plurality of borders, each border having an intensity level that is lower than the first intensity level.
9. The method of claim 8, wherein each of the plurality of borders has a different intensity level.
10. The method of claim 9, wherein the plurality of borders are arranged in order based on their intensity levels such that the border with the highest intensity level is adjacent to the high-intensity region and the border with the lowest intensity level is adjacent to the low-intensity region.
11. The method of claim 8, wherein the plurality of borders are spaced apart such that the high-intensity region is visible between each border.
12. The method of claim 7, wherein the shape of the border is tetragonal.
13. The method of claim 7, wherein the shape of the border is circular.
14. The method of claim 7, wherein the shape of the border matches the shape of at least a portion of the high-intensity region.
15. The method of claim 1, wherein the dithered region completely surrounds the high-intensity region.
16. The method of claim 1, wherein the message code is communicated using the first data bit value and the second data bit value.
17. A method of transmitting a message carrier containing a message code to a user comprising: displaying a high-intensity region on a display screen, wherein the high- intensity region represents a first data bit value; displaying a low-intensity region on the display screen, wherein the low- intensity region represents a second data bit value and is located adjacent to the high- intensity region along an edge; and displaying a dithered region on the display screen, wherein the dithered region is located on the high-intensity region at or near the edge.
18. The method of claim 17, wherein the display screen comprises a television screen or a computer monitor screen.
19. The method of claim 17, wherein the high-intensity region comprises a white block and the low-intensity region comprises a block having a color chosen from the group consisting of black, brown, blue, green, purple, and red.
20. The method of claim 17, wherein the high-intensity region has a first intensity level.
21. The method of claim 20, wherein the dithered region comprises at least one border having a second intensity level that is lower than the first intensity level.
22. The method of claim 21, wherein the dithered region comprises a plurality of borders, each border having an intensity level that is lower than the first intensity level.
23. The method of claim 22, wherein each of the plurality of borders has a different intensity level.
24. The method of claim 23, wherein the plurality of borders are arranged in order based on their intensity levels such that the border with the highest intensity level is adjacent to the high-intensity region and the border with the lowest intensity level is adjacent to the low-intensity region.
25. The method of claim 22, wherein the plurality of borders are spaced apart such that the high-intensity region is visible between each border.
26. The method of claim 21, wherein the shape of the border is tetragonal.
27. The method of claim 21, wherein the shape of the border is circular.
28. The method of claim 21, wherein the shape of the border matches the shape of at least a portion of the high-intensity region.
29. The method of claim 17, wherein the message code is communicated using the first data bit value and the second data bit value.
30. A method of minimizing blooming effects in a digital image of a display screen comprising adding one or more dithered regions to areas on the display screen where a high-intensity region is adjacent to a low-intensity region.
31. The method of claim 30, wherein the display screen comprises a television screen or a computer monitor screen.
32. The method of claim 30, wherein the high-intensity region comprises a white region.
33. The method of claim 30, wherein the low-intensity region comprises a region having a color chosen from the group consisting of black, brown, blue, green, purple, and red.
34. The method of claim 30, wherein the high-intensity region has a first intensity level.
35. The method of claim 34, wherein each dithered region comprises at least one border having a second intensity level that is lower than the first intensity level.
36. The method of claim 35, wherein each dithered region comprises a ' plurality of borders, each border having an intensity level that is lower than the first intensity level.
37. The method of claim 36, wherein each of the plurality of borders has a different intensity level.
38. The method of claim 37, wherein the plurality of borders are arranged in order based on their intensity levels such that the border with the highest intensity level is adjacent to the high-intensity region and the border with the lowest intensity level is adjacent to the low-intensity region.
39. The method of claim 36, wherein the plurality of borders are spaced apart such that the high-intensity region is visible between each border.
40. The method of claim 35, wherein the shape of the border matches the shape of at least a portion of the high-intensity region.
41. The method of claim 30, wherein the dithered region completely surrounds the high-intensity region.
42. A message code comprising: a high-intensity region; a dithered region, wherein the dithered region is located adjacent to the high- intensity region; and a loW'intensity region, wherein the low-intensity region is located adjacent to the dithered region.
43. The message code of claim 42, wherein: the high-intensity region comprises a white block that represents a first data bit value, the low-intensity region comprises a colored block that represents a second data bit value, and the color is chosen from the group consisting of black, brown, blue, green, purple, and red.
44. The message code of claim 42, wherein the high-intensity region has a first intensity level.
45. The message code of claim 44, wherein the dithered region comprises at least one border having a second intensity level that is lower than the first intensity level.
46. The message code of claim 45, wherein the dithered region comprises a plurality of borders, each border having an intensity level that is lower than the first intensity level.
47. The message code of claim 46, wherein each of the plurality of borders has a different intensity level.
48. The message code of claim 47, wherein the plurality of borders are arranged in order based on their intensity levels such that the border with the highest intensity level is adjacent to the high-intensity region and the border with the lowest intensity level is adjacent to the low-intensity region.
49. The message code of claim 46, wherein the plurality of borders are spaced apart such that the high-intensity region is visible between each border.
50. The message code of claim 45, wherein the shape of the border is tetragonal.
51. The message code of claim 45, wherein the shape of the border is circular.
52. The message code of claim 45, wherein the shape of the border matches the shape of at least a portion of the high-intensity region.
53. A method for pushing promotional messages to users comprising: displaying a message carrier on a display screen, wherein the message1 carrier comprises one or more dithered white blocks and one or more black blocks; receiving a message code from a user, wherein the message code was acquired by the user from the message carrier; and pushing a promotional message to the user based on the received message code.
54. The method of claim 53, wherein the message carrier further comprises: an identifying border; and the message code.
55. The method of claim 54, wherein the message code is formed using the one or more dithered white blocks and the one or more black blocks.
56. The method of claim 53, wherein the message code is received over a wireless network.
57. The method of claim 56, wherein the wireless network comprises a GSM network, a CDMA network, or a TDMA network.
58. The method of claim 53, wherein the promotional message is pushed over a wireless network.
59. The method of claim 58, wherein the wireless network comprises a GSM network, a CDMA network, or a TDMA network.
60. A method of transmitting a message carrier to a user comprising: printing a high-intensity region on a print media; printing a low-intensity region on the print media, wherein the low-intensity region is located adjacent to the high-intensity region along an edge; and printing a dithered region on the print media, wherein the dithered region is located on the high-intensity region at or near the edge.
61. The method of claim 60, wherein the print media comprises a high- gloss paper or a highly reflective paper.
62. The method of claim 60, wherein the high-intensity region comprises a white block and the low-intensity region comprises a block having a color chosen from the group consisting of black, brown, blue, green, purple, and red.
63. The method of claim 60, wherein the high-intensity region has a first intensity level.
64. The method of claim 63, wherein the dithered region comprises at least one border having a second intensity level that is lower than the first intensity level.
65. The method of claim 64, wherein the dithered region comprises a plurality of borders, each border having an intensity level that is lower than the first intensity level.
66. The method of claim 65, wherein each of the plurality of borders has a different intensity level.
67. The method of claim 66, wherein the plurality of borders are arranged in order based on their intensity levels such that the border with the highest intensity level is adjacent to the high-intensity region and the border with the lowest intensity level is adjacent to the low-intensity region.
68. The method of claim 65, wherein the plurality of borders are spaced apart such that the high-intensity region is visible between each border.
69. The method of claim 64, wherein the shape of the border matches the shape of at least a portion of the high-intensity region.
70. A broadcast signal comprising: a first portion that defines an image of a high-intensity region; a second portion that defines an image of a dithered region, wherein the dithered region is located adjacent to the high-intensity region; and a third portion that defines an image of a low-intensity region, wherein the low-intensity region is located adjacent to the dithered region.
71. The broadcast signal of claim 70, wherein the high-intensity region represents a first data bit value.
72. The broadcast signal of claim 70, wherein the low-intensity region represents a second data bit value.
PCT/US2005/022617 2004-07-01 2005-06-23 Application for united states letters patent WO2006007485A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/882,533 2004-07-01
US10/882,533 US7672532B2 (en) 2004-07-01 2004-07-01 Dithered encoding and decoding information transference system and method

Publications (2)

Publication Number Publication Date
WO2006007485A2 true WO2006007485A2 (en) 2006-01-19
WO2006007485A3 WO2006007485A3 (en) 2006-06-22

Family

ID=35599499

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2005/022617 WO2006007485A2 (en) 2004-07-01 2005-06-23 Application for united states letters patent

Country Status (2)

Country Link
US (1) US7672532B2 (en)
WO (1) WO2006007485A2 (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4293111B2 (en) * 2004-10-27 2009-07-08 株式会社デンソー Camera driving device, camera driving program, geometric shape code decoding device, and geometric shape code decoding program
US7908397B1 (en) * 2005-02-28 2011-03-15 Adobe Systems Incorporated Application server gateway technology
US7660558B2 (en) 2005-12-31 2010-02-09 Adobe Systems Incorporated Interrupting and resuming a media player
US7603113B2 (en) 2005-12-31 2009-10-13 Adobe Systems Incorporated Using local codecs
FR2906667B1 (en) * 2006-10-02 2009-01-16 Eastman Kodak Co PRODUCTION OF VISUAL CODES FOR THE MATCHING OF ELECTRONIC EQUIPMENT.
US20130167024A1 (en) 2006-12-05 2013-06-27 Adobe Systems Incorporated Embedded document within an application
US7743339B1 (en) 2007-02-01 2010-06-22 Adobe Systems Incorporated Rendering text in a brew device
US20080192006A1 (en) * 2007-02-08 2008-08-14 Silverbrook Research Pty Ltd System for enabling user input and cursor control
US8589779B2 (en) * 2007-03-08 2013-11-19 Adobe Systems Incorporated Event-sensitive content for mobile devices
US8070071B2 (en) * 2007-09-21 2011-12-06 Silverbrook Research Pty Ltd Method of imaging a coding pattern with data elements encoding by multi-pulse position modulation
US8970707B2 (en) * 2008-12-17 2015-03-03 Sony Computer Entertainment Inc. Compensating for blooming of a shape in an image
US9437046B2 (en) 2012-06-06 2016-09-06 Sodyo Ltd. Anchors for location-based navigation and augmented reality applications
US10147214B2 (en) * 2012-06-06 2018-12-04 Sodyo Ltd. Display synchronization using colored anchors
WO2015083055A1 (en) * 2013-12-02 2015-06-11 Sodyo Ltd. Display synchronization using colored anchors
US10635958B2 (en) 2015-01-28 2020-04-28 Sodyo Ltd. Hybrid visual tagging using customized colored tiles
CN112424794A (en) 2018-07-11 2021-02-26 索迪欧有限公司 Detecting high resolution machine-readable labels using mosaic image sensors

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5799111A (en) * 1991-06-14 1998-08-25 D.V.P. Technologies, Ltd. Apparatus and methods for smoothing images
US6028677A (en) * 1997-09-16 2000-02-22 Hewlett-Packard Co. Method and apparatus for converting a gray level pixel image to a binary level pixel image
US6292268B1 (en) * 1997-09-04 2001-09-18 Minolta Co., Ltd. Image processor and image processing method handling multilevel image data
US6499662B1 (en) * 1998-09-14 2002-12-31 Psc Scanning, Inc. Fast edge detection system tolerant of high degree of intersymbol interference
US6608702B1 (en) * 1998-07-13 2003-08-19 Oki Data Corporation Method for generating threshold patterns and gradation image binarizing device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030021437A1 (en) * 2001-07-11 2003-01-30 Hersch Roger David Images and security documents protected by micro-structures

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5799111A (en) * 1991-06-14 1998-08-25 D.V.P. Technologies, Ltd. Apparatus and methods for smoothing images
US6292268B1 (en) * 1997-09-04 2001-09-18 Minolta Co., Ltd. Image processor and image processing method handling multilevel image data
US6028677A (en) * 1997-09-16 2000-02-22 Hewlett-Packard Co. Method and apparatus for converting a gray level pixel image to a binary level pixel image
US6608702B1 (en) * 1998-07-13 2003-08-19 Oki Data Corporation Method for generating threshold patterns and gradation image binarizing device
US6499662B1 (en) * 1998-09-14 2002-12-31 Psc Scanning, Inc. Fast edge detection system tolerant of high degree of intersymbol interference

Also Published As

Publication number Publication date
US7672532B2 (en) 2010-03-02
US20060013502A1 (en) 2006-01-19
WO2006007485A3 (en) 2006-06-22

Similar Documents

Publication Publication Date Title
WO2006007485A2 (en) Application for united states letters patent
US7319862B1 (en) Block-based encoding and decoding information transference system and method
US20040117254A1 (en) Interactive electronic commerce and message interchange system
US20040117255A1 (en) Interactive electronic commerce and message interchange system featuring delivery of messages tailored to individual users
US9747608B2 (en) Third-party provider method and system
US7575171B2 (en) System and method for reliable content access using a cellular/wireless device with imaging capabilities
EP2179368B1 (en) Method and apparatus for internet monitoring by third parties using monitoring implements
US20130026241A1 (en) Device and its use for creation, output and management of 2d barcodes with embedded images
US20040169892A1 (en) Device and method for generating a print, device and method for detecting information, and program for causing a computer to execute the information detecting method
KR100830369B1 (en) Method for Providng Advertisement and Program Recording Medium
US11430011B2 (en) Methods and computer-readable media for carrying out promotional campaigns
WO2009083554A1 (en) Method of decoding, corresponding terminal and computer program, method of translation, corresponding server and computer program
CN107203731B (en) Method and device for generating label set and method and device for scanning label set
EP3202108A1 (en) Method and device for adapting the display of a video stream by a client
CN107070749B (en) Information pushing method and system based on graphic code
US11361183B2 (en) Architecture for faster decoding in a barcode reading system that includes a slow interface between the camera and decoder
CN107786828B (en) Implicit information transmission method, device and system
EP1854050A1 (en) A method for capturing an image using an apparatus and obtaining information regarding the image, the image containing a tag
US20160180390A1 (en) Advertisement-Based Internet Service Method and Controller for Implementing the Same
CN103092611B (en) The interactive system of a kind of moving advertising API and method
WO2004008281A2 (en) Interactive electronic commerce and message interchange system featuring delivery of messages tailored to individual users
KR101280278B1 (en) Difference cost calculation system of advertisement by using image code and method thereof
CN109903139A (en) A kind of e-commerce platform merchandise news acquisition methods according to timing node
US20240022752A1 (en) System and method for managing encoded information in a real-time screen-to-camera communication environment
KR101119644B1 (en) Wired and wireless interworking service providing system and method thereof based on rfid

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 69(1) EPC. EPO FORM 1205A DATED 30-07-07.

122 Ep: pct application non-entry in european phase

Ref document number: 05763211

Country of ref document: EP

Kind code of ref document: A2