US20030212739A1

US20030212739A1 - Store and forward architecture

Info

Publication number: US20030212739A1
Application number: US10/142,553
Authority: US
Inventors: Antoine Boucher; Peter Coumans; Peter Scheyen; Brent Elphick; Vasanthan Gunaratnam; Allan Lodberg
Original assignee: Individual
Current assignee: TVWorks LLC
Priority date: 2002-05-09
Filing date: 2002-05-09
Publication date: 2003-11-13
Also published as: US7577703B2; US20060101153A1

Abstract

A store and forward (S&F) architecture is provided that supports multiple applications within an extensible network to direct information of various formats to any of multiple destinations. In the presently preferred embodiment of the invention, Java applications running on a client send non-priority data to any server on the application network or, alternatively, anywhere on a global telecommunications network such as the Internet, at some time in the future. In the preferred embodiment, S&F allows a client application to send usage statistics to a database on the application network. It also enables T-commerce purchases to be made by the user, where the purchase and other relevant information is sent to a destination web server as if the purchase had taken place on the web via a full web browser on an Internet-connected PC.

Description

BACKGROUND OF THE INVENTION

1. Technical Field

The invention relates to communications systems. More particularly, the invention relates to a store and forward architecture.

2. Description of the Prior Art

The term “store and forward” pertains to communications systems in which messages are received at intermediate routing points and recorded i.e. stored, and then transmitted, i.e. forwarded, to the next routing point or to the ultimate recipient. Such systems are commonly used in the cable and satellite television industry. For example, DirecTV operates a satellite television system that offers such services as Pay-Per-View. In Pay-Per-View mode, a subscriber to DirecTV selects a program to be purchased and viewed. The subscriber typically has a set top box, such as a satellite receiver in the case of DirecTV, that contains information about the subscriber's privileges and, if the subscriber is authorized to purchase Pay-Per-View broadcasts, the set top box decodes the program selected by the subscriber for viewing when it is broadcast, e.g. via the DirecTV satellite. The set top box captures information with regard to the purchase, i.e. it stores purchase information for billing purposes. At an appropriate time, for example during a regularly scheduled upstream communication from the set top box to DirecTV, e.g. via a telephone call, the set top box sends this purchase information to the Pay-Per-View service, i.e. it forwards this stored information to DirecTV to allow the subscriber's account to be billed for their purchase of the program which they selected and viewed.

While such systems provide an effective approach to a dedicated application, such as Pay-Per-View services, they are not useful or readily configurable for execution of multiple applications, e.g. Pay-Per-View and messaging and shopping. Further, such known systems operate within the confines of a well definedwell-defined network architecture, e.g. the Pay-Per-View server and associated network elements. Thus, they are not easily reconfigured to provide disparate services over an extended or extensible network, e.g. they cannot discern whether one destination within the network is more appropriate than another because they are set to communicate with a specified destination for a dedicated purpose. They are therefore not agile at routing information among multiple destinations. Finally, such known systems operate within specified parameters that treat all communications in a similar fashion because all communications relate to the same thing, e.g. Pay-Per-View. Thus, there is no notion of scheduling events, such as upward communication, based upon the nature of information to be communicated, nor are different information formats or protocols handled well within such known systems.

It would be advantageous to provide a store and forward system that supported multiple applications within an extensible network to direct information of various formats to any of multiple destinations.

SUMMARY OF THE INVENTION

The invention provides a store and forward (S&F) architecture that supports multiple applications within an extensible network to direct information of various formats to any of multiple destinations. In the presently preferred embodiment of the invention, Java applications running on a client send non-priority data to any server on the application network or, alternatively, anywhere on a global telecommunications network such as the Internet, at some time in the future.

In the preferred embodiment, S&F allows a client application to send usage statistics to a database on the application network. It also enables Television-based Commerce (T-commerce) purchases of products via interaction with the T.V. to be made by the user, where the purchase and other relevant information is sent to a destination web server as if the purchase had taken place on the web via a full web browser on an Internet-connected PC.

The preferred S&F mechanism consists of five distinct parts:

Client-side code in the micro Java Virtual Machine, the Java bytecode execution engine (uVM); and

A server-side proxy server process

A communications network for transporting information from the client to the S&F proxy (A real-time two-way RF network in the preferred embodiment); and

A server-side Application Server process; and

A communications network for transporting information from the S&F proxy to the Application Server process (An IP network in the preferred embodiment)

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block schematic diagram showing logical architecture and data flow according to the invention; [0015]
FIGS. 2[0016] a-2 c are flow diagrams that show a communication session according to the invention;
FIGS. 3[0017] a-3 f are flow diagrams show a scenario in which a Server cannot be contacted within a prescribed (short) period of time according to the invention;
FIGS. 4[0018] a-4 d are flow diagrams show messages that are exchanged according to the invention;
FIG. 5 is a block schematic diagram that shows a header that is sent at the start of every message both from the client to the proxy and from the proxy down to the client according to the invention; [0019]
FIG. 6 is a block schematic diagram that shows a data request message that is included after the common data header according to the invention; and [0020]
FIG. 7 is a block schematic diagram that shows a message that is sent by the proxy to the client for the purposes of an acknowledgement and to send the result of the transaction.[0021]

DETAILED DESCRIPTION OF THE INVENTION

The invention provides a store and forward (S&F) system that supports multiple applications within an extensible network to direct information of various formats to any of multiple destinations. In the presently preferred embodiment of the invention, Java applications running on a client send non-priority data to any server on the application network or, alternatively, anywhere on a global telecommunications network such as the Internet, at some time in the future. [0022]
In the preferred embodiment, S&F allows a client application to send usage statistics to a database on the application network. It also enables T-commerce purchases to be made by the user, where the purchase and other relevant information is sent to a destination web server as if the purchase had taken place on the web via a full web browser on an Internet-connected PC. [0023]
The preferred S&F mechanism consists of two distinct parts: [0024]
Client-side code in the uVM; and [0025]
A server-side proxy server process. [0026]
Architectural Overview [0027]
FIG. 1 is a block schematic diagram showing logical architecture and data flow according to the invention. The presently preferred architecture consists of nine distinct entities: [0028]
Client Application ([0029] 1)
Stats API ([0030] 2)
Store and Forward API ([0031] 3)
UVM native Store and Forward layer ([0032] 4)
Digital Terminal Operating System (OS) ([0033] 5)
RF Distribution Network ([0034] 6)
Store and Forward Proxy ([0035] 7)
IP Network ([0036] 8)
Application Server ([0037] 9)
The Client Application ([0038] 1), Stats API (2), S&F API (3), uVM native Store and Forward Layer (4) collectively comprise what is called the Client Software.
The Client Application is a Java applet running on the digital terminal, in the preferred embodiment. This application uses the services provided by the Stats API and the S&F API to send statistics or application-specific data to an Application Server. [0039]
The Stats API is middleware that resides on the digital terminal. It facilitates the gathering of user events from different client applications ([0040] 1). For example, when a user starts an application from the main navigation screen (NavShell), the Stats API records the time and the application that was started. The Stats API then uses the S&F API to send the statistics data to the S&F proxy and ultimately to an Application Server that will collect usage statistics.
The Stats API must provide to the S&F API the IP address and port number of the Application Server to which the statistics data should be sent. In the preferred embodiment, the Client Software can be configured with default values for the Statistics Application Server. [0041]
Each digital terminal can be configured to send statistics without identifying the sender. This allows statistics to be collected anonymously, which protects individual subscribers' privacy. In addition, the terminal can be configured with the amount of memory to use for storing statistics. Configuration can also specify that statistics should be sent after a fixed period of time or after a certain amount of data has been collected. [0042]
The S&F API is used by the Stats API but can also be used by a Client Application directly. The S&F API allows the user to specify several attributes that affect how, when and where the data is sent. The user of this API is required to provide the IP address, port number of the Application Server that will receive the data. The user must also provide the transmission protocol that the S&F proxy should use to send the data to the Application Server. The standard Internet protocols UDP, TCP and HTTP are supported. When the S&F API receives a request to send, the data and all optional and required parameters are passed to the uVM native S&F layer. [0043]
The S&F API allows the user to specify when the initial attempt to send the data occurs, how long this transaction has to complete and how many times to retry sending the data to the S&F proxy. [0044]
When the uVM native S&F layer receives data to send, it formats a message that contains the address of the Application Server as well as the data. The structure of the message in the preferred implementation is shown in FIGS. 5, 6. [0045]
The S&F layer can store data in NVRAM to ensure that the data will be saved if the power is lost. The S&F implementation can be configured to not send S&F data during certain times of the day, called blackout periods. These blackout periods are used when the digital terminals are being polled to retrieve PPV purchase information. Any additional traffic during these times would affect the ability to retrieve this information. [0046]
The S&F implementation uses a scheduling algorithm that includes transaction lifetime, initial send delay, and the blackout schedule. In order to avoid having all terminals on a network all attempt to send at the same time (which would result is severe data loss), a random element is also added to the scheduling algorithm. [0047]
Each digital terminal can also be configured with the amount of memory to use for S&F data, the maximum number of messages to store before sending, and the IP address and port of the S&F proxy to send to. [0048]
The Digital Terminal Operating System (OS) is a software entity provided by the manufacturer of the Digital Terminal. It provides access to sending and receiving data over the RF Distribution Network. [0049]
The RF Distribution Network, in the preferred embodiment, is a two-way real-time RF network. Data delivery over this network is unreliable. The Store and Forward system implements additional levels of reliability. The S&F API allows the caller to determine which level of reliability is required. [0050]
The Store and Forward Proxy is a server-side construct that facilitates upstream communication for the digital terminal via the NC1500 An NC-1500 is a upstream communications router (manufactured by Motorola). It is specifically used for to deliver messages sent by digital terminals (via the upstream cable network) to the headend communications network. When user event data is received by the Proxy, the proxy tries to deliver the data to the Application Server specified in the data, via the protocol similarly specified in the data. The S&F proxy can send data to any Application Server that is reachable via the supported Internet transport protocols mentioned above. The S&F Proxy sends data to the Application Server across the IP Network. The S&F Proxy and the Application Server can communicate via UDP, TCP or HTTP, which are standard Internet Protocols. [0051]
An Application Server is an entity connected to the IP Network that can receive data from a Client Application via at least one of UDP, TCP and HTTP. It is responsible for parsing and processing the data sent by the Client Application. The important thing about the Store and Forward Proxy is that it can be used to communicate to any application server via the standard UDP, TCP/IP, or HTTP protocols. [0052]

APPLICATION SERVER EXAMPLE

The presently preferred Application Server is a statstics server. When contacted by the Store and Forward Proxy, it breaks down the received information into single events and saves them in a flat text file. [0053]
The Fetcher is daemon that runs in a central point of the Cable-S architecture. It collects data from one or more Servers and saves it to a central log file. [0054]
The Data Bridge (not shown) is a software utility that can be used to import the Fetcher flat file into an Oracle database. [0055]
At this point, a third party tool such as BrioQuery can be used to create graphical representations of the collected data. [0056]
Log File Format—Server [0057]
The log file format presently consists of a single user event per line, with the following four tab delimited fields: [0058]
Date Event User Date Data [0059]
Log File Parameters—Server [0060]
Date [0061]
The time in seconds since Jan. 1, 1970 (the UNIX Epoch) that the event reached the Server GMT. This allows for the use of standard UNIX function calls for the decoding of time. [0062]
Event [0063]
A four character user event code that describes a single user action. [0064]
User [0065]
The identification of the user that caused the event. The presently preferred environment only allows identification to the set-top level and not the user level. [0066]
Date [0067]
The time in seconds since Jan. 1, 1970 (the UNIX Epoch) that the event occurred GMT. This allows for the use of standard UNIX function calls for the decoding of time. [0068]
Data [0069]
Any optional data associated with an event. [0070]
Current User Event Codes [0071]

Table “A” below outlines the event codes used by the preferred embodiment. Each event is accompanied by a time stamp and the set-top/username. Some events also include additional data in the optional data field. For example, the TVCH event contains the numeric value of the channel that was tuned.

TABLE A


Event Codes

Event		When
Code	Description	Event Is Triggered	Data Field

UNON	Unit On	Set-top is powered on	n/a
UNOF	Unit Off	Set-top is powered off	n/a
ULIN	User Login	User logs in	n/a
ULOF	User Logoff	User logs off	n/a
TVCH	New TV	User tunes to a new	numeric value of channel
	Channel	channel	example: 2
TVLC	Left TV	User leaves a tuned	numeric value of the
	Channel	channel	channel that was left and
			the channel that is
			currently being viewed
			separated by a semicolon
			example: 2;7
TVMT	TV Mute On	TV is muted	n/a
TVUM	TV Mute Off	Sound is turned back	n/a
		on
WWPG	New Web	URL is visited	the URL
	Page		example:
			http://www.cnn.com

Additional User Event Codes [0073]

Table “B” below outlines the additional codes that are added to the existing set to facilitate user interaction with the Guide, uBrowser, TV Ticker, and Games. Each event is accompanied by a time stamp and the set-top name. Some events also include additional data in the optional data field. For example, the APDN contains a application identifier.

TABLE B


Additional User Event Codes

Event		When Event
Code	Description	Is Triggered	Data Field

GDON	Guide On	User enters the	The menu in Guide that is
(future		Guide	selected. At this time, only
use)			the main menu is identified
			in statistics gathering.
			example: main
GDDN	Guide Duration	User exists a	The menu identifier and
(future		menu from	the duration using that
use)		the Guide	menu in seconds (future
			use).
			example: main;16
APDN	Application	User exits the	The application ID and the
	Duration	application	duration in the application
			in seconds
			example: uBrowser;23
UWWW	Micro Browser	User leaves a	The URL visited and the
	URL	URL	duration spent at the URL
			in seconds
			example:
			www.liberate.com;300
TKCV	TVTicker	User cursors to	The category identifier that
(future	Category View	a category to	the user is viewing and
use)		view available	duration
		stories	example: business;5
TKST	TVTicker Story	User selects a	The story identifier that the
	Selection	story	user selected and the
			duration spent reading
			story
			example:
			Sports;tc.jsp?11751;47

Example Event Entries Of Data [0075]
Some examples of typical user events are as follows: [0076]
996588616 APDN 0010304ead 996521265 Menu:0;3 [0077]
996588616 APDN 0010304ead 996521427/interactive/Golf.jar;328 [0078]
996588616 APDN 0010304ead 996521530 Menu:1;8 [0079]
996588616 APDN 0010304ead 996521710/interactive/Spades.jar;175 [0080]
These events can be interpreted as: [0081]
all events arrived at the Server on Tue Jul 31 10:10:16 2001 [0082]
user displayed the menu for 3 seconds and then exited on Jul 30 15:27:45 2001 [0083]
user played Golf game for 328 seconds and exited on Jul 30 15:30:27 2001 [0084]
user displayed the [0085] menu #1 for 8 seconds and then exited on Jul 30 15:32:10 2001
user played Spades game for 175 seconds and exited on Jul 30 15:35:10 2001 [0086]
Technical Specifications—Client Configuration [0087]
All client-side provisioning for statistics gathering is facilitated through a configuration file. The config.props file is uploaded to the/etc directory of the out of band carousel on the Application Server via the Carousel User Interface, which is the user interface for managing files being placed on the data carousel. [0088]
Client features include: [0089]
1. Store and Forward uVM API used by applications—This API allows sending raw data to a thrid party server using UDP, TCP, or HTTP. [0090]
The Store and Forward uVM API interfaces with the native Store and Forward module which is part of the uVM. [0091]
2. Native Store and Forward module—stores data either in dynamic RAM or in non-volatile memory until it is received by the Store and Forward proxy. [0092]
uses a scheduling algorithm which takes into account blackout periods to determine when to send, i.e. forward; data to the Store and Forward proxy. The scheduling algorithm attempts to minimize data loss in the upstream by sharing the upstream communications resource across all settops so as not to overload the upstream channel. [0093]
implements the Store and Forward protocol, formatting data according to the specified protocol. [0094]
The following variables are available for configuration: [0095]
SFProxyAddress [0096]
This defines the IP address of the S&F proxy and must be in the dotted-decimal format. It cannot be a hostname. Also, the IP address must be one that is connected to the application-server network. There is no default value. Therefore, if it is not provided, messages are not sent from the client to the proxy. [0097]
e.g. SFProxyAddress=192.168.14.215 [0098]
SFProxyPort [0099]
This defines the port on which the S&F proxy server is providing the S&F service (default 1022). [0100]
e.g. SFProxyPort=10010 [0101]
SFSchedDelay [0102]
Used in calculating the time at which an upstream message is sent. An upstream message is sent at a random time between blackout periods. The SFSchedDelay determines the window of time within which a random value is chosen. The unit of measure for SFSchedDelay is seconds. [0103]
SFMaxConcMsgs [0104]
Defines the maximum number of S&F messages held in memory. A value of zero is valid but pending messages are not purged (default value is 100). [0105]
SFMaxMemSize—(Recommended Setting: 8192) [0106]
The maximum amount of memory that S&F can use when storing messages. The unit of measure is bytes and (default 8192). [0107]
SFBlackoutSchedule—(Recommended Setting: 6AM-12PM Daily) [0108]
This defines the a times when the client is not allowed to send upstream messages to the S&F proxy and consists of a list of start-time end-time combinations separated by commas: [0109]
e.g. SFBlackoutSchedule=M 12:30 T 13:30, W 14:00 W 18:00 [0110]
Note that the schedule spans for a week and the start-times for a blackout period must be smaller than the end-time. The days of the week are abbreviated by one letter and are ordered as follows: [0111]
S (Sunday)<M (Monday)<T (Tuesday)<W (Wednesday)<H (Thursday)<F (Friday)<A (Saturday) [0112]
The time is specified on a 24 hour clock starting at time 0 and ending and hour 23. [0113]
Restrictions: The separators are space (‘ ’) and comma (‘,’). At this time, multiple separators cannot be used to separate fields, e.g. M[0114] _—1:20 is invalid but M_—1:20 is valid.
Currently, the accepted blackout period for field deployments is 6 PM to 12 AM daily: [0115]
SFBlackoutSchedule=M 18:00 T 0:00, T 18:00 W 0:00, W 18:00H 0:00, H 18:00 F 0 \:00, F 18:00 A 0:00, A 18:00 S 0:00, S 18:00 M 0:00 [0116]
StatsAnonymous [0117]
This specifies whether or not the statistics gathered are anonymous. In other words, if it is anonymous, then the user names are marked with an ‘X’ when they are recorded. A value of “false” results in user names being recorded. The default is to be anonymous. [0118]
e.g. StatsAnonymous=false [0119]
StatsMaxBufferSize—(Recommended Setting: 8192) [0120]
This specifies the amount of S&F memory that can be used for statistics. Once the limit is reached and a new statistic is recorded, pending statistics are cancelled, starting with the oldest. The unit of measure is bytes and default value is 8192. [0121]
StatsBufferSize—(Recommended Setting: 880) [0122]
This specifies the amount of statistics data that is buffered before it is sent to the statistics server. The values supported are in the range 15-880. The unit of measure is bytes. [0123]
StatsFlushTime—(Recommended Setting: 1800) [0124]
This specifies the amount of time statistics are buffered before they are flushed. The value must be greater than zero and the unit of measure is seconds. [0125]
e.g. StatsFlushTime=1800 [0126]
Address [0127]
This specifies the address of the (statistics) server and must be in the dotted-decimal format. It cannot be a hostname. There is no default value. Therefore, if it is not provided, statistics are not sent from the client to the proxy. [0128]
e.g. Address=192.168.14.206 [0129]
Port [0130]
This defines the port on which the server servicing requests (default value is 89). [0131]
Notes: [0132]
To disable the gathering and transmission of stats, set the following variables to 0: [0133]
SFMaxMemSize=0 [0134]
StatsMaxBufferSize=0 [0135]
Store and Forward Client-Proxy Protocol [0136]
The Store and Forward protocol allows for varying levels of reliability, which is achieved through the use of messages passed from the client to the proxy and from the proxy to the client. The client can specify the level of messaging required by setting flags in each request message to the proxy. There are three flags in the preferred embodiment that can be set. They can be set individually or in combination, but some combinations are not legal. Other flags may be provided as well in alternative embodiments of the invention. [0137]
The first flag tells the proxy whether or not to send the client an acknowledgement when it has received the client's request. The purpose of this flag is to let the client know that it is safe to purge the request data from NVRAM. [0138]
The second flag tells the proxy whether or not the client wants to be informed of the result of the request. Low-priority requests may not require a result to be sent back to the client. [0139]
The third flag tells the proxy whether or not to hold the result of the request until the client sends the proxy a message acknowledging the receipt of the request result. This is important if the client must know the result of the request. Without this flag, the proxy sends the result only once before purging the request. If that message gets lost, the client is not able to determine the result. With this flag set, the proxy holds onto the request and the result until the client indicates that it has successfully received the result. [0140]
The following describes the effects of the various combinations of flags: [0141]
ACK−0 RESULT−0 DONE−0 [0142]
The client sends the request, but the proxy does not store this in the database. It does not send any message to the client about this request. The proxy purges all information about this request as soon as it completes or times out. [0143]
ACK−1 RESULT−0 DONE−0 [0144]
When the proxy receives this message, it sends either an ACK or the result, but not both. See the discussion of DELAYED ACK for more details. The proxy writes this request to the database if the request cannot be completed immediately. The proxy purges this request as soon as the result is known or the request timeout occurs. [0145]
ACK−0 RESULT−1 DONE−0 [0146]
When the proxy receives this message, it attempts to carry out the request. See the discussion of DELAYED ACK for more details. The proxy writes this request to the database if the request cannot be completed immediately. When the result of the request is known, the proxy sends the result once to the client and immediately purges this request. [0147]
ACK−0 RESULT−0 DONE−1 [0148]
Invalid combination. If the proxy receives a request with this combination set, it turns off the DONE flag and treats it as if the flags were (0,0,0). [0149]
ACK−1 RESULT−1 DONE−0 [0150]
When the proxy receives this message, it attempts to carry out the request. See the discussion of DELAYED ACK for more details. The proxy writes this request to the database if the request cannot be completed immediately. When the result of the request is known, the proxy sends the result once to the client and immediately purges this request. [0151]
ACK−1 RESULT−0 DONE−1 [0152]
Invalid combination. If the proxy receives a request with this combination set, it turns off the DONE flag and treats it as if the flags were (1,0,0). [0153]
ACK−0 RESULT−1 DONE−1 [0154]
The proxy processes the request immediately, but only writes it to the database if the request cannot be completed in a certain period of time. No ACK is sent to the client once the request has been written to the database. Once the result of the request is known, the result is sent back to the client. The proxy does not purge this result until it receives the DONE message from the client or until the request expiry time elapses. [0155]
ACK−1 RESULT−1 DONE−1 [0156]
The proxy processes the request immediately, but only writes it to the database if the request cannot be completed in a certain period of time. An ACK is sent to the client once the request has been written to the database. Once the result of the request is known, the result is sent back to the client. If the result is known before the ACK has been sent, the request is not written to the database and the result is sent to the client instead of the ACK. See the DELAYED ACK discussion for more details. The proxy does not purge this result until it receives the DONE message from the client or until the request expiry time elapses. [0157]
DELAYED ACK [0158]
Note that if at least one of the three flags are set, the client implements a “delayed ACK” scheme. This means that when the proxy receives a request, it does not send the acknowledgement message right away if the ACK flag is set, but tries to complete the transaction first. [0159]
If the request completes within a few seconds, e.g. ten seconds, the request result is sent instead of the acknowledgement. If the request cannot be completed in that time, the request is written to the database and then the acknowledgement is sent to the client. When the request result is determined later, the result is sent if the result flag is set. [0160]
If the ACK flag is set and the proxy receives a duplicate request, it sends back an ACK immediately. A likely reason for the proxy to receive a duplicate is because the client never received the first ACK, probably due to a lost message. [0161]
Technical Specifications—Communication Session (Result) [0162]
The following outlines typical communication sessions between the DCT and the Server via the Store and Forward Proxy. The specifics of these communication sessions are configurable by using the three bit flags entry in the data_send packet previously mentioned. [0163]
Setting the flags bits to 010 (−, RESULT, −) produces the following communication session: [0164]
First, the DCT sends data to the Proxy (see FIG. 2[0165] a).
The Proxy then transfers the data from to the Server from memory (FIG. 2[0166] b).
The Proxy then sends a RESULT message to the DCT to inform it that the data was sent to the Server so that it may purge the majority of the transaction from memory (FIG. 2[0167] c).
In the event that the initial data does not reach the Proxy or the Proxy goes down between the time the data are received and are sent to the Server, a RESULT message is not sent and the DCT assumes that the data did not reach the Proxy, thereby causing a retransmit from the client. [0168]
This next session outlines the scenario in which the Server cannot be contacted within a prescribed (short) period of time. [0169]
First, the DCT sends data to the Proxy (FIG. 3[0170] a).
The Proxy then attempts to transfer the data to the Server from memory (FIG. 3[0171] b).
After x seconds the Proxy assumes the server is down and writes the transaction to the database (FIG. 3[0172] c).
The proxy sends an ACK message when the proxy has been able to deliver the message to the specified final destination (FIG. 3[0173] d).
The Proxy sends the data to the Server at a later date (FIG. 3[0174] e).
The Proxy then sends a RESULT message to the DCT so that it may purge the majority of the transaction from NVRAM (FIG. 3[0175] f).
In the event that the initial data does not reach the Proxy or the Proxy goes down between the time the data is received and it is sent to the Server, an ACK message is not sent and the DCT assumes that the data did not reach the Proxy causing a retransmit. [0176]
Technical Specifications—Statistics Gathering & Transmission [0177]

As events are triggered by user interaction with the DCT, statistics are gathered in the buffer. At the client level, the fields include:



Event
User	Date	Data

APDN	0010304ead 996521265	Menu:0;3
APDN	0010304ead 996521427	/interactive/Golf.jar;328
APDN	0010304ead 996521530	Menu:1;8
APDN	0010304ead 996521710	/interactive/Spades.jar;175
UWWW	0010304ead 998936906	\http://www.metatv.net/compact_
		att/cnnsi/story.jsp?want=2;13

Typical size of the MENU event is approximately 34 bytes. Because of the variable “data” field, a uBrowser statistic can be as large as a visited URL. In the preferred embodiment, the events are approximately 90 bytes. [0179]
When the client buffer is filled and additional statistics are generated before the current ones are removed, the oldest statistics are removed to create space. Currently, there is no operator indication when this occurs. [0180]
When an upstream statistic transmission is triggered, a single, or group of statistics is sent to the Store and Forward Proxy in a single upstream packet. The packet payload is limited to a maximum of approximately 1000 bytes in the preferred embodiment. [0181]
The algorithm used to send these statistics dictates that a packet is to be sent up to a maximum of ten times between the first transmission and the maximum expiry time, currently twenty four hours. Once an ACK is received from the Store and Forward proxy, the DCT assumes the statistic was successfully sent and removes it from memory. If, after ten attempts to send the statistic, an ACK is not received, the statistic is cleared from memory. [0182]
Store and Forward Summary [0183]
Java features include: [0184]
Asynchronous application layer APIs, e.g. http APIs. [0185]

Java API



	public class StoreAndForward
	{

	/*
	* Used to store and forward data to the 3rd party

application

* server identified by < addr:port> using the

specified protocol.

* The same instance can be used to forward multiple

sets

	* of data.
	*/
	public StoreAndForward (InetAddress addr, int port, int

protocol) ;

/* Returns the IP address of the 3rd party application

server. */

	public InetAddress getInetAddress ( ) ;
	/* Returns the port of the 3rd party application

server.*/

	public int getPort ( ) ;
	/* Returns the protocol used to communicate with the

app. server */

	public int getProtocol ( ) ;
	/* Returns the current timeout value. */
	public int getTimeout ( ) ;
	/* Returns the current maximum retry count.*/
	public int getMaxRetryCount ( ) ;
	/* isTransaction*/
	public boolean isTransaction ( ) ;
	/* Sets the number of retries for forwarding data.
	* After this limit is reached, no more retries are

attempted

* but the transaction state is stored until the expiry

time.

	*/
	public void setMaxRetryCount (int count) ;
	/* Sets the time in seconds after which transaction is

abandoned.*/

	public void setTimeout (int timeout) ;
	/* setTransaction */
	public void setTransaction(boolean isTransaction) ;
	/* Sets the minimum time (in seconds) after which the

first

	* attempt to send data is to be made.
	*/
	public void setInitialSendTime (int time) ;
	/* Add listener to get status on all unfinished

transactions */

	public void addNotify (SFNotifyListener nl) ;
	/* Removes a listener from the list of listeners. */
	public void removeNotify (SFNotifyListener nl) ;
	/* Submits data to be forwarded to the 3rd party

server. */

	/* The transaction-id is returned. */
	public native int forwardData (byte [] data, int

offset, it len) ;

	/* Cancels the transaction. */
	public native int cancelTransaction(int transactionID);

	}

End-to-End Description [0187]
The following describes the steps taken in a store and forward transaction from start to finish. [0188]
1. The Java applet creates an instance of a StoreAndForward object. [0189]
2. The applet fills in the required data and invokes the send( ) method. [0190]
3. The code in the JVM optionally stores the request in NVRAM to guard against power outages. [0191]
4. At some point in the future, the JVM sends a copy of this request to the proxy server, keeping the original in NVRAM (the flags field directs what the Store and Forward proxy does). [0192]
5. The proxy server receives the message from the client and tries to forward the message before it stores a copy in a database to prevent data loss. [0193]
6. If the ACK flag is set, the proxy server sends an acknowledgement back to the client telling it that it has successfully received the data and attempts to deliver it to the application server specified in the data. [0194]
7. The client receives this acknowledgement and can then remove the payload part of this transaction from NVRAM because the proxy has it. It must keep some information about the transaction in NVRAM until it has completed. [0195]
8. The proxy attempts to contact the application server at the IP address and port given in the request data via the protocol that the client requested. In the preferred embodiment, UDP, TCP, and HTTP POST requests are supported. [0196]
9. The proxy continues to try to send the data until the time limit to send the data has expired. [0197]
10. The proxy creates a transaction status message with the final result. If the original request was stored in the database, the final result is modified in the database record. If the request was not stored, it is not added in this phase. [0198]
The proxy only sends the final result if the results flag is set. If the done flag is set, the proxy resends the done message until the time limit is reached, or it gets a done message. [0199]
11. When the client receives the transaction result, it can notify the original applet if it is still running. [0200]
12. The client purges the transaction: [0201]
a) immediately after sending, if both the ACK and RESULTS flags are set; [0202]
b) after receiving an ACK, or after the transaction expires, if the RESULTS flag is set. [0203]
13.When the proxy receives this DONE status message from the client, it purges all records of this transaction from the database. [0204]
Protocol Summary [0205]
The store and forward protocol can be configured on each request to control the level of reliability to make the protocol flexible to handle different client requirements. It does this by setting up to three bit flags in the store and forward protocol header. Each of these flags adds more reliability. It does this by indicating the messages that are exchanged between the client and the server. [0206]
The three possible flags are: [0207]
SEND_DATA_ACK, SEND_RESULT, SEND_DONE. [0208]
The SEND_DATA_ACK flag indicates that the client wants the proxy to send an acknowledgement that the proxy received the client's request successfully. The client keeps retransmitting the request until this acknowledgement is received. [0209]
The SEND_RESULT flag indicates that the client wants to have the result of the transaction sent back to it when the transaction has either completed successfully, failed, or that the lifetime has expired before the transaction could be completed. This flag can be used with the SEND_DATA_ACK flag or can be used on its own. [0210]
The SEND_DONE flag indicates that the client sends a transaction_done message to the proxy after it receives the transaction result. This flag is only valid if the SEND_RESULT flag is also on. Setting this flag allows the proxy to know when the client is done with the transaction so that it can purge all data associated with this transaction. Using this flag keeps the data on the proxy from getting too big. The only difference between having only the ACK flag set and only the RESULT flag set is when a response is sent back to the client if the data could not be delivered to the application immediately. [0211]
FIG. 4[0212] a shows the messages that are exchanged when no flags are set.
FIG. 4[0213] b shows the messages that are exchanged when the SEND_DATA_ACK flag is set.
FIG. 4[0214] c shows the messages that are exchanged when the SEND_RESULT flag is set.
FIG. 4[0215] d shows the messages that are exchanged when the DATA_ACK, SEND_RESULT and SEND_DONE flags are set.
This last sequence provides the most reliability, but as a consequence requires the most amount of messages to be exchanged. [0216]
Transport Protocol Details [0217]
Common Store And Forward Header [0218]
FIG. 5 shows a header that is sent at the start of every message both from the client to the proxy and from the proxy down to the client. [0219]
Client MAC address—The five-byte identifier for the set top box. [0220]
Flags—a one-byte field that consists of three bits for the message type, three bits for flags, and two bits for the transport protocol. [0221]
The message type field can be one of TRANS_DATA, TRANS_STATUS, or TRANS_CANCEL. [0222]
The flags can be any OR-ed combination of TRANS_SEND_DATA_ACK, TRANS_SEND_RESULT, and TRANS_SEND_DONE. [0223]
The protocol field can be one of PROTO_UDP, PROTO_TCP, or PROTO_HTTP. [0224]
Transaction ID—A two-byte field that uniquely identifies the transaction from this client. [0225]
Data Request Message [0226]
The data request message (see FIG. 6) is included after the common data header. [0227]
Send Expiry—The number of minutes, relative to the time the proxy receives the data that the proxy server has to send the data to the application server. If this time elapses without a completed send to the application server, either a success or the application server returned a failure (only for HTTP), then the transaction is considered to have timed out. No further attempts are made to send the data to the application server. The proxy then changes Transaction Expiry—Send Expiry seconds to a fixed period of time, such as 60 seconds in the preferred embodiment, to send this TIMEDOUT result back to the client, if requested by the client. [0228]
Destination IP address—The four-byte IPv4 address of the application server to which to send the data. [0229]
Destination Port—the two-byte port number on the application server to which to send the data. The protocol to use to send the data is specified in the common header. [0230]
Transaction Status Message [0231]
This message (see FIG. 7) is sent by the proxy to the client for the purposes of an acknowledgement and to send the result of the transaction. It is sent from the client to the proxy to indicate that the client has finished with the transaction. This is appended after the common data header. [0232]
Result—The last known result of this request. Can be one of INVALID, INPROGRESS, SUCCEEDED, FAILED, TIMEDOUT, CANCELLED, or TOO_MANY_TRIES. [0233]
State—If the Result was INPROGRESS, then this field is the current state of the transaction. Can be one of DATA_RCVD, DATA_ACKED, SENDING_DATA, SENT_DATA. [0234]
Server Proxy [0235]
The server proxy receives data sent from the client and passes it on to the host specified in the data. The client also specifies the transport mechanism to be used, e.g. TCP, UDP or HTTP. If the proxy server cannot send the data to the final destination (referred to herein as the application server) within a given period of time, e.g. ten seconds, then the proxy server saves the client data to a database, and sends an acknowledgement to the client (in some embodiments) to tell it that the proxy has successfully received the request and is responsible for ensuring its delivery. [0236]
The server proxy continues to try to deliver the data to the destination application server until a timeout for that request has expired, the client cancels the request, or it has been successfully delivered. [0237]
Running the Server Proxy [0238]
The preferred proxy has the following command-line options: [0239]
p specifies the UDP port number that the proxy should listen on to receive requests from clients. If this option is not specified a default of 1022 is used. Note that the use of this default requires the proxy to be run with an EUID of 0. [0240]
c specifies the path to a configuration file. The default is “./.proxyrc”. This configuration file specifies the debugging level to use, as well as the name of the debugging output file to use. The configuration file can be changed at run-time and is reread by sending the server proxy process a SIGHUP. [0241]
d specifies the Data Source Name (DSN) to use. This name must refer to an entry in the odbc.ini configuration file that the server proxy reads. It uses this to determine on which host the database resides and which ODBC driver to use to talk to it. [0242]
u specifies the username the proxy should use when logging into the database. [0243]
w specifies the password, if any, to use with the given username to log in to the database. [0244]
The data the proxy server sends to the set top box (STB) is sent as a UDP packet. This UDP packet is received by the NC and sent downstream to the STB via the out of band channel (OOB). For this to work correctly, there must exist a route or routes for the STB IP addresses to the network controller (NC), which is the gateway to the OAM&P network. OAM&P is the network between the Store and Forward proxy and the NC1500. It stands for Operation, Administration, Management, & Provisioning. [0245]
Statistics Gathering [0246]
The following discussion describes specifications for the format, collection, and presentation of statistics gathered from user interaction with a preferred embodiment of the invention. [0247]
The areas in which statistics are gathered include: [0248]
NavShell use [0249]
Web Browsing [0250]
TV Viewing [0251]
High Level Requirements [0252]
The following discussion outlines the high level requirements for statistics gathering, as well as the type of information that is collected for each event. [0253]
User Identification [0254]
The default configuration sets the user identification field to anonymous. There is, however, a configuration option that allows the Digital Communications Terminal (DCT), e.g. the set-top box to include the actual user name. [0255]
Statistics to be Collected [0256]
NavShell [0257]
Duration and exit time of all applications, including games and enhanced TV (ETV). [0258]
Duration and exit time of NavShell menu display. [0259]
Duration and exit time of ETV icon display. [0260]
TVTicker [0261]
Manual selection of a category view that lasts for at least five seconds. [0262]
Category selected to view a complete story. [0263]
Length of time that a story is viewed. [0264]
uBrowser [0265]
URL [0266]
Duration at URL [0267]
ETV [0268]
none [0269]
Games [0270]
none [0271]
Although the invention is described herein with reference to the preferred embodiment, one skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. Accordingly, the invention should only be limited by the claims included below. [0272]

Claims

1. A store and forward (S&F) system that supports multiple applications within an extensible network to direct information of various formats to any of multiple destinations, comprising:

a client application resident at a client location for storing non-priority data and for sending said data to any one or more servers on said network at some future time; and

a server-side proxy server process.

2. The system of claim 1, wherein said client application allows any of sending usage statistics to a database on said network; and enabling T-commerce purchases to be made by a user, where purchase and other relevant information is sent to a destination server.

3. A store and forward (S&F) system, comprising:

a client module resides on a digital terminal for facilitating gathering of user events from different applications, and for sending said user events to a store and forward proxy in batch;

a store and forward proxy comprising a server-side construct for facilitating upstream communication for said digital terminal; for converting user event data received by said proxy to a standardized format; and for sending said converted data to a server;

a server for facilitating communications from both said client and a fetcher, wherein said server breaks down received information into single events and saves them in a file when contacted by said store and forward proxy; and

a fetcher for collecting data from at least one server and saving said data to a central log file.

4. The system of claim 3, further comprising:

a data bridge for importing said fetcher central log file into a database.

5. The system of claim 3, said client module further comprising:

a configuration file.

6. The system of claim 5, said configuration file comprising any of the following variables:

SFProxyAddress which defines an IP address of said store and forward proxy;

SFProxyPort which defines a port on which said store and forward proxy server is providing store and forward service;

SFschedDelay which is used in calculating a time at which an up stream message is sent, wherein an upstream message is sent at a random time between blackout periods, and wherein SFSchedDelay determines a window of time within which a random value is chosen;

SFMaxConcMsgs which defines a maximum number of store and forward messages held in memory;.

SFMaxMemSize which is a maximum amount of memory that said store and forward system can use when storing messages;

SFBlackoutSchedule which defines a time when said client module is not allowed to send upstream messages to said store and forward proxy, and which consists of a list of start-time end-time combinations;

SFBlackoutSchedule;

StatsAnonymous which specifies whether or not statistics gathered are anonymous;

StatsMaxBufferSize which specifies an amount of store and forward memory that can be used for statistics, wherein once a limit is reached and a new statistic is recorded, pending statistics are cancelled, starting with an oldest;

StatsBufferSize which specifies an amount of statistics data that is buffered before it is sent to a statistics server;

StatsFlushTime which specifies an amount of time statistics are buffered before they are flushed;

Address which specifies an address of a statistics server; and

Port which defines a port on which a server is servicing requests.

7. The system of claim 3, wherein said store and forward proxy facilitates communication between said client module and said server as follows:

said client module sends data to said store and forward proxy;

said store and forward proxy then transfers said data from to said server from memory; and

said store and forward proxy then sends a result message to said client module to inform it that said data was sent to said server so that said client module may purge at least a majority of a transaction from client memory.

8. The system of claim 7, wherein in the event that initial data does not reach said store and forward proxy or said store and forward proxy goes down between a time said data are received and said data are sent to said server, a result message is not sent and said client module assumes that said data did not reach said store and forward proxy, thereby causing a retransmit of said data.

9. The system of claim 3, wherein said store and forward proxy facilitates communication between said client module and said server when said server cannot be contacted within a prescribed period of time as follows:

said client module sends data to said store and forward proxy;

said store and forward proxy then attempts to transfer said data to said server from memory;

after a predetermined interval, said store and forward proxy assumes said server is down and writes a transaction to a database;

said store and forward proxy then sends a result message to said client module to inform it that data was written to said database, so that said client module knows a packet was received and does not require a retransmit;

said store and forward proxy sends said data to said server at a later date; and

said store and forward proxy then sends a result message to said client module so that said client module may purge at least a majority of a transaction from client memory.

10. The system of claim 3, said client module further comprising:

a buffer for gathering statistics as events are triggered by user interaction.

11. The system of claim 10, wherein when said client module buffer is filled and additional statistics are generated before current statistics are removed, oldest statistics are removed to create space.

12. The system of claim 10, wherein when an upstream statistic transmission is triggered, a single, or group of statistics is sent to said store and forward proxy in a single upstream packet.

13. The system of claim 10, wherein when once an acknowledgement is received from said store and forward proxy, said client module assumes a statistic was successfully sent and removes it from memory.

14. A method for performing a store and forward transaction, comprising the steps of:

at a client, using an applet for:

creating an instance of a StoreAndForward object;

filling in required data and invoking a send method;

storing a request in non-volatile memory to guard against power outages; and

at some point in time, sending a copy of said request to a proxy server, while keeping an original in non-volatile memory;

at a proxy server:

receiving said message from said client and storing a copy in a database to prevent data loss;

sending an acknowledgement back to said client telling it that it has successfully received said data; and

attempting to deliver said data to an application server specified in said data.

said client:

receiving said acknowledgement and then removing a payload part of a transaction from non-volatile memory because said proxy server has said payload, said client keeping some information about said transaction in non-volatile memory until said transaction is completed;

said proxy:

attempting to contact said application server at an address and port given in said request data via a protocol that said client requested;

continuing to try to send said data up to a number of request attempts given in said request or until a time limit to send said data is expired;

creating a transaction status message with the final result and stores this message in the database.

sending said message back to said client and waiting for a done message from said client.

wherein, when said client receives said transaction result, it can notify an original applet if said applet is still running;

said client:

sending a transaction status message to said proxy indicating that said transaction is done;

where, when said proxy receives said done status message from said client, said proxy purges all records of said transaction from said database.

15. A store and forward system, comprising:

a client application resident at a client location for storing data and for sending said data to any one or more servers on said network at some future time; and

a server-side proxy server process;

a store and forward header that is sent at the start of every message both from said client to said proxy and from said proxy down to said client; and

means for setting at least one flag in said store and forward protocol header.

16. The system of claim 15, wherein said at least one flag comprises any of:

a SEND_DATA_ACK flag which indicates that said client wants said proxy to send an acknowledgement that said proxy received said client's request successfully, wherein said client keeps retransmitting said request until an acknowledgement is received;

a SEND_RESULT flag which indicates that said client wants to have a result of a transaction sent back to it when said transaction has either completed successfully, failed, or a lifetime has expired before said transaction could be completed; and

a SEND_DONE flag which indicates that said client sends a transaction_done message to said proxy after it receives a transaction result, wherein setting this flag allows said proxy to know when said client is done with said transaction so that it can purge all data associated with said transaction.

17. The system of claim 15, wherein said store and forward header comprises any of:

a Client MAC address which is an identifier for a client appliance;

one or more Flags which indicate any of a message type, a flag, and a transport protocol;

a Transaction ID that uniquely identifies a transaction from a particular client; and

a Data Request Message.

18. The system of claim 17, wherein said data request message is included after a common data header and comprises any of:

a Transaction Expiry which set a time by which a transaction must complete;

a Send Expiry which sets a time said proxy server has to send data to an application server;

a Destination address of an application server to which to send said data;

a Destination Port number on an application server to which to send said data;

a Number of Retransmits which sets a maximum number of times said proxy should attempt to send said data to an application server; and

one or more optional Flags by said client can specify additional options for a request.

19. The system of claim 15, further comprising:

a transaction status message appended after a common data header, which is sent by said proxy to said client as an acknowledgement and to send a result of a transaction; and which is sent from said client to said proxy to indicate that said client has finished with a transaction;

wherein said transaction status message comprises any of:

a Result which is a last known result of a request, and which can comprise any of INVALID, INPROGRESS, SUCCEEDED, FAILED, TIMEDOUT, CANCELLED, or TOO_MANY_TRIES; and

a State wherein, if a Result is INPROGRESS, then this field is a current state of a transaction, and which can comprise any of DATA_RCVD, DATA_ACKED, SENDING_DATA, SENT_DATA.

20. A store and forward system that allows for varying levels of reliability, comprising:

a client;

a proxy; and

a plurality of messages passed from said client to said proxy and from said proxy to said client;

wherein said client can specify a level of messaging required by setting flags in each request message to said proxy, wherein said flags can be set individually or in combination.

21. The system of claim 19, comprising:

a flag that tells said proxy whether or not to send said client an acknowledgement when it has received a client's request, wherein said flag lets said client know that it is safe to purge a request data from memory.

22. The system of claim 19, comprising:

a flag that tells said proxy whether or not said client wants to be informed of a result of a request, wherein low-priority requests may not require a result to be sent back to said client.

23. The system of claim 19, comprising:

a flag that tells said proxy whether or not to hold a result of a request until said client sends the proxy a message acknowledging receipt of said request result, wherein said proxy holds onto a request and a result until said client indicates that it has successfully received said result.