US 20050273508 A1
A method and system for performing a service on a home network, by: connecting a first and a second home device to the home network; providing a database including a plurality of application interface description data objects, where each application interface description data object includes information in a structured format for commanding and controlling of a home device by one or more other home devices connected to the network; the second home device accessing a first application interface description object for the first home device in the database; the first home device accessing a second application interface description object for the second home device in the database; sending control and command data from the first home device to the second home device utilizing the second application interface description object over the network; and sending control and command data from the second home device to the first home device utilizing the first application interface description object over the network. Whereby, the first and second home devices perform said service.
18. A method for performing a service on a network, the method comprising the steps of:
(a) querying a device to obtain application interface description data when the device is connected to the network, wherein the application interface description data includes information for commanding and controlling of the device by another device connected to the network;
(b) storing the obtained application interface description data in a database; and;
(c) providing an application interface description of a first device from the database to a second device, wherein the second devices interacts with that first device utilizing said application interface description of the first device for command and control.
19. The method of
20. In a network system for providing a service, a controller comprising an agent that: (a) queries a device to obtain application interface description data when the device is connected to the network, such that the application interface description data includes information for commanding and controlling of the device by another device connected to the network, (b) stores the obtained application interface description data in a database, and (c) makes accessible to a first device an application interface description of a second device from the database wherein the first device interacts with the second device utilizing said application interface description of the second device for command and control.
21. The controller of
22. The method of
23. The system of
Applicant claims the benefit of U.S. Provisional Application No. 60/084,578 entitled “Command and Control Using XML”, filed on May 7, 1998 which is incorporated herein by reference. The present application is related to the following copending applications that are commonly assigned and which are incorporated herein by reference: U.S. patent application Ser. No. 09/104,299, entitled “Browser Based Command and Control Home Network”; U.S. patent application Ser. No. 09/104,297, entitled “Method and Apparatus for A Home Network Auto-Tree Builder”; U.S. patent application Ser. No. 09/104,298, entitled “Improved Home Network, Browser Based, Command and Control”; U.S. patent application Ser. No. 9/103,469, entitled “Method and Apparatus for Creating Home Network Macros”; and U.S. patent application Ser. No. 09/104,606, entitled “Programming Tool For Home Networks.
The present invention relates to the field of network systems, and more particularly, to home network having multiple devices connected thereto.
A network generally includes a communication link and various devices with communication capability connected to the communication link. The devices include computers, peripheral devices, routers, storage devices, and appliances with processors and communication interfaces. An example of a network is a home network for a household in which various devices are interconnected. A usual household can contain several devices including personal computers and home devices that are typically found in the home. As such the term “device” typically includes logical devices or other units having functionality and an ability to exchange data, and can include not only all home devices but also general purpose computers. Home devices include such electronic devices as security systems, theater equipment, TVS, VCRs, stereo equipment, and direct broadcast satellite services or (DBSS), also known as digital satellite services (DSS), sprinkler systems, lighting systems, micro waves, dish washer, ovens/stoves, washers/dryers, and a processing system in an automobile.
In general, home devices are used to perform tasks that enhance a homeowner's life style and standard of living. For example, a dishwasher performs the task of washing dirty dishes and relieves the homeowner of having to wash the dishes by hand. A VCR can record a TV program to allow a homeowner to watch a particular program at a later time. Security systems protect the homeowner's valuables and can reduce the homeowner's fear of unwanted entry.
Home devices, such as home theater equipment, are often controlled using a single common control unit, namely a remote control device. This single common control unit allows a homeowner to control and command several different home devices using a single interface. Thus, may manufacturers have developed control units for controlling and commanding their home devices from a single interface.
One drawback associated with using the remote control unit to command and control home devices is that it provides static and command logic for controlling and commanding each home device. Another drawback associated with using remote control units is that known remote control units cannot control a plurality of diverse devices, and more particularly, cannot control a plurality of devices having different capabilities to communicate with each other in order to accomplish tasks or provide a service.
In conventional network systems a user provides commands using a remote control unit or device control panel. Once the user ceases, there is no controller unit or device in the network to provide commands for automatic operation. After a user initially controls and commands a first set of devices, conventional systems do not provide a mechanism for the first set of devices to automatically communicate with a second set of devices in the network as necessary in order to accomplish tasks without direct user control and command of the second set of devices. Further, conventional systems do not provide an efficient method for various network devices to obtain information about other network devices in the network for command and control.
There is, therefore, a need for a method and a system which provides dynamic control and command devices in a home network. There is also a need for such a method and system to provide the ability to control a plurality of diverse devices having different capabilities to communicate with each other in order to accomplish tasks or provide a service. There is also a need for such a method and system to provide the ability for various network devices to automatically command and control other various network devices. There is also a need for such a method and system to provide universally accessible command and control information for inter-device communication.
The present invention satisfies these needs. In one embodiment the present invention provides a method and system for performing a service on a home network, by: connecting a first and a second home device to the home network; providing a database including a plurality of application interface description data objects, where each application interface description data object includes information in a structured format for commanding and controlling of a home device by one or more other home devices connected to the network; the second home device accessing a first application interface description object for the first home device in the database; the first home device accessing a second application interface description object for the second home device in the database; sending control and command data from the first home device to the second home device utilizing the second application interface description object over the network; and sending control and command data from the second home device to the first home device utilizing the first application interface description object over the network. Whereby, the first and second home devices perform said service.
In one version of the invention, the first home device stores first application interface data therein, and the second home device stores second application interface data therein. The database is formed by querying the first and second home devices to transfer said application interface data for the first and second home devices to the database device. The database can be stored in a database device and connected to the network for universal access by network devices. In that case, the first application interface description object for the first home device can be provided from the data base to the second home device over the network. Further, the second application interface description object can be provided from the data base to the first home device over the network.
Further, three or more home devices can be connected to the network, wherein at least one home device accesses the database to query the application interface description objects of a plurality of home devices for sending command and control data to the plurality of home devices over the network. Each application interface description object can include data in a structured format. The structured format can include XML format.
These and other features, aspects and advantages of the present invention will become better understood with regard to the following description, appended claims and accompanying drawings where:
In one aspect, the present invention provides inter-device communication in a network such as a home network. As home devices become more intelligent and can share information, inter-device communication allows a user to interconnect devices in a network to take advantage of the information sharing capabilities of those devices. As such, inter-device communication plays a crucial role in affording a user with the ability to fully and flexibly utilize the networked devices.
Each client device 12 may communicate with one or more server devices 14 in the network 10. Further, each server device 14 may communicate with one or more other server devices 14, and one or more client devices 12, in the network 10. Each client device 12 can include a user communication interface including input devices such as a mouse and keyboard for receiving user input, and a display for providing a control user interface for a user to interact with the networked devices. The user interface can include a graphical user interface (GUI) display 18 for providing information to the user. Referring to
For control between a controlling client device 12 and a controlled server device 14, the client device 12 accesses the GCO 22 of the server device 14 by, for example, transferring the GCO 22 from the server device 14 to the client device 12 over the network. The client device 12 then uses the transferred GCO 22 to create a control user interface GUI 18 for the user to communicate with the control program 20 of the server device 14 from the client device 12 over the network. The user provides command and control to at least the control program 20 of the server device 14 from the client device 12.
Storing the GCO 22 of each server device 14 in the server device itself may reduce the processing and storage requirements of the client devices 12 in networks with several server devices 14. Further, storing the GCOs 22 in the server devices 14 may allow each server device 14 to provide its own GUI look and feel to the user, and allows for modification or updating of the GCOs 22 without modifications to client devices 12.
In addition, the GUI is 18 of various server devices 14 may include commonalities such as: (1) a common GCO model type for the client device renderer 24 to display GUIs 18, (2) common communication protocols for transferring the GCOs 22 from various server devices 14 to the client device 12, and (3) common communication protocols for GUI interaction from the client device 12 to the control program 20 of the corresponding server device 14, wherein the client device 12 does not require a built-in knowledge of a particular server device 14 being controlled.
Referring still to
For example, the control state data 26 can include the status of control information in the GUI 18 for the server device 14, such as timer setup for a recording action in a VCR server device. The control state data 26 is stored in the controlled server device 14, and displayed to a user through the GUI 18 of the server device 14 at the controlling client device 12, for user control of the server device 14. Preferably, the controlling client device 12 for displaying the GUI 18 of the server device 14 does not retain knowledge of the control state data 26 for the controlled server device 14.
Each server device 14 can be controlled by one or more client devices 12. As such, the control state data 26 stored in the server device 14 includes status of the information in the GUI 18 of the server device 14 at each of the controlling client devices 12. For example, when the user controls a server device 14 using a first client device 12, upon completion of the user control, the information in the GUI 18 of the server device 14 at the first client device 12 is saved by the server device 14 in the control state data 26 of the server device 14.
Alternatively, while the user is interacting with the GUI 18 of the server device 14 at the first client device 12, the control state data 26 of the server device 14 is updated with the information in the GUI 18 of the server device 14 at the first client device 12, and upon completion of user control, the control state data 26 is retained in the server device 14. When the user controls the server device 14 using a second client device 12, the control state data 26 is made available to the user via the GUI 18 of the server device 14 at the second client device 12 for further control. The user can also use the first client device 12 at a later time to control the server device 14, whereupon the control state data 26 is made available to the user via the GUI 18 of the server device 14 at the first client device 12 for further control. The server device 14 can also include a clock 28, or maintains the current time, to allow time delay action based on time or clock input from a user, as described below.
A client device 12 and a server device 14 can be physically bundled together as one unit such as a DTV. In that case, the server device 14 includes a control program 20 for controlling the server hardware, and the client device 12 provides control user interface to the server control program 20 for control and command of at least the server hardware.
For example, for automatic time-delay operation, a user can “log on” to a client device 12 to control a first set of server devices 14 and specify desired services. The user then “logs off” from the client device 12. The first set of server devices 14 perform communication and control among themselves, and at a later time, one or more of server devices 14 in the first set automatically control a second set of server devices 14 as necessary to collectively provide the desired services without user involvement.
For example, the data stream source hardware 32 can include a Tuner-Access Device such as a Direct Broadcast Satellite (DBS). A DBS is a multi-channel alternative to cable television and provides cable-like television programming directly from satellites on small (18 inch to 3-foot diameter) satellite dishes. With DBS, several standard analog television signals are digitally compressed onto a single satellite transponder thereby allowing up to 200 or more channels receivable with a dish pointed at a fixed position in the sky. The data stream sink hardware 34 can include a Digital Video Cassette Recorder (DVCR) which comprises s digital VCR that is able to decode compressed digital video signals on playback. The user provides command and control data including “time-delay record” event data for the DVCR and a “time-delay select a program” event data for the Tuner-Access Device. After the time-delay, the Tuner-Access Device selects the desired program, and sources program data to the DVCR which receives and records the program data without further control actions from the user.
In one example operation, upon selection of the server devices 14 SERVER1 and SERVER2, the client device 12 transfers the GCO 22 of each server device 14 to the client device and displays a corresponding GUI 18 for each of the server devices 14 SERVER1 and SERVER2. The user can interact with the GUI 18 of each server device 14 to provide command and control to the corresponding server device 14 for service. Each server device 14 can provide service alone or in combination with other server devices 14. Further, the session manager 36 transfers control state data 26 between the GUIs 18 of the server devices 14 in the client device 12 as necessary for the corresponding server devices 14 to perform a service. Based on the user command and control information, two or more of the server devices 14 can communicate command and control information therebetween to provide a user requested service.
The session manager 36 can include a software agent which functions to access and display available home network services provided by various server devices 14 in the network 10. The software agent can additionally match the capabilities of various server devices 14 in the network 10 and display selection information for only those server devices 14 that have compatible capabilities. Further, the session manager 36 can match the selections made in the GUI 18 of one server device 14 to the selections in GUI 18 of another server device 18 to help the user provide meaningful command and control information to the server devices 14.
In another example operation, the session manager 36 executes the software agent which searches the network and discovers the server devices 14 connected to the network. The software agent also accesses capabilities data stored in each server device 14 to determine the capabilities of the server devices 14 and provide information about those capabilities to the user. The session manager 36 then displays the selection icons Serv1, Serv2, Serv3 and Serv4 for the server devices SERVER1, SERVER2, SERVER3 and SERVER 4 as shown in
The session manager 36 initially enables all the selection icons Serv1, Serv2, Serv3 and Serv4 to allow the user to select from among all four selection icons. After the user selects the server device SERVER1 by clicking on the Serv1 selection icon, the session manager 36 determines that the server devices SERVER3 and SERVER 4 are incompatible in capability with the server device SERVER1. As such, the session manager 36 disables the selection icons Serv3 and Serv 4 for server devices SERVER3 and SERVER4, respectively. The user can then click on the icon Serv2 to command and control the server device SERVER2.
As the user interacts with the GUI 18 of a selected server device 14, control and command information input by the user into each GUI 18 provide additional capabilities information which affect further server device selections by the user. For example, if a VCR server device 14 is selected, further action by the session manager 36 in enabling or disabling selection icons for other server devices 14 is affected by a user decision to play or record.
Each server device 14 in the network has one or more service capabilities as discussed above by way of example with reference to the server devices in
Since each device 14 can be a source or sink for several different services on the network, each device 14 stores a capabilities data table (Capabilities Table 1) as shown by example in
In one implementation, a Device Manager conducts a matching or comparison of device source and sink services. For example, the Device Manager can be implemented as a software agent to compare the capabilities or properties of various devices 14 and locate devices 14 with matching capabilities. For example, in a case where the service is a media stream from a first device 14 across the network to a second device 14, the Device Manager compares the capabilities of the first and second devices 14 to assist the user in making a sensible selection of the second device 14 which is compatible with the capabilities of the first device 14. The following is an example list of service capabilities for an embodiment of a server device 14:
Each device 14 can further store an attribute data table (Attribute Table 2) including pertinent attributes of the device, shown by example in
In the user-to-client device control model described above, attribute data can be displayed on the GUI page of the server device 14 at the client device 12. Alternatively, a second level device information home page can be utilized to display said attribute data. Further, the attribute data in the form of a text or Extensible Markup Language (XML) file can be accessed by a software agent. For the device-to-device control model, the attribute data for the controlled device is stored in the device interface application interface.
The Device Location attribute field in the Attribute Table 2 is used to store the location or group for each device 14. The Device Type attribute field specifies the device type, such as VCR, DVD, DTV, Camcorder, PC, Security System, etc. for the particular device 14. The Device Type attributes field is used to select a default device icon to represent the device within the Device Page if the device itself does not supply one. The Attribute Table 2 can include multiple entries for the Default Source and the Default Sink attributes fields. Each such entry represents a different default source or sink device 14 for each data type handled by the device 14.
Preferably, the capabilities and attributes data are packaged into structured data using a hierarchical language. This provides a common method of retrieving the capabilities and attributes data that are used for other purposes such as in GCO transfer and server device-to-server device control. As an example, the attributes data can include the following structured data format:
As an example, the capabilities data can include the following structured format:
An application interface language is utilized to allow different server devices 14 to perform device-to-device control, including sever device-to-server device control. The application interface language includes command languages, and can be described using XML, as detailed below. The control program 20 of one server device 14 remotely controls the control program 20 of another server device 14 over the network, without using GUIs 18 or user involvement. An example of device-to-device control is automatic operation. A user initially provides control through a client device 12 for a desired service, and subsequently two or more server devices 14 automatically communicate and control one another without further user interaction to provide the service.
In the second instance “B”, the user is replaced by a software client control program 50. The software client control program 50 generates XML based command postings to the service application “S” and receives back XML command postings. And, in the third instance “C”, the software client control program 50 is replaced by an application such a server device control program 20, wherein commands and responses are exchanged between two service applications 20. In that regard, instance “B” is a special case of instance “C” with a null service.
An application interface language based on XML is used for control between a first server device 14 and a second sever device 14 (device-to-device or service-to-service) for devices or services that are world wide web (Web) enabled and Internet enabled. The application interface language is based on the Web standard, middleware layer. In one embodiment, device-to-device control includes remotely controlling the control program 20 or Application, in one server device 14 from another server device 14 in the network 10. As such, the interfaces (API) to such Applications 20 are made available over the network using API extensions. Preferably, the API extensions utilize a standard format, such as an XML-based interface, to provide overall interoperability.
Referring to the API extensions for the Service A, the first upper-most block 52 provides a comprehensive definition or data base of CE objects and methods using English words to describe CE devices. The comprehensive definition or data base can also be in C, XML or other formats capable of representing objects and their respective methods. The comprehensive definition or data base utilizing XML is termed XCE definition. The second block 54 provides a format for representation of an API in XML form for all devices 14, designated as an interface data type definition INTERFACE.DTD.
A software agent, designated as Tool A, utilizes a subset of the XCE definition for Service A, and uses the interface data type INTERFACE.DTD for Service A to create an XML form document, INTERFACE-A.XML. The document INTERFACE-A.XML describes the objects and methods supported by the Service A according to the document type definition INTERFACE.DTD for Service A. Other data type definitions can also be used to create the INTERFACE-A.XML document.
The software Tool A also creates a look-up table 56 to convert from XML messages from Service B on the network interface, to API definitions for Service A, programmed in C for example, and complied to executable binary. Preferably, the look-up table 56 is created at compile time, whereby during run-time, incoming XML form method messages (commands) from Service B are converted to the API format created by the complied application C code for Service A. The look-up 56 table provides run-time translation of XML object method calls from Service B into device native language calls for Service A. The look-up table 56 is complied with the device control program 20 for local execution on the server device A for Service A.
The INTERFACE-A.XML can be used by Service A for validity checks if it encounters an error in a received message. The INTERFACE-A.XML can also be used by a foreign Application such as Service B to determine the message format for Service A before communicating with Service A. Further, if a message from Service B to Service A causes an error, Service B can access the INTERFACE-A.XML document to diagnose the error.
Referring to the API extensions for the Service B, the first block 58 provides a comprehensive definition or data base of CE objects such as the XCE definition for Service A above. The next block 60 provides a language definition for making XML form method (command) calls to remote API services or devices such as the API for Service A. The language definition is a document type definition Method Request CALL.DTD which describes interaction with objects on the network.
A software agent, designated as Tool B, utilizes at least a subset of the objects and methods in the XCE definition for Service B and the CALL.DTD document, to generate a look-up table 62 for converting commands from a complied C program code for Service B into XML form method requests. As such, the look-up table 62 provides conversion between a method invoked by Service B (e.g., “PLAY”) and the XML document or message that carries the method call across the network interface to Service A, for example. The subset of the XCE definition used by software Tool B depends on the extent and nature of use of the network. For example, the subset can be selected to provide global or restricted use of all available services on a home network.
Therefore, the API extensions provide for communication between various devices on the network using XML. In the example above, the program code 20 for Service B generates method calls to an API, and the API calls are converted to XML form to comply with the Web/Internet standard XML for inter-device communication. The XML method calls (messages) are sent to Service A over the network, and Service A reconverts the XML method calls from the network interface to program code API definitions for Service A. This conversion and re-conversion provides Web/Internet compatibility for diverse devices in the network with program code APIs which would otherwise require binary compatibility between different devices. Appendix 1 shows examples of the XML interface blocks utilizing the block diagrams in of
Further, Appendix 1 provides examples of interface definitions INTERFACE.DTD and CALL.DTD used to create description documents of available services, INTERFACE.XML, described above. The CALL.DTD definition includes a rule set for generating method call or function call message such as XML Remote Procedure Call (RPC) or XMLRPC messages. The CALL.DTD definition describes an output interface of a controller service 14. In a home network, for example, INTERFACE.XML represents the services available on the home network. The available services are a subset of the entire services in the CE space.
In a One-Touch-Record (OTR) scenario, a user is in control of a Tuner-Access-Device such as a Satellite STB. The user controls the tuning using an Electronic Program Guide (EPG) such as a graphical user interface representation of program listings. OTR record provides the user with a service including selection of a future program from the EPG for recording without the user accessing the VCR graphical user interface to program the VCR for a Time Delayed Recording. OTR automates the control of the VCR. Below is an example control list of actions in OTR.XML shown in Appendix 2: (1) StreamOpen=play the selected program stream output to the network from a Satellite STB; for OTR this control is local to the STB device; (2) StorageOpen=open a storage service; and (3) StorageRecord=Send the Record command across the network to the VCR.
As discussed above in relation to
In one scenario, the first server device B wishes to control the second sever device A in the network. The INTERFACE-A.XML document of the second device A is transferred from the second server device A to the first server device B and used by Application B to query the capabilities and API interface methods of the second server device A. This allows the first server device B to control the second server device A utilizing XML remote procedure calls XMLRPC. In another scenario, the first server device B performs the above steps after attempting to communicate with the second server device A at least once, and failed to establish communication. Yet in another scenario the first server device B queries the INTERFACE-A.XML document in the second server device A remotely without transferring the INTERFACE-A.XML document to the first server device B.
Upon examining the contents of the INTERFACE-A.XML document, the first server device B can create commands for sending to the second server device A in XML format as described above. Generally, the first server device B can interpret at least a portion of the contents of the INTERFACE-A.XML document that overlaps with a subset of the XCE definition used by the first and second server devices B and A as described above. If the first server device B is unable to interpret a portion of the contents of the INTERFACE-A.XML document, then the first server device B can ignore that portion, or fetch an application to assist it in interpreting that portion, by translation as described further below.
In a first example, the application E accesses the application interface description A by remote query over the network. In a second example, the application E accesses the application interface description A by transferring a copy of the application interface description A from the controlled device 14 to the controller device 14. The application E then queries the interface description A locally. In a third example, the application interface description A is transferred to a library device 64 which provides library space for interface descriptions, and the application E remotely queries the interface description A in the library. The library device 64 stored the address (URI) of the associated applications available for direct control action and responses.
A first XML layer API, designated as XML Layer OUT 68, is used for sending messages, and a second XML layer API, designated as XML Layer IN 70, is used for receiving messages. The XCE definition and the XML definition of a method call, namely the document type definition CALL.DTD described above, are used to create the XML Layer OUT 68. Further the XCE definition and the XML definition for a method call, namely document type definition INTERFACE.DTD described above, are used to create the XML Layer IN 70. For example a controller application utilizes the XML Layer OUT 68 and a controlled application utilizes the XML Layer IN 70.
Each of the server devices A and B includes hardware and software for controlling other server devices over the network and for being controlled by other server devices over the network. In
Each of the devices A and B also includes an XML parser 74, comprising program code for parsing and validating XML messages, such as XML interface and XMLRPC commands. The XML parser 74 is similar to said XML Layer IN 70 described above with reference to
An Interface Fetcher comprising program code, is utilized by each of the devices A and B to fetch the device interface of another device directly from another device or from a home network Interface Library 80. When a device 14 is a controller device, a controller application program code 82 in the controller device 14 effects command and control of other devices 14 over the network, by supervising software and hardware in the controller device 14 such as the XML parser 74, the interface fetcher 78 and the XMLRPC codec 76. When a device 14 is a controlled device, a controlled application program code 84 in the controlled device 14 supervises software and hardware in the device 14 for the device 14 to be controlled by other devices 14. A Home Network Device Web server 86 in each of the devices A and B manages communication between the devices over the network. An XML to Native Lookup Table 88 in each of the devices A and B is used by the controlled application 84 to convert information in XMLRPC messages (e.g., method name, parameters name and type) to native interface of the device (e.g., native method name, parameters name and type). Said table 88 is not used when the names of methods and parameters in XML messages and the native interface of the device are the same.
Each of device the devices A and B further includes one or more Handlers 90, wherein each Handler 90 includes a pointer from within the controlled application 84 to a native implementation of one specific device functionality. In most devices, native implementations of device functionality include binary code at run-time. The binary code can be generated from higher level languages at compile time, including C and Java, for example. As such, consumer electronics manufacturers can add more Handlers 90 for new functions without affecting existing Handlers and function implementations. A hardware service 92 in each of the devices A and B includes native implementations of device functions. Each of the devices A and B also includes a Native Interface 94 which comprises the API of native implementation of the device functions.
Further, a Network Object Request Broker such as a Home Network Object Request Broker (HNORB) 79 and Interface Library (IL) 80 provides a middleware layer 98 for the home network 10. As shown in
The middleware layer, comprising the HNORB 79 and the IL 80, can be connected directly to the Internet, such that selected home devices can be accessed from outside of a local home network 10. The middleware layer 98 in one local home network can be connected to the middleware layer 98 in other local home networks over the Internet to provide an integrated network comprising two home networks 10. In that case, authorized users with the appropriate stream encryption can access a DVD changer in the user's primary home, from a TV in the user's secondary home to play a video and view it on the TV.
To use the Interface Library 80, at least one HNORB&IL should be running on the local home network 10. More than one HNORB&IL may also be available. For example, a cable modem, several DTVs, and a central home hub can all have their own HNORB&IL software agents. To locate the HNORB&IL, a device 14 sends a broadcast message over the local home network. The first HNORB&IL to respond to the device 14 is utilized by the device 14. Once a HNORB&IL is located, the device 14 and the HNORB&IL can establish a point-to-point Transmission Control Protocol (TCP) or User Datagram Protocol (UDP) connection for registration, interface request and fetch, and device lookup services. If a UDP protocol is not available, a TCP protocol can be used for high bandwidth connections such as IEEE 1394. HTTP-based XMLRPC can also be utilized for device to HNORB&IL communications. For example, a device 14 can remotely call a “register” method of HNORB to pass the device interface as one or more parameters, or, a XMLRPC call can retrieve a partial or entire device interface from the IL as a XMLRPC response or return value.
As aforementioned, more than one HNORB&ILs can run in a local home network 10 simultaneously, wherein each HNORB&IL recognizes a subset of available devices and one HNORB&IL can communicate with other HNORB&1Ls to locate the devices 14 it can not find. Multiple HNORB&ILs on one local home network 10 can locate each other automatically by using broadcasting messages, such as UDP or TCP. In this case, multiple HNORBs construct a distributed object request broker, while multiple Interface Libraries 80 construct a distributed interface library. To provide fault tolerance, if one of the HNORB&IL should terminate unexpectedly, all devices registered with this HNORB&IL are notified and said devices can automatically register with another available HNORB&IL.
Each device interface has an associated consistent, unique logical name. Other devices can use said consistent, unique, logical name to recognize and access a device, even after said device's location or real network address has changed. The mapping of the logical names and real device addresses are handled by a software agent for naming service in HNORB. Preferably, a standardized naming method is utilized. More preferably, a hierarchical naming structure is used to organize device names into a hierarchical tree. This hierarchical structure can be expressed using “/”, similar to that in a file system. The structure can be generated by different methods, such as by different service types as a home/MPEG2/TV; or by different locations, such as home/livingroomVCR. Several naming trees may coexist for performance and efficiency.
In the example command and control between the controller server device A and the controller server device B in
The controller application 82 of the device A attempts to query all or part of the device interface of the controlled device B. If an Interface Library 80 is not available, the controller device A can request and fetch the device interface of the controlled device B directly from the controller device B by first sending a request to device B over the network, and then receiving the XML interface of device B from the device B. However, if an Interface Library 80 is available, the controller device A can request all or part of the device interface of the controlled device B from the Interface Library 80. The software agent of HNORB obtains the XML device interface of the device B from the Interface Library 80 structure and sends it back to the controller device A.
Once the controller device A receives the XML device interface of the controlled device B, the controller application of device A uses the XML parser 74 of device A to parse and interpret the device interface of the device B. The XMLRPC codec 76 of device A then generates desired XMLRPC command messages using the parser results. The XMLRPC command messages are sent to the controlled device B over the network. Upon receiving said XMLRPC command messages, the controlled application 84 of device B uses the XML parser 74 of device B to parse and interpret the received XML command. messages. The XMLRPC codec 76 of device B then decodes the parser results to obtain the method call information in the command message, including a method name and parameters for the device B functions to perform requested services.
The controlled application 84 of device B then uses the XML to Native Lookup Table 88 and Handlers 90 in the device B to access and launch the native function implementations of device B through the native interface of device B. If a function generates any responses or return values, said responses or return values are encoded into XML or XMLRPC messages and sent to the controller device A. Further, the middleware layer HNORB and IL can provide the controller device A with a reference to the controlled device B, whereby the device A can generate remote calls to the device B native functions just as calls to the local device A native function.
Preferably, a standard XMLRPC format is utilized so that all devices can interpret and decode RPC calls over the network. Because the device interface of a controlled device 14 can be queried and examined by a controller device 14, preferably a simplified XMLRPC format with sufficient device interface information is utilized to improve efficiency. The following example shows two possible formats of XMLRPC calls for One Touch Record (OTR) and Time Delayed Record (TDR) operations.
XML RPC call, example format including detailed tag and interface information:
XML RPC call, example format with reduced tags and interface information:
Controller and controlled applications 82, 84 are programmed using a standard interface subset of the XML based XCE data base 104. Each device interface is stored with said applications 82, 84 in XML form. Although the XCE data base 104 need not be in XML, said subset interface produced at compile time is in XML in an embodiment of the invention, as described above in reference to
The XML interfaces 72 of the CE devices, including said XML interface document and said DTD document, are stored in a universally accessible library such as the home network Interface Library 106. A software agent 110 collects the device interfaces 72 of all accessible devices 14 over the network and places them into the searchable structured Interface Library 106 along with the device name/address information. The Interface Library 106 is a subset of the XCE database 104 and the process of generating the Interface Library 106 is similar to that of rebuilding part or all of the XCE database 104. The Interface Library 106 can function as a collection of device interfaces 72 of all devices 14 in the home network, or as a cache depending on availability of storage space, wherein only the most recently used device interfaces 72 are stored therein. In cases where a device 14 updates its device interface 72 due to an event, such as disk change in a DVD player, part of the device interface 72 is updated based on an event service.
In a typical command and control model for a server device 14 to control another server device 14 according to the present invention, a first device 14 attempts to query the device interface of a second device 14 at the second interface layer 114. After selecting function categories (FC), the first device 14 queries the interface layer 116 of a specific function category in the second device 14, such as Record Category. Further, the first device 14 can query the interface layer 118 of a specific function, such as OTR or TDR, to make calls to said functions. The hierarchical or tree structure makes finding of an interface function more efficient and saves network bandwidth. An example interface file structure and layers can be:
Similarly, the home network Interface Library 106 is preferably hierarchical and can be structured in a variety of ways such as by different service type of devices or by different locations such as rooms. Said hierarchical structure is the interface of a local home network 10 to other home networks or the Internet.
Appendix 5 shows an example hierarchical device interface definition 72 which can be implemented in XML syntax. Said hierarchical device interface definition 72 can include the following fields:
In the above description, ‘Services_home’ provides the main structure including A/V consumer electronics. A branch of the structure is expanded in detail for a particular example of a video services sink, and stream destination (e.g., DVCR) control interface. The control interfaces in a typical home network can include:
In the above description, ‘server_av’ is part of the structure for the control interfaces for A/V appliances offering A/V stream service, and is subdivided into ‘controls-gen’, ‘source’ and ‘sink’ capabilities.
Referenced above, ‘service_id’ or ‘application_interface_id’ includes the name, address or Web address or URL location of one or more devices 14. Because the XCE database 104 comprises the totality of agreed upon interfaces, typically a Dynamic Host Configuration Protocol (DHCP) software agent executes to assign an address and a default name to each device, and the address and a default name are added the interface of the service or device. The software agent 110 then collects device interfaces 72 which include subset or ‘device partial XCE’ definitions from all the devices locally connected to the home network to generate a network partial XCE’. Additional relevant external interfaces can be added to the structure for external control. For example ‘service_id’ can be a name/address in a received structure or in a network Interface Library 106 including entries from the software agent according to the device interfaces of the devices connected to the network. Thereafter, a user can search for a service in the database and access an application whose interface includes a particular data branch of the library using said name/address. As such, the network can include multiple identical services distinguished by said name/address information.
The above description can apply equally to client devices 12. An alternative syntax XCE definition or database for the CE space can be utilized. The alternative syntax XCE data base includes full service descriptions including home automation, appliances and automobile, for example. In cases where a service object provides flexibility and parameters for control, a control method is utilized to control the object as desired. Appendix 3 shows example commands in the AV/C and CAL command languages including binary and hex data strings.
In another aspect, the present invention provides for use of existing command language implementations for device-to-device command and control in a network. Devices can include internal objects and APIs which, at run time, create binary strings according to existing transport mechanisms. In that case, in order to provide XML remote procedure calls (XML RPC) from one device 14 to another device 14 in the network, the exiting application interface implementation is replaced with calls to the XML service API. As such, the original implementation is equivalent to a wrapper for the XML service API.
In one embodiment, a method of common presentation or packaging of data and command protocol is provided, whereby a receiving device 122 can determine the native format of transmitted data. If the receiving device 122 can interpret that native format, then it can accept the data directly. Otherwise, the receiving device 122 can request a translator device 124 or application to translate the data into a desired format which the requesting device 122 can interpret. The translator device 124 or application determines the native format of the original data, translates the data into said desired format, and sends the translated data to the requesting device 122.
The requesting device 122 then processes that data as though the data had originally been provided in the requesting device's native language format by the sending device 120. The requesting device 122 can also send a response back to the sending device 120 in the requesting device's native format, or send a response by proxy through the translator device 124 or application for translation into the native format of the sending device 120. The translation method can be utilized for information including command protocols, data files and audio/video streams.
For devices that do not utilize the common format described above, the present invention provides for translation of data including command protocols to, and from, such non-compliant devices. For example, when a non-compliant device 120 sends data to a compliant device 122, the compliant device 122 can translate the data based on a determination of the native format of the data. For example, the compliant device 122 can examine the data for particular bit patterns within the data. When a compliant device sends data to a known non-compliant device, the compliant device can translate the data before transmission based on a determination of the native format of the non-compliant device.
An example implementation can be for a home network which supports the IP and HTTP protocols. The home network can be connected to the Internet to obtain applications' and services of various types for desired functionality. As such, the common format method can be made compatible with Internet protocols and procedure for operation over the Internet and the home network.
One example of providing a common data format is utilizing XML to create a package for the data for transmission over the home network. The data can include command protocol, streaming audio or video, graphics or applications. The data is ‘wrapped’ with a standard header identifying the native format of the data and contents of the package, in XML form. The header allows unique identification of the data type the data portion of the XML code, whereby the data can be translated if necessary and provided to appropriate applications upon receipt.
Under the Web standard, the identification process is performed by browsers using file name extensions to identify the type and contents of a file transmission. The browsers then launch appropriate plug-in modules to process the file. In the home network, XML is utilized to identify data transmissions which provides all home network transmissions over IP with a common identification method as described above.
Alternatively, a software layer can be provided in the home network protocol stack to uniquely identify the contents of all data transmissions over the home network. The software layer can be used instead of XML. The common format and identification principles of the present invention apply equally in either embodiment using XML or said software layer as identification methods.
The translation server device 124 translates the data and converts the translated data into an XML translation response package. The translation server 124 then transmits the response package back to the requesting device 122. In case of a translation error, the translation server 124 can provide a translation response error condition to the requesting device 122. Upon receiving the translated data, the requesting device 122 processes the translated data in the response package.
Example of an XML data package or packet can be:
Example of a translation request package or packet can be:
Example of a translation request package or packet can be:
Example of a translation response error condition package or packet can be.
Further, Table 3 in
To provide translation services, a translation server 124 is identified in the network during network configuration in a manner similar to that of DHCP servers. The translation server 124 broadcasts its IP address to all devices in the network for a period of time after the network is configured. All devices 120, 122 compatible with the translation services store the IP address of the translation server 124 as it is broadcast over the network during network boot up.
Alternatively, the requesting device 122 can broadcast a translation request over the home network. All translation servers 124 in the network that receive the translation request can respond to the translation request by sending a translation response to the requesting device 122. The requesting device 122 then selects one translation server 124 among the responding translation servers. In one example, the requesting device 122 selects the first translation server 124 that responds to the translation request. In another example, the translation servers 124 can negotiate among themselves and/or with the requesting device 122 for the selection of a translation server 124 for satisfying the translation request.
In another embodiment of the invention, multiple translation servers 124 are utilized to fulfill all translation requests. For example, a single translation server 124 may not have the capability to translate all requests. In such cases, it is necessary to identify the address of each translation server 124 and the type of translation service each translation server 124 can provide. Each device 120, 122 can store a list of all translation server IP addresses and a corresponding list of the types of translation services each translation server 124 provides, and optionally the associated translation application.
For efficiency, if a sending device 120 wishes to send data to a receiving device 122 which is known to use a different native format than that of the sending device 120, the sending device 120 can send the data to the receiving device 122 by proxy through a translation server 124. The sending device 120 transmits a command to the translation server 124 similar to the translation request command and includes the address of the receiving device 122 as the destination for the translated data.
In cases where a receiving device 122 requires translation of a data stream, the sending device 120 can route the data stream directly to a translation server 124, and the translation server 124 in turn transmits the translated data to the receiving device 122 as described above. Alternatively, the sending device 120 can send the data stream to the receiving device 122, and the receiving device 122 then routes the data stream to the translation server 124 for translation and return of the translated data back to the receiving device 122.
In the description herein, the control mechanism is based on the Hypertext Transfer Protocol (HTTP 1.1) which provides an application-level protocol for distributed, collaborative, hypermedia information systems. HTTP is a generic, stateless, object-oriented protocol in wide use for many tasks. A feature of HTTP is the typing and negotiation of data representation, allowing systems to be built independently of the data being transferred. Preferably, the network protocol used by devices and applications on the home network is IP (Internet Protocol). However, other protocols can also be utilized.
Although the present invention has been described in considerable detail with regard to the preferred versions thereof, other versions are possible. Therefore, the appended claims should not be limited to the descriptions of the preferred versions contained herein.