US 20100070448 A1
The present invention is directed to an integrated implementation framework and resulting medium for knowledge retrieval, management, delivery and presentation. The system includes a first server component that is responsible for adding and maintaining domain-specific semantic information and a second server component that hosts semantic and other knowledge for use by the first server component that work together to provide context and time-sensitive semantic information retrieval services to clients operating a presentation platform via a communication medium. Within the system, all objects or events in a given hierarchy are active Agents semantically related to each other and representing queries (comprised of underlying action code) that return data objects for presentation to the client according to a predetermined and customizable theme or “Skin.” This system provides various means for the client to customize and “blend” Agents and the underlying related queries to optimize the presentation of the resulting information. The present invention is directed to a framework or medium for knowledge retrieval, management, delivery and/or presentation. The system maintains semantic information and other knowledge to provide retrieval services to clients via a communication medium. Within the system, objects or events in a hierarchy are semantically related to each other, and agents implementing queries return data objects for presentation to the client according to a semantically influenced or determined theme. This system provides various means for the client to customize agents and/or the underlying related queries to optimize the presentation of the resulting information.
3. A system for knowledge retrieval, management, delivery and presentation, implemented on at least one computer capable of presenting at least one semantic relationship as part of a search result that presents at least one document in response to a query, the computer system comprising a computer storage medium having a plurality of computer softwar components embodied thereon, the computer software components comprising:
a knowledge indexing and classification component wherein information from both structured and unstructured information sources are semantically encoded to create a plurality of knowledge objects;
a knowledge integration component to perform the steps of:
creating a semantic network based on semantic associations between the plurality of knowledge objects having semantic encoded information;
hosting domain-specific, episodic and contextual information;
dynamically linking at least one knowledge object to domain-specific information creating a linkage network;
maintaining the semantic attributes and dynamic linkage network of knowledge objects in a data store;
a semantic query processing component to perform the steps of:
receiving at least one user input query for processing;
extracting at least one semantic query based on user input query;
inspecting the data store to determine at least one semantic relationship between the semantic query and the dynamically linked knowledge object in the linkage network based on one or more rules for determining the one semantic relationship;
semantically linking the semantic query with the dynamically linked knowledge object in the linkage network to create a relational node;
delivering a representation of the semantically linked relational node based on the user query to a client according to customizable user preferences.
4. A method for creating a semantic network of knowledge objects in a computer memory capable of storing at least one knowledge object having schema and semantic links and hosting domain-specific semantic information used to classify and categorize domain-specific information;
evaluating a schema of a first knowledge object;
obtaining domain-specific semantic information from a memory related to the first knowledge object schema if the schema of the first knowledge object lacks a domain-specific meaning; and
creating a semantic link between the first knowledge object and the domain-specific semantic information if the schema of the first knowledge object suggests association with domain-specific information.
5. A method for searching data products stored on a computer readable medium, implemented on at least one computer comprising:
building a natural language relationship of a plurality of data products forming a semantic linkage map, further comprising:
analyzing the text within the plurality of data products based on a series of predefined ontologies to determine at least one semantic concept, the semantic concept built from analysis of the language, word patterns, and a context of the text within each data product, the semantic concept containing text not found within the data product but inferentially related by connection in the semantic linkage map;
creating semantic metadata using the determined semantic concepts;
determining associations between the semantic metadata in the plurality of data products;
applying a semantic ranking to the semantic metadata;
indexing the ranked semantic metadata;
linking the ranked semantic metadata to create a linkage map
receiving a search query of the built semantic linkage map further comprising:
analyzing the search query based on the series of predefined ontologies to determine at least one semantic concept in the search query based on at least one of a meaning and a context, the semantic concept containing text not found within the data product but inferentially related by connection in the semantic linkage map;
creating semantic metadata using the determined concepts;
comparing the semantic metadata to the indexed and ranked semantic metadata; and
displaying a list of data products in a rank order based on the compared semantic metadata.
This application is a Continuation-in-Part of and claims priority to co-pending U.S. patent application Ser. Nos. 11/505,261 filed Aug. 16, 2006, Ser. No. 11/462,688 filed Aug. 4, 2006, Ser. No. 11/561,320 filed Nov. 17, 2006, Ser. No. 11/829,880 filed Jul 27, 2007, Ser. No. 11/931,659 filed Oct. 31, 2007; Ser. No. 11/931,793 filed Oct. 31, 2007, Ser. No. 12/134,003 filed Juni 5, 2008, Ser. No. 12/206,695 filed Sep. 8, 2008, and Ser. No. 12/206,656 filed Sep. 8, 2008.
This application is also a Continuation-In-Part Application of and claims priority to U.S. Provisional Patent Application No. 60/970,498 filed Sep. 6, 2007 and U.S. Provisional Patent Application No. 60/820,606 filed Jul. 27, 2006. This application also claims priority to U.S. patent application Ser. No. 11/383,736 (NERV-1-1013), filed May 16, 2006 which application claims priority to U.S. Provisional Patent Application No. 60/681,892 filed May 16, 2005. This application also claim priority to U.S. patent application Ser. No. 11/127,021 filed May 10, 2005; which application claims priority to U.S. Provisional Application Ser. Nos. 60/569,663 (Attorney Docket No. NERV-1-1007) and/or U.S. Provisional Application Ser. No. 60/569,665 (Attorney Docket No. NERV-1-1008).
This application also claims priority to U.S. application Ser. No. 10/179,651 (Attorney Docket No. FORE-1-1001) filed Jun. 24, 2002, which application claims priority to U.S. Provisional Application No. 60/360,610 (Attorney Docket No. NERV-1-1003) filed Feb. 28, 2002 and/or to U.S. Provisional Application No. 60/300,385 (Attorney Docket No. FORE-1-1002) filed Jun. 22, 2001. This Application also claims priority to U.S. Provisional Application No. 60/447,736 (Attorney Docket No. NERV-1-1004) filed Feb. 14, 2003. This Application also claims priority to PCT/US02/20249 (Attorney Docket No. FORE-11-1001) filed Jun. 24, 2002.
This application claims priority to U.S. application Ser. No. 10/781,053 (Attorney Docket No. NERV-1-1006) filed Feb. 17, 2004, which application is a Continuation-In-Part of U.S. application Ser. No. 10/179,651 filed Jun. 24, 2002, which claims priority to U.S. Provisional Application No. 60/360,610 filed Feb. 28, 2002 and/or to U.S. Provisional Application No. 60/300,385 filed Jun. 22, 2001. This Application also claims priority to U.S. Provisional Application No. 60/447,736 filed Feb. 14, 2003. This Application also claims priority to PCT/US02/20249 filed Jun. 24, 2002. This Application also claims priority to PCT/US2004/004380 (Attorney Ref. No. NERV-11-1012) and/or U.S. application Ser. No. 10/779,533 (Attorney Ref. No. NERV-1-1005), both filed Feb. 14, 2004. This application claims priority to PCT/US04/004674 (Attorney Docket No. NERV-11-1013) filed Feb. 14, 2004, which application is a Continuation-In-Part of U.S. application Ser. No. 10/179,651 filed Jun. 24, 2002, which claims priority to U.S. Provisional Application No. 60/360,610 filed Feb. 28, 2002 and/or to U.S. Provisional Application No. 60/300,385 filed Jun. 22, 2001. This Application also claims priority to U.S. Provisional Application No. 60/447,736 filed Feb. 14, 2003. This Application also claims priority to PCT/US02/20249 filed Jun. 24, 2002. This Application also claims priority to PCT/US2004/004380 (Attorney Ref. No. NERV-11-1012) and/or U.S. application Ser. No. 10/779,533 (Attorney Ref. No. NERV-1-1005), both filed Feb. 14, 2004.
All of the foregoing applications are hereby incorporated by reference in their entirety as if fully set forth herein.
This disclosure is protected under United States and International Copyright Laws.© 2002-2009 Nosa Omoigui. All Rights Reserved. A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
Knowledge is now widely recognized as a core asset for organizations around the world, and as a tool for competitive advantage. In today's connected, information-based world, knowledge-workers must have access to the knowledge and the tools they need to make better, faster, and more-informed decisions to improve their productivity, enhance customer relationships, and to make their businesses more competitive. In addition, industry observers have touted “agility” and the “real-time enterprise” as important business goals to have in the information economy.
Many organizations have begun to realize the value of disseminating knowledge within their organizations in order to improve products and customer service, and the value of having a well-trained workforce. The investments businesses are making in e-Learning and corporate training provides some evidence of this. Companies have also invested in tools for content management, search, collaboration, and business intelligence. Companies are also spending significant resources on digitizing their business processes, particularly with respect to acquiring and retaining customers.
However, many knowledge/learning and customer-relationship assets are still stored in a diverse set of repositories that do not understand each other's language, and as a result are managed and interacted with as independent islands of information. As such, what many organizations call “knowledge” is merely data and information. The information economy in large part is a struggle to find a way to provide context, meaning and efficient access to this ever increasing body of data and information. Or, stated differently, to turn the mass of available data and information into usable knowledge.
Information has been long accessible in a variety of forms, such as in newspapers, books, radio and television media, and in electronic form, with varying degrees of proliferation. Information management and access changed dramatically with the use of computers and computer networks. Networked computer systems provide access throughout the system to information maintained at any point along the system. Users need only establish the requisite connection to the network, provide proper authorization and identify the desired information to obtain access.
Information access further improved with the advent of the Internet, which connects a large number of computers across diverse geography to provide access to a vast body of information. The most wide spread method of providing information over the Internet is via the World Wide Web. The Web consists of a subset of the computers or Web servers connected to the Internet that typically run Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), GOPHER or other servers. Web servers host Web pages at Web sites. Web pages are encoded using one or more languages, such as the original Hypertext Markup Language (HTML) or the more current eXtensible Markup Language (XML) or the Standard Generic Markup Language (SGML). The published specifications for these languages are incorporated by reference herein. Web pages in these formatting languages may be accessed by Internet users via web browsing software such as Microsoft's Internet Explorer or Netscape's Navigator.
The Web has largely been organized based on syntax and structure, rather than context and semantics. As a result, information is typically accessed via search engines and Web directories. Current search engines use keyword and corresponding search techniques that rely on textual or basic subject matter information and indices without associated context and semantic information. Unfortunately, such searching methods produce thousands of largely unresponsive results; documents as opposed to actionable knowledge. Advanced searching techniques have been developed to focus queries and improve the relevance of search results. Many such techniques rely on historical user search trends to make basic assumptions as to desired information. Alternatively, other search techniques rely on categorization of Web sites to further focus the search results to areas anticipated to be most relevant. Regardless of the search technique, the underlying organization of searchable information is index-driven rather than context-driven. The frequency or type of textual information associated the document determines the search results, as opposed to the attributes of the subject matter of the document and how those attributes relate to the user's context. The result is continued ambiguity and inefficiency surrounding the use of the Web as a tool for acquiring actionable knowledge.
In enterprises around the world today, the Web is the information platform for knowledge-workers. And there lies the problem. The Web as we know it is a platform for data and information while its users operate at the level of “knowledge.” This disconnect is a very fundamental one and cannot be understated. The Web, in large measure, has fulfilled the dream of “information at your fingertips.” However, knowledge-workers demand “knowledge at your fingertips” as opposed to mere “information at your fingertips.” Unfortunately, today's knowledge-workers use the Web to browse and search for documents—compilations of data and information—rather than actual knowledge relevant to their inquiry. To achieve improved knowledge requires providing proper context, meaning and efficient access to data and information, all of which are missing with the traditional Web.
Efforts have been made to achieve the goal of “knowledge at your fingertips.” One example is a new concept for information organization and distribution referred to as the Semantic Web. The Semantic Web is an extension of the current Web in which information is given well-defined meaning, better enabling computers and people to work in cooperation. While conceptually a significant step forward in supporting improved context, meaning and access of information on the Internet, the Semantic Web has yet to find successful implementation that lives up to its stated potential.
Both the current Web and the Semantic Web fail to provide proper context, meaning and efficient access to data and information to allow users to acquire actionable knowledge. This is partially a problem related to the ways in which Today's Web and the contemplated Semantic Web are structured or, in other words, related to their technology layers. As shown in
In addition, various properties must be present in a comprehensive information management system to provide an integrated and seamless implementation framework and resulting medium for knowledge retrieval, management and delivery. A non-exhaustive list of these properties include: Semantics/Meaning; Context-Sensitivity; Time-Sensitivity; Automatic and intelligent Discoverability; Dynamic Linking; User-Controlled Navigation and Browsing; Non-HTML and Local Document Participation in the Network; Flexible Presentation that Smartly Conveys the Semantics of the Information being Displayed; Logic, Inference, and Reasoning; Flexible User-Driven Information Analysis; Flexible Semantic Queries; Read/Write Support; Annotations; “Web of Trust”; Information Packages (“Blenders”); Context Templates, and User-Oriented Information Aggregation. Each of these properties will be discussed below in the context of their application to both Today's Web and the Semantic Web.
Today's Web lacks semantics as an intrinsic part of the platform and user experience. Web pages convey only textual and graphical data rather than the semantics of the data they contain. As a result, users cannot issue semantic queries such as those that one might expect with natural language—for example, “find me all books less than hundred pages long, about Latin Jazz, and published in the last five years.” To be able to process such a query, a Web site or search engine must “know” it contains books and must be able to intelligently filter its contents based on the semantics of the query request. Such a query is not possible on the Web today. Instead, users are forced to rely on text-based searches. These searches usually result in information overload or information loss because the user is forced to pick search terms that might not match the text in the information base. In the aforementioned example, a user might pick the search term “Books Latin Jazz” and hope that the search engine can make the connection. The user is usually then left to independently filter the search results. This sort of text-based search also implies that terms that might convey the same meaning. In the above example, results from search terms such as “Books on South or Central American Jazz” or “Publications on Jazz from Latino Lands” might be ignored during the processing of the search query.
The lack of semantics also implies that Today's Web does not allow users to navigate based on they way humans think. For example, one might want to navigate a corporate intranet using the organizational structure. For example, from people to the documents they create to the experts on that documents to the direct reports of those experts to the distribution lists the direct reports are members of to the members of the distribution lists to the documents those members created, etc. This “web” is semantic and is based on actual information classification (“things”) and not just “pages” as Today's Web is.
The lack of semantics also has other implications. First, it means that the Web is not programmable. With semantics, the Web can be consumed by Smart Agents that can make sense of the pages and the links and then make inferences, recommendations, etc. With Today's Web, the only “Agent” that can make inferences is the human brain. As such, the Web does not employ the enormous processing power that computers are capable of—because it is not represented in a way that computers can understand.
The lack of semantics also implies that information is not actionable. A search engine does not “understand” the results it spits out. As such, once a user receives search results, he or she is “on his or her own.” Also, a web browser does not “understand” the information it is displaying and as such cannot do smart things with the information. With semantics in place, a smart display, for example, will “know” that an event is an event and might do interesting things like check if the event is already in the user's calendar, display free/busy information, or allow the user to automatically insert the event into his/her calendar thereby making the information actionable. Information presented without semantics is not actionable or might require that the semantics be inferred, which might result in an unpleasant user experience.
The Semantic Web seeks to address semantics/meaning limitations with Today's Web by encoding information with well-defined semantics. Web pages on the Semantic Web include metadata and semantic links to other metadata, thereby allowing search engines to perform more intelligent and accurate searches. In addition, the Semantic Web includes ontologies that will be employed for knowledge representation, thereby allowing a semantic search engine to interpret terms based on meaning and not merely on text. For example, in the previous example, Latin Jazz ontology might be employed on a Semantic Web site and would allow a search engine on the site to “know” that the terms “Books on South or Central American Jazz” or “Publications on Jazz from Latino Lands” have the same meaning as the term “Books on Latin Jazz.” While conceptually overcoming many of the deficiencies with Today's Web, there has not to date been a successful implementation of a well-defined data model providing context and meaning, including in particular the necessary semantic links, ontologies, etc. to provide for additional characteristics such as context-sensitivity and time-sensitivity.
Today's Web lacks context-sensitivity. The implication of a lack of context is that Today's Web is not personal. For example, documents in accessible storage are independently static and therefore stupid. Information relevant to the subject matter of the document has already been published, is being newly published, or will soon be published. Because the document in storage is static, however, there is no way to dynamically associate its subject matter with this relevant information in real-time. Stated differently, users have no way to dynamically connect their private context with external information in real-time. Information sources (such as the document) that form context sit in their own islands, totally isolated from other relevant information sources. This results in information and productivity losses.
The primary reason for this is that Today's Web is a presentation-oriented medium designed to present views of information to a dumb client (e.g., remote computer). The client has virtually no role to play in the user experience, aside from merely displaying what the server tells it to display. Even in cases where there is client-side code (like Java applets and ActiveX controls), the controls usually do one specific thing and do not have coordinated action with the remote server such that code on the client is being orchestrated with code on the server.
From a productivity standpoint, the implication of this is that knowledge-workers and information consumers are totally at the mercy of information authors. Today, knowledge-workers have portals that are maintained and updated to provide custom views of corporate information, external data, etc. However, this is still very limiting because knowledge-workers are completely helpless if nothing dynamically and intelligently connects relevant information in the context of their task with information that users have access to.
If a knowledge-worker does not see a link to a relevant piece of information on his of her portal, of if a friend or colleague does not email him or her the link, the information gets dropped; information does not connect with or adapt to the user context or the context in which it is displayed. Likewise, it is not enough to just notify a user that new data for an entire portal is available and shove it down to their local hard drive. It lacks a customizable presentation with context sensitive alert notifications.
The Semantic Web suffers from the same limitations as Today's Web when it comes to context-sensitivity. On the Semantic Web, users are likewise at the mercy of information authors. The Semantic Web itself will be authored, but the authoring will include semantics. As a result, users are still largely on their own to locate and evaluate the relevance of available information. The Semantic Web, as a standalone entity, will not be able to make these dynamic connections with other information sources.
Today's Web lacks time-sensitivity. The Web platform (e.g., browser) is a dumb piece of software that merely presents information, without any regard to the time-sensitivity of the information. The user is left to infer time sensitivity or do without it. This results in a huge loss in productivity because the Web platform cannot make time-sensitive connections in real-time. While some Web sites focus on presenting time-sensitive information, for example, by indexing information past a predetermined date, the Web browser itself has no notion of time-sensitivity. Instead, it is left to individual Web sites to include time-sensitivity in the information they display in their own island. In other words, there is no axis of time on a Web link.
The Semantic Web, like Today's Web, also does not address time-sensitivity. A Semantic Web can have semantic links that do not internalize time. This is largely because the Semantic Web implicitly has no notion of software Web services that address context and time-sensitivity.
Today's Web lacks automatic and intelligent discoverability of newly created information. There is currently no way to know what Web sites started anew today or yesterday. Unless the user is notified or the user serendipitously discovers a new site when he or she does a search, he or she might not have any clue as to whether there are any new Web sites or pages. The same problem exists in enterprises. On Intranets, knowledge-workers have no way of knowing when new Web sites come up unless informed via some external means. The Web platform itself has no notion of announcements or discovery. In addition, there is no context-sensitive discovery to determine new sites or pages within the context of the user's task or current information space.
The Semantic Web, like Today's Web, does not address the lack of automatic discoverability. Semantic Web sites suffer from the same problem—users either will have to find out about the existence of new information sources from external sources or through personal discovery when they perform a search.
Today's Web employs a pure network or graph “data structure” for its information model. Each Web page represents a node in the network and each page can contain links to other nodes in the network. Each link is manually authored into each page. This has several problems. First, it means that the network needs to be maintained for it to have continuous value. If Web pages are not updated or if Web page or site authors do not have the discipline to add links to their pages based on relevance, the network loses value. Today's Web is essentially prone to having dead links, old links, etc. Another problem with a pure network or graph information model is that the information consumer is at the mercy of—rather than in control of—the presentation of the Web page or site. In other words, if a Web page or site does not contain any links, the user has no recourse to find relevant information. Search engines are of little help because they merely return pages or nodes into the network. The network itself does not have any independent or dynamic linking ability. Thus, a search engine can easily return links to Web pages that themselves have no links or dead, stale or irrelevant links. Once users obtain search results, they are on their own and are completely at the mercy of whether the author of the returned pages inserted relevant, time-sensitive links into the page.
The Semantic Web suffers from the same problem as Today's Web because the Semantic Web is merely Today's Web plus semantics. Even though users will be able to navigate the network semantically (which they cannot currently do with the Web), they will still be at the mercy of how the information has been authored. In other words, the Semantic Web is also dependent on the discipline of the authors and hence suffers from the same aforementioned problems of Today's Web. If the Semantic Web includes pages with ontologies and metadata, but those pages are not well maintained or do not include links to other relevant sources, the user will still be unable to obtain current links and other information. The Semantic Web, as currently contemplated, will not be a smart, dynamic, self-authoring, self-healing network.
With Today's Web, the user has no control over the navigation and browsing experience, but rather is completely at the mercy of a Web page and how it is authored with links (if any). As shown with reference to prior art
The Semantic Web suffers from a similar problem as Today's Web in that there is no user-controlled browsing. Instead, as shown with reference to prior art
Another problem with Today's Web is the requirement that only documents that are authored as HTML can participate in the Web, in addition to the fact that those documents have to contain links. The implication is that other information objects like non-HTML documents (e.g., PDF, Microsoft Word, PowerPoint, and Excel documents, etc.)—especially those on users” hard drives—are excluded from the benefits of linking to other objects in the network. This is very limiting, especially since there might be semantic relevance between information objects that are not HTML and which do not contain links.
Furthermore, search engines do not return results for the entire universe of information since vast amount of content available on the web is inaccessible to standard web crawlers. This includes, for example, content stored in databases, unindexed file repositories, subscription sites, local machines and devices, proprietary file formats (such as Microsoft Office documents and email), and non-text multimedia files. These form a vast constellation of inaccessible matter on the Internet, referred to as “the invisible Intranet” inside corporations. Today's Web servers do not provide web crawler tools that address this problem.
The Semantic Web also suffers from this limitation. It does not address the millions of non-HTML documents that are already out there, especially those on users” hard drives. The implication is that documents that do not have RDF metadata equivalents or proxies cannot be dynamically linked to the network.
Flexible Presentation that Smartly Conveys the Semantics of the Information being Displayed
Today's Web does not allow users to customize or “skin” a Web site or page. This is because Today's Web servers return information that is already formatted for presentation by the browser. The end user has no flexibility in choosing the best means of displaying the information—based on different criteria (e.g., the type of information, the available amount of real estate, etc.)
The Semantic Web does not address the issue of flexible presentation. While a semantic Web site conceptually employs RDF and ontologies, it still sends HTML to the browser. Essentially, the Semantic Web does not provide for specific user empowerment for presentation. As such, a Semantic Web site, viewed by Today's Web platform, will still not empower the user with flexible presentation. Moreover, despite industry movement towards XML, only a new platform can dictate that data will be separated from presentation and define guidelines for making the data programmable. Authors building content for the Semantic Web either return XML and avoid issues with presentation entirely, or focus their efforts on a single presentation style (vertical industry scenario) for rendering. Neither approach allows the Semantic Web to achieve an optimum degree of knowledge distribution.
Because Today's Web does not have any semantics, metadata, or knowledge representation, computers cannot process Web pages using logic and inference to infer new links, issue notifications, etc. Today's Web was designed and built for human consumption, not for computer consumption. As such, Today's Web cannot operate on the information fabric without resorting to brittle, unreliable techniques such as screen scraping to try to extract metadata and apply logic and inference.
While the Semantic Web conceptually uses metadata and meaning to provide Web pages and sites with encoded information that can be processed by computers, there is no current implementation that is able to successfully achieve this computer processing and which illustrates new or improved scenarios that benefit the information consumer or producer.
Today's Web lacks user-driven information analysis. Today's Web does not allow users to display different “views” of the links, using different filters and conditions. For example, Web search engines do not allow users to test the results of searches under different scenarios. Users cannot view results using different pivots such as information type (e.g., documents, email, etc.), context (e.g., “Headlines,” “Best Bets,” etc.), category (e.g., “wireless,” “technology,” etc.) etc.
While providing a greater degree of flexible information analysis, the Semantic Web does not describe how the presentation layer can interact with the Web itself in an interactive fashion to provide flexible analysis.
Today's Web only allows text-based queries or queries that are tied to the schema of a particular Web site. These queries lack flexibility. Today's Web does not allow a user to issue queries that approximate natural language or incorporate semantics and local context. For example, a query such as “Find me all email messages written by my boss or anyone in research and which relate to this specification on my hard disk” is not possible with Today's Web.
By employing metadata and ontologies, the conceptual Semantic Web allows a user to issue more flexible queries than Today's Web. For example, users will be able to issue a query such as “Find me all email messages written by my boss or anyone in research.” However, users will not be able to incorporate local context. In addition, the Semantic Web does not define an easy manner with which users will query the Web without using natural language. Natural language technology is an option but is far from being a reliable technology. As such, a query user interface that approximates natural language yet does not rely on natural language is required. The Semantic Web does not address this.
Today's Web is a read-only Web. For example, if users encounter a dead link (e.g., via the “404” error), they cannot “fix” the link by pointing it to an updated target that might be known to the user. This can be limiting, especially in cases where users might have important knowledge to be shared with others and where users might want to have input as to how the network should be represented and evolve.
While the Semantic Web conceptually allows for read/write scenarios as provided by independent participating applications, there is no current implementation that provides this ability.
Today's Web has no implicit support for annotations. And while some specific Web sites support annotations, they do so in a very restricted and self-contained way. Today's Web medium itself does not address annotations. In other words, it is not possible for users to annotate any link with their comments or additional information that they have access to. This results in potential information loss.
While the Semantic Web conceptually allows for annotations to be built into the system subject to security constraints, there is no current implementation that provides this ability.
Today's Web lacks seamless integration of authentication, access control, and authorization into the Web, or what has been referred to as a “Web of Trust.” With a Web of Trust, for example, users are able to make assertions, fix and update links to the Web and have access control restrictions built in for such operations. On Today's Web, this lack of trust also means that Web services remain independent islands that must implement a proprietary user subscription authorization, access control or payment system. Grand schemes for centralizing this information on 3rd party servers meet with consumer and vendor distrust because of privacy concerns. To gain access to rich content, asset users must log in individually and provide identity information at each site.
While the Semantic Web conceptually allows for a Web of Trust, there is no current implementation that provides for this ability.
Neither Today's Web nor the Semantic Web allows users to deal with related semantic information as a whole unit by combining characteristics of potentially divergent semantic information to produce overlapping results (for example, like creating a custom, personal newspaper or TV channel).
Neither Today's Web nor the Semantic Web allows users to independently create and map to specific and familiar semantic models for information access and retrieval.
Today's Web lacks support for user-oriented information aggregation. The user can only access one Web site or one search engine at a time, within the context of one browsing session. As such, even if there is context or time-sensitive information on other information sources that relate to the information that the user is currently viewing, those sources cannot be presented in a holistic fashion in the current context of the user's task.
The Semantic Web also suffers from a lack of user-oriented information aggregation. The medium itself is an extension of Today's Web. As such, users will still access one site or one search engine at a time and will not be able to aggregate information across information repositories in a context or time-sensitive manner.
Given the growing demand for “knowledge at your fingertips” as well as the deficiencies in Today's Web and the conceptual Semantic Web, many of which are noted above, there is a need for a new and comprehensive system and method of knowledge retrieval, management and delivery. Preferred embodiments of the present invention are directed in part to a semantically integrated knowledge retrieval, management, delivery and/or presentation system. Preferred embodiments of the present invention and system include several additional improved features, enhancements and/or properties, including, without limitation, semantic advertisements, spider RSS integration, pivot views, watch lists, context extraction methods, context ranking methods, client duplication management methods, a server data and index model, improved metadata indexing methods, adaptive ranking methods, and content transformation methods.
The explosive growth of digital information is increasingly impeding knowledge-worker productivity due to information overload. Online information is virtually doubling every year and/or most of that information is unstructured—usually in the form of text. Traditional search engines have been unable to keep up with the pace of information growth primarily because they lack the intelligence to “understand,” semantically process, mine, infer, connect, and/or contextually interpret information in order to transform it to—and/or expose it as—knowledge. Furthermore, end-users want a simple yet powerful user-interface that allows them to flexibly express their context and/or intent and/or be able to “ask” natural questions on the one hand, but which also has the power to guide them to answers for questions they wouldn't know to ask in the first place. Today's search interfaces, while easy-to-use, do not provide such power and/or flexibility.
Now that the Web has reached critical mass, the primary problem in information management has evolved from one of access to one of intelligent retrieval and/or filtering. Computer users are now faced with too much information, in various formats and/or via multiple applications, with little or no help in transforming that information into useful knowledge.
Search engines such as Google™ provide some help in filtering information by indexing content based on keywords. Google™, in particular, has gone a step further by mining the hypertext links in Web pages in order to draw inferences of relevance based on page popularity. These techniques, while helpful, are far from sufficient and/or still leave end-users with little help in separating wheat from chaff. The primary reason for this is that current search engines do not truly “understand” what they index or what users want. Keywords are very poor approximations of meaning and/or user intent. Furthermore, popularity, while useful, is no guarantee of relevance: Popular garbage is still garbage.
Furthermore, knowledge has multiple axes, and/or search is only one of those axes. Knowledge-workers also wish to discover information they might not know they need ahead of time, share information with others (especially those that have similar interests), annotate information in order to provide commentary, and/or have information presented to them in a way that is contextual, intuitive, and/or dynamic—allowing for further (and/or potentially endless) exploration and/or navigation based on their context. Even within the search axis, there are multiple sub-axes, for instance, based on time-sensitivity, semantic-sensitivity, popularity, quality, brand, trust, etc. The axis of choice depends on the scenario at hand.
Search engines are appropriately named because they focus on search. However, merely improving search quality without reformulating the core goal of search will leave the information overload problem unaddressed.
The preferred and alternative embodiments of the present invention are described in detail below with reference to the following drawings.
The preferred and alternative embodiments of the present invention are described in detail below with reference to the following drawings.
The Appendix attached hereto and referenced herein is incorporated by reference. This Appendix includes exemplar code illustrating a preferred embodiment of the present invention.
The present invention is directed in part to an integrated and seamless implementation framework and resulting medium for knowledge retrieval, management, delivery and presentation. The system includes a server comprised of several components that work together to provide context and time-sensitive semantic information retrieval services to clients operating a presentation platform via a communication medium. The server includes a first server component that is responsible for adding and maintaining domain-specific semantic information or intelligence. The first server component preferably includes structure or methodology directed to providing the following: a Semantic Network, a Semantic Data Gatherer, a Semantic Network Consistency Checker, an Inference Engine, a Semantic Query Processor, a Natural Language Parser, an Email Knowledge Agent and a Knowledge Domain Manager. The server includes a second server component that hosts domain-specific information that is used to classify and categorize semantic information. The first and second server components work together and may be physically integrated or separate.
Within the system, all objects or events in a given hierarchy are active Agents semantically related to each other and representing queries (comprised of underlying action code) that return data objects for presentation to the client according to a predetermined and customizable theme or “Skin.” This system provides various means for the client to customize and “blend” Agents and the underlying related queries to optimize the presentation of the resulting information.
The end-to-end system architecture of the present invention provides multiple client access means of communication between diverse knowledge information sources via an independent Semantic Web platform or via a traditional Web portal (e.g., Today's Web access browser) as modified by the present invention providing additional SDK layers that enable programmatic integration with a custom client.
The methodology of the present invention is directed in part to the operational aspects of the entire system, including the retrieval, management, delivery and presentation of knowledge. This preferably includes securing information from information sources, semantically linking the information from the information sources, maintaining the semantic attributes of the body of semantically linked information, delivering requested semantic information based upon user queries and presenting semantic information according to customizable user preferences. Alternative embodiments of the methodology of the present invention are directed to the operation of Agents representing queries that are used with server-side and client-side applications to enable efficient, inferential-based queries producing semantically relevant information.
The present invention is directed in part to a semantically integrated knowledge retrieval, management, delivery and presentation system, as is more fully described in my co-pending parent application (U.S. application Ser. No. 10/179,651 filed Jun. 24, 2002). The present invention and system includes several additional improved features, enhancements and/or properties, including, without limitation, Entities, Profiles and Semantic Threads, as are more fully described in the Detailed Description below.
ActionScript. Scripting language of Macromedia Flash. This two-way communication assists users in creating interactive movies. See also http://www.macromedia.com/support/flash/action_scripts/actionscript_tutorial/.
Agency. A named instance of a Knowledge Integration Server (KIS) that is the semantic equivalent of a website.
Agency Directory. A directory that stores metadata information for Agencies and allows clients to add, remove, search, and browse Agencies stored within. Agencies can be published on directories like LDAP or the Microsoft Active Directory. Agencies can also be published on a proprietary directory built specifically for Agencies.
Agent. A semantic filter query that returns XML information for a particular semantic object type (e.g., documents, email, people, etc.), context (e.g., Headlines, Conversations, etc.) or Blender.
Agent Discovery. The property of the information medium of the present invention that allows users to easily and automatically discover new server-side Agents or client-side Agents created by others (friends or colleagues). Also see “Discoverability.”
Annotations. Notes, comments, or explanations that are used to add personal context to an information object. In the preferred embodiment, annotations are email messages that are linked to the object they qualify, and which can have attachments (just like regular email messages). In addition, annotations are first class information objects in the system and as such can be annotated themselves, thereby resulting in threaded annotations or a tree of annotations with the initial object as the root.
Application Programming Interface (API). Defines how software programmers utilize a particular computer feature. APIs exist for windowing systems, file systems, database systems, networking systems, and other systems.
Calendar Access Protocol (CAP). Internet protocol that permits users to digitally access a calendar store based on the iCalendar standard.
Compound Agent Manager™. Trademarked name for an Agency component that programmatically allows the user to create and delete Compound Agents and to manage them by adding and deleting Agents.
Context. Information surrounding a particular item that provides meaning and otherwise assists the information consumer in interpreting the item as well as finding other relevant information related to the item.
Context Results Pane. A Results Pane that displays results for context-based queries. These include results for Context Palettes, Smart Lenses, Deep Information, etc. See “Results Pane.”
Context-Sensitivity. The property of an information medium that enables it to intelligently and dynamically perceive the context of all the information it presents and to present additional, relevant information given that context. A context-sensitive system or medium understands the semantics of the information it presents and provide appropriate behaviors (proactive and reactive based on the user's actions) in order to present information in its proper context (both intrinsically and relationally).
Context Template™. Trademarked name for scenario-driven information query templates that map to specific and familiar semantic models for information access and retrieval. For example, a “Headlines” template in the preferred embodiment has parameters that are consistent with the delivery of “Headlines” (where freshness and the likelihood of a high interest level are the primary axes for retrieval). An “Upcoming Events” template has parameters that are consistent with the delivery of “Upcoming Events.” And so on. Essentially, Context Templates can be analogized to personal, digital semantic information retrieval “channels” that deliver information to the user by employing a well-known semantic template.
Deep Information™. Trademarked name for a feature of the present invention that enables the Information Agent to display intrinsic, contextual information relating to an information object. The contextual information that includes information that is mined from the Semantic Network of the Agency from whence the object came.
Discoverability. The ability of the information medium of the present invention to intelligently and proactively make information known or visible to the user without the user having to explicitly look for the information.
Domain Agent Wizard™. Trademarked name for a system component and its user interface for allowing the Agency administrator to create and manage Domain Agents.
DOTNET (.NET). Microsoft® .NET is a set of Microsoft software technologies for connecting information, people, systems, and devices. It enables software integration through the use of XML Web Services: small, discrete, building-block applications that connect to each other, as well as to other, larger applications, via the Internet. .NET-connected software facilitates the creation and integration of XML Web Services. See http://www.microsoft. com/net/defined/default.asp).
Dynamic Linking™. Trademarked name for the ability of the Information Nervous System of the present invention to allow users to link information dynamically, semantically, and at the speed of thought, even if those information items do not contain links themselves. By virtue of employing smart objects that have intrinsic behavior and using recursive intelligence embedded in the Information Agency's XML Web Service, each node in the Semantic Network is much smarter than a regular link or node on Today's Web or the conceptual Semantic Web. In other words, each node in the Smart Virtual Network or Web of the present invention can link to other nodes, independent of authoring. Each node has behavior that can dynamically link to Agencies and Smart Agents via drag and drop and smart copy and paste, create links to Agencies in the Semantic Environment, respond to lens requests from Smart Agents to create new links, include intrinsic alerts that will dynamically create links to context and time-sensitive information on its Agency, include presentation hints for breaking news (wherein the node can automatically link to breaking news Agents in the namespace), form the basis for deep info that can allow the user to find new links, etc. A user of the present invention is therefore not at the mercy of the author of the metadata. Once the user reaches a node in the network, the user has many semantic means of navigating dynamically and automatically—using context, time, relatedness to Smart Agencies and Agents, etc.
Email XML Object. An information object with the “Email” information object type. The XML object has the “Email” SRML schema (which uses XML).
Environment Browser. See Information Agent.
Favorite Agents Manager™. Trademarked name for a system component and user interface element that allows the Agency administrator to manage server-side Favorite Agents.
Flash. Macromedia Flash user interface platform that enables developers and content authors to embed sophisticated graphics and animations in their content. See http://www.macromedia.com/flash.
Flash MX. Macromedia Flash MX is a text, graphics, and animation design and development environment for creating a broad range of high-impact content and rich applications for the Internet. See http://www.macromedia.com/software/flash/productinfo/product_overview/.
Global Agency Directory™. Trademarked name for an instance of an Agency Directory that runs on the Internet (or other global network). The Global Agency Directory allows users to find, search, and browse Internet-based Agencies using their Information Agent (directly in their semantic environment). Also, see “Agency Directory.”
HTTP. Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia information systems. It is a generic, stateless, protocol that can be used for many tasks beyond its use for hypertext, such as name servers and distributed object management systems, through extension of its request methods, error codes and headers. A feature of HTTP is the typing and negotiation of data representation, allowing systems to be built independently of the data being transferred. See http://www.w3.org/Protocols/ and http://www.w3 .org/Protocols/Specs.html.
Inference Engine™. Trademarked name for the methodology of the present invention that observes patterns and data to arrive at relevant and logically sound conclusions by reasoning. Preferably utilizes Inference Rules (a predetermined set of heuristics) to add semantic links to the Semantic Network of the present invention.
Information. A quantitative or qualitative measure of the relevance and intelligence of content or data and which conveys knowledge.
Information Agent™. Trademarked name for the semantic client or browser of the present invention that provides context and time-sensitive delivery and presentment of actionable information (or knowledge) from multiple sources, information types, and templates, and which allows dynamic linking of information across various repositories.
Information Nervous System™. Trademarked name for the dynamic, self-authoring, context and time-sensitive information system of the present invention that enables users to intelligently and dynamically link information at the speed of thought, and with context and time-sensitivity, in order to maximize the acquisition and use of knowledge for the task at hand.
Information Object™ (or Item or Packet). Trademarked name for a unit of information of a particular type and which conveys knowledge in a given context.
Information Object Pivot™. Trademarked name for an information object that users employ as a navigational pivot to find other relevant information in the same context.
Information Object Type. See Object Type.
Intelligent Agent. Software Agents that act on behalf of the user to find and filter information, negotiate for services, easily automate complex tasks, or collaborate with other software Agents to solve complex problems. By definition, Intelligent Agents must be autonomous or, in other words, freely able to execute without user intervention. Additionally, Intelligent Agents must be able to communicate with other software or human Agents and must have the ability to perceive and monitor the environment in which they reside. See http://www.findarticles.com/cf_dls/m
Internet Calendaring and Scheduling (iCalendar). Protocol that enables the deployment of interoperable calendaring and scheduling services for the Internet. The protocol provides the definition of a common format for openly exchanging calendaring and scheduling information across the Internet.
Internet Message Access Protocol (IMAP). Communications mechanism for mail clients to interact with mail servers, and manipulate mailboxes thereon. Perhaps the most popular mail access protocol currently is the Post Office Protocol (POP), which also addresses remote mail access needs. IMAP offers a superset of POP features, which allow much more complex interactions and provides for much more efficient access than the POP model. See http://www-smi.stanford.edu/projects/imap/m1/imap.html.
Intrinsic Semantic Link™. Trademarked name for semantic links that are intrinsic to the schema of a particular information object. For instance, an email information object has intrinsic links like “from,” “to,” “cc,” “bcc,” and “attachments” that are native to the object itself and are defined in the schema for the email information object type.
Island. An information repository that is isolated from other repositories which may contain relevant, semantically related, context and time-sensitive information but which are disconnected from other contexts in which such information might be relevant.
J2EE. The Java™ 2 Platform, Enterprise Edition (J2EE) used for developing multi-tier enterprise applications. J2EE bases enterprise applications on standardized, modular components by providing a set of services to those components and by handling many details of application behavior automatically. See http://java.sun.com/j2ee/overview.html.
Knowledge. Information presented in a context and time-sensitive manner that enables the information consumer to learn from the information and apply the information in order to make smarter and more timely decisions for relevant tasks.
Knowledge Agent™. See Information Agent.
Knowledge Base Server™ (KBS). Trademarked name for a server that hosts knowledge for the Knowledge Integration Server (KIS).
Knowledge Domain Manager™ (KDM). Trademarked name for a component of the Knowledge Integration Server that is responsible for adding and maintaining domain-specific intelligence on the Semantic Network.
Knowledge Integration Server™ (KIS). Trademarked name for a server that semantically integrates data from multiple diverse sources into a Semantic Network, which can also host server-side Agents that provide access to the network and which hosts XML Web Services that provide context and time-sensitive access to knowledge on the server.
Knowledge Web™. See Information Nervous System.
Liberty Alliance. The vision of the Liberty Alliance is to enable a networked world in which individuals and businesses can more easily conduct transactions while protecting the privacy and security of vital identity information. To accomplish its vision, the Liberty Alliance seeks to establish an open standard for federated network identity through open technical specifications. See http://www.projectliberty.org/index.html.
Lightweight Directory Access Protocol (LDAP). Technology for accessing common directory information. LDAP has been embraced and implemented in most network-oriented middleware. As an open, vendor-neutral standard, LDAP provides an extendable architecture for centralized storage and management of information that needs to be available for today's distributed systems and services. LDAP is currently supported in most network operating systems, groupware and even shrink-wrapped network applications. See http://publib-b.boulder.ibm.com/Redbooks.nsf/RedbookAbstracts/sg244986.html? Open.
Link Template™. See Context Template.
Local Context. Local Context refers to client-side information objects and Agents accessible to the users. This includes Agents in the Semantic Environment, local files, folders, email items in users' email inboxes, users' favorite and recent Web pages, the current Web page(s), currently opened documents, and other information objects that represent users' current task, location, time, or condition.
Meaning. The attributes of behavior of information that allows the consumer of the information to locate and navigate to it based on its relevant information content (as opposed to its text or data) and to act on it in a context and time-sensitive manner, in order to maximize the utility of the information.
Metadata. “Data about data.” It includes those data fields, links, and attributes that fully describe an information object.
Natural Language Parser. Parsing and interpreting software component that understands natural language queries and can translate them to structured semantic information queries.
Nervana™. Trademarked name for a proprietary, end-to-end implementation of the Information Nervous System information medium/platform. The name also defines a proprietary namespace for resource type and predicate name qualifiers.
Network Effects. This exists when the number of other users affects the value of a product or service to a particular user. Telephone service provides a clear example. The value of telephone service to users is a function of the number of other subscribers. Few would be interested in telephones that were not connected to anyone, and most would assess higher value to a phone service linked to a national network rather than just a local network. Similarly, many computer users prize a computer system that allows them to exchange information readily with other users.
Network Effects are thus demand-side externalities that generate a positive feedback effect in which successful products become more successful. In this way, Network Effects are analogous to supply-side economies of scale and scope. As a firm increases output, economies of scale lead to lower average costs, permitting the firm to lower prices and gain additional business from rivals. Continued expansion results in even lower average costs, justifying even lower prices. Similarly, the positive feedback from Network Effects builds upon previous successes. In the computer industry, for example, users pay more for a more popular computer system, all else equal, or opt for a system with a larger installed base if the prices and other features of two competing systems are equivalent. See http://www.ei.com/publications/1996/fall1.htm.
Network News Transfer Protocol (NNTP). Protocol for the distribution, inquiry, retrieval, and posting of news articles using a reliable stream-based transmission of news among the ARPA-Intemet community. NNTP is designed so that news articles are stored in a central database allowing subscribers to select only those items they wish to read. Indexing, cross-referencing, and expiration of aged messages are also provided.
Notifications. Notifications are alerts that are sent by the Information Agent or an Agency to indicate to a user that there is new information on an Agent (either a client-side Agent or a server-side Agent). Users can request notifications from Agents in their Semantic Environment. Users can indicate that they have received the notification. The notification source (the client or server) stores information for the user and the Agent indicating the last time the user acknowledged a notification for the Agent. The notification source polls the Agent to check if there is new information since the last acknowledge time. If there is, the notification source alerts the user. Alerts can be sent via email, pager, voice, or a custom alert mechanism such as Microsoft's .NET Alerts service. Users have the option of indicating their preferred notification mechanism for the entire notification source (client or server)—which applies to all Agents on the notification source—on a per-Agent basis (which overrides the indicated preference on the notification source.
Object. See Information Object.
Object Type. Identification data associated with information that allows the consumer to understand the nature of the information, to interpret its contents, to predict how the information can be acted upon, and to link it to other relevant information items based on how the object types typically relate in the real world. Examples include documents, events, email messages, people, etc.
Ontology. Hierarchical structuring of knowledge according to essential qualities. Ontology is an explicit specification of a conceptualization. The term is borrowed from philosophy, where “Ontology” is a systematic account of Existence. For artificial intelligence systems, what “exists” is that which can be represented. When the knowledge of a domain is represented in a declarative formalism, the set of objects that can be represented is called the universe of discourse. This set of objects, and the describable relationships among them, are reflected in the representational vocabulary with which a knowledge-based program represents knowledge. Thus, in the context of artificial intelligence, the ontology of a program is described by defining a set of representational terms. In such ontology, definitions associate the names of entities in the universe of discourse (e.g., classes, relations, functions, or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. Formally, ontology is the statement of a logical theory.
The subject of ontology is the study of the categories of things that exist or may exist in some domain. The product of such a study, called ontology, is a catalog of the types of things that are assumed to exist in a domain of interest D from the perspective of a person who uses a language L for the purpose of talking about D. The types in the ontology represent the predicates, word senses, or concept and relation types of the language L when used to discuss topics in the domain D. See, generally, http://www-ks1.stanford.edu/kst/what-is-an-ontology.html and http://users.bestweb.net/˜sowa/ontology/).
Predicates. A Predicate is an attribute or link whose result represents the truth or falsehood of some condition. For example, the predicate “authored by” links a person with an information object and indicates whether a person authored the object.
Presenter™. System component in the Information Agent (semantic browser) of the present invention that handles the aggregation and presentation of results from the semantic query processor (that preferably interprets SQML). The Presenter handles layout management, aggregation, navigation, Skin management, the presentation of Context Palettes, interactivity, animations, etc.
RDF. Resource Description Framework (RDF) is a foundation for processing metadata; it provides interoperability between applications that exchange machine-understandable information on the Web. RDF emphasizes facilities to enable automated processing of Web resources. RDF defines a simple model for describing relationships among resources in terms of named properties and values. RDF properties may be thought of as attributes of resources and in this sense correspond to traditional attribute-value pairs. RDF properties also represent relationships between resources. As such, the RDF data model can therefore resemble an entity-relationship diagram.
RDF can be used in a variety of application areas including, for example: in resource discovery to provide better search engine capabilities, in cataloging for describing the content and content relationships available at a particular Web site, page, or digital library, by intelligent software Agents to facilitate knowledge sharing and exchange, in content rating, in describing collections of pages that represent a single logical “document”, for describing intellectual property rights of Web pages, and for expressing the privacy preferences of a user as well as the privacy policies of a Web site. RDF with digital signatures is preferably a component of building the “Web of Trust” for electronic commerce, collaboration, and other applications. See, generally, http://www.w3.org/TR/PR-rdf-syntax/ and http://www.w3.org/TR/rdf-schema/.
RDFS. Acronym for RDF Schema. Resource description communities require the ability to say certain things about certain kinds of resources. For describing bibliographic resources, for example, descriptive attributes including “author”, “title”, and “subject” are common. For digital certification, attributes such as “checksum” and “authorization” are often required. The declaration of these properties (attributes) and their corresponding semantics are defined in the context of RDF as an RDF schema. A schema defines not only the properties of the resource (e.g., title, author, subject, size, color, etc.) but may also define the kinds of resources being described (books, Web pages, people, companies, etc.). See http://www.w3.org/TR/rdf-schema/).
Results Pane™. Trademarked name for the graphical display area within the Information Agent (semantic browser) that displays results of an SQML query. For example, an optional player control/navigation/filter toolbar, a “Server-Side Agents Dialog” can allow users to browse and open server-side Agents, and display sample results with the “Documents” information object type from the server-side Agent.
Semantics. Connotative meaning.
Semantic Environment™. This refers to all the data stored on users' local machines, in addition to user-specific data on an Agency server (e.g., subscribed server-side Agencies, server-side Favorite Agents, etc.). Client-side state includes favorite and recent Agents and authentication and authorization information (e.g., user names and passwords for various Agencies), in addition to the SQML files and buffers for each client-side (user-created) Agent. The Information Agent is preferably configured to store Agents for a set amount of time before automatically deleting them, except those that have been added to the “favorites” list. For example, users may configure the Information Agent to store Agents for two weeks. In this case, Agents older than two weeks are automatically purged from the system and the Semantic Environment is adjusted accordingly. The Semantic Environment is employed for Context Palettes (Context Palettes use the Agencies in the “recent” and “favorites” list in order to predict what default Agencies users want to view context from).
Semantic Environment Manager™. Trademarked name for a software component that manages all the local state for the Semantic Environment (in the Information Agent). This includes storing and managing the metadata for all the client-side Agents (and the history and favorites Agent sub-lists), per-Agent state (e.g., Agent Skins, Agent preferences, etc.), notification management, Agency browsing (on Agency directories), listening for Agencies via multicast and peer-to-peer announcement protocols, services to allow users to browse the Semantic Environment via the semantic browser (via the Tree View, the “Open Agent” dialog, and the Results Pane), etc.
Semantic Data Gatherer™ (SDG). Trademarked name for XML Web Service used by the Knowledge Integration Server (KIS) and which is responsible for adding, removing and updating entries in the Semantic Network via the Semantic Metadata Store (SMS).
Semantic Metadata Store™ (SMS). Trademarked name for a software component on the KIS that employs a database (e.g., SQL Server, Oracle, DB2) having tables for each primary object type to store all the metadata on the KIS.
Semantic Network. System and method of linking objects associated with schemas together in a semantic way via the database tables on the Semantic Metadata Store.
Semantic Network Consistency Checker™. Trademarked name for a software component that runs on an Agency of the present invention that is tasked with maintaining the integrity and consistency of the Semantic Network. The checker runs periodically and ensures that entries in the “SemanticLinks” table exist in the native object tables, that entries in the “objects” table exist in the native object tables and that all entries in the Semantic Metadata Store still exist at the repositories from where they were gathered.
Semantic Queries. Queries that incorporate meaning, context, time-sensitivity, context-templates, and richness that approach natural language. Much more powerful than simple, keyword-based queries in that they are context and time-sensitive and incorporate meaning or semantics.
Semantic Query Markup Language (SQML). A proprietary XML-based query language used by this invention to define, store, interpret and execute client-side semantic queries. SQML includes tags to define a query that gets its data from diverse resources (that represent data sources) such as files, folders, application repositories, and references to Agency XML Web Services (via resource identifiers and URLs). In addition, SQML includes tags that enable semantic filtering (via custom links and predicates) which indicate how data is to be queried and filtered from the resources, and arguments that indicate how the resources are to be queried and how the results are to be filtered. In particular, the arguments can include references to local or remote context. The context arguments are then resolved by the client-side SQP at run-time to XML metadata. The XML metadata is then passed to the appropriate resource (e.g., an Agency's XML Web Service) as a method call along with the reference to the resource and the semantic links and predicates that indicate how the query is to be resolved by the resource (e.g., the Agency's XML Web Service). SQML is to the Information Nervous System as HTML is to Today's Web. The main difference is that SQML defines the rules for semantic querying while HTML defines the rules for Hypertext presentation. However, SQML is superior in that it enables the client to recursively create new semantic queries from existing ones (by creating new SQML with new links derived from an existing SQML query), e.g., via drag and drop and smart copy and paste, the Smart Lens, Context Templates and Palettes, etc. In addition, because SQML does not define the rules for presentation, the results of the semantic query can be presented in multiple ways, using a “skin” that takes the results (in SRML) to generate presentation based on the user's preferences, interests, condition, or context. Furthermore, SQML can contain abstract links and predicates such as those that refer to or employ Context Templates. The resource (e.g., the Agency's XML Web Service) then resolves the SQML to an appropriate query format (e.g., SQL or the equivalent in the case of an Agency's XML Web Service) and then invokes the “actual” query in order to generate the results (which will then account for the user's context or Context Template). Also, an SQML buffer or file can refer to multiple resources (and Agencies), thereby empowering the client to view results in an aggregated fashion (e.g., based on context or time-sensitivity), rather than based on the source of the data—this is a powerful feature of the invention that enables user-controlled browsing and information aggregation (see the sections on both below). Lastly, every client-side Agent has an SQML definition and file, just as every Web page has an HTML file.
Semantic Query Processor™ (SQP). Trademarked name for the server-side semantic query processor (XML Web Service in the preferred embodiment) that takes SQML and converts it to SQL (in the preferred embodiment) and then returns the results as XML. On the Knowledge Integration Server (KIS), the SQP is the main entry point to the Semantic Network of the present invention responsible for responding to semantic queries from clients of the KIS. On the server, this is the software component that processes semantic queries represented as SQML from the client. On the client, the client-side SQP takes aggregate SQML and compiles or maps it to individual SQML queries that can be sent to a server (or Agency) XML Web Service.
Semantic Results Markup Language (SRML). A proprietary XML-based data schema and format used by this invention to define, store, interpret and present semantic results. On the client, SRML is returned from the SQP via semantic resource handlers that interpret, format, and issue query requests to semantic data sources. Semantic data sources will include an Agency's XML Web Service, local files, local folders, custom data sources from local or remote applications (e.g., a Microsoft Outlook email application inbox), etc. The XML Web Service will return SRML to a client, in response to the client's semantic query. This way, the XML Web Service will not “care” how the results are being presented at the client. This is in contrast with Today's Web and the Semantic Web where servers return already-formatted HTML for a client to present and where clients merely present presentation data (as opposed to semantic data) and cannot customize the presentation of the data. In this invention, two clients can render the same SRML in completely different ways, based on the current “skin” that has been selected or applied by the user of either client. The “skin” then converts the SRML to a presentation-ready format such as XHTML, DHTML+TIME, SVG, Flash MX, etc.
SRML is a meta-schema, meaning that it is a container format that can include data for different information object types (e.g., documents, email, people, events, etc.). An SRML file or buffer can contain intertwined results for each of these object types. Well-formed SRML will contain well-formed XML document sections that are consistent with the schema of the information object types that are contained in the semantic result the SRML represents. See Sample A of the Appendix hereto.
Semantic Web. Extension of Today's Web in which information is given well-defined meaning, better enabling computers and people to work in cooperation. See Tim Berners-Lee, James Hendler, Ora Lassila, The Semantic Web, Scientific American, May 2000.
Facilities to put machine-understandable data on Today's Web are becoming a high priority for many communities. The Web can reach its full potential only if it becomes a place where data can be shared and processed by automated tools as well as by people. For the Web to scale, tomorrow's programs must be able to share and process data even when these programs have been designed totally independently. The Semantic Web is a conceptual vision: the idea of having data on the Web defined and linked in a way that it can be used by machines not just for display purposes, but for automation, integration and reuse of data across various applications. See also http://www.w3.org/2001/sw/.
Session Announcement Protocol (SAP). In order to assist the advertisement of multicast multimedia conferences and other multicast sessions, and to communicate the relevant session setup information to prospective participants, a distributed session directory may be used. An instance of such a session directory periodically multicasts packets containing a description of the session, and these advertisements are received by other session directories such that potential remote participants can use the session description to start the tools required to participate in the session.
In its simplest form, this involved periodically multicasting a session announcement packet describing a particular session. To receive SAP, a receiver simply listens on a well-known multicast address and port. Sessions are described using the Session Description Protocol (ftp://ftp.isi.edu/in-notes/rfc2327.txt). If a receiver receives a session announcement packet it simply decodes the SDP message, and then can display the session information for the user. The interval between repeats of the same session description message depends on the number of sessions being announced (each sender at a particular scope can hear the other senders in the same scope) such that the bandwidth being used for session announcements of a particular scope is kept approximately constant. If a receiver has been listening for a set time, and fails to hear a session announcement, then the receiver can conclude that the session has been deleted and no longer exists. The set period is based on the receivers' estimate of how often the sender should be sending.
See, generally, http://www.faqs.org/rfcs/rfc2974.html, http://www.video.ja.net/mice/archive/sdr_docs/node1.html, ftp://ftp.isi.edu/in-notes/rfc2327.txt.
Simple Mail Transfer Protocol (SMTP). Protocol designed to transfer mail reliably and efficiently. SMTP is independent of the particular transmission subsystem and requires only a reliable ordered data stream channel. An important feature of SMTP is its capability to relay mail across transport environments. See http://www.ietf.org/rfc/rfc0821.txt.
Skins. Presentation templates that are used to customize the user experience on a per-Agent basis or which customizes the presentation of the entire layout (independent of the Agent), or object (based on the information object type), context (based on the Context Template), Blender (for Agents that are Blenders), for the semantic domain name/path or ontology, and other considerations. Each Agent will include a Skin which in turn will have an XML metadata representation of parameters to customize the layout of the XML results that represent information objects (the layout Skin), for example, whether or not those results are animated, the manner in which each result is displayed, including a representation of the object type (the object Skin), styles, colors, graphics, filters, transforms, effects, animations (and so on) that indicate the ontology of the current results (the ontology Skin), styles that indicate the Context Template of the current results (the context Skin) and styles that indicate how to view and navigate results from Blenders (i.e., the Blender Skin)
Smart Lens™. Trademarked name for a proprietary feature of this invention that allows users to select a Smart Agent or an object as a context with which to view another object or Agent. The lens then displays metadata, links, and result previews that give users an indication of what they should expect if the context is invoked. Essentially, the Smart Lens displays the results of a “potential query.” The Smart Lens allows users to quickly preview context results without actually invoking queries (thereby increasing their productivity). In addition, the Smart Lens can display views that are consistent with the context, using pivots, templates and preview windows, thereby allowing users to analyze the context in different ways before invoking a query.
Smart Virtual Web™. Trademarked name for the property of the present invention to integrate semantics, context-sensitivity, time-sensitivity, and dynamism in order to empower users to browse a dynamic, virtual, “on-the-fly,” user-controlled “Web” that they control and can customize. This is in contrast with Today's Web and the conceptual Semantic Web, both of which employ a manually authored network wherein users are at the mercy of the authors of the information on the network.
Structured Query Language (SQL). Pronounced “ess-que-el.” SQL is used to communicate with a database. According to ANSI (American National Standards Institute), it is the standard language for relational database management systems. SQL statements are used to perform tasks such as update data on a database, or retrieve data from a database. Some common relational database management systems that use SQL are: Oracle, Sybase, Microsoft SQL Server, Access, Ingres, etc. Although most database systems use SQL, most of them also have their own additional proprietary extensions that are usually only used on their system. However, the standard SQL commands such as “Select”, “Insert”, “Update”, “Delete”, “Create”, and “Drop” can be used to accomplish almost everything that one needs to do with a database.
SQL works with relational databases. A relational database stores data in tables (relations). A database is a collection of tables. A table consists of a list of records, each record in a table preferably includes the same structure, and each has a fixed number of “fields” of a given type.
See, generally, http://www.sqlcourse.com/intro.html and http://www.dcs.napier.ac.uk/˜andrew/sql/0/w.htm.
Scalable Vector Graphics (SVG). Language for describing two-dimensional graphics in XML. SVG allows for three types of graphic objects: vector graphic shapes (e.g., paths consisting of straight lines and curves), images and text. Graphical objects can be grouped, styled, transformed and composited into previously rendered objects. Text can be in any XML namespace suitable to the application, which enhances searchability and accessibility of the SVG graphics. The feature set includes nested transformations, clipping paths, alpha masks, filter effects, template objects and extensibility. SVG drawings can be dynamic and interactive. The Document Object Model (DOM) for SVG, which includes the full XML DOM, allows for straightforward and efficient vector graphics animation via scripting. A rich set of event handlers such as onmouseover and onclick can be assigned to any SVG graphical object. Because of its compatibility and leveraging of other Web standards, features like scripting can be done on SVG elements and other XML elements from different namespaces simultaneously within the same Web page. See http://www.w3.org/Graphics/SVG/Overview.htm8.
Taxonomy. An organizational structure wherein divisions are ordered into groups or categories.
Time-Sensitivity. Property of an information medium to deliver and present information based on when the information would be most relevant in time. For instance, freshness is an attribute that denotes time-sensitivity. In addition, the delivery and presentation of upcoming events (which, by definition, are time-sensitive) and the manner in which the time-criticality of the events are displayed are properties of a time-sensitive medium.
Today's Web. This refers to the World Wide Web as we know it today. Today's Web is a universe of hypertext servers (HTTP servers), which are the servers that allow text, graphics, sound files, etc. to be linked together. Hypertext is simply a non-linear way of presenting information. Rather than reading or learning about things in the order that an author, or editor, or publisher sets out for us, readers of hypertext may follow their own path, create their own order or meaning out the material. This is accomplished by creating “links” between information. These links are provided so that user may “jump” to further information about a specific topic being discussed (which may have more links, leading each reader off into a different direction). The Hypertext medium can incorporate pictures, sound, and video present a multimedia approach to presenting information, also referred to as hypermedia. See, generally, http://www.w3.org/History.html and http://www.umassd.edu/Public/People/KAmaral/Thesis/hypertext.html.
Multicast Time to Live (TTL). Multicast routing protocol uses the field of datagrams to decide how “far” from a sending host a given multicast packet should be forwarded. The default TTL for multicast datagrams is 1, which will result in multicast packets going only to other hosts on the local network. A setsockopt(2) call may be used to change the TTL. As the value for TTL increases, routers will expand the number of hops they will forward a multicast packet. To provide meaningful scope control, multicast routers typically enforce the following “thresholds” on forwarding based on the TTL field:
User State. This refers to all state that is either created by a user or which is needed to cache a user's preferences, favorites, or other personal information on a client or server. Client-side User State includes authentication credential information, users' Agent lists (and all the metadata including the SQML queries for the Agents), home Agent, configuration options, preferences such as Skins, etc. Essentially, client-side User State is a persisted form of users' Semantic Environment. Server-side User State includes information such as users' Favorite Agents, subscribed Agents, Default Agent, semantic links to information objects on the server (e.g., “favorites” links) etc. Server-side User State is optional for servers but support for it is preferred. Servers preferably support user logon and a “people” object type (even without server-side Agents) because these are needed for features such as favorites, recommendations, and for Context Templates such as “Newsmakers,” “Experts,” “Recommendations,” “Favorites,” and “Classics.”
Virtual Information Object Type™. Trademarked name for object types that do not map to distinct object types, yet are semantically of interest to users.
Virtual Parameter™. Trademarked name for variables, parameters, arguments, or names that are dynamically interpreted at runtime by the semantic query processor. This allows the Agency administrator to store Agents that refer to virtual names and then have those names be converted to actual relevant terms when the query is invoked.
Web of Trust. Term coined by members of the Semantic Web research community that refers to a chain of authorization that users of the Semantic Web can use to validate assertions and statements. Based on work in mathematics and cryptography, digital signatures provide proof that a certain person wrote (or agrees with) a document or statement. Users can preferably digitally sign all of their RDF statements. That way, users can be sure that they wrote them (or at least vouch for their authenticity). Users simply tell the program whose signatures to trust. Each can set their own levels of trust (or paranoia), and the computer can decide how much of what it reads to believe.
By way of example, with a Web of Trust, a user can tell a computer that he or she trusts his or her best friend, Robert. Robert happens to be a rather popular guy on the Net, and trusts quite a number of people. All the people he trusts in turn trust another set of people. Each of these measures of trust is to a certain degree (Robert can trust Wendy a whole lot, but Sally only a little). In addition to trust, levels of distrust can be factored in. If a user's computer discovers a document which no one explicitly trusts, but no one has said it has totally false either, it will probably trust that information a little more than one which many people have said is false. The computer takes all these factors into account when deciding the trustworthy of a piece of information. Preferably, the computer combines all this information into a simple display (thumbs-up/thumbs-down) or a more complex explanation (a description of all the various trust factors involved). See http://blogspace.com/rdf/SwartzHendler.
Web Services-Interoperability (WS-I). An open industry organization chartered to promote Web services interoperability across platforms, operating systems, and programming languages. The organization works across the industry and standards organizations to respond to user needs by providing guidance, best practices, and resources for developing Web services solutions. See http://www.ws-i.org.
Web Services Security (WS-Security). Enhancements to SOAP messaging providing quality of protection through message integrity, message confidentiality, and single message authentication. These mechanisms can be used to accommodate a wide variety of security models and encryption technologies. WS-Security also provides a general-purpose mechanism for associating security tokens with messages. No specific type of security token is required by WS-Security. It is designed to be extensible (e.g. support multiple security token formats). For example, a client might provide proof of identity and proof that they have a particular business certification. Additionally, WS-Security describes how to encode binary security tokens. Specifically, the specification describes how to encode X.509 certificates and Kerberos tickets as well as how to include opaque encrypted keys. It also includes extensibility mechanisms that can be used to further describe the characteristics of the credentials that are included with a message. See http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnglobspec/html/ws-security.asp.
Extensible Markup Language (XML). Universal format for structured documents and data on the Web. Structured data includes things like spreadsheets, address books, configuration parameters, financial transactions, and technical drawings. XML is a set of rules (you may also think of them as guidelines or conventions) for designing text formats that let you structure your data. XML is not a programming language, and one does not have to be a programmer to use it or learn it. XML makes it easy for a computer to generate data, read data, and ensure that the data structure is unambiguous. XML avoids common pitfalls in language design: it is extensible, platform-independent, and it supports internationalization and localization. XML is fully Unicode-compliant. See http://www.w3.org/XML/1999/XML-in-10-points.
XML Web Service (also known as “Web Service”). Service providing a standard means of communication among different software applications involved in presenting dynamic context-driven information to the user. More specific definitions include:
1. A software application identified by a URI whose interfaces and binding are capable of being defined, described and discovered by XML artifacts. Supports direct interactions with other software applications using XML based messages via Internet-based protocols.
3. Programmable application logic accessible using standard Internet protocols. Web
Services combine aspects of component-based development and the Web. Like components, Web Services represent black-box functionality that can be reused without worrying about how the service is implemented. Unlike current component technologies, Web Services are not accessed via object-model-specific protocols, such as DCOM, RMI, or HOP. Instead, Web Services are accessed via ubiquitous Web protocols (ex: HTTP) and data formats (ex: XML).
See http://www.xmlwebservices.cc/, http://www.perfectxml.com/WebSvc1.asp and http://www.w3.org/2002/ws/arch/2/06/wd-wsa-reqs-20020605.html.
XQuery. Query language that uses the structure of XML to intelligently express queries across all these kinds of data, whether physically stored in XML or viewed as XML via middleware. See http://www.w3.org/TR/xquery/ and http://www-106.ibm.com/developerworks/xml/library/x-xquery.html.
XPath. The result of an effort to provide a common syntax and semantics for functionality shared between XSL Transformations (http://www.w3.org/TR/XSLT) and XPointer (http://www.w3.org/TR/xpath#XPTR). The primary purpose of XPath is to address parts of an XML [XML] document. In support of this primary purpose, it also provides basic facilities for manipulation of strings, numbers and Booleans. XPath uses a compact, non-XML syntax to facilitate use of XPath within URIs and XML attribute values. XPath operates on the abstract, logical structure of an XML document, rather than its surface syntax. XPath gets its name from its use of a path notation as in URLs for navigating through the hierarchical structure of an XML document.
In addition to its use for addressing, XPath is also designed so that it has a natural subset that can be used for matching (testing whether or not a node matches a pattern); this use of XPath is described in XSLT. XPath models an XML document as a tree of nodes. There are different types of nodes, including element nodes, attribute nodes and text nodes. XPath defines a way to compute a string-value for each type of node. Some types of nodes also have names. XPath fully supports XML Namespaces (http://www.w3.org/TR/xpath#XMLNAMES). Thus, the name of a node is modeled as a pair consisting of a local part and a possibly null namespace URI; this is called an (http://www.w3.org/TR/xpath#dt-expanded-name). See http://www.w3.org/TR/xpath#XPTR.
XSL. A style sheet language for XML that includes an XML vocabulary for specifying formatting. See http://www.w3.org/TR/xslt11/.
XSLT. Used by XSL to describe how a document is transformed into another XML document that uses the formatting vocabulary. See http://www.w3.org/TR/xslt11/.
1. Invention Context
There is a misconception that the Holy Grail for information access is the provision of natural language searching capability. Prior technologies for information access have focused principally on improving the interface for searching for or accessing information to optimize information retrieval. The presumption has largely been that providing a natural language interface to information will perfectly solve users' information access problems and end the frustration users have with finding information.
In truth, however, many axes of analysis are involved in how people acquire knowledge in the real world. One example is context. There are many things people know only because of where they were at a certain place and time. If they were not at that place at that time, they would not know what is in fact known or, indeed, might not care to know. Having the ability to search for what is presently known with natural language does not assist in uncovering the knowledge related to that particular time and place. There are simply no natural parameters that form the correct query to retrieve the desired information.
The conundrum is that a person cannot ask for what he or she might not even know would have value until after the fact. Stated differently, one cannot query for what they do not know they do not know, or for what they do not know that they might want to know. Context-sensitivity, time-sensitivity, discovery, dynamic linking, user-controlled browsing, users' “Semantic Environment,” flexible presentation, Context Skins, context attributes, Context Palettes (which bring up relevant, context and time-sensitive information based on Context Templates) and other aspects of this invention recognize and correct this fundamental deficiency with existing information systems.
For example, people may have many CDs in their library (thereby adding to the “knowledge” of music) because they attended certain parties and spoke with certain people. Those people at those parties mentioned the CDs to the person, thereby increasing the person's knowledge of music. As another example, a person may purchase a book (if read, increasing the person's knowledge on the particular topic of the book), based on a recommendation from a hitherto unknown stranger the person happened to sit beside on an airplane flight. In the real world, people acquire knowledge based not just on what they read and search for, but also based on the friends they keep, the people with whom they interact and the people whose judgment they trust. The “knowledge environment” is arguably as critical if not more critical for knowledge dissemination and acquisition as the model for retrieval (whether digital or analog).
The present invention mirrors virtually every real-world knowledge-acquisition scenario in the digital world. The resulting Information Nervous System™ is the medium doing most of the work but the scenarios map very cleanly to the analog (real) world. The inability of efforts such as natural-language search techniques of Today's Web as well as the Semantic Web to recognize the many ways in which knowledge is disseminated and acquired render them ultimately ineffective. The present invention accounts for the variety of ways in which humans have always acquired knowledge—independent of the actual technology used for information delivery.
By way of example, there has always been context and there has always been time. Likewise there has always been the notion of discovery and the need to link information dynamically and with user control. There have always been certain Context Templates, albeit in different mediums that presented herein, including “classics,” “history,” “timelines,” “upcoming events,” “headlines.” These templates existed before the creation of the Internet, Today's Web, Email, e-Learning, etc. Nevertheless, prior to the present invention, there was no ability in the electronic medium to focus on the mode, protocol and presentation of knowledge delivery which maps to real-world scenarios (for example, via Context Templates, context-sensitivity, time-sensitivity, dynamic linking, flexible presentation, Context Skins, context attributes, etc.) as opposed to actual information types, semantic links, metadata, etc. There will always be new information types. But the dissemination and acquisition axes of knowledge (e.g., Context Templates) have always and will always remain the same. The present invention captures this reality.
In addition, the present invention provides the ability to disseminate knowledge via serendipity. Serendipity plays a large part in knowledge acquisition in the real world and it is a first-class mode of knowledge delivery. The present invention enables a user to acquire information serendipitously (albeit intelligently) by its support for context, time, Context Templates, etc.
Information models or mediums that employ a strict, static structure like a “Web” break down because they assume the presence of an authored “network” or “Web” and fail to account for the various axes of knowledge formation. Such information models are not user-focused, do not incorporate context, time, dynamism and templates, and do not map to real-world knowledge acquisition and dissemination scenarios. The present invention minimizes information loss and maximizes information retained, even without the presence of a “Web” per se, and even if no natural language is employed to find information. This is possible because, unlike existing mediums for information access, a preferred embodiment of the present invention focuses on the knowledge dissemination models that incorporate context, time, dynamism, and templates (for the benefit of both the end-user and the content producer) and not on the specifics of the access interface, or the linking (semantic or non-semantic) of information resources based on static data models or human-based authoring. In many scenarios, a “Web” (semantic or non-semantic) is necessary as a means of navigation, but is far from being sufficient as a means of knowledge dissemination and acquisition. The Information Nervous System of the present invention incorporates “knowledge axes” described in the invention (including but not limited to link-based navigation) and intelligently and seamlessly integrates them to facilitate the dissemination and acquisition of knowledge and to benefit all parties involved in the transfer of knowledge.
2. Value Propositions
Today, knowledge must be “manually hard-coded” into the digital fabric of an information structure, whether it be for an enterprise, a consumer or the general inquiring population. If it is not authored and distributed properly, no one knows of its existence, knows how it relates to other sources of intelligence, or knows how to act on it in real-time and in the proper fashion. This is largely because Today's Web was not designed to be a platform for knowledge. It was designed to be a platform for presentation and is intentionally dumb, static, and reactive. Today, knowledge-workers—those who seek to use information by adding context—and meaningare at the mercy of knowledge-authors.
A significant aspect of knowledge interaction is to have knowledge-workers be able to navigate their way through a knowledge space in a very intuitive manner, and at the speed at which they wish to make decisions and act on the knowledge. In other words, knowledge-workers do not have to “think” about an e-Learning island as being separate from documents in their organizations, e-mail that contains customer feedback, media files, upcoming video-conferences, a meeting they had recently, information stored in newsgroups, or related books. The preferred situation is to relegate the information “type” and “source” and to create a “seamless knowledge experience” that cuts across all those islands in a semantic way.
In creating a knowledge experience, it is also preferred to be able to integrate knowledge assets across content-provider, partner, supplier, customer and people boundaries. In the enterprise scenario, for example, no single organization has all the knowledge it needs to remain competitive. Knowledge is stored in industry reports, research documents from consulting firms and investment banks, media companies like Reuters™ and Bloomberg™, etc. All this constitutes “knowledge.” It is not enough to deploy an e-Learning repository to train users on a one-time or periodic basis. Users should have always-on access to knowledge from a variety of sources, in-place, and in an intelligent context that is relevant to their current task.
All this requires a layer of intelligence and pro-activity that is not available today. Today, for example, enterprises use information portals, such as intranets and the Internet, as a way of disseminating information to their employees. However, this is far from being enough, as it provides only presentation-level integration. This is akin to subscribing to newsletters to keep updated with information, as opposed to having an Agent that manages your information for you, helps you discover new information on-the-fly, helps you capture and share information with colleagues, etc.
To accomplish the desired level of knowledge interaction requires Agents working in the background, reasoning, learning, inferring, matching users together based on their profiles, capturing new knowledge and automatically deducing new knowledge, and federating knowledge from external sources so that they become a seamless part of the knowledge experience. This in turn requires the semantic integration of knowledge assets so that they all make sense in a holistic fashion, rather than merely providing the basis for presentation-level integration and document searching. The implementation framework and resulting medium must provide real-time, agile discovery and recommendation services so that context and time-sensitive information is “honored” and such that knowledge-workers can be more productive and get more done faster and with less. And lastly, the system must work with existing information sources in a plug-n-play manner, must seamlessly and automatically classify and integrate known knowledge assets, and must embed the knowledge tools in the knowledge themselves, thereby adding another “dimension” into knowledge assets.
The present invention is designed to be an intelligent, proactive, real-time knowledge platform that co-exists with Today's Web (or any other layer of presentation). Incorporation and use of the present invention will allow knowledge-workers to be in control of their knowledge experiences because authoring (via “connections”) will be done intelligently, dynamically, automatically, and at the speed of thought.
3. Today's “Information” Web vs. The Information Nervous System of the Present Inveniton
With Today's Web environment, the semantics of information presented are lost upon conversation of the structured data to HTML at the server, meaning that the “knowledge” is stripped from the objects before the user has an opportunity to interact with them. In addition, Today's Web is authored and “hard-coded” on the server based on how the author “believes” the information will be navigated and consumed. Users consume only information as it is presented to them.
The present invention adds a layer of intelligence and layers of customization that Today's HTML-based Web environment cannot support. The present invention provides an XML-based dynamic Web of smart knowledge objects rather than dumb Web pages wherein the semantics of the objects are preserved between the server and the client, thereby giving users much more power and control over their knowledge experience. In addition, with the Web of the present invention, knowledge-workers are able to consume and act on information on their own terms because they will interactively author their own knowledge experiences via “dynamic linking” and “user-controlled browsing.”
The Information Agent (semantic browser) of the present invention is designed to co-exist with Today's Web and to integrate with and augment all facets of private and public intranets as well as the Internet. The technology platform stacks of Today's Web and the
Information Nervous System of the present invention are summarized in
Apart from overlapping layers of processing, the present invention uniquely handles information from the bottommost level of operation in a manner that preserves the semantics of the underlying information sources. At both the Structured and Unstructured Information Sources Layers, the system 10 handles information uniformly, taking into account metadata and semantics associated with the information. At the Information Indexing Layer, information metadata and semantics are extracted from unstructured. The system 10 adds three additional platform layers not present in Today's Web: Knowledge Indexing and Classification Layer, wherein information from both structured and unstructured sources are semantically encoded; Knowledge Representation Layer, wherein associations are created that allows maintenance of a self-correcting or healing Semantic Network of knowledge objects; and Knowledge Ontology and Inference Layer, wherein new connections and properties are inferred in the Semantic Network. At the Logic Layer a knowledge-base is created that allows for programmability at a semantic level. At the Application Layer, server-side scripts are used in association with the knowledge-base. These scripts dynamically generate knowledge objects based on user input, and may include semantic commands for retrieval, notifications and logic. This Layer may also include Smart Agents to optimize the handling of semantic user input. The Presentation Layer of the system 10 preserves the semantics that are tracked from the bottommost layers. Presentation at this Layer is dynamically generated on the client computer system and completely customizable.
By the maintenance, integration and use of semantics in all technology layers, the present invention creates a virtual Web of actionable “objects” that directly correspond to “things” that humans interact with physically or virtually or, in other words, as familiar “Context Templates.” As opposed to Today's Web, which is a dumb Web of documents, the present invention provides for a smart virtual Web of actionable objects that have properties and relationships, and in which events can dynamically cause changes in other parts of the virtual Web.
The present invention provides a programmable Web. Unlike Today's Web which is a dumb Web of documents, the Web of the present invention is programmable akin to a database—it is able to process logic and rules, and will be able to initiate events.
While Today's Web is encoded for human, and thus is focused primarily on presentation of static information, the virtual Web of the present invention is encoded primarily for machines, albeit ultimately presented to humans as the end of the knowledge delivery chain. The present invention provides an intelligent, learning Web. This means that the virtual Web of the present invention will be able to learn new connections and become smarter over time. The Web is dynamic, virtual and self-authoring, thereby providing much more power to knowledge-workers by intelligently and proactively making semantic connections that Today's Web is unable to provide, thereby leading to a reduction in and eventual elimination of information loss.
The Web of the present invention is a self-healing Web. Unlike Today's web which has to be manually maintained by document authors, the present invention provides a Web that is self-maintained by machines. This feature rectifies broken links because the Web will fix disconnections in the network automatically.
Finally, as will be set forth in greater detail below, the various embodiments of the present invention incorporate some or all of the axes of knowledge acquisition described above to provide substantial advantages over existing systems directed to Today's Web or the conceptual Semantic Web.
1. System Overview
The present invention is directed to a system and method for knowledge retrieval, management and delivery. This system and method is referred to herein by the trademarked term Information Nervous System™. With reference to
2. System Architecture
The end-to-end system architecture for the Information Nervous System of the present invention is shown with reference to
The system architecture for the KIS of the Information Nervous System, including components thereof, are shown with reference to
3. Technology Stacks
The significant differences between Today's Web and the conceptual Semantic Web are further highlighted by reference to the technology stacks of each as shown with reference to
4. System Hererogeneity
Heterogeneity is an advantage of the present invention. In the preferred embodiment, the KIS Agency XML Web Service is portable. This means that it supports open standards such as XML, XML Web Services that are interoperable (e.g., that employ the WS-I standard for interoperability), standards for data storage and access (e.g., SQL and ODBC/JDBC) and standard protocols for the information repositories from which the DSAs gather data (e.g., LDAP, SMTP, HTTP, etc.), etc.
For example, in a preferred embodiment, a KIS (on which an Agency is running) is able to:
In an alternative embodiment, the KIS Agency may be configured to extract metadata stored in a proprietary repository (via an appropriate DSA).
To achieve heterogeneity, in the preferred embodiment, for client-server communications, the system 10 uses XML Web Service standards that work in an interoperable manner (across platforms). These include appropriate open and interoperable standards for SOAP, XML, Web Services Security (WS-Security), Web Services Caching (WS-Caching), etc.
In the preferred embodiment of the present invention, the semantic browser (also referred to by the trademarked term Information Agent™) is able to operate cross-platform and in different environments, such as Windows, .NET, J2EE, Unix, etc. This ability is consistent with the notion of a semantic user experience in that users do not and should not care about what “platform” the browser is running on or what platform the Agency (server) is running on. The semantic browser of the present invention provides users with a consistent experience regardless whether they are “talking” to a Windows (or .NET) server or a J2EE server. Users are not required to take any extra steps while installing or using the client based on the platform on which any of the Agencies they are interacting with is running.
The Information Agent preferably uses open standards for its Skins and other presentation effects. These include standards such as XSLT, SVG, and proprietary presentation formats that work across platforms (e.g., appropriate versions of Flash MX/ActionScript).
A sample, heterogeneous, end-to-end implementation of a preferred embodiment of the Information Nervous System of the present invention is shown with reference to
The preferred embodiment of the Information Nervous System provides support for all aspects of security: authentication, authorization, auditing, data privacy, data integrity, availability, and non-repudiation. This is accomplished by employing standards such as WS-Security, which provides a platform for security with XML Web Service applications. Security is preferably handled at the protocol layer via security standards in the XML Web Service protocol stack. This includes encrypting method calls from clients (semantic browsers) to servers (Agencies), support for digital signatures, authenticating the calling user before granting access to an Agency's Semantic Network and XML Web Service methods, etc.
The preferred embodiment that the present invention supports local (client-side) credential management. This is preferably implemented by requiring users to enter a list of their usernames and passwords that they use on multiple Agencies (within an Intranet) or over the Internet. The semantic browser aggregates information from multiple Agencies that may have different authentication credentials for the user. Supported authentication credentials optionally include common schemes such as basic authentication using a username and password, basic authentication over SSL, Microsoft's .NET Passport authentication service, the new Liberty Alliance authentication service, client certificates over SSL, digest authentication, and integrated Windows authentication (for use in Windows environments).
In the preferred embodiment, with the users' credentials cached at the client, the semantic browser uses the appropriate credentials for a given Agency by checking the supported authentication level and scheme for the Agency (which is part of the Agency's schema). For example, if an Agency supports integrated Windows authentication, the semantic browser invokes the XML Web Service method with the logon handle or other identifier for the current user. If the Agency supports only basic authentication over SSL, the semantic browser passes either the username and password or a cached copy of the logon handle (if the client was previously logged on and the logon handle has not expired) in order to logon. The preferred embodiment employs techniques such as logon handle caching, aging and expiration on the KIS in order to speed up the authentication process (and logon handle lookups) and in order to provide more security by guarding against hijacked logon handles.
The Agency XML Web Service preferably supports different authentication schemes either implicitly (if the feature is natively supported by the server operating system or application server) or at the application-level by the XML Web Service implementation itself. Alternative embodiments of the KIS Agency's XML Web Service preferably employ a variety of authentication schemes such as basic authentication, basic over SSL, digest, integrated Windows authentication, and client certificates over SSL, and integrated .NET passport authentication.
6. Efficiency Considerations
Client-Side and Server-Side Query and Object Caches. The present invention provides for query caches, which are responsible for caching queries for quick access. On the client, the client-side query cache caches the results of SQML queries with specified arguments. The cache is preferably configured to purge its contents after a predetermined amount of time (e.g., a few minutes). The amount of time is preferably set by modeling system usage and arriving at an optimal value for the cache time limit. Other parameters may also be considered, such as the data arrival rate on the Agency (in the case of per-Agency caches, which is another implementation option), the usage model (e.g., navigation rate) of the user, etc.
Caching improves performance because the client does not have to needlessly access recently used servers as the user navigates the semantic environment. In the preferred embodiment, the client employs standard XML Web Service Caching technologies (e.g., WS-Caching). In addition, on the client, there is preferably an object cache. This cache caches the results of each SQML resource and is tagged with the resource reference (e.g., the file path, the URL, etc.). This optimizes SQML processing because the client can get the XML metadata for an SQML resource directly from the object cache, without having to access the resource itself. The resource may be the local file system, a local application (e.g., Microsoft Outlook), or an Agency's XML Web Service. Like the query cache, the object cache may be configured to purge its contents after a set amount of time (e.g., a few minutes).
In an alternative embodiment, on the server, the server-side query cache caches the category results for XML arguments. This speeds up the query response time because the server does not have to ask the KDM to categorize XML arguments (via the one or more instances of the KBS that the KIS is configured to get its domain knowledge from) on each query request. In addition, the server can cache the SQL equivalents of the SQML arguments it receives from clients. This speeds up the query response time because the server would not have to convert SQML arguments to SQL each time it receives a request from a client. In the preferred embodiment, aggressive client-side caching is employed and server-side caching is avoided unless it clearly improves performance. This is because client-side caching scales better than server-side caching since the client caches requests based on its local context.
Virtual, Distributed Queries. The present invention employs virtual, distributed queries. This is consistent with its “dynamic linking” and “user-controlled browsing” functionality. The system does not require static networks that link—or massive individual databases that house—all the metadata for the system. This precludes the need for manual authoring and maintenance on a local or global scope. In addition, this precludes the need for integrated (or universal) storage, wherein all the metadata is required to be stored on a single metadata store and accessible through one database query interface (e.g., SQL). Rather, the present invention employs the principle of “Dynamic Access” via its use of XML Web Services to dynamically distribute queries across various Agencies (in a context and time-sensitive manner), and to aggregate the results of those queries in a consistent and user-friendly manner on the client.
1. Agencies and Agents
The present invention introduces a unique approach to using Agencies and Agents to retrieve, manage and deliver knowledge.
In a preferred embodiment of the present invention, the Agency is an instance of the Knowledge Integration Server (KIS) 50 and is the invention's equivalent of a Web site. An Agency is preferably installed as a Web application (on a Web server) so as to expose XML Web Services. An Agency will preferably include an Agency administrator. In a preferred embodiment of the present invention, an Agency has the following primary components:
Corresponding to schemas for the respective classes.
Server-Side User State. In the preferred embodiment of the present invention, Agencies support server-side User State, which associates related concepts including “people” metadata and user authentication. Server-side User State facilitates many of the implementation details of the present invention, including the storage of user favorites (by semantic links between people objects and information objects), the inference of favorites in order to generate new links (e.g., recommendations), Annotations (that map users' comments to information objects), and the inference of “experts” based on semantic links that map users to information (e.g., posted emails, annotations, etc.). Server-side User State is preferably used with some Context Templates like “Experts,” “Favorites,” Recommendations,” and “Newsmakers.”
Client-Side User State. The Information Agent (semantic browser) preferably supports roaming of local client-side User State. This includes users' Semantic Environment and users' credentials (securely transferred). In the preferred embodiment, users are able to easily export their client-side User State to another machine in order to replicate their Semantic Environment onto another machine. This is preferably achieved by transferring users' Agent list (recent and favorites), the metadata for the Agents (including the SQML buffers), users' local security credentials, etc. to an XML format that serializes all this state and enables the state to be easily transferred. Alternatively, an XML schema may be developed for all the local client-side User State. Caching the User State on a server and synchronizing the User State using common synchronization techniques can also facilitate roaming. The semantic browser preferably downloads and uploads all client-side User State onto the server, rather than storing the state locally (in an XML file or a proprietary store like the Windows registry).
An Agent is the main entry point into the Semantic Network of the present invention. An Agent preferably consists of a semantic filter query that returns XML information for a particular semantic object type (e.g., documents, email, people, etc.). In other words, an Agent is preferably configured with a specific object type (described below). Agents can also be configured with a Context Template (described below). In this case, the query will return an object type, but it will incorporate the semantics of the Context Template. For example, Agents configured with a “Headlines” Context Template will be sorted by time and relevance, etc. Agents are also used to filter notifications, alerts and announcements. Agents can be given any name. However, in the preferred embodiment of the present invention, the naming format for most Agents is:
There will also be Domain Agents (see below) that may follow a different naming convention (see below). At the semantic browser of the present invention, a fully qualified Domain Agent name will have the format:
For example, the Email Domain Agent on the Agency http://research.Agency.asp configured with the category wireless.all from the knowledge-base ABC.com/kb.asp with the semantic domain name industries.informationtechnology will be fully named as:
The semantic browser of the present invention is preferably configurable to use only the Agent name or to include the “Agency” and “kb” qualifiers.
Agent Types. There are three primary types of Agents created on server 20: Standard Agents, Compound Agents, and Domain Agents. A Standard Agent is a standalone Agent that encapsulates structured, non-semantic queries, i.e., without domain knowledge (or an ontology/taxonomy mapping). For example, on the server, the Agent All.PostedToday.All is a simple Agent that is resolved by filtering all objects based on the CreationTime property. Standard Agents can also be more complex. For example, the Agent All.PostedByAnyMemberOfMyTeam.All may resolve into a complicated query that involves joins and sub-queries from the Objects table and the Users table (see below).
A Compound Agent contains other Agents and allows the Agency administrator to create queries that generate results that are the UNION or the INTERSECTION of the results of their contained Agents (depending on the configuration). Compound Agents can also contain other Compound Agents. In the presently preferred embodiment, Compound Agents contain Agents from the same Agency. However, the present invention anticipates the integration of Agents from different Agencies. By way of example, a Compound Agent All.Technology.Wireless.All might be created by compounding the following Agents:
As described above, a Domain Agent is an Agent that belongs to a semantic domain. A Domain Agent is initialized with an Agent query, just like any other Agent. However, this query includes the CATEGORIES table, which is populated by the Knowledge Domain Manager (see below). While the preferred embodiment of the present invention utilizes a KBS 80 having proprietary ontologies corresponding to a private Semantic Environment, the present invention contemplates integrated support of ontology interchange standards that will enable an Agency to connect to one or more custom private KBS, for example within an organization where the Agency was previously initialized with a proprietary ontology for that organization.
An example of a Domain Agent is Email.Technology.Wireless.A11. This Agent is preferably created with a knowledge source URL such as:
This knowledge source URL corresponds to the Technology.Wireless.All category for the default domain on the knowledge base installed on the ABC.com/marketingknowledge.asp Web service. This is resolved to the following HTTP URL: http://ABC.com/marketingknowledge.asp?category=“technology.wireless.all.” In this example, a fully qualified version of the category URL may be:
Domain Agents are preferably created via a Domain Agent Wizard, and the Agency administrator is able to add Domain Agents from the KBS 80 to the Semantic Network of the present invention. The Domain Agent Wizard allows users to create Domain Agents for specific categories (using a category URL) or for an entire semantic domain name. In the latter case, the Agency is preferably configured to automatically create Domain Agents as new categories are added to the semantic domain on the KBS. This feature allows domains and categories to remain dynamic and therefore easily adaptable to the user's needs over time. When Domain Agents are managed in this fashion, the Agency is configurable so as to remove Agents that are no longer in the semantic domain. Essentially, in this mode, the Domain Agents are synchronized with the CATEGORIES table (which in turn is synchronized with the CATEGORIES list at the relevant KBS by the Knowledge Domain Manager, described below).
A Domain Agent is initialized with a structured query that filters the data the Agent manages based on a category name or URL. In this situation, the structured query is identical to the queries for Standard Agents. An example of a resultant query for a category Agent is:
The Domain Agent Wizard asks the user whether he or she wants to name the Agent based on the short category name or a friendly version of the fully qualified category name. An example of the latter is: Marketing.Technology.Wireless.All [@ABC]. The fully qualified Domain Agent naming convention is:
Blenders. Blenders are users' personal super-Agents. Users are able to create a Blender and add and remove Agents (across Agencies) to and from the Blender. This is analogous to users having their own “Personal Agency”. Blenders are preferably invoked only on the system client since they include Agents from multiple Agencies. The client of the present invention aggregates all objects from a Blender's Agents and presents them appropriately. Blenders preferably include all manipulation characteristics of other types of Agents, e.g., drag and drop, Smart Lens (see below). A Blender can contain any type of Agent (e.g., Standard Agents, Search Agents, Special Agents, as well as other Blenders).
The present invention provides for a Blender Wizard, which is a user interface designed to facilitate users in creating Blenders. For example, from an Information Agent interface wizard that allows users to create and manage a new Blender. The “Add Blender” shows the second page of the Add Blender wizard where a user can enter the name and description of the Blender and optionally select information object type filters at a second page. At a third page, the Add Blender wizard, in accordance with a preferred embodiment of the present invention, users add and remove Agents from the Semantic Environment to or from the Blender. When the “Add Agents” option is selected, the “Open Agent” dialog is displayed from which users can add a new Agent, Blender or Agency to the new Blender.
Breaking News Agents. A Breaking News Agent is a specially tagged Smart Agent. In addition to the option of having time-criticality being defined by the Agency administrator, the user has the option of indicating which Agents refer to information that he or she wants to be alerted about. Any information being displayed will show alerts if there is breaking news that relates to it on a Breaking News Agent. For example, a user will be able to create an Agent as:
“All Documents Posted on Reuters today” or “All Events relating to computer technology and holding in Seattle in the next 24 hours” as Breaking News Agents. This feature functions in an individual way because each Breaking News Agent is personal (“breaking” is subjective and depends on the user). For example, a user in Seattle perhaps would want to be notified on events in Seattle in the next 24 hours, events on the West Coast in the next week (during which time he or she can find a cheap flight), events in the United States in the next 14 days (the advance notice for most U.S. air carriers to get a modestly priced cross-continental flight), events in Europe in the next month (likely because he or she needs that amount of time to get a hotel reservation), and events anywhere in the world in the next six months.
In a preferred embodiment, the present invention automatically checks the Semantic Environment for breaking news by querying each Breaking News Agent or by querying the “Breaking News” Context Template. It will do this for all objects displayed in the semantic browser window. If a Breaking News Agent indicates that there is breaking news, the Information Agent object Skin so indicates by flashing the window or by showing a user interface that clearly indicates that there is an alert that relates to the object. When the user clicks on the breaking news icon, a breaking news pane or a Context Palette for the “Breaking News” Context Template is displayed allowing the user to see the breaking news, select the Breaking News Agent (if there are multiple with breaking news), select predicates, and select other options. An exemplar pane of a Breaking News Agent user interface is shown in
Default Agents. In an alternative embodiment, each Agency exposes a list of default Agents. Default Agents are similar to the default page on a Web site; authors of the Agency determine which Agents they want users to always sees. Alternatively, on the client, Default Agents may be invoked when users click on the root of the Information Agent's Environment (which preferably corresponds to a “Home Agent,” for example, the equivalent of the “Home Page” on Today's Web browser). Combined Default Agents may also be configured by users.
Default Special (or Context) Agents. In the preferred embodiment, the client or the Agency support a Default Special or Context Agent that maps to each Context Template (discussed below). These Agents preferably use the appropriate Context Template without any filter. For example, a Default Special Agent called “Today” returns all items on all Agencies in the “recent” and “favorites” lists (or on a configured list of Agencies) that were posted today. In yet another example, the Default Special Agent called “Variety” shows random sets of results for every Agency in the Semantic Environment corresponding to the “variety” Context Template.
Default Special Agents preferably function as a starting point for most users to familiarize themselves with the Information Nervous System of the present invention. In addition, Default Special Agents retain the same functionality as Smart Agents, such as use of drag and drop, copy and past, Smart Lens, Deep Information, etc.
Horizontal Decision Agents. In the preferred embodiment, Agents utilized by the client to assist with user interaction, including:
Public versus Local Agents. Agents that are created by the Agency administrator are “Public Agents.” Agents created and managed by users are “Local Agents.” Local Agents can refer to remote Agencies via SQML that includes references to Agency XML Web Service URLs, or can refer to local Agencies that run a local instance of the KIS with a local metadata store.
Saved Agents—Users' My Agents List. In the preferred embodiment, users are able to save a copy of an invoked Agent or a query result as a local Agent. For example, users may drag and drop a document on their hard drive to an Agent folder to generate a semantic relational query. Users could save that result as an Agent named “Documents.Technology.Wireless.RelatedToMyDocument.” This will then allow the user to navigate to that Agent to see a personalized semantic query. Users would then be able to use that Agent to create new personal Agents, and so on. Personal Agents can also be “published” to the Agency. Other users are preferably able to discover the Agent and to subscribe to it.
In the preferred embodiment, a local Agent is created by a “Save as Agent” button that appears on the client anytime a semantic relational query result is displayed. This is analogous to users saving a new document. Once the Agent is saved, it is added to the users' My Agents list. An Agent responds to a semantic query based on the semantic domain of the Agency on which it is hosted. Essentially, a semantic query to an Agent is analogous to asking whether the Agent “understands the query.” The Agent responds to a query to the best of its “understanding.” As a further illustration, an Agent that manages “People” responds to a semantic query asking for experts for a document based on its own internal mapping of people in its semantic domain to the categories in that domain.
Alternatively, the system client may be configured to use non-semantic queries. In this case, the Agency will use extracted keywords for the query. All Agents support non-semantic queries. Preferably only Agents on Agencies that belong to a semantic domain will support semantic queries. In other words, semantic searches degrade to searches.
Each Agent has an attribute that indicates whether it is “smart” or not. A Smart Agent is preferably created on an Agency if that Agency belongs to a semantic domain. In addition, a Smart Agent only returns objects it fully “understands.” In the preferred embodiment, when an Agency is installed, there are several default Smart Agents that the Agency administrator may optionally choose to install, including: All.Understood.All; Documents.Understood.All; and Email.Understood.All.
For example, Email.Understood.All only returns email objects that the Agency can semantically understand based on its semantic domain (or ontology).
The present invention preferably includes the capability for users to display all objects and only those the Agency understands
Search Agents. A Search Agent is an Agent that is initialized with a search string. In the preferred embodiment, on invocation, the client issues the search request. A Search Agent is configurable so as to search any part of the Semantic Environment, including: Frequently Used Agents; Recently Used Agents; Recently Created Agents; Favorite; All [Saved] Agents; Deleted Agents; Agents on the local area network; Agents on the Global Agency Directory; Agents on any user-customized Agency directories; and All Agents in the entire Semantic Environment. The client issues the search request based on the scope of the Search Agent. If users indicate that they want the search to cover the entire Semantic Environment, the client issues the request to all Agents in the Semantic Environment Manager (see below) and all Agents on the local area network, the Global Agency Directory and user-customized Agency Directories.
Server-Side Favorite Agents. In yet an alternative embodiment, the Agency supports User States support Favorite Agents. In the analogous context of Today's Web, a Web site allows users to customize their favorite links, stocks, etc. When initially queried, an Agency displays both its Default Agents and the Favorite Agents of the calling user (if there is a User State).
Smart Agents. A Smart Agent is a standalone Agent that encapsulates structured, semantic queries that refer to an Agency via its XML Web Service. In the preferred embodiment, user on the client are able to create and edit Smart Agents via a “Create Smart Agent” wizard that allows them to browse the Semantic Environment via the Open Agent dialog, and add links from specified Agencies. Essentially, this corresponds to users creating the SQML query from the user interface. In the preferred embodiment, the user interface only allows users to add links from the same Agency resource. However, users can create Agents of the same categories across Agencies, in addition to Special Agents and Blenders (which are also preferably cross-Agency). The user interface allows the user to add links using existing Smart Agents as Information Object Pivots provided that the Smart Agent refers to the same Agency for the current query. In a preferred embodiment the Open Agent dialog with the user interface controls allow a user to select link (predicate) templates, the links themselves, and the objects. In one embodiment, the Open Agent dialog allows users to browse the Semantic Environment and open an Agent. Agencies can be navigated in the Semantic Environment using the “Open Agent” dialog with the “Small Preview” view. An “Open” tool on the toolbar can show a user new options to open Agents form the Semantic Environment or to import regular information (e.g., from the file system) to the Semantic Environment by creating Dumb Agents.
The link templates essentially allow the user to navigate predicate for the current object type using predefined filters, thus allowing the user to avoid going through all the predicates for the object type. Examples of link templates include: All; Breaking News (links that refer to time-sensitivity, e.g., “posted in the last”); Categorization; Definite (non-probabilistic links); Probable (probabilistic links); and Annotations.
In the preferred embodiment, the Open Agent dialog allows the user to select the object to “link to” and, depending on the type of the object, allows the user to browse the object (e.g., from a calendar control if it is a date/time, from a text box if it is text, from the file system if it is a file or folder path, etc.) The wizard user interface also allows the user to preview the results of the query. A temporary SQML entry is created with the current predicate list and that is loaded in a mini-browser window within the wizard dialog box. The user is able to add and remove predicates, and will also have the option of indicating whether he or she wants a union (an “OR”) or an intersection (an “AND”) of the predicates. The user interface will also check for duplicate predicates.
Once the user finishes the wizard to create the Smart Agent, the Smart Agent is added to the Semantic Environment and the SQML is also saved with the associated object entry. In the preferred embodiment, the user can later browse the Smart Agent using the Agent property inspector property sheet. This allows the user to view the simple Semantic Environment properties (e.g., name, description, creation time, etc.) and also to view the resource URL (the WSDL URL to the XML Web Service of the Agency being queried) and the predicate list. The user can edit the list from the property sheet.
Default Smart Agent. A Default Smart Agent is similar to a Default Special Agent except that it is based on information object types and not on Context Templates. By way of example, “Documents” would return all documents on all Agencies in the users' Semantic Environment; “Email” would return all email messages in user's Semantic Environment, etc.
Special Agent. A Special Agent is a Smart Agent created by users based on a Context Template (see below). A Special Agent is preferably initialized with an Agent name, albeit without a specific Agent reference. For example, a Special Agent “Email.Technology.Wireless.A11” may be created even if there are no Agents of that name in the Semantic Environment. Like a Search Agent, a Special Agent is scoped to search for any Agent with its name on any part of the Semantic Environment. In the preferred embodiment, when a Special Agent is invoked by users, the client searches for any Agents that bear its name. If or when it finds any Agents with the name, the client invoke the Agent.
In the preferred embodiment, users enter parameters consistent with a Context Template, indicating the category fillers (if required) and what Agency(ies) to query. These can be manually entered using the Open Agent dialog, or users can indicate that they want to query the “recent” Agencies, “favorite” Agencies, or both. In an alternative embodiment, users have the choice of selecting categories (if required) that are in the union or intersection of the selected Agencies, or all categories known to the Global Agency Directory. In yet an alternative embodiment, users are able to select the information type (as opposed to a Context Template) and keywords to search (as opposed to predicates or categories).
Default Special Agents. In the preferred embodiment, the system client installs Default Special Agents that map to all supported Context Templates. By way of example, in the preferred embodiment, Default Special Agents including the following: Headlines, Breaking News, Conversations, Newsmakers, Upcoming Events, Discovery, History, All Bets, Best Bets, Experts, Favorites, Classics, Recommendations, Today, Variety, Timeline, Upcoming Events and Guide.
Custom Special Agents. In contrast to user-created Special Agents, Custom Special Agents are Special Agents specially developed and signed in order to guarantee that the Special Agents are safe, secure, and of high-performance. The present invention provides for a plug-in layer to allow organizations and developers to create their own custom blenders. An example of a custom blender is “All.CriticalPriority.All that relates to my most recent documents or email.” This Custom Blender may be implemented by an SQML file with a resource entry as follows:
In the preferred embodiment, the Presenter (see below) resolves the “link” entry locally and initiates XML Web Service requests to the target resource with XML arguments corresponding to the newest documents or email messages. This allows the target Agent to focus on responding to semantic queries purely with XML filters without knowing the semantics related to filter origination. In an alternative embodiment, a Custom Blender such as the above example is a Default Agent.
Vertical Decision Agents. Vertical Decision Agents are Agents that provide decision-support for vertical industry scenarios.
Agent Schema. Agents operate within specified parameters and exhibit predetermined characteristics that comprise the Agent schema. Agent schemas may vary widely with being equally applicable within the technology of the present invention. By way of example only, the Agent schema of the preferred embodiment of the present invention is shown in
In the preferred embodiment, SQL query formats are used. However, multiple query formats, for example XQL, XQuery, etc., are contemplated within the scope of the present invention.
The KIS 50 preferably hosts an Agents table (for server-side Agents) in its data store corresponding to this schema.
As explained in greater detail below, Agents may optionally include their own Skins An Agent Skin is represented as an URL to an XSLT file or equivalent Flash MX or ActionScript. If the Agent's Skin URL is not specified, a default Skin for the Agent's object type is presumed.
Agent Query Rules. Each server-side Agent query must be specified to return the OBJECTID column. Each table has this column for it is what links the Objects table with the tables for the derived object type. Objects and other tables are described in greater detail below.
Because each Agent query can form the basis of a sub-query, cascaded query or a join, it is preferable that each query follow this format. By way of example, the query for News.All will be may appear as “SELECT OBJECTID FROM NEWS” (where “NEWS” is the name of the table hosting metadata for news articles, with the “news” schema). As a result, the server 10 can then use this query as part of a complex query. For example, if the user drags and drops a document onto the Agent, the server might execute this query as:
This example assumes that the document is classified to belong to categories in the CATEGORIES table with object identifiers 50, 67, and 89 and that a link probability of 0.9 is the threshold to establish that a document belongs to a category. In this example, the document is used as a filter for the News.All query and the query text is used as part of the complex query.
Having a consistent standard for queries allows the semantic query processor to merge queries until they finally have to be presented. For example, each call to the semantic query processor must indicate what object type in which to return the results. The query processor then returns XML information consistent with the schema for the requested object type. In other words, the query processor preferably returns schema-specific results for presentation. Each query is stored at the semantic layer (to return an OBJECTID). To use the last example, when the user invokes the News.All Agent, the browser calls the query processor on the Agency XML Web Service. The query processor will then invoke the query and filter it with the ‘News Article’ object type, as such:
Query Virtual Parameters. Agent queries preferably contain special Virtual Parameter. A typical example may include: ‘%USERNAME%. In this example, the Semantic Query Processor (SQP) resolves the Virtual Parameter to a real argument before invoking the query. An Agent People.MyTeam.All is configured with the SQL query:
In this example, the Agent name includes “MyTeam” even though the Agent can apply to any user. The %USERNAME% variable is resolved to the actual calling user's name by the SQP. The SQL call is resolved to as follows:
Simple Agent Search. Each Agent will support simple search functionality. In the preferred embodiment, a user is able to right-click on a Smart Agent in the Information Agent and hit “Search.” This will bring up a dialog box where the user enters search text. This creates the appropriate SQML with the associated predicate, e.g., “nervana:contains”. The present invention provides a simple, fast way for users to search Agents (and create Smart Agents from there) without going through the “Create Smart Agent” wizard and selecting the “contains text” predicate (which alternatively achieves the same result).
Agency Agent Views. An alternative embodiment of the present invention includes Agency Agent Views. An Agency Agent View is a query that filters Agents based on predefined criteria. For example, the Agent view “Documents” returns only Agents that manage objects of the document semantic class. The Agent view “Reuters News” returns a list of Agents that manage news objects with “Reuters” as the publisher. Agency Agent Views are important in order to give users an easy way to navigate through Agents. The Agency administrator is able to create and delete Agent views.
Agent Publishing and Sharing. The preferred embodiment makes it easy for Agents to be published and shared. This is preferably implemented by serializing the Semantic Environment into an XML document containing the recent and Favorite Agents, their schema, their SQML buffers, etc. and publishing the document to a publishing point. This XML document may also be emailed to colleagues, friends, etc. in order to facilitate the propagation and sharing of local (user-created) Agents. This is analogous to how Web pages are published today and how web URLs and links are shared by sending links and attachments via email.
2. Knowledge Intefration Server
The Knowledge Integration Server (KIS) 50 is the heart of the server-side of the system 10. The KIS semantically integrates data from multiple diverse sources into a Semantic Network and hosts Agents that provide access to the network. The KIS also hosts semantic XML Web Services to provide clients with access to the Semantic Network via Agents. To users, a KIS installation may be viewed as an Agency. The KIS is preferably initialized with the following properties:
a. Semantic Network
The Semantic Network is the core data component of the KIS. The Semantic Network links objects of the defined schemas of the present invention together in a semantic way via database tables. The Semantic Network consists of schemas and the Semantic Metadata Store (SMS). The Semantic Network is preferably comprised of two data schemas: Objects and SemanticLinks Additional data schemas may be included based on system requirements and enterprise needs. The SMS is preferably a standard database (SQL Server, Oracle, DB2, etc.) where all semantic data is stored and updated via database tables. The SMS preferably includes tables for each primary object type (described below).
By way of example, a sample Semantic Network directed towards an enterprise situation is shown with reference to
Objects. The Objects table contains every object in the Semantic Network. The “Object” can be thought of as the “base class” from which every semantic object type will be derived. The preferred schema of the Object type is shown with reference to
The SourceID refers to the identifier for the Semantic Data Adapter (SDA) from which the object was gathered. The Semantic Data Gatherer (SDG) uses this information to periodically check whether the object still exists by requesting status information from the SDA from which the object was retrieved.
SemanticLinks. The SMS preferably includes a SemanticLinks schema (and corresponding database table) that will store semantic links. These links will annotate the objects in the other data tables of the SMS and will preferably constitute the data model for the Semantic Network. Each semantic link will have a semantic link ID. The SemanticLinks table preferably includes the field names and types as shown with reference to
By way of example, the semantic link “Steve reports to Patrick” will be represented in the table with a subject ID corresponding to Steve's ID in the Users table, a predicate type of PREDICATETYPEID_REPORTSTO (see table below), Patrick's object ID in the Users table, a link score of 100 (indicating that it is a “truth” and that the link is not probabilistic) and a Reference Date that qualifies the link.
The KIS creates, updates, and maintains database tables for each object type (via the SMS). The following illustrates preferred but nonexclusive list of primary and derived object types: Person, User, Customer, Category, Document, Analyst Brief, Analyst Report, Case Study, White Paper, Company Profile, E-Book, E-Magazine, Email Message, Email Annotation, Email News Posting, Email Distribution list, Email Public Folder, Email Public folder Newsgroup, News Article, Event, Meeting, Corporate Event, Industry Event, TV Event, Radio Event, Print Media Event, Online Meeting, Arts and Entertainment Event, Online Course, Media, Book, Magazine, Multimedia, Online Broadcast and/or Online Conference. Object types are preferably expresses as hierarchical paths. The path can be extended, e.g., “events\meetings” can be extended with “qualified Meetings,” e.g., “events\meetings\company meetings.” This schema model is indefinitely extensible and configurable.
Virtual Information Object Types. Virtual Information Object Types are object types that do not map to distinct object types, yet are semantically of interest to users. An example is the “Customer Email” object type, which derives from the “Email” object type. This object type is “virtual” in that it does not have a distinct schema and, as a consequence, does not have a distinct table in the SMS on the KIS. Rather, it uses the “Email” table on the SMS, since it derives from the “Email” object type. Even though it is not a distinct object type, users will be interested in browsing and searching for “Customer Email” as though it were indeed distinct.
In the preferred embodiment, Virtual Object Types are implemented by storing the metadata in the appropriate table on the SMS (in this case, the “Email” table, since the object type derives from “Email”). However, the resolution of queries for the object type is accomplished differently from regular queries for distinct object types. When the server SQP receives a semantic query request (via the XML Web Service) for a virtual information object type (such as “Customer Email”), it resolves the request by joining the tables that together form the object type. For instance, in the preferred embodiment, in the case of “Customer Email,” the server will resolve in query with the SQL sub-query:
The present invention contemplates a variety of schemas associated with each object type. Other schemas are in development that will have comparable applicability to the present invention. The “Document” schema, for example, may be extended with fields from the Dublin Core schema (http://www.cis.ohio-state.edu/cgi-bin/rfc/rfc2413.html) and other industry standard schemas. In yet another example, “News Article” schema may be an extension of the NewsML schema (http://www.newsml.org). By way of example only, preferred user object schema made in accordance with the present invention are shown with reference to
By way of example only, the preferred category object schema made in accordance with the present invention is shown with reference to
By way of example only, the preferred document object schema made in accordance with the present invention is shown with reference to
By way of example only, the preferred email message list object schema made in accordance with the present invention is shown with reference to
By way of example only, the preferred event object schema message list object schema made in accordance with the present invention is shown with reference to
By way of example only, the preferred media object schema message list object schema made in accordance with the present invention is shown with reference to
By way of example,
b. Semantic Data Gatherer
In the preferred embodiment, the Semantic Data Gatherer (SDG) is responsible for adding, removing, and updating entries in the Semantic Network via the SMS. The SDG consists of a list of XML Web Service references. These form an Information Source Abstraction Layer (ISAL). Each of these references is initialized to gather data from via a Data Source Adapter (DSA). A data source adapter is an XML Web Service that gathers information from a local or remote semantic data source for a give object type. It then returns the XML corresponding to object entries at the data source. All DSAs preferably support the same interface via which the SDG will gather XML data. This interface includes methods to:
If each call to the DSA XML Web Service will be stateless, the API should include information, preferably via a string with command parameters, which qualifies the request. For example, a DSA for an email inbox includes parameters such as the name of the user whose inbox is to be gathered. A DSA for a Web site or document store will have to include information on the URL or directory path to be crawled.
Each DSA is required to retrieve information in the schema for its object type. Because a DSA must be implemented for a particular object type, the SDG will expect XML for the schema for that object type when it invokes a gather call to the DSA.
The SDG is responsible for maintaining the integrity and consistency of all the database tables in the SMS (the Semantic Network). In this embodiment, the SDG is also referred to as a Semantic Network Manager (SNM). The database tables preferably do not contain redundant or stale entries. Because the SDG retrieves objects with well-known schemas the semantics of each of the object types is understood, and the SDG maintains the consistency of the tables accordingly. For example, the SDG preferably does not add redundant Document XML metadata to the DOCUMENTS table. The SDG uses the semantics of documents to check for redundancy. In the preferred embodiment this is accomplished by comparing the author name, creation date/time, file path, etc. The SDG also performs this check for other tables (e.g., EVENTS, CUSTOMERS, NEWS, etc.). For example, the SDG will perform redundancy checking for events by examining the title, the location, and the date/time. Other tables are maintained accordingly. The SDG will also update objects in the database tables that have been changed.
The SDG is also preferably responsible for cleaning up the database tables. The SDG periodically queries the DSA to determine whether all of the objects in each table managed by the DSA still exists. For example, for a DSA that retrieves documents, the SDG will pass the XML metadata to the DSA Web service and query whether the object still exists. The DSA attempts to open the URL for the document. If the document does not exist anymore, the DSA will indicate this to the SDG. Individual DSAs, and not the SDG, are responsible for object validation to avoid security restrictions that are data source specific. For example, there might be data source restrictions that prevent remote access to local resources. In such a case, only the DSA XML Web Service (which is preferably running locally, relative to the data source) will have access to the data source. Alternatively, some DSAs might run on the Agency server, alongside the SDG and other server components, and retrieve their data remotely.
Having the DSAs handle object validation also provides additional efficiency and security in that the DSA prevents the SDG from knowing the details of how to open each data source to check whether an object still exists. Since the DSA needs to know this (since it retrieves the XML data from the data source and therefore has code specific to the data source), it is more appropriate for the DSA to handle this task.
The SDG preferably maintains a gather list that will point to DSA XML Web Service URLs. The KIS administrator is able to add, delete, and update DSA entries from the SDG gather list. Each gather list entry is preferably configured with:
4. The gather schedule—this indicates how often the SDG should ‘crawl’ the DSA to gather XML metadata.
In a preferred embodiment, the Agency is initialized with a user directory domain and group name. In this case, the SDG preferably automatically enters a gather list entry for the user directory DSA. For example, if the Agency is configured with a Exchange 2000 User Directory with Domain Name “Foo” and Address Book or group name “Everyone,” the SDG creates a gather list entry with the Exchange 2000 Users DSA (initialized with these parameters). Alternatively, the Agency can be configured to obtain its user directory from any email application server (e.g., Microsoft Exchange or Lotus Notes). The SDG initializes gather list entries with an Email Inbox and Calendar DSA for the system user (and Email Knowledge Agent, described below). These three gather list entry DSAs (Users, Inbox, and Calendar) are initialized by default. The Inbox is preferably used to store Agency email postings and annotation and the Calendar DSA is used to store events posted to the Agency by users. Other custom DSAs can be added by the Agency administrator.
The SDG also keeps track of the last time the SDA reported to it that objects have been added or deleted to or from the data source. This date/time information is preferably based on the SDA's clock. Each time the SDA reports that there is new or deleted data, the SDG will update the date/time information in its entry for the SDA and gather all the new or deleted information in the SDA. The SDG will then update the database tables.
The SDG preferably maps the XML information it receives from the SDAs to the Semantic Network of the present invention. The SDG stores all the XML metadata in the database tables in the SMS. In addition, the SDG parses the XML it receives from the SDA and, where necessary, maps semantic links to specific XML fields. The SDG adds or updates semantic links in cases where the XML includes information that “links” objects together. For example, the schema for an email object preferably includes fields including “From,” “To,” “Cc,” “Bcc,” and “Attachments.” In the case of the “From,” “To,” “Cc” and “Bcc” columns, the fields in the XML refer to email addresses (separated by delimiters such as “;” or “,” or a space). In the case of the “Attachments” column, this field will refer to the file paths of the files that are attached to the email message (separated by delimiters such as “,”). This raw XML is stored in the EMAIL database table, along with the other columns. In addition, the SDG parses the fields of the email object and adds semantic links to other objects that are identified by the contents of those fields. For example, if the “to” field contains “firstname.lastname@example.org” and the attachments field contains the string “c:\foo.doc, c:\bar.doc,” the SDG will process the email as follows:
In the case of attachments, the SDG extracts the XML metadata for the attached documents. If an XML object with the file path already exists in the SMS (or, in other words, the Semantic Network), the SDG will update the metadata. If the XML object does not already exist, the SDG creates a new document object with the XML metadata. The SDG will adds an entry to the SEMANTICLINKS table with the email object ID as the subject, the new document's object ID as the subject, and the predicate PREDICATETYPEID_ATTACHEDTO. This allows the user to be able to navigate from an email message to its attachments and then use the attachments as pivots to continue to browse the Semantic Network, for example using semantic tools like the Smart Lens (discussed below).
The SDG does not create any objects in the event for which it does not find user objects that match the entries in the XML fields. Preferably, the SDG gathers information from a Directory SDA when a user is manually added to the Agency. The Agency administrator preferably adds users to the Agency via the user group on the Agency properties.
The following illustrates an example of mapping raw email XML metadata to the Semantic Network.
c. Semantic Network Consistency Checker
The Semantic Network Consistency Checker (CC) complements the consistency checking that is performed by the SDG. As described above, the SDG maintains the integrity of the database tables by precluding the addition of redundant entries into the Semantic Network (from various data sources). The CC also ensures the consistency of the OBJECTS and SEMANTICLINKS tables. The CC periodically checks the OBJECTS table to ensure that each object exists in the native table (preferably by checking the OBJECTID field value). For example, a document object entry in the OBJECTS table preferably also exists in the DOCUMENTS table (with the same object ID). The CC removes any object in the OBJECTS table without a corresponding object in the native table (DOCUMENTS, EVENTS, EMAIL, etc.) and vice-versa.
The CC is also responsible for maintaining the consistency of the SEMANTICLINKS table. The semantics of this table are preferably as follows: A semantic link cannot exist if either its subject (“linked from”) or its object (“linked to”) do not exist. To illustrate this, if object A links to object B with predicate P, and either A or B is deleted, the link should be deleted. The CC periodically checks the SEMANTICLINKS table. If any of the subjects or objects has been deleted, the CC deletes the semantic link entry.
Consistency checks may be implemented in code in the KIS itself or as stored procedures or constraints at the database level.
d. Inference Engine
The Inference Engine is responsible for adding semantic links to the Semantic Network. The Inference Engine employs Inference Rules, which consist of a set of heuristics, to add semantic links based on ongoing semantic activity. The Inference Engine is preferably allowed to remove semantic links. Decision Agents (described below) use the Inference Engine to assist knowledge-workers in making decisions.
The Inference Engine operates by mining the Semantic Network and adding new semantic links that are based on probabilistic inferences. For example, the Inference Engine preferably monitors the Semantic Network and observes patterns in how email is sent, the type of email sent and by whom. The Inference Engine infers from this information background information, such as the expertise of the user, related to various subject matter categories within the monitoring purview of the Inference Engine. For example, the Inference Engine adds semantics links with the predicate PREDICATETYPEID_EXPERTON to indicate that a user is an expert in a particular category. The subject in this case will be a user object and the object will be a category object. To infer this, the Inference Engine is preferably configured to observe semantic activity for at least a certain period of time (e.g., two weeks), or to only infer links after users have sent at least a certain predetermined number of messages or authored a certain number of documents. The Inference Engine infers the new link by keeping statistics on the PREDICATETYPEID_CREATOR and PREDICATETYPEID_CONTRIBUTOR links.
By way of example, the Inference Engine may infer that users are an expert on a category if:
More sophisticated inference models with which to accurately infer this data are contemplated. For example, probability distributions as well as statistical correlation models may be employed. Preferably these models will be developed on a per-scenario basis over time.
The Inference Engine is also responsible for removing links that it might have added. For example, if an employee changes jobs, he or she might “cease” to be an expert on a specific category (relative to other employees). Once the Inference Engine detects this (e.g., by observing email patterns), it removes semantic links that indicate that the person is an expert on the category.
Inferred semantic links are important for scenarios that involve probabilistic semantic queries. For example, in one embodiment of the present invention, using the Information Agent, users may drag and drop a document from their file-system onto an Agent (say, People.Research.All). In this case, users will want to know the people in the Research department that are experts on the document. The browser will then invoke an SQML query with the Agent as resource (or subject), the predicate nervana:experton, and the document path as the object. The Presenter will then retrieve the XML metadata for the document and call the XML Web Service, residing on the Agency that hosts the Agent, with the predicate ID and the document's XML metadata as arguments. The server-side semantic query processor on the Agency processes this XML Web Service call and translates the call to a SQL query consistent with the data model of the Semantic Network. In this example, the call is preferably resolved as follows:
e. Server-Side Semantic Query Processor
The server-side Semantic Query Processor (SQP) responds to semantic queries from clients of the KIS. The SQP is preferably the main entry point to the Semantic Network on the KIS (or Agency). The SQP is exposed via the Agency's XML Web Service. The SQP processes direct Agent semantic queries and generic (client-generated) semantic queries with semantic link filters (see below). For queries with server-side Agent filters, the Information Agent passes the Agent name and object index arguments to the SQP to be invoked. For example, the browser may ask for objects 0-24 on Agent Documents.Technology.Wireless.All. In this example, the SQP looks up the Agent query in the Agents table and invokes the query onto the database that hosts the Semantic Metadata Store (SMS). The Agent query is preferably stored as SQL or another well-known query format like XQuery or XQL. The SQP may convert the query format to a format that the database (that holds all the tables) understands. Because most commercial databases understand SQL, it will preferably operate as the default Agent query format.
The Agent query preferably follows the query rules described above. Therefore, the query returns the object ID rather than the schema fields for the Agent's object type. In the above-described example, Documents.Technology.Wireless.All invokes the Agent query “SELECT OBJECTID FROM DOCUMENTS WHERE . . . ” The SQP is responsible for issuing a query that is filtered with the Agent query, but which returns the actual metadata for the object type (in this case, the “document” object type). In this example, the query appears as follows:
This query returns the data columns for the “document” schema for all the objects with an object ID that matches those in the original Agent query. The SQP reviews the metadata results of the database query and translates them to well-formed XML using the appropriate schema for the object type of the Agent (in this case, “document”). In the event that the database supports raw XML retrieval, the SQP optimizes the query by asking the database to give it XML results. This results in better performance since the SQP does not have to perform the extra translation step. The SQP passes the XML back to the caller via the Agency's XML Web Service.
The SQP preferably handles more complex queries that are passed by the semantic browser (or other client of the XML Web Service). By way of example, such queries may take the form of the following XML Web Service API:
In this example, the “[ ]” symbols refer to arrays. The API takes a zero-based begin index, a zero-based end index, an optional Agent name, an integer indicating the number of semantic links, an array of operator names, an array of link predicate names, an array of link type names, and an array of strings that refer to the link objects. If the Agent name is NULL (“ ”), the SQP processes the query “as is”; without any preconceived Agent filter. This will be the case with queries that are wholly generated form the client. The arrays are variable sized because the “NumberOfLinks” parameter indicates the size of each array. The operator names include valid predetermined operators, including logical operators, which can be used to qualify queries in SQL or other query formats. Examples include term:or and term:and. The link predicate names may include one or more predefined predicates (e.g., term:relevantto, term:reportsto, term:sentto, term:annotates, term:annotatedby, term:withcontext, etc.). The link type names indicate the type of link objects. Common examples include term:url and term:object. In the case of term:url, the link object string refers to a well-formed URL comprising objects:// . . . or Agent:// . . . . In the case of term:object, the argument will be a well-formed XML metadata instruction referring to a object defined within the present invention. This object is preferably resolved from the client or from another Agency. The API returns a string that contains the XML results (in addition to the return value for the XML Web Service method call itself).
By way of example, the SQML with the data:
f. Natural Language Parser
The Natural Language Parser (NLP) preferably converts natural language text to either an API call that the SQP understands or to raw SQL (or a similar query format) that can be processed by the database. The Natural Language Parser is passed text directly from the semantic browser or by email via the Email Knowledge Agent (see below).
g. Email Knowledge Agent
The KIS preferably includes one primary publishing component, referred to as the Email Knowledge Agent (or Enterprise Information Agent (EIA)). This Agent functions, in essence, as a digital employee, and preferably includes a unique email address (e.g., a custom name selected by the Agency administrator). The Email Knowledge Agent complements existing publishing tools such as Microsoft Office, SharePoint, etc. by adding a “Fire and Forget” method of publishing information and sharing knowledge. This is especially useful in cases where the person publishing the information does not know who might be interested in it.
In a preferred embodiment of the present invention, users send email to the Email Knowledge Agent to publish comments, annotations, documents, attachments, etc. The Email Knowledge Agent extracts meaning from the email and properly adds it to the Semantic Network. Other users are able to access published information via Agents of other platform presentation tools such as drag and drop, the Smart Lens, etc. (discussed below).
The Email Knowledge Agent is a system component that is created by the Agency administrator. The system user name is indicated when the server is first installed. The system user preferably corresponds to an email user in the enterprise email system (e.g., Microsoft Exchange, Lotus Notes, etc.) In this embodiment, the Email Agent has its own mailbox, calendar, address book, etc. These in turn correspond to the objects on the Email Server for the system user. When the server is installed, the KIS installs the appropriate DSA for the system inbox (depending on the email application). The KIS preferably automatically adds a gatherer list entry in the SDG indicating that the system inbox should be periodically crawled for email.
Because the Email Knowledge Agent is a first-class email address, it also serves as a notification source and a query source (for natural-language and instant messaging). Notifications from an Agency are preferably sent by the Email Knowledge Agent (indicating that there is new and relevant information the user might be interested in, etc.). The Email Knowledge Agent may also receive email from users as natural language queries. These messages are parsed by the SQP and processed. The XML results are preferably sent to the user as an HTML file (with the appropriate default Skin) generated with XSLT processed over the XML results of the natural-language query.
Because the Email Knowledge Agent is a regular familiar component or “employee,” the Agency administrator preferably adds the address to distribution lists. This step allows the SDG to semantically index all the email in these distribution lists, thereby populating the Semantic Network by seamlessly integrating the Email Knowledge Agent into distribution lists useful to users. This is a very seamless way of integrating the digital Information Nervous System of the present invention with the way people already work in an organization.
Annotations. The Email Knowledge Agent is preferably used to publish annotations. In the present invention, annotations are preferably email messages. In the preferred embodiment, the annotation object type is a subclass of the email object type. This allows users to use email, typically the most common publishing tool, to annotate objects in the semantic browser. Users are able to annotate objects and add attachments to the annotations. These attachments are semantically indexed by the SDG on the KIS. This makes possible scenarios where a user is able to navigate from, say, a document, to an annotation, to its document attachment, to an article on Reuters, to an industry event that starts next week.
The process described for semantically indexing email (by mapping the email XML schema to the Semantic Network) also applied to annotations. However, in the case of annotations in a preferred embodiment of the present invention, additionally processing is desirable. Specifically, when the user clicks “Annotate” on an object in the Presenter window in the semantic browser (described below), the browser loads the registered email client on the local machine (e.g., Microsoft Outlook, Microsoft Outlook Express, etc.). The “to”' field is populated with the address of the system user for the Agency that hosts the object. The subject field is populated with a special string, for example, “annotation: object=[objectid]”. When the email arrives in the Email Knowledge Agent's inbox, the DSA for the email inbox will pick it up (e.g., via a server event). The SDG retrieves the new email XML metadata from the DSA by receiving an event, or from the DSA the next time it asks the DSA for more data. In a preferred embodiment, this polling process occurs frequently. The DSA returns the XML metadata of the email object, oblivious to the fact that the email object refers to an email object type or an annotation object type. The SDG processes the email XML metadata, and examines the “subject” field. If the SDG “sees” the “annotation:” prefix, it knows that the email is actually an annotation, and proceeds to extract the object ID argument from the subject text. The SDG updates the Semantic Network for remaining email messages (adding each message to the OBJECTS and EMAIL tables, adding semantic links for the “from,” “to,” “cc,” “bcc,” and “attachments” fields, where necessary, etc.). In the preferred embodiment, the SDG performs an extra step. Specifically, it adds a semantic link entry that links the email object with the object indicated by the object ID argument in the subject text (with the PREDICATETYPEID_ANNOTATES predicate).
With the present invention, an annotation is treated as another semantic link with a special predicate. As a result, all the semantic features apply to annotations, such as semantic navigation via semantic links, semantic queries, etc. For example, a user can query for all annotations written by every member of his of her team in the last six months. This can be accomplished in the semantic browser by dragging, for example, the Agent Annotations.All on top of the Agent People.MyTeam.All and then sorting the results, or by creating a Smart Agent, which in turn invokes the “Create Smart Agent” wizard to create the query.
h. Knowledge Domain Manager
The Knowledge Domain Manager is the component on the KIS that is responsible for adding and maintaining domain-specific intelligence on the Semantic Network. The KDM essentially “annotates” the Semantic Network with domain-intelligence. The KDM is initialized with URLs associated with one or more instances of the Knowledge Base Server (KBS), which in turn effectively stores “knowledge” for one or more semantic domains. The KBS has ontology and categories corresponding to taxonomy for each semantic domain that it supports. In addition, an Agent with a semantic domain (connected to a KBS) responds to semantic queries. If an Agent does not belong to a semantic domain, it cannot correspond to semantic queries (that require an ontology or taxonomy). Rather, it only responds to keyword-based queries (albeit it will still provide context and time-sensitive retrieval services, but the available contexts will be limited).
Each entry in the KDM is a semantic domain entry. The semantic domain entry has the URL to the KBS and a semantic domain name. The semantic domain name maps to a specific ontology on the KBS. In the preferred embodiment of the present invention, semantic domain names follow the convention:
<Top Level Domain Name>\<Secondary Level Domain Name> . . .
Examples of semantic domain names include: Industries, Industries\Pharmaceuticals\LifeSciences, Industries\InformationTechnology, General\Sports.Basketball\NBA, General\Sports.Basketball\CBA.
Alternatively, semantic domains names can be referred to as “domain paths” as long as they are fully qualified. Full qualification is achieved by adding an Internet domain name prefix to the beginning of the path. This indicates the “owner” or “source” of the semantic domain. For example, “Nervana.NET\Industries\Pharmaceuticals” refers to “Industries \Pharmaceuticals” semantic domain according to the “NERVANA.NET” Internet domain name. In another example, “Reuters.com\Sports\Basketball” refers to “Sports\Basketball” on “Reuters.com.” Using this approach, domain names and paths are maintained globally unique.
The Knowledge Domain Manager (KDM) periodically requests each KBS in its domain entry list for the categories in the knowledge domain. The KDM is preferably implemented as an XML Web Service on the KIS. The KDM includes configuration options for each semantic domain entry. One of these options may include the schedule with which the KDM will update the Semantic Network with domain-specific intelligence corresponding to the semantic domain entry. For example, the Agency administrator may configure the KDM (via the KIS) to crawl a semantic domain on a KBS every day at 1 pm. The update schedule should be consistent with how often the administrator believes the ontology or taxonomy on the KBS changes.
The KIS preferably invokes the KDM periodically and asks it to update the CATEGORIES table. In the preferred embodiment, the KDM calls the KBS (via an XML Web Service API call) to obtain updated categories for the semantic domain name in the semantic domain entry, which corresponds to a particular taxonomy. An example of an API call follows: GetCategoriesForSemanticDomain (String SemanticDomainName). The KBS returns an XML-based list of all the categories in the semantic domain referred to by the semantic domain name. This XML list is consistent with the CATEGORIES schema shown above (category URL, name, description, the KBS URL and the semantic domain name). The KDM updates the CATEGORIES table with this information. For category entries that already exist in the table, the KDM updates the name and description. For new entries, the KDM requests a new object ID from the object manager and assigns that to the category entry. Since, in the preferred embodiment, a category is an “object,” it inherits from the Object type and therefore has an object ID.
The KDM synchronizes the CATEGORIES table to the CATEGORIES list on the KBS (for a particular semantic domain) by deleting entries in the CATEGORIES table not present in the new list after examining the URL of the category entries and obtaining the relevant KBS URL and semantic domain name. If a semantic domain entry is deleted from the KIS, the KDM deletes all category entries with a corresponding semantic domain name and KBS URL. Essentially, this will be akin to ridding the Agency of existing knowledge.
The KDM periodically categorizes all “knowledge objects” in the Semantic Network based on its semantic domain entries. When new objects are added to the Semantic Network by the SDG, the SDG requests that the KDM categorize the objects. The KDM enumerate all KBS instances in its semantic domain entries and invokes XML Web Service calls with the XML of the object as the argument. In the preferred embodiment, the KBS returns a result in an XML buffer similar to:
This information indicates the semantic categorization weights of the XML object for the categories in the semantic domain on the KBS. In a preferred embodiment of the present invention, the semantic domain entry is initialized with a threshold (0-100) indicating the minimum weight that the KDM should request from the KBS. The KBS returns scores that exceed the predetermined threshold. The KDM annotates the Semantic Network based on these categorization results. This is preferably accomplished by adding or updating a semantic link with the predicate type ID of “belongs to category” with the object ID of the category in the result. The KDM will update the SEMANTICLINKS table. Assuming by way of example that the object that is categorized has an object ID value of 56, the update query appears as follows:
The KDM periodically scans and categorizes all the “knowledge objects” (documents, news articles, events, email, etc., preferably not including objects like people). This process preferably occurs even if an object in the Semantic Network has previously been categorized as the KBS might have become “smarter” and therefore provides superior categorization. In such a case, the results could change even if the same categorization request is repeated. This will occur, for example, if the ontology on the KBS has been updated. Thus, in the preferred embodiment, categorization will be performed both when an object is added to the Semantic Network by the Semantic Data Gatherer and periodically to ensure that the Semantic Network has the most up-to-date domain knowledge.
i. Other Components
The Favorite Agents Manager. On Agencies that support User States, a Favorite Agents Manager manages a list of per-user favorite Agents. In the preferred embodiment, the Favorites Agent Manager stores a mapping of user names to favorite Agents in a UserFavoriteAgents table.
Compound Agent Manager. A Compound Agent Manager manages the creation, deletion, and update of compound Agents. As described above, compound Agents are Agents that are comprised of other Agents in the system, and are initialized to return the union or intersection of the query results in the contained Agents. The Compound Agent Manager manages all compound Agents in the system and maps compound Agents to the Agents they contain via the CompoundAgentMap table.
The Compound Agent Manager exposes functions to create compound Agents, delete, rename, add to and remove Agents from them, and indicate whether a union or an intersection is desired. Compound Agents can be added to other compound Agents. On invocation, the semantic query processor asks the Compound Agent Manager for its compound query. The Compound Agent Manager navigates through its Agent map graph and returns a complex query of all the queries of all Agents that it contains. If Agents are deleted, compound Agents “pick up” the new state when they are invoked, ignoring the Agent query. In other words, the compounding of queries is only done for Agents that still exist. If the compound Agent observes that one of its Agents has been deleted, it will delete the entry from its map.
User Profile Manager. The User Profile Manager (UPM) preferably uses the Inference Engine to infer the user's profile on an ongoing basis. The UPM annotates the Semantic Network based on feedback from users as to their explicit preferences. In the preferred embodiment, this process involved use of the PREDICATEID_ISINTERESTEDIN predicate. The UPM infers semantic links and annotate the Semantic Network with the PREDICATEID_ISLIKELYTOBEINTERESTEDIN predicate. All query results to the user will be qualified (out-of-band) with a query to the Semantic Network for the PREDICATEID_ISLIKELYTOBEINTERESTEDIN predicate. Query results are based on the user's habits, as the Inference Engine learns them over time.
Alternatively, the UPM may be configured with user profile information stored in the User State Store (USS). This is information manually entered at the client indicating the user's preferences. This information is transferred and stored at the server that the user is interacting with. These preferences are tied to different schema. For example, for documents, the schema may be based on the preferred categories. For email messages, the schema may be based on preferred categories, authors, or attachments. These are two of many possible examples. The UPS annotates the Semantic Network based on the manually entered information in the USS.
Server Notification Manager. The Server Notification Manager (SNM) is responsible for batching server-side notifications and forwarding them to users. In the preferred embodiment, users register for server-side notifications at the Agent level. Each Agent is capable of firing notifications of its query results. The Server Notification Manager determines how to filter the query results and format them for delivery via email, voice, pager or any other notification mechanism, e.g., the Microsoft .NET Alerts notification services. The Server Notification Manager maintains information on the last time users “read” the notification. This is preferably indicated from the client via a user interface. The SNM preferably only notifies a user when there is new information on the Agent since the last “read” time for the particular user.
Agent Discovery. Using multicast-based Agent discovery, each Agency sends multicast announcements indicating its presence on the local multicast network. The Agency administrator sets the multicast TTL. The present invention preferably uses either use the Session Announcement Protocol (SAP) with a well-known port of 9875 and a TTL of 255, or a proprietary announcement port with a customizable TTL. For details on SAP, see http://sunsite.cnlab-switch.ch/ftp/doc/standard/rfc/29xx/2974, which is incorporated by reference.
The Information Agent preferably includes a listener component that receives SAP announcements. In the preferred embodiment, the announcements are sent as XML and will include the following information: The server ID (this is a unique identifier);The server URL (this is the HTTP URL to the Agency's XML Web Service); The announcement period (T)—this indicates the time between each announcement; and Whether there are any new Agents in the Agency since the last announcement and the last Agent creation time (on the Agency's clock).
Each Agency sends the XML announcement and uses Forward Error Correction (FEC) or Forward Erasure Correction to encode the packet. This makes the system robust to dropped packets. Alternatively, the Agency can be configured to send the XML announcements several times in succession (per announcement).
The Information Agent multicast listener exposes directory-like semantics to the Semantic Environment Manager. The listener aggregates all the XML announcements from the Agencies from which it receives announcements. It will also cache the last time it received an announcement from each Agency. The listener flags Agencies that it thinks might be dead or inactive. It does this when it has not heard from the Agency for a time longer than the Agency's announcement period. The listener might be configured to wait for several periods before flagging the Agency as inactive. This will handle the case of dropped announcements (due, perhaps, to traffic congestion). The listener will update the Agency list in the Semantic Environment Manager each time it receives announcements.
The Semantic Environment Manager periodically inquiries of the listener whether there are any new Agents. The Semantic Environment Manager checks the Agency list and asks each Agent that is active whether it has new Agents. The Semantic Environment Manager qualifies this request with the Agency's last Agent creation time maintained locally and the current time based on the Agency's clock. The Agency responds and also sends the new value of the last Agent creation time. The Semantic Environment Manager caches this value in the Agency entry. If there are new Agents, the browser inform the user via a dialog box and asks the user whether he or she wants to view the new Agents.
The present invention also supports Agency announcements using a peer-to-peer Agent discovery. In this model, announcements are sent either to a directory server that all clients check or directly to the clients via a standard peer-to-peer publishing protocol.
3. Knowledge Base Server
The Knowledge Base Server (KBS) is the server that hosts knowledge for the KIS. In most applications, many instances of the KIS will be deployed, but only few (or one) KBS will be deployed for any given organization. This is because KBS can be reused (they are domain-specific but data-independent). For example, a pharmaceutical firm might deploy one KBS initialized with a pharmaceuticals ontology, but have several KIS installations; perhaps per employee division or per employee group. The KIS preferably includes the following components:
As explained above, the KIS (via the KDM) periodically sends XML objects to the KBS to categorize them for a given semantic domain.
4. Information Agent (Semantic Browser Platform)
The system client, in the preferred embodiment the Information Agent of the present invention, includes the semantic browser components and user interface that provide a semantic user experience. In the preferred embodiment, the Information Agent provides the following high-level services:
An advantage of the Information Agent of the present invention is that users open up Agents similar how users open up documents from their file-system namespace. The Information Agent will have its own environment that opens up semantic “worlds” of information. For example, ABC company may have an internal KIS Agency that has Agents for internal documents, email, etc. In addition, third-parties may host Agencies on the Internet to hold information on industry reports, industry events, etc. In a preferred embodiment of the present invention, ABC company employees open Agents to discover information on the Internet that relates to their work as well as to semantically relate information that is internal to ABC company to information that is external but relevant to ABC company.
b. Client Configuration
In the preferred embodiment, the system client is able to semantically link information found locally as well as on remote Agencies. This is preferably accomplished through the use of an exposed Semantic Environment comprised of Agencies from a Global Agency Directory, Agencies on the local area network (published via multicast or a peer-to-peer publishing system) and Agencies from a custom Agency Directory using Agent Discovery. The preferred client configuration is based on a framework having Agents and local Agencies, and includes a Semantic Environment Manager, which manages locally saved Agents and Favorite Agents, essentially integrating the history and favorites metaphors. The Semantic Environment Manager uses Semantic Query Documents within the Semantic Environment to present knowledge to users via the Semantic Environment Browser. The client configuration will also include the Agent Discovery information (e.g., Agency lists, Agency directory information, etc.).
c. Client Framework Specification
Overview. The client framework specification provides the service infrastructure for the Information Agent user interface, and defines basic services and interfaces, includes core user interface components, and provides an extensible, configurable environment for the main building blocks of the user interface of the Information Agent. This section described the client framework specification according to a preferred embodiment of the present invention. The Framework Core defines base services, configuration, preferences and security mechanisms. The Core User Interface Components define the user interface services and modules that support server and Agent configuration, control and invocation, and some configuration for the Semantic Browser Framework. The Core User Interface Components are implemented as a Windows Shell extension and associated user interface (described below). The Semantic Browser Framework provides base query and results management services, and the framework for results presentation. The specifics of the user interface related to semantic object presentation are preferably configurable and extensible; even default presentation support is provided as a pre-installed “extension.” The Semantic Browser Framework is preferably implemented as a set of behavior extensions to existing platforms used in Today's Web (e.g., Internet Explorer), and leverages the supported XML, XSLT, HTML/CSS and DOM functionality.
Context. The client framework builds upon semantic services components of the present invention including semantic query support, context and time-sensitive semantic processing and linking of information, etc. The client framework is preferably built as a shell extension and platform (e.g., Internet Explorer) extensions, which provides functionality to users in the context of their existing tools and environment. For example, the Information Agent may be implemented as a Shell Extension (which extends the Windows Shell and employs the standard Explorer view and user interface models). In an alternative embodiment, the present invention is equally applicable in a standalone semantic browser application.
Requirements. The preferred requirements for the client framework relate to flexibility and extensibility. This ensures that the user interface can be easily and quickly adapted as there are more information object types, user profiles, etc. Included are the following:
Semantic Environment Manager (SEM). The SEM manages the creation, deletion, updating and browsing of Agents, Blenders, and Agencies on users' local machines. In addition, the SEM is responsible for listening to Agency multicast announcements, browsing Agencies on the enterprise directory (e.g., via LDAP), browsing Agencies on a custom directory, and browsing Agencies on the Global Agency Directory.
The SEM includes a storage layer that stores the metadata of every Agent on the system, including all the Agent attributes (such as the Agent name, description, creation time, last usage time, the Agent type (Smart, Dumb, Special, etc.), the information object type the Agent represents (for Agents created based on information type), the context type the Agent represents (for Special Agents or Agents created based on a Context Template), the attributes of the Agent, a reference to the XSLT or other script file that represents the Agent's Skin (including filter/sort preferences and other presentation schemes), the notification information and method (if requested for the Agent), and the buffer or file-path/URL to the Agent's SQML query. The Information Agent (semantic browser) may store this Agent metadata in a local database, a store like the Windows registry, or in an XML file store on the local file-system.
The SEM also uses the Agent attribute to indicate whether an Agent is a Favorite Agent. In addition, the SEM automatically deletes Agents that are not favorites and which are older than a configurable age limit (e.g., two weeks).
The Information Agent's Shell Extension and other components (such as the toolbar and the Open Agent dialog) employ the SEM to provide Agent creation, deletion, browsing, updating, and management of Agents via its user interface.
Preferences Manager. This component manages all client-side preferences, providing services to persist the preferences, communicates with servers as needed to share preferences or support roaming, and supports setting and obtaining preference values from other components. This component has associated user interface as well as some more specific preferences user interface components. The preferences are divided into sub-components, and may abstract the preferences for associated client classes. These include:
Notification Manager. Notifications provide a means to indicate to users that there is new information available on a given Smart Agent. Users optionally configure a specific Smart Agent to support or provide notifications (it will be OFF by default for most Smart Agents), and will also configure how to present notifications to users. These notifications are presented by the Notification user interface component.
The Notification Manager is responsible for managing background, polling queries for the appropriate set of Smart Agents. The Live Information Manager is a parallel component that provides similar services to the Results Browser.
The Notification Manager gathers the list of Smart Agents marked for notification, and periodically polls the associated servers for new information. “New” is defined as “since the last poll [or query].” Each time the poll responds, it includes a timestamp indicator that the Notification Manager must persist, associated with the Agent.
The user interface associated with configuring the Notification Manager is preferably implemented in coordination with the Agent Tree View. This enables notifications (e.g., a “Notify” popup menu option of each Smart Agent). The Notification Manager may also support alternatives for notifying the user when there are new results available. Some options include a display style (e.g. bold, colored, etc.) for the Agent in the Agent Tree View, a reminder dialog, audio notification, or more exotic actions like email, IM or SMS notification.
Client-Side Security. Client-side security issues relate to extension code and Skins The Skins are preferably XSLT, but may also support script. In addition, the generated HTML may include references to ActiveX components and behaviors. The presentation sandbox may include security restrictions that prevent Skins from running potentially malicious code via script. For example, the implementation may completely disallow any unsigned code (including ActiveX and DHTML behaviors).
All client-server communication with Agencies are preferably hidden from the published interfaces (for Skins), which third parties will customize to provide custom Skins By isolating the functionality outside of the primary client runtime, the risk of security compromise can be reduced.
Core User Interface Components
Agent Tree View. This is a Shell Extension Tree View that supports much of the core user interface for controlling and invoking Agents.
Semantic Environment Browsing User Interface. This provides user interface to allow users to browse the Semantic Environment. An example of this is the “Open Agent Dialog.” This complements the Agent Tree View, which also displays a hierarchical view of the namespace (see screenshots).
Agent Inspector. This provides user interface to view the properties or edit (in the case of user-created Smart Agents) an individual Agent, Blender or Agency.
Browser Host. This is preferably a “wrapper” on the semantic browser core (e.g., the Internet Explorer browser runtime), which allows the presentation of a custom view of the Agents, Agencies, and Blenders in the Agent Tree View. It preferably does not have any user interface itself, but is a bridge component between the Shell Extension and the Browser Framework. This component is also preferably responsible for coordinating certain browser functionality with the Windows Shell user Interface, including in particular the navigation (“back/forward”) mechanism, in order to provide a seamless “back/forward” user experience (wherein the user only has to deal with one “back/forward” history list).
Core Preferences UI. This provides a user interface for preferences related to Semantic Environment, server, persona and Agent management, as well as any other miscellaneous preference settings. This preferably includes primitive property sheet dialog, possibly divided up into separate sheets by functional area. In the preferred embodiment, this should be a tabbed dialog user interface.
Skin Preferences UI. This provides a user interface for preferences related to Skin management. This is preferably a property sheet dialog. The list of available Skins should be presented as a list, for selection. This user interface allows users to set the current Skins, as distinct from the default Skins It preferably allows users to make the current Skin be the default. For per-Agent Skin preferences, this preferably allows users to select a Skin for the currently selected or opened Agent.
Notification UI. The user interface associated with configuring the Notification Manager is preferably implemented in coordination with the Agent Tree View. The Notification Manager may also support alternatives for notifying users when there are new results available. Some options include a display style (e.g. bold, colored, etc.) for the Agent in the Agent Tree View, a reminder dialog, audio notification, or more exotic actions like email, IM or SMS notification. In the preferred embodiment, the user interface should include a tabbed dialog (or equivalent) to allow users to select out of the aforementioned notification schemes (and the like).
Screen Saver. The user interface preferably provides a special modality to the Results Browser that function like a screen saver, filling the screen in a theater-mode display. In the preferred embodiment, special Skins should be used for the screen-saver mode. These Skins could emphasize a dynamic display that can leverage a larger screen area, but could also use larger fonts and more widely spaced layout.
Results Browser. The Results Browser is responsible for displaying the results of queries, and the information on any local resources opened. The Results Browser preferably obtains one or more XML files from the Query Manager and merges these into a single XML file that represents a list of objects. The list itself may be filtered or sorted as an initial step. The list as a structure is transformed by a special class of Skin (an XSLT transform sheet, possibly including some script) that handles lists. The list-Skin creates the primary DHTML (or the like) structure, e.g., a list, a table or perhaps a timed sequence. Object Skins manage the individual DHTML items that present the information for each object instance. List-Skins may handle the dispatch of individual object Skins (mapping object class to Skin), but the Results Brower preferably provides default mappings of class to Skin for simplicity.
Users may prefer a given form of presentation, and may choose default Skins (both for the list as well as for object classes). The original query (i.e. the SQML) may also include parameters that indicate which Skins should be used (especially which list-Skin) These will be passed to the Results Browser along with the results. The Results Brower uses the facilities of the Skin Manager to select the right Skin to apply. Different rules may be employed for how user preferences and Agent (author) preferences are combined and prioritized.
When query results are composed of multiple distinct XML files, the Results Browser must merge these into a single XML document to provide a seamless user experience. The preferred embodiment provides for handling additional results dynamically. This dynamic update mode is preferably implemented by using a different template or perhaps a script method within the XSLT template. Alternatively, the list Skins may require a behavior (or local runtime component) to manage the logic of adding to the document without disturbing user context.
Query Manager (or Client-Side Semantic Query Processor). The Query Manager is responsible for handling the communication with the server(s), executing the requests for information and gathering the XML results. The resulting XML is passed to the Results Browser for presentation to users.
The Query Manager preferably provides the services to support the Smart Lens functionality. When a Smart Lens request is made, the results are returned as XML and are passed to the Results Browser, preferably marked to indicate that they are Smart Lens results for a given object. The Query Manager preferably includes the following sub-components that provide individual services to fulfill the query requests.
Filter/Sort Manager. The Filter/Sort Manager supports the application of filters and sorts to the lists of results provided to the Results Browser. The Filter/Sort Manager leverages the services of the Filter/Sort Preferences component to obtain user preferences for current settings. The main function of this component is to resolve general preferences, per-Agent preferences, and any settings defined in the actual results (this may or may not be supported). This component is notified by the Filter/Sort Preferences component when users change the currently applied filters and sorts. Because the associated user interface is part of a tool bar associated with the Shell Extension (i.e. its right-pane View), but the application of the functions happens in the Results Browser space, the control is typically indirect.
Lens Mode. When a Smart Lens is invoked, the Results Browser must generate Lens requests (queries) for objects that users choose. The queries are asynchronous so that users can select Smart Lens queries for various objects and view the results as they are returned. A suggested user interface for this is to reserve some real-estate for a Smart Lens icon. When in Smart Lens mode and the user clicks (or hovers) over the Smart Lens icon, a query is issued, and the icon changes to indicate that the query is in progress. When results are returned, they are handled by the Results Browser and dedicated Smart Lens templates in the Skins, and the Smart Lens icon for an object changes to indicate that results are available. Clicking or hovering over the icon again will display the Smart Lens results in a Skin specific manner (see sample Smart Lens pane user interface). If the query is returned quickly enough, then the whole function preferably feels like a popup activated by a hover or single click.
Deep Info View. If Deep Information is not available in the original results, this component generates the associated query. The query is preferably asynchronous. When results are returned to the Results Browser, they are processed through the appropriate Skin (using a special Deep Information template for each Skin), and the resulting HTML is incorporated into the results document under the associated object. The primary Skin for the schema inserts a Deep Information element in the HTML for the object so that the Results Browser knows where to incorporate the results. When Deep Information is available (whether as part of the original results or in response to a Deep Information query), the Skin either displays it directly or will indicate that it is present, and some Skin-defined user interface will allow users to enable the display (e.g. as a popup window).
Context Info Manager. For objects currently displayed in the Results Browser, certain notifications are preferably provided by default. Two classes of new or additional info will be provided to users:
Skin Manager. Maintain user preferences for list Skins, object Skins, and dependencies between list and object Skins (certain object Skins may only make sense for a given list-Skin) The Skin Manager also maintains parameters for each Skin that indicate constraints for the Skin, e.g. how much screen real-estate it requires, or modalities it best applies to. Considerable intelligence is preferably built in that assists the Results Browser to choose Skins for a range of screen and window size constraints, as well as for modalities, accessibility, language and other constraints. Initial versions will likely be much simpler.
Skin Templates. This describes the structure of a Skin and how it is applied from within the Results Browser. A Skin is preferably XSLT templates that convert the results XML to XHTML (and/or other languages like SVG) or proprietary presentation platforms like Flash MX and ActionScript. The templates can also insert styling information, e.g. for CSS styling. The resulting presentation code (e.g., XHTML) can restrict the inclusion of code, for security reasons. Framework code in the Results Browser invokes the Skins The preferred embodiment includes the following classes of Skins.
Skins preferably model constraints such as modality and presentation display area by handling the constraints (passed as parameters either statically or dynamically by events within the browser core itself). This is preferably supported by imposing a restriction that list Skins must specify only acceptable object Skins In an alternative approach, object Skins may be designed for a given list Skin, and the Results Browser/Skin Manager chooses object Skins for the current list Skin.
List Skin Details. Users may choose a single list Skin for the current view and make it the default. List Skins may also be associated with individual Agents, in which case the generic default is overridden. The Results Browser invokes the list Skin to process the list of results, although the list Skin preferably does not actually handle the individual objects. It creates some per-object instance in the framework presentation (e.g., a timed entry in a sequence, or a cell in a table, or an item in a list), and then the object Skins will fill in the details.
Object Skin Details. The object Skins convert a particular schema to XHTML. Support for asynchronous query results for things like Deep Information and Context Template information are provided by invoking associated templates from the Results Browser (through the DOM) on the query results XML, and then inserting the resulting XHTML into the results document through DOM interfaces. There are preferably several individual templates within an object Skin, including:
Information. It may be called for inline deep info provided with original results, or it may be called to handle asynchronously requested Deep Information. Either way, it preferably generates XHTML in some form, which is inserted under the wrapper element for Deep Information. The insertion probably happens in XSLT for inline deep info, and is effected through DOM insertion for Deep Information query results.
In the preferred embodiment, the template cannot modify the other contents of the XHTML (even for the same object), so it will be up to the Results Browser to coordinate the user interface changes that indicate when Deep Information, live information or Smart Lens results are available. The framework requires certain icons to be used (also for consistency), and for these to have regular names or element types, which will allow the Results Browser to find and modify them as needed. In addition, the Results Browser can create and raise events to indicate the state changes. The template-generated script can respond to these events, and display the associated information as desired.
Default Skins. In the preferred embodiment, a set of default Skins is provided. This preferably includes Skins for the basic object classes and a small set of list-Skins that allow a variety of views of query results. Preferable list-Skins include:
e. Client Framework
In the preferred embodiment, the system client includes Shell Extensions, a Presenter, and Skins used by the Presenter to display information with context and meaning
Shell Extension. An Explorer Shell Extension is a Microsoft Windows software component that extends the Windows Shell with custom code. Shell Extensions allow applications to use the Shell as a custom client, and also provide services such as clean integration with the desktop, the file-system, Internet Explorer, etc. Examples of default shell extensions include “My Documents,” “My Computer,” “My Network Places,” “Recycle Bin,” and “Internet Explorer.”
The use of a Shell Extension in the preferred embodiment of the present invention has several advantages:
The Shell Extensions of the present invention provide a view of users' Semantic Environment (e.g., history, favorites and other views). In the preferred embodiment, the Shell Extension provides for the following:
Presenter. The Presenter is a set of local components (e.g., browser plug-ins) that take semantic queries from scripts (or other plug-ins) and pass them off to a KIS Agency XML Web Service. The present invention translates the results of semantic queries and passes XML to other behaviors or scripts for eventual presentation to users.
In the preferred embodiment, the Presenter is invoked by the Shell Extension with an SQML file. The system preferably communicates with the XML Web Service directly. The system resolves the SQML file and invokes calls to open XML information sourced locally or remotely (via XML Web Services on Agencies referred to in the SQML file). Alternatively, if an Agent URL is passed to the system, the Presenter directly opens the URL by invoking it via a call to the XML Web Service of the Agency on which the Agent is hosted. In the preferred embodiment, the system calls the appropriate method with the appropriate semantic object type. Examples of default semantic object types are SEMANTICOBJECTYPEID_EVENT, SEMANTICOBJECTTYPEID_EMAILMESSAGE, etc, which are defined in the header file (semanticruntime.h). The preferred embodiment allows registration of new semantic object types via the RegisterSemanticObjectType API. This semantic query processor on the Agency returns the appropriate XML results using the semantic object type as a filter.
In the preferred embodiment, a Skin according to the present invention (see below) uses XSLT (and/or script) to transform the XML returned from the framework (en-route from the XML Web Service) into DHTML. The Shell Extension allows users to select a new Skin for the current query.
Skins are preferably object-type specific, Context Template specific (for Special Agents) or Blender specific (for Blenders) Skins can also be customized based on the semantic domain name/path or ontology of the Agent, and based on other attributes such as the user's persona, condition, location, etc. Each Agent is configured on an Agency with a default Skin. The present invention further contemplates custom Skins that may be published onto the root Agency (e.g., on the Global Agency Directory). The client preferably downloads the Skin either from the Agency for the declared Agent or from a central server (e.g., the Global Agency Directory), and applies it to the current presentation. The client optionally includes user preferences to ignore Agent Skins or to confine them to a portion of the user interface.
Aside from the Skin type (e.g., object Skin, list/layout Skin, Context Skin, Blender Skin, etc.), in the preferred embodiment, Skins are categorized as follows:
Semantic Skins are preferably required to be interactive, except when they are displayed as part of a tear-off (see above) or screensaver. Each Skin allows users to seek to a particular point in the “semantic presentation.” For example, if the Skin initially displays only the first 25 items, the Skin must have a seek-bar (or other user interface mechanism) to allow the user to seek to the next 25 items, to fast-forward, to rewind, etc. Some Skins have a “Real-Time Mode” option. In this mode, the Skin continuously fetches new objects from the XML Web Service (via pull) Skins are responsible for polling the XML Web Service for new information based on the schema of the desired objects. In the preferred embodiment there are no notifications to the client since the Agency does not maintain any client-specific state for scalability reasons.
Skins optionally include a real-time mode. These Skins are required to be intelligent in that they must cycle through (i.e., present, order or highlight) objects based on priority. For example, if the Presenter relays information indicating that a new object is posted on the Agency, the Skin immediately displays/reorders/highlights this and continues the presentation of the remaining objects. The Presenter determines the ordering and the Skin deals with dynamism given various sort and filter settings. This creates the perception that the semantic presentation is occurring in real-time. In the preferred embodiment, this occurs when there is new data that users are allowed to access using Skins If the list is time-sorted, the real-time presentation may confuse users due to jumping the user interface into an interactive mode. A user preference option in some modes (e.g., screen saver mode) automatically resets the Skin to display the new data (e.g. scrolling to the top of a sorted list when new data is inserted at the top of the list).
In an alternative embodiment, Skins are designed to customize their presentation based on the amount of available presentation window. For example, a Skin may change from static mode to dynamic mode by displaying information using fade-in and fade-out if, for example, the presentation window is relatively small Skins are preferably modal depending upon the expected level of user interaction. For example, a screen saver works differently from a browser; a docked view is similarly different (not only because it is smaller, but because it is assumed to be a kind of background view rather than a focus of user interaction). When a view is minimized or hidden, an alternate mode may be used (especially to indicate new information). Examples are audio notification, reminder-like alerts, start-bar show and blink (like outlook reminders). Agents may be used to send email, telephony or Instant Messenger (IM) notifications. In an alternative embodiment, the present invention contemplates an Agent that posts to a Web site (e.g., automatic HTML content generation for event calendars).
Alternatively, Skins may generate audio-visual information. For example, a text-to-speech Skin may read out an email object. This feature has great potential value for disabled users and for users of auto-PCs, etc., as well as other uses.
In the preferred embodiment, the Skins framework exposes the following services:
Skins. As introduced above, Skins are presentation templates that are used to customize users experience on a per-Agent basis. In the preferred embodiment, Skins are XSLT templates and/or scripts that are hosted on a centralized server Skins according to the present invention preferably generate XHTML+TIME code (e.g., for Presenter display, text-to-speech, Structured Vector Graphics (SVG) via a plug-in, etc.) and access various system services. In the preferred embodiment, Skins support the following features:
1. Display some or all of the fields corresponding to the XML schema of the object(s) being displayed. The Skin optionally provides users a way to uniquely distinguish objects in a returned set or provides users with any conventional access means, for example, filename, URL or personal name (for people).
2. Display a user interface indicating whether the object is understood by the host Agency. Each object preferably includes an “understood” field that indicates this information.
3. For the semantic object type SEMANTICOBJECTTYPE_OBJECT, the Skin optionally displays the raw object metadata or displays the metadata for the XML schema for the class-specific objects that the raw objects represent. For Skins that display class-specific XML schema for queries that refer to raw objects, the Skins must be “smart” to display the class-specific information in different panes. Preferred ways of accomplishing this uses frames, tabbed boxes, or other user interface techniques. Since every semantic query points to raw objects, the Skin preferably either loads the query with the filter SEMANTICOBJECTTYPE_OBJECT (which simply returns raw objects) or the required object type ID. In the preferred embodiment, in order to prepare the presentation of an object list with raw objects of many classes, the Skin should first:
Get the object query
For each semantic object type, determine how many objects exist in the Agent resource for the given object type. This is preferably obtained by calling the Agency XML Web Service method GetNumObjectsOfClassInAgent with the Agent URL and the object type ID name (email, document, event, etc.) as argument. The XML Web Service returns the number of objects in the Agent, satisfying the object type ID filter.
Depending on how many object types there are in the Agent query, the Skin displays frames or other user interface that are appropriate for the number of object types. In the preferred embodiment, when the Skin is ready to load the object type-specific metadata, it calls the Agency's XML Web Service method ExecuteSemanticQuery with the Agent URL and the semantic object type as the arguments
4. When users hover over an object, more metadata for the object is displayable.
5. If a Smart Agent Smart Lens is selected, the Information Agent of the present invention displays contextual metadata that maps the object in the Smart Lens with the object underneath the mouse. In one embodiment, the Smart Lens applies to objects displayed within the Presenter. In alternative embodiment, the present invention allows the Smart Lens to be invoked in other applications (e.g., Microsoft Office applications, the desktop, etc.). This involve installing system hooks to track the mouse and invoke a Smart Lens application when the mouse moves anywhere in the system. The “hook” is called on all mouse events and the hook will also capture the mouse. The Smart Lens may alternatively be invoked asynchronously. In this embodiment, anytime the Presenter displays new results, it checks the clipboard to see if there is any semantic Smart Lens information present. In the asynchronous embodiment, the Presenter automatically caches all the Smart Lens results for all objects in its view. It displays an icon beside each object it presents indicating that there is context-specific related information therein. In a preferred embodiment, users are able to invoke a Smart Lens for any object in the view.
6. Breaking Information. Each object preferably displays a user interface indicating whether there is “breaking information” relating to the object. This is the semantic equivalent of “breaking news.” The user interface is preferably presented to indicate the criticality of the information, yet must not be too intrusive in case users do not want to see the information. For example, the user interface may be shown as an icon that slowly blinks at a corner of the object display window. When users hover over the icon, metadata on the “breaking information” is displayed. In the preferred embodiment, “breaking information” is implemented by an implicit Special Agent that invokes calls to all Agents using the Breaking News Context Template.
7. Each object is preferably displayable with a user interface indicating whether the object has any Annotations. This information is included as a field in all query results for all objects.
8. Preferably, each object is displayable with a user interface indicating whether there is related information on any predefined Context Template or Special Agent on the client. This preferably includes Special Agents created by users, as well as default Special Agents (e.g., installed by the client). In the preferred embodiment, Context Palettes for the Context Templates are displayed with the user having the option of displaying one or more of the Context Palettes, hiding them, scrolling them (in order to navigate the Context Palettes), etc. Context Templates and Context Palettes are discussed in further detail below. In an alternative embodiment, Agency priorities preferably include the following:
Critical priority. This is the highest priority. For example, for a given document, this flag will be TRUE (on the Agency) if a related email message was just posted (in this example with a few minutes) or if there is an upcoming event that is imminent.
High Priority. This is the next highest priority. The user interface feedback preferably makes it clear that the priority is high enough to warrant users' attention, albeit the feedback must not be very intrusive. The priority is optionally different for different Users, e.g., if there is an event that is local to users the priority might be higher than if the event is remote (particularly if there is no way for the remote user to participate in the event).
Medium Priority. This may merely indicate that there is information that users should look at if they have the time. The user interface feedback must make this clear.
Low Priority. This may indicate that there is related information that is germane but not recent.
The four priority virtual Blenders are preferably installed by default on the client. These Blenders automatically aggregate information from corresponding priority Agents on each Agency in the My Agencies list. There is preferably default priority Agents on every Agency. In the preferred embodiment, relational semantic queries take the context and the user into consideration.
In the preferred embodiment for each Context Template (or the currently selected Context Template), the Presenter enumerates the Agencies that users add to their My Favorite Agencies list or the recent Agencies, and queries appropriate Agencies using dynamically generated SQML to find out if there are any objects that relate to the current object based on the Context Template. If any of the Agencies in the favorites or recent lists are not accessible, the user interface preferably transparently handles this by ignoring the Agency. In the preferred embodiment, by default, the dynamically generated SQML is created by indexing the SQML of the currently selected object's SRML and inserting the resource in the SQML as a link filter in the SQML of the Context Template (preferably using the default predicate “relevant to”). This intelligently handles the mapping of the object type of the currently selected object to the semantics of the displayed Context Palette. For example, if the currently selected object is a document, the Headlines Context Palette uses the SQML based on a derivation of the SQML for the Headlines Context Template. Each Agency in the Semantic Environment semantically processes the resulting SQML appropriately using the default predicate. In another example, if the selected object is a person, the Headlines Palette shows the Headlines relevant to the person, e.g., the “Headlines” authored or annotated by the person, etc. Alternatively, if the currently selected object is a document or email message, the SQML (with the default predicate) produces semantic results that represent semantically related Headlines on each Agency. These results are preferably displayed in the Context Palette. The same applies to other Context Palettes (e.g., Classics, Newsmakers, etc.).
For a person object, the priority flag preferably refers either to objects the person has posted or to objects the person authored or is hosting. In this example, only metadata fields with semantic uniqueness are preferably used to make this determination (e.g., the person's email address).
9. Each object preferably displays a user interface including a number of manipulation options. By way of example only, a sample user interface illustrating an information object displayed in the Information Agent (semantic browser) Results Pane is shown in
Intrinsic Semantic Links. These are links that are intrinsic to the semantic class of the object. If there are no Intrinsic Semantic Links, nothing needs to be displayed. By was of example, an email object of the preferred embodiment includes the following Intrinsic Semantic Links:
In the preferred embodiment, when any of these semantic links are invoked by users, the client fetches the metadata for the associated object (and not the object itself). This allows users to explore the semantic information for aspects of the original object. The Skin preferably calls the XML Web Service of the Agency that hosts the object with the appropriate method. In the preferred embodiment, the form of this method is ISemanticRuntimeService::LoadNativeSemanticLink. This embodiment includes the semantic class ID, the name of the semantic link, the name of the argument, and the string form of the argument. For example, to “navigate” to the third attachment (with a zero-based index), the Skin should call LoadNativeSemanticLink(SEMANTICCLASS_EMAILMESSAGE, “Attachments”, “Index”, 2). This preferably generated the SQML that represents this relational semantic query, creates a new temporary Smart Agent that has this SQML and loads the Smart Agent. This illustrates preferred semantic navigation. The process is optionally recursive. The user can navigate off the new results using any of the new objects and pivots, etc.
An example of a balloon popup associated with an Intrinsic Semantic Link showing an email sample according to the present invention is shown in
Annotations. This preferably allows users to navigate to a summary view for all the Annotations for the current object. In the preferred embodiment, the Skin displays all the Annotations by calling the ISemanticRuntimeService::EnumAnnotations (with the object metadata as argument). This returns an XML representation of the property table containing the metadata for the Annotation objects. The Skin preferably displays some representation of the Annotation summary being displayed (e.g., names or titles of the Annotations). When an Annotation link is invoked by users, the Skin displays metadata for the Annotation object. These functions preferably come from filters applied on the client. Alternatively, these functions can be created as an Agent. This aspect of the present invention further illustrates semantic navigation. The Annotations are preferably loaded using an SQML representation of the “Annotations” query. This creates a new Smart Agent with this SQML. The Smart Agent is then added to the “recent” list and loaded (or navigated to). The process is optionally recursive. The user can navigate using the newly displayed Annotation(s) as pivots, etc.
Related Objects. In the preferred embodiment, this optionally allows users to find related information on each Agency included in the users' My Agencies list using the current object as an Information Object Pivot. This is preferably accomplished without resorting to a copy and paste or reliance on the Shell Extension user interface). In the preferred embodiment, the user interface popup shows information in the following format:
The “All my agencies” list is obtained by the Presenter simply by enumerating the Agencies users have registered locally. The Presenter returns the “Agencies that understand this object” list by “asking” each locally registered Agency whether it understands the object in question. The Presenter passes the XML representation of the object to the Agency, which attempts to semantically process the XML representation. The Agency returns a flag indicating whether it understands the object. The Presenter optimizes the returned list by excluding the Agency on which the object itself is hosted since each object has a field that indicates whether the Agency understands its contents.
Verbs. This allows users to invoke any actions that relate directly to the current object. For example, a document or an email message can have an “Open” verb. This opens the word processor or email client and displays the information. An event can have an “Add to Outlook Calendar” verb. In the preferred embodiment, verbs, preferably class-specific, are invoked on the client by the system framework. The Agency need know nothing about verbs. In the preferred embodiment of the present invention, there are several verbs for every object. These verbs are preferably displayed first in the popup menu. In the preferred embodiment the verbs include:
1. Annotate. When the user invokes this verb, the Skin preferably communicates with the client runtime and calls the Annotate method. This method initiates the default mail client with the appropriate subject line (which the Agency parses to interpret the Annotation). Users send a regular email message as an Annotation for the object. Email Annotations optionally include attachments that also constitute semantic links. This allows users to navigate from an object (e.g., a document) to its Annotation to its attachment and then to an external content source (e.g., via a Smart Lens). Alternative embodiments are also supported for Annotations, e.g., simple form-based or dialog-based annotations. But email provides the most semantic richness.
2. Copy. This copies the object XML to the system clipboard.
3. Hide. This indicates that users have no interest in viewing the object.
4. Open. This is qualified with the link of what is being opened. In the example of a document, “Open Document” may be displayed. For an email message, “Open Email” may be displayed. The client opens the object with the default application registered in the system for the link's MIME type. In an alternative embodiment, the present invention support other related open verb form, such as “Open with . . . ”, which allows users to open the object with a specific application.
5. Mark as Favorite. This is preferably displayed if the Agency supports User State and if the object is not a favorite.
6. Unmark as Favorite. This is preferably displayed if the Agency supports User State and if the object is a favorite.
An example of a balloon popup associated with a Verb user interface according to the present invention is shown in
Annotate (Opens Outlook; if the object is from an Agency, the Agency's Email Agent address is filled in the “to” field; if not, the “to” field is left blank so the user can indicate the Agency for object annotation association). If the object is not from an Agency, the object should be attached to the email message either as an URL or as a full-blown attachment).
Mark as Favorite (stored on the client)
Unmark as Favorite
Person and Customer: +=“Send Email”
10. When a Skin loads a new query or the metadata for one or more objects, the Skin preferably calls the framework with the query or the metadata. In the preferred embodiment Skins do not perform queries, but passes queries to the Presenter runtime which then managers the results.
11. Deep Information (or Presentation) Mode. An alternative embodiment the present invention provides Skin support for Deep Presentation Mode. In this embodiment, the Skin displays a user interface indicating whether there is related information for the current object. The Skin also displays text describing the information. For example, for a given document object, the Skin may display a popup with the text “Jane Doe posted the most recent email message that relates to this object: <summary of email message>” In this embodiment, the Skin shows details for specific information, such as the last recently posted related object or the most imminent upcoming object. The Skin may optionally display other “truths” or inferred data that might be interesting to users. Examples include:
Lisa Heilborn recently posted a related document: <summary>
The most likely author of this document is <foo>
Steve Judkins reports to Patrick Schmitz. Patrick has posted 54 critical priority objects that relate to this one.
This document has 3 likely experts: <names>
Yuying Chen appears to have the most expertise on this document.
The present invention framework exposes several “semantic depth” levels that Skins use to obtain information. Smart Lenses may also be configured to support Deep Presentation Mode. In other words, in the preferred embodiment, invoking a Smart Lens on an object returns the deep information similar to what is shown above. The Skin shows an icon at a corner of the object display window. Users are able to click that icon to display the “deep information.” Metadata for the “deep information” can optionally be fetched asynchronously.
An example of a balloon popup associated with a Deep Information Mode user interface according to the present invention is shown in
e. Semantic Query Document
From the client's perspective, every thing it understands is a query document. In the present invention, the client opens “query documents” in a way analogous to how a word processor opens “textual and compound documents.” The client is primarily responsible for processing a Semantic Query Document and rendering the results. A Semantic Query Document is preferably expressed and stored in form of the Semantic Query Markup Language (SQML). This is akin to a “semantic file format.” In the preferred embodiment, the SQML semantic file format consists of the following:
By way of example, SAMPLE B of the Appendix hereto illustrates a Semantic Query Document in accordance with the present invention.
In the preferred embodiment, the Presenter includes an SQML interpreter. When the Presenter opens an SQML file, it preferably interprets it by first parsing it, validating it, creating a master entry table, and then executing the entries in the entry table. Effectively, it “compiles” the SQML file before “executing” it, not unlike how a language compiler compiles source code into an object module before it is then linked with other modules and executed. In the case of the SQML interpreter, this process optionally involves loading other SQML files via references. This process is preferably not cyclical. The client uses the XSLT templates in the “<skin>” tags (if available and if not overridden by default or Agent Skins) to display the information for each declared object type. Any returned objects that do not have a declared Skin are displayed with the default Skin of the object type or, in the case of a single Agent entry, that of the Agent (if one is specified).
In an alternative embodiment, the client may load a new Skin to display each object type even after the Semantic Query Document is opened. In this embodiment, the “<skin>” tags preferably inform the client which Skin to load the query with initially. In this embodiment, the specified Skin is preferably appropriate for the declared object type.
In the preferred embodiment, the framework executes the document in two phases: the validation phase and the execution phase. For the validation phase, the interpreter first builds a master semantic entry table. The table is keyed with the resource URL and also has columns for the operator, the resource, the resource type, the predicate, the predicate type, and the link. The interpreter excludes all redundant entries as it adds entries into the table. Also, interpreter preferably canonicalizes all URLs before it adds them into the table. For example, the URLs “http://www.abccorp.com” and “www.abccorp.com!” are interpreted as being identical since they both share the same canonical form. The interpreter builds and maintains a separate SQML reference table. This table includes the canonical path to the SQML file. When the interpreter loads the original SQML file, it adds the canonical file path to the reference table. If the SQML file points to itself, the interpreter ignores the entry or returns an error. If the SQML file points to another SQML resource, it adds the new file to the reference table. It then recursively loads the new resource and the process repeats itself If, during the process, the interpreter comes across an SQML entry that is already in the reference table, the interpreter returns an error to the calling application (indicating that there is a recursive loop in the SQML document). As the interpreter finds more resources in the document graph path, it adds them to the master entry table for the given resource. It dynamically adds links for a given resource to that resource's entry in the entry table. As a result, the interpreter effectively flattens out the document link graph for each resource in the graph.
The interpreter then proceeds to the execution phase. In this phase, the interpreter reviews the semantic entry table and executes all the resource queries asynchronously, or in sequential fashion. Next, it processes each resource based on the resource type. For example, for file resources, it opens the property metadata for the file and displays the metadata. For HTTP resources that refer to understood types (e.g., documents), the interpreter downloads the URL, extracts it, and displays it. For Agent resources, it calls the XML Web Service for each Agent and passes the links as XML arguments, qualifying each link with the operator. In the preferred embodiment, operators for links that cross document boundaries are always AND. In other words, the interpreter will AND all links for identical resources that are not declared together because recursive queries are assumed to be filters. The interpreter issues as many calls to a component representing the resource as there are Agent resources. For each link, the interpreter resolves the link by converting it into a query suitable for processing by the resource. For example, an Agent with a link with the attributes:
In order to optimize the query, the Agency XML Web Service exposes methods for passing several arguments qualified with operators (and, or, etc.). The interpreter preferably issues one call to the XML Web Service for the Agent resource with all the link arguments.
Semantic Query Implementation Scenarios. The following are exemplar scenarios illustrating the implementation and operation of Semantic Query Documents according to a preferred embodiment of the present invention.
Scenario 1: Loading an SQML Document. The client creates a temporary file and writes into it a buffer containing the attributes of simple, local HTML page. This page includes the client framework component (e.g., an ActiveX control, a Java applet, an Internet Explorer behavior, etc.). The page is initialized with this component opening the SQML file and a unique ID identifying the Information Agent instance. The component itself opens the SQML file. In other words, the client framework tells the plug-in what SQML query document to open. The plug-in opens the Semantic Query Document by interpreting it as described above.
Scenario 2: Open Documents. The client opens the standard dialog box, which allows users to select files to be opened. The dialog box is initialized with standard document file extensions (e.g., PDF, DOC, HTM, etc.). When users select the documents, the dialog box returns a list of all the opened documents. The client creates a new SQML file and adds resource entries with the paths of the opened documents. The new SQML file is given a unique name (preferably based on a globally unique identifier (GUID)). Because this is a temporary file, the name is preferably not exposed to users. The methodology proceeds to Scenario 1 as described above.
Scenario 3: Open Folder in Documents. The client creates an SQML file (as described above) and initializes it with one resource entry: file://<folderpath>?includesubfolders=(true|false). The SQML file is loaded (as in Scenario 1) by enumerating all the documents in the folder and displaying the metadata for the documents.
Scenario 4: Save as Agent. The client opens a dialog box allowing users to set the Agent name. The client renames the Agent in the Semantic Environment (see below) to the new name. The Agent being saved may be temporary or may already have been saved under a different name. The Information Agent preferably suggests an Agent name.
Scenario 5: Save into Blender. The client opens a dialog box that allows users to select a Blender. The dialog box preferably allows users to create a new Blender. When the Blender is selected, the client opens the Blender's SQML file into the SQML object model and adds the new entry (the currently loaded SQML file). It then increments the reference count of the current entry.
Scenario 6: Drag and Drop. The client creates and opens an SQML file with a single resource entry, for example, similar to the following:
Scenario 7: Multiple Drag and Drop. The client creates and opens an SQML file with a single resource entry, for example, similar to the following:
Scenario 8: Smart Lens. When a Smart Lens is selected in the Information Agent, the Information Agent indicates to the Semantic Environment Manager (see below) that a Smart Lens has been selected for the Information Agent identifier. When the Skin notices that the mouse is over an object (e.g., via the “onmouseover” event in the document object model (DOM)), it calls the Presenter first to find out whether the Information Agent is in Smart Lens mode. The client framework determines this by asking the Semantic Environment Manager if an Information Agent with the identifier is in Smart Lens mode. Because the Semantic Environment Manager caches this information from the Information Agent itself, it can answer the question on behalf of the Information Agent. If the Information Agent is in Smart Lens mode, the client framework preferably obtains the SQML buffer from the system clipboard via the Semantic Environment Manager. This is because a Smart Lens is a virtual “paste” in that it obtains its information from the clipboard. In other words, any object or Agent that is copied to the clipboard can be used as a Smart Lens (even regular text). The framework obtains the SQML buffer and instantiates resource components for every resource in the SQML buffer. The client framework calls the resource API GetInformationForSmartLens passing the XML information for the currently displayed object to the resource. All resources preferably return Smart Lens metadata to the client framework. Each resource preferably returns metadata in the form of a list of Smart Lens information nuggets. Each nugget contains a text entry and a list of query buffers (in SQML). The text entry contains simple text or a custom text format, for example, similar to the following:
Each “<A>” tag pair preferably includes a corresponding SQML query buffer in the information nugget. The client framework formats the text into DHTML (or similar presentation format) for display in the Information Agent (e.g., as a balloon popup or other user interface, preferably not to block or conceal the object that the mouse is over). The client framework displays a user interface for links (analogous to HTML links) where the containing “<A>” and “</A>” tags are found. When a link is invoked, the client framework calls the Semantic Environment Manager to create a new cache entry. The Semantic Environment Manager indicates what file-path the entry should be stored in. The client framework writes the SQML buffer for the <A>tag that was clicked into the file. The client framework pushes the SQML document to the Semantic Environment Manager and loads the SQML into the Information Agent (via Dynamic HTML). Because the Semantic Environment Manager includes this SQML document as the current document, users are able to save the document via the “save” button in the Information Agent (e.g., “save as Agent” or “save into Blender”). An example of information that a Smart Lens can display is as follows:
In the preferred embodiment, any information that can be contained in SQML can be invoked as a Smart Lens (e.g., Agents, people, documents, Headlines, Classics, Agencies, text, HTTP URLs, FTP URLs, files from the file-system, folders from the file-system, email URLs from an email application such as Microsoft Outlook, email folder URLs, etc). For example, users are able to copy regular text from text-based applications to the clipboard. If users enter the Information Agent and select the Smart Lens, the SQML version of the text will be invoked as a Smart Lens (via a “document” resource). If the “text Smart Lens” is then hovered over a document object, the document resource representing the text Smart Lens optionally displays the similarity quotient, indicating to users similarities between the Smart Lens object and the object underneath the mouse. If the object underneath the mouse is a person object, the document resource may decide to “ask” the Agent representing the person object whether the Agent is an expert on the information contained in the text. Alternatively, the Smart Lens might display links to similar documents or email messages the person has authored that relate to the text.
Scenario 9: Copy and Paste.
Copy: On invocation of a Copy command from within the Semantic Environment, the client framework copies an SQML buffer to the system clipboard with a custom clipboard format. This ensures that other applications (e.g., Microsoft Word, Excel, Notepad, etc.) do not recognize the format and attempt to paste the information. The SQML buffer is preferably consistent with the semantics of the object being copied. For example, a copy operation from an object being displayed in the Presenter is copied as a resource with the appropriate resource type and URL from whence the metadata came. Copying an icon representing an Agent copies the URL of the Agent or the cache entry referring to the Agent's entry in the Semantic Environment. Copying information from a desktop application (e.g., Microsoft Outlook) copies SQML with a resource type referring to the source application and URLs pointing to the objects within the application. These URLs are preferably resolvable at runtime by the interpreter to objects within that application. For example, copying an email message from Outlook to be copied into the Semantic Environment may create a resource entry as follows:
Paste: On the invocation of a Paste command, the client framework creates an SQML file based on the clipboard format of the information being pasted. For example, if the clipboard contains a file path, the SQML file contains a link (from the resource on which the Paste was invoked) to an object with the file path. This file is opened as described above. If the clipboard format is an URL, the object is of the URL object type. If the format is regular text, the object contains the actual text with, in this example, the resource type nervana:text. Alternatively, the client framework creates a temporary cache entry, stores the text there (e.g., as a .TXT file), and stores the SQML object with a reference to the file path and the object type, in this example, nervana:filepath. When the interpreter is invoked, it creates an XML metadata version of the text and invokes the resource with the XML link argument. If the clipboard format is the SQML clipboard format of the present invention, a similar process is performed, except that if a file is created, the extension will be .SQM (or .SQML). This indicates to the interpreter that the object is an SQML file and not just a regular text file.
f. Semantic Environment
A preferred embodiment of the Semantic Environment of the present invention provides a view of every Agent and Agency available to user via the Information Agent. This preferably includes Agents that have been saved locally into the favorites “My Agents” list, recently used Agents, Agents on local Agencies, and Agents on remote Agencies. Remote Agencies include Agencies that announce their presence via multicast on the local area network, Agencies available on a Global Agency Directory, and Agencies available on a custom Agency Directory. Agents can be dynamically added to the Semantic Environment by invoking their URL. In the preferred embodiment, the Semantic Environment hierarchy has the pattern shown in SAMPLE C of the Appendix hereto. “Recently Used,” “Recently Created” Agents are preferably collapsed to “Recent Agents.” Optionally, “All Agents,” “Deleted Agents,” and “Custom View” may be added.
The Agencies view allows users to see the Agents in the main view by Agency. The object type view allows users to see the same Agents, but filtered by object type. Other views operate in similar fashion, e.g., “By Context” (based on Context Templates) and “By Time.” The Semantic Environment merges the notion of “favorites” with the notion of “history.” The Semantic Environment optionally adds dynamically managed views such as “recently used Agents,” etc. These views are preferably updated by code running within the Semantic Environment Manager (see below).
Icons incorporated into the Semantic Environment may include the following:
Snapshots. Users are preferably able to save a snapshot of the Semantic Environment. A Semantic Environment snapshot essentially is a time-based cache of the state of the Semantic Environment. In the preferred embodiment, a snapshot includes locally stored state with the following information:
g. Semantic Environment Manager
The present invention provides a Semantic Environment Manager that exposes APIs to manage the Semantic Environment objects. In the preferred embodiment, the managed Semantic Environment objects are comprised primarily of Agent references via SQML buffers. The Semantic Environment Manager also exposes APIs to navigate the Semantic Environment. In the preferred embodiment, the Semantic Environment Manager allows instances of the Information Agent to:
Agent in the Semantic Environment (e.g., saved or local Agents, Standard Agents, Blenders, etc.). In the preferred embodiment, notification methods include sending email, instant messages, pager messages, telephony messages, etc. The Semantic Environment Manager includes a Notification Manager (see below), which will manage notification requests from users via the Information Agent. The Notification Manager stores a list of notification requests. A notification request preferably includes the Semantic Environment object ID (which identifies the Agent), the type of notification (email, IM, etc.) and the destination, e.g., the email address, etc. The Notification Manager periodically polls each Agent in the notification request list to “ask” if there are any new objects. The Notification Manager also passes the “last requested time” (based on the destination Agent's clock). The Agent responds with the number of new objects (by invoking its stored query and passing back the number of objects in the query results that were created since the “last requested time”). The Agent responds with the current time (on its clock). The Notification Manager stores the Agent's time to avoid time synchronization problems. Alternatively, the client and all Agencies use the same time server (a time Web service) to get their time to ensure that all time comparisons will be on the same scale.
Agency Directories. In the preferred embodiment, the Semantic Environment Manager preferably maintains an Agency list for each Agency “directory.” The multicast network preferably looks to the Semantic Environment Manager as a directory of Agencies. In the preferred embodiment, there is a default Global Agency Directory configured with the URL to an XML Web Service on a public system. This XML Web Service stores a cache of all registered Agencies (preferably with the information described above, including ID, URL, etc.). The XML Web Service exposes methods to allow Agencies to register their presence on the Agency Directory. The XML Web Service filters redundant entries. The XML Web Service also exposes methods to allow users to enumerate all Agencies on the Agency Directory. The Semantic Environment Manager enumerates the directory in this manner. Preferably, the Information Agent considers the Agency Directory as an extension of the Semantic Environment, and allows users to browse and open Agents on the Agencies listed on the Agency Directory. Users are preferable able to add URLs to custom Agency Directories that may be installed on the internal network. The present invention contemplates the creation and integration of customizable Agency Directories. This essentially is an alternative to using multicast for discovery in cases where multicast may not be enabled on the network (for bandwidth conservation reasons) or where certain subnets on the wide area network do not support multicast.
h. Environment Browser (Semantic Browser or Information Agent™)
The Environment Browser, or Information Agent, hosts a regular Web browser component (such as the Internet Explorer ActiveX control), and is primarily responsible for taking an SQML file and rendering the results via the Presenter. In the preferred embodiment, it does this by opening a local HTML file initialized with a reference to the SQML document cache entry of the SQML file. The HTML file loads the Presenter through a control (e.g., ActiveX, Java, Internet Explorer behavior, etc.). This control retrieves the SQML document from the cache (via the Semantic Environment Manager) and loads the SQML file as described above. The control adds objects to the Web browser document object model (DOM) as it received callbacks from resources indicating that objects are available to be converted to XHTML (or equivalent presentation format, preferably via the current XSLT and/or script-based Skin, and pushed into the DOM for presentation. The Information Agent allows users to open an SQML file or an entry in the cache (via the cache ID). The Information Agent also allows users to navigate back and forward, and to navigate the first document in the stack (analogous to the “back,” “forward,” and “home” options in Today's Web browsers, the difference being that in this case SQML documents are being opened for interpretation and display (of the results) as opposed to HTML and other documents).
In a preferred embodiment, the Semantic Environment can include a toolbar popup menu option having tools allowing users to import local search results into the Semantic Environment, e.g., via a Dumb Agent, to create a new Special Agent, a new Blender, or a new local Agency. Alternatively, these tools can be collapsed into one tool button that invokes a wizard from which users can select the kind of Agent (Dumb, Smart, Special) or Agency they wish to create. A sample dialog can allow users to search the Semantic Environment using keywords, for example, creating a new Smart Agent (with the appropriate SQML). Users are preferably able to customize the name of the new Smart Agent and add an optional description. A “Save” tool popup menu options of the toolbar that can allow users to save a newly created or opened Agent permanently into the Semantic Environment (e.g., into the “favorites” list), or to save the Agent into a Blender. A Smart Lens tool menu option of the toolbar can allow users to invoke the Smart Lens (based on the Smart Agent or object that is currently on the clipboard, communicating to the Presenter that the user wishes to use the clipboard contents as a Smart Lens. The Presenter can preferably automatically invoke the Smart Lens functionality for any object users hover over (e.g., with the mouse). The menu can also show a “Paste as Smart Lens and Pin” option that keeps the Smart Lens turned on (even across Agent navigations) until the user explicitly turns off the Smart Lens. An “Open Agent” dialog view allows users to open server-side Agents from the Semantic Environment and change the “view” of the Environment (e.g., Large Icons, Small Icons, List, etc.). A standard Windows “Open” dialog allows users to import a “regular” document from the file system into the Semantic Environment of the Information Nervous System. For example, a Dumb Agent can be created that refers to the document(s). In use, when the Dumb Agent is invoked, the document(s) is opened in the Information Agent and all of the semantic tools (e.g., smart copy and paste, Context Templates, etc.) are enabled with the document(s). By this method, the browser can make a regular, “stupid” document on the file system semantically “smart.” A custom “Open Documents in Folder” dialog can allow users to search for documents on a folder on the local file system and import them into the Semantic Environment. This makes the documents “smart” by “exposing” them via the semantic tools of the Information Nervous System (e.g., smart copy and paste, Context Templates, etc.). A “Browse for Folder” dialog box can be displayed when users select a browse option, allowing users to select a folder to open (from the local file system). A page from the “Add Blender” wizard can allow users to select whether they want to create a standard Blender or a virtual Blender.
i. Additional Application Features
Application Menu Extensions and other Framework Features. The system client preferably installs a menu extension to applications that support programmatic extensions but that do not already support copying data to the clipboard. These include applications such as Microsoft Windows Media Player and Microsoft Outlook (for email message headers). In the preferred embodiment the menu extension reads “Copy.” The system copies the selected object as an XML object to the Windows system clipboard. For example, the system plug-in for an email Microsoft Outlook copies a selected email object as an Email XML Object. For applications that already support the clipboard, no extension is needed.
Server-Side Favorite Objects. On Agencies that support User State, users are able to mark objects as “favorites.” When an object is marked as a favorite, the Presenter invokes a method on the Agency's XML Web Service. The XML Web Service adds a semantic link between the user object and the object in question. In the preferred embodiment, users are able to view favorite objects via the All.MyFavorites.All Default Agent. This Agent returns all objects that have been marked as favorites. The Agency administrator is able to create sub-Agents such as All.MyFavorites.Technology.XML.All.
The Presenter allows users to mark and unmark favorites, which is also a means of redefining the structure that the servers and Agencies export. The use of “favorites” scenario is especially valuable in cases where users may see objects of interest and not want to navigate them immediately. The favorites feature may optionally be also used by the Agency to recommend objects to users. In the preferred embodiment, these recommended objects are retrievable via the All.Recommended.All Agent. The Agency recommends objects based primarily on objects that users have marked as being favorites. Server-side favorites will also preferably be used with the “favorites,” Classics and Recommendations Context Templates.
Agent Screen Savers. A preferred embodiment of the present invention allows users to select any subscribed Agent as a screen-saver. Users are preferably warned that Agents may expose sensitive data and given an opportunity to determine whether it is safe to use a particular Agent as a screen-saver. In the preferred embodiment, the system client is capable of loading any subscribed Agent as a screen-saver. In an alternative embodiment, users may combine Agents to provide a desired screen-saver presentation. Alternatively, a screen-saver may be a structured Skin that includes displayed parallel Agents, for example, in four quadrants of the screen.
Agent-Agent Smart Lens. In an alternative embodiment, the system client supports the use of a Smart Lens (invoked either through an Agent or a Blender) as a context to invoke another Agent or Blender. For example, users may select All.CriticalPriority.All and want to use that Agent as a Smart Lens to browse All.Understood.All in order to find out all objects that are critical priority and which are also understood by the destination Agency.
Smart Lens Sample User Interface Illustrations.
Blender Skin User Interface Illustrations.
Multiple Drag and Drop. In an alternative embodiment, the system client allows users to select multiple documents or folders from the desktop and use them as the basis of relational queries on an Agent or Blender. This allows users to further refine a query using multiple documents as the refining tool. For example, the user may optionally indicate whether they want the union or intersection of the results (using each of the documents as a filter). This creates an SQML file with one resource (the object over which the links were dragged) and multiple links (one per document or dragged object). The client's SQP preferably interprets this by retrieving the XLM metadata for all the object filters and calling the destination Smart Agent's XML Web Service with the XML arguments. In the preferred embodiment, the Agency's XML Web Service categorizes the XML metadata arguments, forms the proper SQL representation of the query and returns the results.
URL Shortcut Conventions. Agencies of the present invention may share the Internet Web since they are optionally installed as Web applications. As a result, Agencies can be referred to using the Web's naming scheme (e.g., a regular HTTP URL). In the preferred embodiment, the present invention exposes shortcut naming conventions and URLs that are specific to the Information Agent's Semantic Environment.
Sharing and Roaming Client Information. In the preferred embodiment, users are able to share Agents (including Blenders) with others by sending them via email, instant messaging, etc. Local information users are preferably able to either store Agent information locally or have the information roam with them (e.g., via AbccorpliMirror support in Windows 2000 for department-wide roaming, via a proprietary XML Web Service on a Global Agency Directory (using passwords for identity), or via integration with Microsoft .NET My Services, which employs Microsoft's Passport identity service).
Local Agencies. The system client preferably also allows users to create and add local Agencies that run a local instance of the KIS to the “My Agencies” list. In this embodiment, the client also allows users to delete a personal Agency.
User-Experience Consistency and Non-Disruptiveness. The Information Agent (semantic browser) of the present invention provides a consistent and undisruptive user experience. In other words, the Information Agent seamlessly coexists with Today's Web browser. Tools such as “Back,” “Forward,” “Home,” “Stop,” “Refresh,” and “Print” preferable work as they do with Today's Web browser so as not to confuse the user. Many of the tools remain the same albeit the functionality is different. In addition, new tools are preferably added to the toolbar and menu options reflecting the new functionality in the semantic browser (these can be seen by observing the toolbar in the screenshots).
5. Providing Context in the Present Invention
a. Context Templates
The present invention provides Context Templates, or scenario-driven information query templates that map to specific semantic models for information access and retrieval. Essentially, Context Templates can be thought of as personal, digital semantic information retrieval “channels” that deliver information to a user by employing a predefined semantic template. In the preferred embodiment, the semantic browser 30 allows the user to create a new “Special Agent” using Context Templates to initialize the properties of the Agent. Context Templates preferably aggregate information across one or more Agencies.
By way of example only, the present invention defines the following Context Templates. Additional Context Templates directed towards the integration and dissemination of varied types of semantic information are contemplated within the scope of the present invention (examples includes Context Templates related to emotion, e.g., “Angry,” “Sad,” etc.; Context Templates for location, mobility, ambient conditions, users tasks, etc.).
“Headlines” Context Template. The Headlines Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of CNN's “Headline News” program in how it conveys semantic information. The Context Template allows a user to access information headlines from one or more Agencies, sorted according to the information creation or publishing time and a configurable amount of time that defines information “freshness.” For example, CNN's “Headline News” displays headlines every 30 minutes (around the clock). In a preferred embodiment, the Information Agent 30 of the present invention allows users to create a Headlines Special Agent using the following filters and parameters:
In addition to freshness, the Headlines Context Template preferably incorporates how “hot” the result items are in order to determine the ranking of the results. This may be accomplished by querying the Agency to find out the number of semantically related objects on the Agency, which is a good indicator of whether an object's topic is “hot.” In addition, returned objects (or items) are preferably sorted by freshness or as new.
By way of example, SAMPLE D of the Appendix hereto illustrates an SQML output from a Headlines Context Template of the preferred embodiment. In this example, the Context Template retrieves all information from four different Agencies (marketing, research, sales, and human resources), with a freshness time span of 30 minutes, and with a “relevant to” predicate (indicating a semantic query). In the preferred embodiment, the SQML of this example, as for all Context Templates, can optionally form the basis of a Smart Lens, smart copy and paste, drag and drop and other tools in the semantic toolbox.
“Breaking News” Context Template. The Breaking News Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of CNN's “Breaking News” program inserts that interrupt regularly scheduled programming in how it conveys semantic information. Like CNN's “Breaking News” inserts, this Context Template allows users to access “breaking,” time-critical information from one or more Agencies, preferably sorted by the information creation or publishing time or the event occurrence time (in the case of event), and with a configurable amount of time that defines freshness and a configurable “deadline” for events to define time-criticality. For example, the Context Template can be defined to filter information objects posted in the last one-hour, or events holding in the next one day.
In the preferred embodiment, the Breaking News Context Template is different from Breaking News Agents. The Context Template is a template that defines static query parameters that are passed to one or more Agencies. A Breaking News Agent is any Smart Agent users may have created and is essentially user-created and user-customizable. By way of example, a Breaking News Special Agent based on the Breaking News Context Template may inform users of information objects posted in the last hour or events holding in the next day that relate to a local document (or any other local context, if specified). But a Breaking News Agent gives users the flexibility of receiving alerts for “Events on wireless technology being given by a member of my team and holding either Seattle or Portland in the next 24 hours and which relate to this document on my hard drive.” The Breaking News Agent provides users much greater flexibility and personalization than the Breaking News Context Template. An advantage of the Breaking News Context Template is that it preferably forms the basis for intrinsic alerts by using parameters that qualify as “breaking” for typical users.
“Conversations” Context Template. The Conversations Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of CNN's “Crossfire” program in how it conveys semantic information. Like “Crossfire,” which uses conversations and debates as the context for information dissemination, in the preferred embodiment, the Conversations Special Agent tracks email postings, annotations, and threads for relevant information. The Conversations Context Template may be thought of as the Headlines Context Template filtered with email object type. In addition to the “Headlines” parameters, the Conversations Context Template preferably (but optionally) contains the following parameters:
“Newsmakers” Context Template. The Newsmakers Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of NBC's “Meet the Press” program in how it conveys semantic information. In this case, the emphasis is on “people in the news,” as opposed to the news itself or conversations. Users navigate the network using the returned people as Information Object Pivots. The Newsmakers Context Template can be thought of as the Headlines Context Template, preferably with the “People” or “Users” object type filters, and the “authored by,” “possibly authored by,” “hosted by,” “annotated by,” “expert on,” etc. predicates (predicates that relate people to information). The “relevant to” default predicate preferably is used to cover all the germane specific predicates. The sort order of the relevant information, e.g., the newsmakers, is sorted based on the order of the “news they make,” e.g., headlines. In addition to the Headlines Context Template parameters, the Newsmakers Context Template preferably contains the following optional parameters:
“Upcoming Events” Context Template. The Upcoming Events Context Template (and its resulting Special Agent) can be analogized to a personal digital version of special programs that convey information about upcoming events. Examples include specials for events such as “The World Series,” “The NBA Finals,” “The Soccer World Cup Finals,” etc. The equivalent in a knowledge-worker scenario is a user that wants to monitor all upcoming industry events that relate to one or more categories, documents or other Information Object Pivots. The Upcoming Events Context Template is preferably identical to the Headlines Context Template except that only upcoming events are filtered and displayed (preferably using a semantically appropriate “context Skin” that connotes events and time-criticality). Returned objects are preferably sorted based on time-criticality with the most impending events listed first.
“Discovery” Context Template. The Discovery Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of the “Discovery Channel.” In this case, the emphasis is on “documentaries” about particular topics. Unlike in the case of “Headline News,” the primary axis for semantic information access and retrieval is not time. Rather, it is one or more category with an intelligent aggregation of information around those categories. In a preferred embodiment of the present invention, the Discovery Context Template simulates intelligent aggregation of information by randomly selecting information objects that relate to a given set of categories and which are posted within an optionally predetermined, configurable time period. While there is an optional configurable time period, the semantic weight as opposed to the time is the preferred consideration for determining how the information is to be ordered or presented. The present invention allows for different axes to be used, for example, the semantic weight for the category or categories being “discovered,” time, randomness, or a combination of all axes (which would likely increase the effectiveness of the “discovery”). The Discovery Context Template preferably has the same parameters as the Headlines Context Template, except that the freshness time span is replaced by an optional maximum age limit, which indicates the maximum age of information (posted to the Agency) that the Agent should return.
“History” Context Template. The History Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of the “History Channel.” In this case, the emphasis is on disseminating information not just about particular topics, but also with a historical context. For this template, the preferred axes are category and time. The History Context Template is similar to the Discovery Context Template, further in concert with “a minimum age limit.” The parameters are preferably the same as that of the Discovery Context Template, except that the “maximum age limit” parameter is replaced with a “minimum age limit” parameter (or an optional “history time span” parameter). In addition, returned objects are preferably sorted in reverse order based on their age in the system or their age since creation.
“All Bets” Context Template. The All Bets Context Template (and its resulting Special Agent) represents context that returns any information that is relevant based on either semantics or based on a keyword or text-based search. In this case, the emphasis is on disseminating information that may be even remotely relevant to the context. The primary axis for the All Bets Context Template is preferably the mere possibility of relevance. In the preferred embodiment, the All Bets Context Template employs both a semantic and text-based query in order to return the broadest possible set of results that may be relevant.
“Best Bets” Context Template. The Best Bets Context Template (and its resulting Special Agent) represents context that returns only highly relevant information. In a preferred embodiment, the emphasis is on disseminating information that is deemed to be highly relevant and semantically significant. For this Context Template, the primary axis is relevance. In essence, the Best Bets Context Template employs a semantic query and will not use text-based queries since it cannot guarantee the relevance of text-based query results. The Best Bets Context Template is preferably initialized with a category filter or keywords. If keywords are specified, categorization is performed by the server dynamically. Results are preferably sorted based on the relevance score, or the strength of the “belongs to category” semantic link from the object to the category filter.
“Favorites” Context Template. The Favorites Context Template (and its resulting Special Agent) represents context that returns “favorite” or “popular” information. In this case, the emphasis is on disseminating information that has been endorsed by others and has been favorably accepted. In the preferred embodiment, the axes for the Favorites Context Template include the level of readership interest, the “reviews” the object received, and the depth of the annotation thread on the object. In one embodiment, the Favorites Context Template returns only information that has the “favorites” semantic link, and is sorted by counting the number of “votes” for the object (based on this semantic link).
“Classics” Context Template. The Classics Context Template (and its resulting Special Agent) represents context that returns “classical” information, or information that is of recognized value. Like the Favorites Context Template, the emphasis is on disseminating information that has been endorsed by others and has been favorably accepted. For this Context Template, the preferred axes includes a historical context, the level of readership interest, the “reviews” the object received and the depth of the annotation thread on the object. The Classics Context Template is preferably implemented based on the Favorites Context Template but with an additional minimum age limit filter, essentially functioning as an “Old Favorites” Context Template.
“Recommendations” Context Template. The Recommendations Context Template (and its resulting Special Agent) represents context that returns “recommended” information, or information that the Agencies have inferred would be of interest to a user. Recommendations will be inserted by adding “recommendation” semantic links to the “SemanticLinks” table and by mining the favorite semantic links that users indicate. Recommendations are preferably made using techniques such as machine learning and collaborative filtering. The emphasis of this Context Template is on disseminating information that would likely be of interest to the user but which the user might not have already seen. For this Context Template, the primary axes preferably include the likelihood of interest and freshness. In the preferred embodiment, the Context Template is implemented by generating SQML that has the PREDICATETYPEID_ISLIKELYTOBEINTERESTEDIN predicate as the primary predicate filter on the Agencies in the Semantic Environment.
“Today” Context Template. The Today Context Template (and its resulting Special Agent) represents context that returns information posted or holding (in the case of events) “today.” The emphasis with this Context Template is preferably on disseminating information that is deemed to be current based on “today” being the filter to determine freshness. In the preferred embodiment, the Today Context Template results are a subset of the Headlines Context Template results wherein the results posted “today” or events holding “today” are displayed.
“Variety” Context Template. The Variety Context Template (and its resulting Special Agent) represents context that returns random information. The emphasis with this Context Template is preferably on disseminating information that is random in order for the user to get a wide range of possible information items. In the preferred embodiment, the primary axis is randomness, albeit the “random” items will be semantically relevant to the query filter (using the “relevant to” predicate).
b. Context Skins
The present invention includes a special class of Skins called “Context Skins ” Context Skins include presentation information that conveys the semantics of the context that they represent. For example, a Context Skin for the Today Context Template may display a background or filter effects with a clock pointing to midnight, or some other representation of “Today.” In yet additional examples, a Context Skin for the Variety Context Template may show transform effects like bowling balls falling over randomly (indicating the randomness of the results); the Breaking News Context Skin may show effects and light animations with flashing text, ambulance red lights, etc. to indicate the criticality of the context; and the History Context Skin may show graphics that indicate “age”; for example, old cars, clocks, etc.
Context Skins preferably “honor” the presentation template for object types being displayed. For example, email objects may be displayed with a background showing stamps or a post office truck in addition to graphics that indicate the Context Template. Because some Context Templates cut across Agencies—and therefore cut across ontologies—they need not display any information that indicates ontology (e.g., industry information). However, Context Skins that are initialized with a category filter preferably indicate the category or ontology of the Context Template. Typically this will be represented with graphics elements (and filters, transforms, etc.) that indicate the industry or genre of the ontology. For example, a Pharmaceuticals Context Skin may have filter effects showing laboratory equipment; an Oil and Gas Context Skin may show pictures of oil rigs; and a Sports Context Skin may show pictures of sports gear, etc.
c. Skin Templates
The present invention allows a user to select different kinds of Skins, depending on the task at hand. The implication of having flexible presentation is that the user can select the best presentation mode based on the current task. For example, users may select a subtle Skin when working on their main machine and where productivity is most critical and where effects are not. Users may select a moderate Skin in cases where productivity is also important but where effects will also be nice to have as well. Users may select an exciting Skin for scenarios like second machines, for example where users are viewing information in their peripheral vision, and features such as text-to-speech to alert them on breaking news is important. Exciting Skins may feature animations, storyboard like effects for deep information, objects displayed on motion paths, and other effects. Exciting Skins are most likely going to be used with screensavers. The choice of Skins is preferably user-definable.
d. Default Predicates
In the preferred embodiment, each object type includes a default predicate that links it with other object types. This provides users with an intuitive method of dynamically linking objects together without requiring a separate evaluation of the predicate to use for the semantic link. For example, a drag and drop operation from a document object to an Agent that returns documents can have the predicates “Related To” and “Possibly Related To.” When a document object is dragged on top of a document Agent, the semantic browser of the present invention displays a popup menu option that allows users to select the predicate to use for the semantic query. In an alternative embodiment, other related popup menus may be incorporated, e.g., a first popup menu that allows users to select the link or predicate template; child popup menus that display the actual predicates for the selected template. The default predicate is preferably inserted in the dynamically generated SQML from which the query will be invoked.
By way of example, a default predicate may be “relevant to.” This predicate maps to a query that returns information in the document Agent that is relevant to the object being dragged. The advantage of having a default predicate in this case is that the semantic browser of the present invention may display a popup menu option named “Open” that in turn invokes a query using this predicate. The semantic browser may also display a popup menu option named “Open with Link” that has submenu options with specific predicates. The default predicate makes the system easier to use because users are able to browse the system using dynamic linking, knowing that the default predicate will be the sensible option giving the source object and that target Agent or object.
In addition to being used in drag and drop scenarios, Default Predicates are optionally used in Smart Lenses, smart copy and paste, etc. Default Predicates may be analogized to degenerate smart links that return “the right thing” given the context. Preferably the default predicate will be “relevant to,” which may in turn produce “The right thing” as the appropriate query result for a semantic distance of one. In an alternative embodiment, the Default Predicate may be a merger of several specific predicates. For example, the Default Predicate for a document-to-people drag or drop, copy or paste, or Smart Lens may be “relevant to” and may be interpreted by the KIS Agency XML Web Service as, for example, a cascaded query involving “authored,” “expert on,” and “annotated” predicates. In other words, “relevance” is interpreted smartly by the present invention and may involve merging together different predicates.
Default Predicates allow users to navigate the system quickly and efficiently and with little thought. Default Predicates provide the system with simplicity and make it intuitive to use. In addition, users are comfortable with Default Predicates because users are already used to invoking HTML links on Today's Web where there is only one predicate: “invoke”.
e. Context Predicates
Context Predicates are predicates that are defined at a high level of abstraction and which map to a relevant subset of the Context Templates. Context Predicates allow a user to select a predicate filter based on a Context Template, rather than on a low-level system predicate. When the query is invoked with the Context Predicate, filtering the containing SQML with the filter parameters of the Context Template generates a new SQML query. For example, the Context Predicate “Best Bets” maps to the Context Template of the same name and filters a query with those information objects that are “best bets” (typically, these will be those items that are returned from a semantic query and not from a text-based query). Similarly, the Breaking News Context Predicate filters items based on whether they qualify with the filter conditions of the Breaking News Context Template. In general, Context Predicates are applied for object types that are consistent with the Context Template (for example, the Context Predicates “Experts” and “Newsmakers” will only be valid for queries that return “Person” objects).
f. Context Attributes
Context Attributes are “virtual attributes” that are cached as part of each XML object that an Agency returns to the client. These attributes are dynamic in that they reflect the current context in which the results are being displayed. For example, where relevant, the Context Attribute “Best Bet” is attached to each XML result that satisfies the semantic query filter in the SQML of the current query. The results of a semantic query with default predicates might include both semantic and non-semantic (text-based query) results. The Agency processing the query may cache Context Attributes for the XML results that are “Best Bets” by running a semantic sub-query on the SQML with the result object as a filter. In this case, the schemas for the “Object” and derived types should include attribute fields for each relevant Context Template (e.g., a “Best Bet” attribute, “Headline” attribute, etc.). This is the preferred implementation. Alternatively, the semantic browser calls the Agency, passes each XML object as an argument and “asks” whether the object satisfies the Context Attribute. Other examples are a Headline Context Attribute that indicates whether the object qualifies as a “Headline” in the context of the current query, a “Classics” attribute, etc. The semantic browser should display a user interface indicating whether the context attribute is set or not.
Context Attributes provides further benefits over the prior art systems in that they make the system easier to use. For example, a user can perform a drag and drop operation to generate a relational query that includes both semantic and non-semantic query filters (as processed by the Agency when it receives the SQML arguments from the client). In one embodiment, the browser “asks” the user whether he or she desires a broad query or a “Best Bets” query. In this mode, the user effectively applies for an additional filter before the query is issued. Alternatively, the Agency, in concert with the semantic browser, preferably returns the results of the broad query, and also qualifies each result with a context attribute and corresponding user interface indicating whether each result object is “broad” or a “Best Bet.” The same applies to other object types like the “Person” object type. Rather than having the user indicate whether a relational query to a Person Agent should return “authors,” “experts,” or “annotators,” the browser can issue a broad query and than qualify the results (with help from the Agency) with whether each returned “Person” object is an “author,” “expert,” or “annotator,” for the current context.
g. Context Palettes
Context Palettes are a very powerful feature of the present invention that involves invoking Context Templates dynamically for the currently selected object within the semantic browser. Essentially, Context Palettes are preferably automatically invoked and displayed when users select any object in the Results Pane. Context Palettes enable users to always have the context for the currently displayed results at their disposal. In addition, the semantic browser constantly refreshes the palette for the currently selected object, thereby guaranteeing that the context for the object is always up to date. In a preferred embodiment, this is accomplished via a timer that triggers a refresh action or by querying the SQML query processor for the Context Palette for whether there is any new object since the last time the palette was refreshed.
In the preferred embodiment, results displayed in Context Palettes are “first-class” information objects in the same way as the information objects displayed in the main Results Pane. In other words, Context Palette results are preferably used with all of the present invention's semantic tools, e.g., smart copy and paste, Smart Lens, Deep Information, etc. The same preferably is true for results displayed in other context panes anticipated in the present invention.
The present invention preferably includes the following Context Palettes. In the preferred embodiment, users have the option to “scroll” through the different Context Palettes for a selected object. The incorporation of additional and different Context Palettes is expressly anticipated, and may parallel the addition of Context Templates.
“Headlines” Context Palette. This uses the Headlines Context Template and employs SQML that has the SQML of the Headlines Context Template with an additional link to the currently selected object, and the default predicate for the object-type combination. In particular, the SQML will be keyed off resources that map to all the favorite Agents or recent Agents in the Semantic Environment. The user configures whether he or she wants Favorite Agents, recent Agents, or both to be used when generating the Context Palette. In addition, the Headlines Context Palette is also configurable to show headlines without any filter for the number of objects to be displayed or the “freshness” time limit. In this case, the palette will allow the user to navigate all the relational results sorted by the publication or post time.
“Breaking News” Context Palette. Contains relational results from every Breaking News Agent in the Semantic Environment using the default predicate of the object-type combination, and linked with the currently selected object. In addition, results for the default Breaking News Context Palette are displayed. The semantic browser of the present invention will dynamically generate SQML with as many (and identical) resource or link combinations as there are Breaking News Agents, with additional links that have the default predicate and the resource qualifier of the currently selected object (a file-path, folder-path, object://URL, etc.). The semantic browser of the present invention invokes the generated SQML query and loads the palette windows with the SRML results. The Breaking News Context Palette preferably contains navigation controls to allow users to navigate the results in the Context Palette.
“Conversations” Context Palette. Similar to the Headlines Context Palette except utilizing the Conversations Context Template.
“Newsmakers” Context Palette. Similar to the Headlines Context Palette except utilizing the Newsmakers Context Template.
“Upcoming Events” Context Palette. Similar to the Headlines Context Palette except utilizing the Upcoming Events Context Template.
“Discovery” Context Palette. Similar to the Headlines Context Palette except utilizing the Discovery Context Template.
“History” Context Palette. Similar to the Headlines Context Palette except utilizing the History Context Template.
“All Bets” Context Palette. Similar to the Headlines Context Palette except utilizing the All Bets Context Palette.
“Best Bets” Context Palette. Similar to the Headlines Context Palette except utilizing the Best Bets Context Template.
“Favorites” Context Palette. Similar to the Headlines Context Palette except utilizing the Favorites Context Template.
“Classics” Context Palette. Similar to the Headlines Context Palette except utilizing the Classics Context Template.
“Recommendations” Context Palette. Similar to the Headlines Context Palette except utilizing the Recommendations Context Template.
“Today” Context Palette. Similar to the Headlines Context Palette except utilizing the Today Context Template.
“Variety” Context Palette. Similar to the Headlines Context Palette except utilizing the Variety Context Template.
“Timeline” Context Palette. This Context Palette preferably contains merged results from the Headlines, Best Bets, History, and Upcoming Events Context Templates. The Timeline Context Palette preferably allows the user to navigate all objects on the semantic timeline based on the currently selected object. The timeline may contain information items based on their publish/post time, event items based on their appointment time, etc. Essentially, with the Timeline Context Palette, the user navigates relevant (and perhaps other semantically related) objects using time as the primary axis for information conveyance.
“Guide” Context Palette. The preferred embodiment of the present invention includes a unified Guide Context Palette. This Context Palette combines all Context Palettes. In other words, each window in the Guide Context Palette corresponds to one result from each of the other system Context Palettes. The user interface for the Guide Context Palette allows the user to scroll through the results for each Context Palette in each window or to animate the results using animation techniques, for example, fade-in/fade-out techniques. A preferred use of the Guide Context Palette is to view context for the currently selected object in a minimal viewing space. In the preferred embodiment, the use has the option of viewing all Context Palettes side-by-side (vertically, horizontally, diagonally, etc.), docked, or in other arrangement formats.
Context Palette User Interface. The user interface for Context Palettes is preferably configurable based on the layout Skin for the currently displayed Agent. In the preferred embodiment, Context Palettes may be docked on the left, right, top or bottom of the Results Pane. Context Palettes may be collapsed in order to minimize intrusion into the viewing area and dynamically re-expanded to full view. Skins may also allow Context Palette windows to be resized to variable sizes or preset, fixed sizes. Alternatively, some Skins may also animate Context Palettes results.
By way of example,
h. Intrinsic Alerts
In a preferred embodiment, in addition to the Breaking News Agent, the present invention provides for Intrinsic Alerts. While conceptually similar to Breaking News Agents, Intrinsic Alerts are fundamentally different in operation. In the case of Breaking News Agents, the present invention signals the user as to breaking news notifications after polling each Breaking News Agent specified by the user and querying it to find if there is anything related to the current object that is breaking An Intrinsic Alert does not require the user to specify a Breaking News Agent or otherwise perform any action in order to introduce breaking news notification. An Intrinsic Alert is automatically signaled in the user interface (for all currently displayed objects) when there is an event that relates to the object at issue in a fundamental, intrinsic way. For example, if the current object is a document, the present invention polls the Agency from whence the document came and asks the Agency if there is any recently posted information on the Agency that relates to the object. If the current object is a person, the present invention may poll the Agency and ask if the person recently sent email, recently posted a document, recently annotated a document, recently joined or exited a distribution list, etc. This allows the user to have in-place information within the native context of the object in a time-sensitive manner.
In the preferred embodiment, the default implementation for Intrinsic Alerts will poll only the Agency from whence the object came. This has the advantage of simplifying the user interface; if the user wants to perform cross-Agency queries, he or she has the option to drag and drop, copy and paste, etc. in order to invoke relational queries. In alternative embodiments, Intrinsic Alerts will poll multiple Agencies, including Agencies other than from whence the object came, in an effort to locate breaking news notifications.
In an alternative embodiment, the present invention is configurable to maintain information as to whether a user has accessed an object. This may be analogized to how an email server keeps track of what email messages a user has read. In an embodiment in which the Agency supports per-object, per-user server-side state, Intrinsic Alerts are always accurate because the Agency indicates that there is “intrinsic breaking news” only if there is information on the Agency that relates to the object in question that has not been accessed or read by the user. This alternative is preferably accomplished means of an additional filter on the SQML query.
The alternative of a per-object, per-user server-side state required for this embodiment has disadvantages, especially for Agencies that will hold massive amounts of information and will have a huge number of users (e.g., Internet-based Agencies). In this situation, the system does not scale well if state is maintained per object and per user.
In an alternative embodiment where the Agency does not support per-object, per-user server-side state, the Agency may be configured with a static freshness time limit for Intrinsic Alerts. For example, the server may be configured with a freshness time limit of thirty minutes, in which case the server would respond in the affirmative if an Intrinsic Alert query is received within thirty minutes of the arrival of a new object that relates to the object in the query. In a preferred embodiment, the KIS Agency maintains information on the average information arrival rate. This way, a busy server will have a lower freshness time limit than a server that seldom receives new information. This embodiment is not as accurate as if the server kept per-object, per-User State because the average arrival rate produces only an approximation of whether an alert should be signaled. This embodiment will still result in reduced information loss. In the preferred embodiment, the present invention optionally signals Intrinsic Alerts in a non-intrusive manner that suggests their probabilistic nature (i.e., that an alert is only a best guess).
i. Smart Recommendations
Smart Recommendations represent semantic queries to the Semantic Network, for inferred semantic links, using an object as an Information Object Pivot. For example, the Inference Engine may infer that users would like to attend a certain event, based on events they have attended in the past, the fact that they have been engaged in many email conversations with the presenter of the event, etc. By way of example, in the preferred embodiment, this information is available in a Smart Recommendations popup context Results Pane such as that shown in
In the preferred embodiment, each link is generated by the object Skin or a special recommendations information pane Skin and will link to SQML containing the predicates for the inferred semantic links
6. Property Benefits of the Present Invention
The Information Nervous System of the present invention provides proper context, meaning and efficient access to data and information to allow users to acquire actionable knowledge. Many of the advantages of the Information Nervous System over Today's Web and the conceptual Semantic Web are derived from its use of the technology layers shown in
The present invention employs semantic links, ontologies, and other well-defined data models using XML. As a result, an Agency as described above has the power of a semantic Web site in that its information includes semantics. In addition, by providing meaning as an intrinsic part of the XML Web Service, it further provides context-sensitivity, time-sensitivity, etc. associated with the subject matter information.
Intelligent system Agents described above monitor the private context of users and automatically alert users when there is relevant information on an information source (or sources) related to the specific context. By way of example, these specific contexts may include the following: My Documents, My Web Portal, My Favorite Websites, My Email, My Contacts, My Calendar, My Customers, My Music, My Location, “This” document, “This” Web site/page, “This” email message, “This” contact, “This” event in my calendar, “This” customer, and “This” music track, album or play-list.
The present invention provides a context-sensitive user experience via the use of information Agents associated with the server 10 and via the semantic browser 30 and associated XML Web Service. For example, users automatically connect information in “My Documents,” “My Email,” etc. (from application islands such as the file system, Microsoft Outlook, etc.) to remote information sources that have semantically relevant information. Users have the flexibility to make these connections in real-time via application-level innovations that reside on top of the Semantic Network such as the new query tools described above, for example, drag and drop, Smart Lenses, smart copy and paste, etc. It is also contemplated that such application tools can be used independent of a Semantic Network, for example, integrated into an existing browser of Today's Web.
In a preferred embodiment, the KIS of the present invention pulls semantic information from the Semantic Web or other repository with semantic markup (preferably via RDF plug-ins) into its Semantic Network. Alternatively, the system 10 of the present invention exists without the Semantic Web. In this situation, the KIS builds its own Semantic Network (e.g., a private semantic web) from data sources that the system administrator selects (e.g., email, documents, etc.). The system 10 of the present invention is able to utilize the actual semantic applications with a semantic backend (which can optionally include the Semantic Web). The system 10 thus provides context-sensitivity via integration with client-side applications (including the proprietary semantic browser 30), location-tracking tools, etc. and the proprietary XML Web Service (which the Semantic Web does not describe). More specifically, while the conceptual Semantic Web describes architecture for semantic linking and knowledge representation, it does not address scenarios and innovations using XML Web Services to provide context-sensitivity, time-sensitivity, dynamic linking, Context Templates, Context Palettes, etc. In contrast, the present invention addresses semantic linking via the semantic data model and Semantic Network as well as provides software services for context sensitivity, time-sensitivity, semantic queries, dynamic linking, Context Templates, Context Palettes, etc. via integration with its proprietary XML Web Service.
The present invention has an intrinsic notion of time-sensitivity. For example, by providing features related to time-sensitivity such as Breaking News Agents, Breaking News Context Templates, Breaking News Context Palettes and intrinsic alerts, the present invention demonstrates the importance of time as an element in semantics and presentation. While not universally true, generally speaking old information is usually not as relevant as new information. For example, when CNN interrupts news broadcast to show breaking news, the interruption is based on a combination of semantics (the relevance of the breaking news about to be displayed) and the fact that the news is indeed breaking Except is those rare cases where the Web author specifically builds in time-prioritized analysis, this time-sensitivity element as an axis for alerts and presentation is totally lacking in Today's Web and in the conceptual Semantic Web.
The present invention allows users to select Smart Agents as Breaking News Agents. Any information being displayed will show alerts if there is relevant breaking news on a breaking-news Agent. For example, with the present invention, a user is able to create an Agent as: “All Documents Posted on Reuters today” or “All Events relating to computer technology and holding in Seattle in the next 24 hours” as Breaking News Agents. Because these Agents are personal (“breaking” is subjective and depends on the user), the browser provides uniquely individual support. In yet another example, a user in Seattle would be able to schedule notification on events in Seattle in the next 24 hours, events on the West Coast in the next week (during which time he or she can find an inexpensive flight), events in the United States in the next fourteen days (the advance notice for most U.S. air carriers to obtain a competitively priced cross-continental flight), events in Europe in the next month (likely because he or she needs that amount of time to get a hotel reservation), and events anywhere in the world in the next six months.
The present invention further supports a Breaking News Context Template based on which users can create Breaking News Agents. In addition, the present invention supports a Breaking News Context Palette that allows users to view all displayed results in the context of a template-based definition of “breaking news,” thereby seamlessly and intelligently integrating context and time-sensitivity.
The present invention further provides a powerful personal historian tool for performing historical analyses. Using browse history, past events, and document creation times, the system 10 can compensate for faulty memory by recalling details from an event, for example, showing results to the query “The coworkers who attended the design meeting from 6/1/98 through 6/1/99”. Alternatively, the system may seek for a cluster of events. For example, investigators may ask for “All stock market transactions greater than $10M related to the airline stocks from 7/1/01 up to 9/11/01” or “Show all documents created within a ten day window of this event”.
The system 10 of the present invention has an intrinsic notion of discovery. In a preferred embodiment, the KIS automatically announces its presence on a local multicast network, an enterprise directory (e.g., an LDAP directory or the Windows 2000 Active Directory), a peer-to-peer system or other system. Ideally, the semantic browser 30 periodically listens for multicast or peer-to-peer announcements and checks an enterprise directory or a Global Agency Directory. The browser also allows the user to navigate the system in a hierarchical fashion to locate additional Agencies. This way, users are notified when new Agencies are available and when existing Agencies expire. The semantic browser of the present invention preferably notifies users instantly when new Agencies are available via namespace snapshots and periodic checks for announcements and directory presence.
The peer-to-peer aspect allows the system 10 to scale and automatically populate the enterprise directory without any centralized maintenance (which is a large ongoing cost for organizations). The system preferably uses programmatic queries for new classes of servers, thereby eliminating the needs for Web logs.
The present system 10 provides fundamental advantages over Today's Web and the conceptual Semantic Web by employing smart objects having intrinsic behavior. The system embeds behavioral characteristics in each Agency's XML Web Service, thereby make each node in the Semantic Network much smarter than a regular link or node on Today's Web or the Semantic Web. In other words, in the preferred embodiment, each node in the Semantic Network of the present invention links to other nodes independent of authoring. Each node has behavior that dynamically links to Agencies. Smart Agents also allow for such additional features as drag and drop and smart copy and paste, creating links to Agencies in the Semantic Environment, responding to lens requests from Smart Agents to create new links, including intrinsic alerts that will dynamically create links to time-sensitive information on its Agency, including presentation hints for breaking news (wherein the node can automatically link to breaking news Agents in the namespace), etc. These features dramatically increase the user's ability to, for example, find and navigate new links. Once the user reaches a node in the network, the user has many semantic means of navigating dynamically and automatically using context, time, relatedness to smart Agencies and Agents, etc. By making each node in the network smarter, the entire Semantic Network becomes a smart, virtual, self-healing and self-authoring network.
The dynamic linking technology of the present invention allows users to issue queries across local/remote information boundaries. For example, the present invention (preferably using SQML technology) allows a user to issue a query like: “Find me all email messages written by my boss or anyone in research and which relate to this specification on my hard disk.” The client-side query processing technology (preferably via SQML) allows this flexible query because the processor links the metadata from the client with the remote XML Web Service that processes the relational query.
Smart and Dynamic Information Propagation. Dynamic linking as provided for in the present invention provide for intelligent information propagation. Because the Semantic Network can be navigated from many more axes than Today's Web or the Semantic Web, information sharing and propagation becomes much more efficient and information loss is minimized.
The dynamic linking property of the present invention allows for continuous semantic browsing as opposed to with Today's Web and the Semantic Web, where static links result in browsing “dead-ends.” With Today's Web and the Semantic Web, the user typically browses to the desired location or effectively reaches an impasse where no further links are available. With dynamic linking, the user can, depending on the nature of the information space at that point in time, continue browsing indefinitely since the node itself includes intelligence to dynamically update links.
For example, via the seamless integration of linking and semantic XML Web Services provided for by the present invention, users drag and drop files, links, etc. to Smart Agents to create new Smart Agents. Preferably this occurs recursively. Smart Agents, in turn, can, where appropriate, be made Breaking News Agents. Other nodes in the presentation display presentation hints indicating whether there is breaking news on any Breaking News Agent. To continue the example, the results of the Breaking News Agent query can be used as a Smart Lens, which shows further results. These results preferably include intrinsic alerts that provide the user with a context and time-sensitive path through the network. Subsequent results can be copied and pasted to any Agency, as well as dragged and dropped on other Smart Agents.
In the preferred embodiment, the dynamic linking of the present invention is applied both to objects within the semantic “sandbox” (objects that are in the system 10 environment and displayed within the semantic browser 30) as well as to external objects that can be dynamically added to the environment. This provides a seamless, dynamic migration path from existing documents (on the file system, Today's Web, or other environments) to the system 10 of the present invention.
The present invention does not require that documents be encoded as RDF or XML before inclusion in the network. Rather, the KIS (or Agency server) automatically extracts metadata from all sorts of documents and adds them to the Semantic Network. In addition, client-side dynamic linking, preferably via such features as drag and drop, smart copy and paste and Smart Lens, ensures that local documents of all types are linked to the network, thereby increasing the value and scope of the network. The present invention automatically extracts metadata from local documents and calls the KIS (via its XML Web Service) to retrieve semantically related information. Thus, the local document is not excluded from the network. The present invention empowers a user to drag and drop a document from a dumb environment (e.g., Today's Web or file system) into the system 10, thereby providing it semantic intelligence. Once the metadata is in the system 10, semantic tools such as semantic lenses, smart copy and paste, etc. may be performed to and with the object. Drag and drop is also supported directly from the user's file system and Today's Web into the system 10.
Flexible Presentation that Smartly Conveys the Semantics of the Information Being Displayed
The present invention empowers users with flexible presentation. Because the XML Web Service sends back XML, rather than HTML, and because the presentation is dynamically generated on the client, the user selects different “skins” with which to view semantic information. Skins preferably convert XML to a format suitable for presentation (e.g., XHTML+TIME, SVG, etc.), allowing the user to dynamically select Skins based on the capability of various display technologies. For example, SVG has many features that XHTML+TIME does not, and vice-versa. The user is able to select an SVG Skin for scenarios in which SVG is optimized. Alternatively, the user is able to select XHTML+TIME for other scenarios.
The flexibility of Skins as part of the present invention provide for application in additional situations. In various alternative embodiments, the use is empowered by text-to-speech Skins that may be running the semantic browser 30 on a second machine concurrently with a first or main machine, for example to assist blind users; dynamically resizable Skins that adapt to the size of the current view-port (thereby allowing the user to resize the window and yet retain a pleasant user experience); Skins that check local state to display semantic hints (e.g., the user's calendar in the case of event information, e.g., free/busy information); Skins that display inline preview windows that save user navigation time and increase productivity; Skins that display different customizable hints for intrinsic alerts, breaking news, deep information, smart recommendations, intrinsic links, lens info, etc. Users are also allowed to select Skins to be used with smart screensavers, for example where users desire to view an Agent in screensaver mode. In an alternative embodiment, the system 10 supports Skins for Context Templates (described above), e.g., Headlines, Newsmakers, Conversations, etc.
By virtue of allowing for flexible presentation, the present invention allows the user to select the best presentation mode based on the current task. For example, users can select a subtle Skin when working on their main machine where productivity is a higher priority than aesthetic effect. Users can select a moderate Skin in cases where productivity is important but where effects are desired or allowed. Users can select an exciting Skin for scenarios like wherein secondary machines are utilized—for example, where users are viewing information in their peripheral vision and desires features such as text-to-speech to alert them of breaking news, etc. Exciting Skins may alternatively feature animations, storyboard like effects for deep information, objects displayed on motion paths, and other special effects.
In addition, Skins according to the present invention are optionally configured with include and exclude object type filters. For example, a Skin may be configured to include only “documents” but exclude “analyst reports.” Because the Skin takes XML results to determine the ultimate presentation, the Skin can include or exclude objects in the XML (SRML) results based on an examination of the object type (or other attributes) of the returned objects.
The present invention provides for logic, inference, and reasoning. The semantic data model on KIS Agency preferably offers support for logic via database processing of the Semantic Network, conversion of semantic queries to SQL and other database query languages for logic processing, etc. In addition, the system 10 of the present invention preferably includes an Inference Engine for inferring links such as the experts on a particular category or information item, recommendations, probabilistic links (e.g., the probability that a person wrote a document), etc. As described above, an Inference Engine according to the present invention preferably observes the Semantic Network, mines it to infer new semantic links and represents resulting links in the SemanticLinks table.
The present invention provides native support for flexible information analysis on the client. The Presenter of the present invention preferably utilizes Smart Lenses to allow a user to preview the results of a semantic query prior to issuing the query. The user is able to change relevant predicates and other filters in order to preview the results. In an alternative embodiment, the user has the option of invoking the query and using that as the basis of a new sub-query, if desired.
The present invention allows a user to issue very flexible semantic queries. The user is able to incorporate local context into queries, e.g., by using filters such as “relates to this document on my hard drive.” Neither Today's Web nor the Semantic Web allow for this. In addition, the present invention preferably incorporates Smart Agents, which utilize references to a proprietary semantic query language (SQML) and includes local and remote resources, predicates, category references and objects. The present invention preferably incorporates the easy to use user interface for creating and editing Smart Agents (representing semantic queries) using a simple wizard model. As discussed above, the system 10 allows semantic queries to form the basis of new queries via the recursive drag and drop feature, e.g., a document or an HTML link can be dragged to an existing or new Smart Agents, thereby creating successive new Smart Agents. Smart Agents are alternatively used as lenses, can have objects pasted onto them to form new semantic queries, and can be added to Blenders, which in themselves are semantic query containers and which, in turn, can be filtered thereby creating sub-Blenders or containers of sub-Agents.
The system 10 of the present invention offers support for read/write functionality by providing an XML Web Service that allows a user to publish information directly into the Semantic Network. This could be any document, an annotation, or a semantic link that corrects a broken link or provides a new link. This is all subject to security restrictions at the XML Web Service and operating system layer. The system 10 employs authentication, access control, and other services from the operating system and application server that sit underneath the XML Web Service layer. These security services are preferably used to secure read and write access to the Semantic Network.
The present invention includes built-in support for Annotations. There is a special predicate “Annotated By” that defines an Annotation semantic link between a person object and any other information object (e.g., a document, email posting, online course, etc.). The system 10 includes presentation-layer support for Annotations by allowing users to navigate to Annotations via intrinsic links, Smart Lenses, etc. The manner in which the present invention incorporates Annotations provides advantages of existing techniques (such as in-place Annotation techniques that embed the Annotation as part of the information object it annotates). In the preferred embodiment of the present invention, Annotations are “first-class” information objects. This means that they can be linked to and from, “lens” over (using Smart Lens), copied and pasted (using smart copy and paste), etc. The present invention exposes Annotations to all of the semantic tools of the present invention, thereby facilitating a user experience more powerful than capable with standard Annotation techniques. In addition, Annotations of the present invention are used with Context Templates. As a result, the Inference Engine is able to employ them to make the system smarter over time. In addition, the system 10 provides a unique and easy means of annotating objects by sending specially formatted email (with a qualified message body) to the email Agent of an Agency.
The present invention provides a “Web of Trust” via the XML Web Service. This service authenticates a user that wants to update the Semantic Network, make assertions, fix/update links, etc. This also allows rich content to be made available via the KIS Agency to registered subscribers for pay-per-view content. The value of the entire network increases when one can utilize the same platform tools to navigate seamlessly across many rich content sources.
The present invention provides for information packages or “Blenders.” Blenders are semantic containers that include references to semantic queries from Smart Agents. This allows a user to deal with related semantic information as a whole unit. The user is able to separately view the individual Agents within the Blenders or view the entire Blender as though the information therein was from one aggregate Agent. This is preferably accomplished by driving each Agent via calls to the XML Web Service. In the preferred embodiment, users drag and drop objects onto Blenders to create sub-Blenders. This is preferably accomplished recursively. Blenders can be created, deleted, and edited. The user is able to add and remove smart Agents to or from Blenders.
Blenders can be thought of as a digital equivalent of a personal newspaper that contains different sections. For example, the USA Today, New York Times, Wall Street Journal, etc. contain different sections such as News, Business, Sports, Life/Entertainment, etc. Each of these sections corresponds to a Smart Agent entry in a Blender and the entire newspaper corresponds to the Blender. The flexible viewing and navigation provided by the present invention can be thought of as the digital equivalent of the user being able to browse each newspaper section completely and sequentially, one at a time, or browse the entire newspaper by starting as page one of each section, followed by page two of each section, etc.
As described in detail above, the present invention provides Context Templates, which are scenario-driven information query templates that map to specific semantic models for information access and retrieval. Essentially, Context Templates can be thought of as personal, digital semantic information retrieval “channels” that deliver information to a user by employing a predefined semantic template. In the preferred embodiment, the semantic browser 30 allows the user to create a new Blender or Special Agent using Context Templates to initialize the properties of the Agent. Context Templates preferably aggregate information across one or more Agencies. In addition, Context Templates are preferably used with Context Palettes to provide intelligent, dynamic, in-place context for any information object that is displayed or selected by the user.
The present invention has intrinsic support for user-oriented information aggregation. Scenarios empower a user to view context and time-sensitive information as though they came from one source even if they cut across information repositories. This provides a significantly more productive user experience that with Today's Web and the conceptual Semantic Web by providing user-oriented computing wherein the user is presented with the right information in the right context and at the right time, regardless of the source of the information. The Information Agent aggregates information dynamically, across information sources, using client-side semantic queries via SQML and aggregating the XML results that come from different Agencies' response to SQML.
The following provides exemplar scenarios of the operation of preferred and alternative embodiments of the present invention as applied in different pragmatic situations.
a. Find All Context that Relate to the Specification on the File Path c:\spec.doc
Drag and drop the icon representing a document to the icon representing the Information Agent. The file is opened in the semantic browser and the Context Palettes are displayed. In the preferred embodiment, these include some or all of the following Context Templates: Headlines, Discovery, Newsmakers, Upcoming Events, Timeline, Conversations, Variety, Classics, Best Bets, Today, Breaking News, etc. These palettes include relevant context from Agencies in the “recent” and “favorite” lists in the namespace.
b. Find All Experts on the Agency Titled “R&D” That Have Expertise on Wireless Technology
Start the “New Smart Agent” wizard and select the “Use Context Template” option when creating the Agent. Select the “R&D” Agency from the “Select Agency” dialog and select the category called “wireless” from the category browser. Open the newly created Smart Agent.
c. Find All Information on Reuters That is Relevant to a Link on the Currently Viewed Web Page
Drag and drop the link to the Agency icon representing “Reuters.” A new Smart Agent is created titled “Information on Reuters relevant to [link title]” and opened in the Information Agent.
d. Find All Information on Reuters That is Relevant to a Link on the Current Web Page and Which is Relevant to the Specification on the File Path c:\spec.doc
Drag and drop the icon representing the document to the Agent that was just created above (“All information on Reuters relevant to [link title]”). This creates a new Smart Agent titled “Information on Reuters relevant to [link title] and relevant to spec.doc.” This illustrates user-controlled browsing and dynamic linking
e. Find All Email on the Internal Agency Titled “Marketing” Relevant to the First Article on Reuters That Was Returned in the Previous Query
Highlight the Reuters article object and click on the button for “Verbs.” This displays a popup menu. Select “Copy.” Find the icon representing the Agency titled “Marketing” (on the Shell Extension Tree View). Right-click the icon. Hit “Paste.” This creates and opens a new Smart Agent titled “Information on ‘Marketing’ relevant to [Reuter's article title].” Focus on the frame in the results window showing email objects.
f. Navigate to the Author of the Email
Highlight the email object and click on the button for “Links.” This displays a popup menu showing the intrinsic links. Navigate to the menu item titled “From:” This displays a popup menu showing the person object on the “from” line of the email object. Select the desired object. This opens a new Smart Agent in the Information Agent showing the metadata of the person that authored the email object. The context of the person is also displayed in the Context Palettes. Users are able to continue browsing using the person object or its context (on any of the Context Palettes).
g. Navigate to the Attachments in the Email
Highlight the email object and click on the button for “Links.” This displays a popup menu showing the intrinsic links of the email object. Navigate to the menu item titled “Attachments.” This displays a popup menu showing the titles of the attachments. Select the desired attachment. This opens the attachment as a new Smart Agent in the Information Agent window. The context for the attachment is displayed in the Context Palettes.
h. Find All Events on the “Energy Industry Events” Agency That are Relevant to the Attachment
Highlight the attachment object and click on the button for “Verbs.” This displays a popup menu. Select “Copy.” Find the icon representing the Agency titled “Energy Industry Events” (on the Shell Extension Tree View). Right-click the icon. Hit “Paste.” This creates and opens a new Smart Agent titled “Information on Energy Industry Events relevant to [email attachment title].”
i. Browse the “My Documents” Folder Using Reuters as a Context
In the Information Agent, select “Open Documents in Folder.” Alternatively, drag and drop the “My Documents” folder to the icon representing the Information Agent. Indicate whether sub-folders are to be included. This creates and opens a new Dumb Agent titled “My Documents.” When you click this Agent, the metadata for the documents in this folder are opened in the Information Agent. When one of the documents is selected, the Context Palettes for the document are displayed. To browse the documents using Reuters as a context, the user finds the icon representing the Reuters Agency, right-clicks on the icon and hits “Copy.” The user hovers over any of the results showing the documents metadata in the Information Agent and selects the icon indicating the Smart Lens. A Smart Lens window is displayed showing information on the results of the relational query. The number of items found on Reuters that are relevant to the document is displayed, in addition to information such as the most recently posted item. In addition, a preview control is displayed to allow the user to preview the results in place. The user is able to choose to click on the results to open an Agent representing the new, relational query. If done, the context for the first object in the results is displayed using the Context Palettes.
j. Notify by Email, Voice or Pager When There is Breaking News That Relates to Anything on XML Technology and Which Relates to This Document
Create a new Smart Agent using the “Breaking News” context and using the “XML” category as a category filter. Drag and drop the icon representing this document to the Agent. This creates a new Smart Agent with an appropriate title. Go to the “Options” menu in the Information Agent and enter the proper information in the notification section (your email address, pager number, telephone number, etc.). Right-click the Smart Agent and select “Notify.”
a. Information Access
Today's Web. John Head-Master works at FastServe, a marketing consulting services company in San Diego. Everyday, he comes in to work and fires up his Web browser. On this day, he decides to browse the corporate Web to see if he can discover new and interesting information. The browser home page is set (using an Enterprise Information Portal) to the corporate home page. The corporate home page has links for the home pages for different divisions within the company. John navigates to these links and from there, keeps clicking links After a while, he gets frustrated because he knows that there are more sources of information that he cannot navigate to, only because he does not know what paths to take. Eventually, he gives up.
Information Nervous System. John fires up his Information Agent (semantic browser). This opens the home Agent. On the page, he sees a list of knowledge links corresponding to products, product groups, reports, corporate events, online courses, and video presentations. He hovers over the “product groups” link. Automatically, a balloon popup appears indicating the number of product groups and other data about the link. He then opens the link. A list of product group objects is then displayed with a customizable look or “skin.” He then hovers his mouse over the first one. A popup menu immediately appears over the link with the actions: “Show Members,” “List Similar Product Groups,” and “Subscribe to Group Events.” He then clicks on “Subscribe to Group Events” and he will now be notified by email (via the Enterprise Information Agent) about all events that relate to this product group. He then clicks “Show Members.” This then opens a new “Knowledge Page” with icons corresponding to people. He then hovers over the icon for Susan Group-Leader. A balloon pop-up then appears showing information on Susan. A right-click menu then appears with the actions, “Reports To,” “List Direct Reports,” “Member Of,” “Authored Documents,” and “Recently Attended Meetings.” John then selects “Recently Attended Meetings.” This opens up a new knowledge page with one meeting object. John then hovers over this and continues browsing.
At some point, John decides to search for a co-worker he met the previous day. He then types in “Wilbur Jones.” This then returns a person object corresponding to Wilbur. John then continues to browse using Wilbur as an Information Knowledge Pivot.
Eventually, John realizes that Wilbur does not seem to have the information he (John) needs. John then types the following query into the search box on his Information Agent: “List all online courses and documents that relate to the upcoming 2002 sales meeting.” The Information Agent (via the Email Agent) then returns a list of actionable online courses and documents that conform to the knowledge query.
b. Knowledge-Driven Customer Relationship Management
Customer Touch-Points. AnySoft is a software manufacturer with 50 products in 100 different languages. They employ their web-site (anysoft.com) to provide up-to-date information to their customers. However, customers have complained that their Web site is very hard to navigate and that they find it very hard to find information on products and to subscribe for notifications.
By deploying an Information Nervous System based on an embodiment of the present invention, AnySoft has deployed an Information Nervous System that co-exists with their existing Web site. The Information Agent is accessible from the home page and from the search bar. Customers now have a much more intuitive way of navigating the Web site for products, relevant white papers, announcements, press releases, corporate events, etc. Customers can now issue natural language queries that return self-navigable and actionable knowledge objects. This feature alone gives customers access to knowledge at their fingertips. Customers can also now use natural language to navigate the AnySoft.com Web site from their handheld devices.
Customer Feedback and Tracking. Comp-Mart is a reseller of computer peripherals with multiple distribution channels. The Company gets customer feedback from its Web site, its call center, its direct sales force, its telemarketing agents, etc. The feedback comes in as documents and email. The Company has identified a problem wherein customer feedback does not get properly routed around the Company to the people that need the information. Employees in product development have complained to management that they find it hard to integrate customer feedback into the product development process because they don't know where to find the information and because critical knowledge is not shared within the organization.
With an Information Nervous System in place, email that contains customer feedback now gets semantically integrated into the Company's Semantic Environment. The KIS of the present invention automatically adds semantic links between customer feedback email and semantic objects like documents, projects, and employees that work on the germane products. Customer feedback intelligently bubbles up in the right places in the knowledge space. The Email Agent sends out periodic notifications to people that are likely be interested in reading customer feedback email.
Also, with the Information Nervous System, the customer becomes an Information Knowledge Pivot. This makes it much quicker and easier to act on customer feedback and to track customer-related knowledge across the organization. The Information Nervous System automatically annotates the customer object with relevant email messages, documents, similar customers, etc. This way, links to the customer can be forwarded via email and co-workers can navigate relevant information from there. The customer object can be searched for, can be browsed, etc.
c. Knowledge-Driven Direct-Sales/Field-Service
Marsha Mindset is a customer service agent for JustInTime Support Services, a computer service firm in Kansas City, Mo. Marsha visits customers around the Kansas City metro area, and always takes her wireless PDA so she can send email to the support headquarters anytime she is in difficulty. JustInTime recently deployed the KIS and the Email Agent. Now, whenever she has support questions, Marsha can now email the Email Agent and ask it questions in natural language. The Email Agent replies to her email with direct answers or with “knowledge links” that allows Marsha to instantly access relevant support email, documents, or people that she could then email or call up on the phone. The JustInTime Direct Sales force also uses the technology of the present invention when in the field selling solutions to customers. The sales representatives also carry wireless PDAs and can issue requests to the Email Agent.
d. Case Studies
Corporate Training, Knowledge Transfer, and Sharing. WaveGen is a biotech company providing “managed care” solutions to doctors around the United States. The company recently deployed the Saba Learning Management System platform for training its employees (especially its sales reps). This reduces travel costs and enables the Company's sales-force to be better prepared to serve physicians in different healthcare regions in the country. It also assists the Company's researchers to be regularly informed of recent discoveries in the biotech research community.
The Company also has other software assets in place that hold valuable sources of knowledge. It has deployed content management solutions that host documents and media files, Microsoft Exchange for email, and collaboration software for online conferences. However, the Company has noticed that knowledge transfer is not very effective because it is not integrated across all these solutions. Sales representatives have indicated that they do not have the tools to discover important sources of knowledge within and outside the organization to assist them in pitching the Company's products to doctors. Enterprise Information Portals are currently used to inform the sales force of upcoming online courses and of important events. However, the sales reps complain that a lot of knowledge (stored in email, documents, etc.) is not brought to their attention because no one knows who else might need them.
In addition, the sales representatives use Microsoft Outlook to add appointments to their calendars for upcoming doctor visits. However, they complain that they only get reminders for the appointments, and that a lot of information that could help them sell products more effectively is not made available to them automatically, ahead of their doctors' appointments.
WaveGen recently deployed an Information Agent based on technology from the present invention. The company deployed the KIS and the Email Agent to facilitate intelligent information connections and routing to help their sales and research teams make better decisions to serve customers and improve the Company's products. Using the Information Agent, the sales force has instant access not just to documents but to “knowledge objects” that are more directly tied to their task at hand. For instance, the sales representatives now have an Agent with “Doctor Jones” as an XML object. This is not a document or a Web page. Rather, it is a semantic representation of the customer. A sales representative can then see semantic links like “Recent Email Messages”, “Relevant Documents,” “Properties,” “Important Dates,” “Relevant upcoming online courses,” etc. This way, the customer becomes the pivot with which the sales agent is navigating the internal Web. These links might generate results from file-shares, Email stores, Microsoft Exchange, etc. But rather than searching or navigating for these knowledge sources as islands, the sales representative can discover new knowledge based on semantic relationships as they relate to the sales representative's task.
This way, the sales representative can have much more powerful knowledge at the sales representative's fingertips, thereby enabling much better customer service. And this knowledge emanates from co-workers, documents that were published by other sales agents, email sent on distribution lists that might not be known to exist, etc. The KIS does the smart thing by automatically making semantic connections from all these disparate sources. The sales representative can then email this “page” to a co-worker. This then becomes a very powerful form of knowledge sharing because the co-worker can then navigate the Information Agent using the same “Dr. Jones” pivot.
The Email Agent also allows the sales representative to issue knowledge queries via natural language. The query results are derived from the Inference Engine and could be based on knowledge that was deduced from existing knowledge. A powerful feature of the Information Nervous System of the present is that knowledge transfer, sharing, discovery all happen automatically based on the Semantic Network.
a. Semantic Information Discovery, Retrieval, and Navigation
Joe Knowledge-Worker starts the Information Agent (the XML-based semantic browser of the present invention). When he logs in, he is prompted with a dialog box indicating that there are new Agents available on the semantic intranet. He then sees a list of Agents from within and outside the organization that may include the following:
He then selects Meetings.ThisWeek.All. The Information Agent then displays a list of objects that represents meetings that he attended this week. This information comes from Microsoft Exchange but this is not exposed to him. Joe then hovers over a link for the first meeting object. A balloon pop-up is then displayed indicating that a new training course was just made available on the intranet. The balloon also indicates that there is a new report on IDC that might be relevant to Joe. In addition to the balloon, a pop-up menu is displayed to the right of the object. This menu has the following verbs:
Joe then selects “Subscribe for follow-up.” This contacts the Meeting Follow-up Agent on the server. This Agent then sends periodic updates of relevant information to the participants of the meeting. This could be done either through the browser or through email. Joe then selects related objects on Events.Corporate.Today.All. This then displays a list of event information objects. Joe then hovers over the first object and a pop-up menu gets displayed. Joe then selects “Add to calendar” and the event is added to his calendar. Joe then decides that he wants to find all industry events that relate to the corporate event. He then drags the object to the Agent Events.Technology.All and releases his mouse. When the mouse is released, the browser then loads information objects from Events.Technology.All (across web-sites and other islands) and which are related to the corporate event the object of which he dragged.
The next week, Joe gets email from the Email Agent. In the email, the Agent informs Joe that it has noticed that everyone that added the event to his or her calendar also watched a corporate training video from the corporate media server. The email contains an XML link, which takes Joe back into the Information Agent. The browser then displays the metadata for the video. One of the items on the pop-up is “Watch Video.” Joe then selects it and watches the video.
The next time Joe logs in to his workstation, he notices that there are new Agents. He then subscribes to Books.Ebay.Computers.All and adds it to his My Agent list. Automatically, an embodiment of the present invention adds this Agent into Joe's Semantic Environment. The Information Agent performs implicit queries and provides recommendations (ranked by relevance and time-sensitivity) that include this Agent. He then clicks on this Agent and semantic information objects (representing books) are displayed in the Results Pane. When he hovers over one of the objects, a pop-up balloon is immediately displayed, alerting him to the fact that there is a related industry conference being hosted by the author of the book. When he clicks the pop-up link, the event object is loaded in the browser, complete with verbs that allow him to add the event to his calendar (either Microsoft Outlook or an Internet-based calendar like the MSN Calendar (accessible via Microsoft's HailStorm Web services), AOL Calendar, etc.)
Explanation of the Scenario. This scenario shows how with the present invention, knowledge-workers are able to obtain access to “federated knowledge.” In this example, Joe's company has “imported” knowledge Agents from Gartner, IDC, Reuters, Ebay, etc. into its knowledge space. As such, these Agents automatically add knowledge into the company's Semantic Network. The scenario also showed how Joe was able to get an “object model” view of the entire organization's knowledge-space via intuitively named Smart Agents. Joe was able to use these Agents to “enter” the Semantic Environment, and then navigate his way from there. All the information objects were delivered in real-time and were actionable (with relevant verbs that were displayed in place). This way, Joe did not have to care about what information islands the objects were coming from, or what applications generated them.
The scenario also shows how Joe was able to discover not just new information but also new Agents. And the scenario shows knowledge collaboration in action—via collaborative filtering—wherein the Information Agent gave recommendations to Joe based on what it noticed others in the enterprise were doing.
Lastly, the scenario illustrates how time-sensitive information is automatically brought to the user's attention at the point of context where it makes sense. The Email Agent automatically connected the book from Ebay with the upcoming industry event, inferred and assigned a relevance and time-sensitivity ranking to the event, and decided that the event was critical enough to warrant displaying the information immediately via an alert in the semantic browser.
b. Peer-to-Peer Knowledge Sharing and Capture
Nancy Hard-worker works at a Fortune 500 company with 40,000 employees. She subscribes to a variety of Web sites and has information forwarded to her by email from friends and co-workers. She just got a bunch of documents from someone at a partner company and she would like to share the information within the organization. She sends the documents to all the distribution lists of which she is a member. The Enterprise Information Agent is a member of these lists also (the Agent adds itself to all public distribution lists when the server is installed). When the Agent receives the information, it classifies it and adds it to the Semantic Network. The Inference Engine then picks up the information.
Several thousand co-workers are not members of any of the distribution lists to which Nancy forwarded the documents. However, they all use the Integrator and all of them have subscribed to the Email.Public.All Agent. While they browse other related parts of the knowledge-web, a balloon popup gets displayed indicating that there is new and relevant email on the Email.Public.All Agent. The co-workers then open up the Agent and the email object is displayed. One of the menu items on the email item is “Show distribution lists to which message was forwarded.” The co-workers then select this and the distribution list information objects are then displayed in the browser. The co-worker then hovers over the distribution list and a pop-up menu item gets displayed. The first item is “Show Members.” The second is “Join.” The co-workers then join the distribution list.
Explanation of the Scenario. This scenario illustrates how information was published, shared and captured via email and how, by use of the Semantic Network, other co-workers found out about this information (and about distribution lists the existence of which they were not aware) from different but related “knowledge angles.” The scenario shows peer-to-peer knowledge sharing in a way that is completely seamless and does not require users to public information to repositories, or to classify information themselves. With certain embodiments of the present invention, everything just happens automatically (in the background) and the knowledge gets bubbled up in relevant places.
The following scenarios help to explain the utility and operation of the system, and will thereby make the rest of the detailed description easier to follow and understand.
1. Patent Examiner Prior Art Search Tool
Largely because of PTO fee diversion, there is a great deal of pressure on U.S. Patent Examiners to conduct a robust prior art search in very little time. And, while the research tools available to Examiners have improved dramatically in the last several years, those tools still have many shortcomings. Among the shortcomings are that most of the research tools are text based, rather than meaning based. So, for example, the search tool on the PTO website will search for particular words in particular fields in a document. Similarly, the advanced search tool on Google™ enables the Examiner to locate documents with particular words, or particular strings of words, or documents without a particular word or words. However, in each case, the search engine does not allow the Examiner to locate documents on the basis of meaning. So, for example, if there is a relevant reference that teaches essentially the same idea, but uses completely different words (e.g., a synonym, or worse yet, a synonymous phrase) than those in the query, the reference, even though perhaps anticipating, may well not be discovered. Even if the Examiner could spare the time to imagine and search every possible synonym, or even synonymous phrase to the key words critical to the invention, it could still overlook references because sometimes the same idea can be expressed without using any of the same words at all, and sometimes the synonymous idea is not neatly compressed into a phrase, but distributed over several sentences or paragraphs.
The reason for this is that words do not denote or connote meaning one to one as, for example, numerals tend to do. Put differently, certain meanings can be denoted or connoted by several different words or an essentially infinite combination of words, and, conversely, certain words or combinations of words can denote or connote several different meanings Despite this infinite many-to-many network of possibilities human beings can isolate (because of context, experience, reasoning, inference, deduction, judgment, learning and the like) isolate probable meanings, at least tolerably effectively most of the time. The current prior art computer-automated search tools (e.g. the PTO website, or Google™, or Lexis™), cannot. The presently preferred embodiment of my invention bridges this gap considerably because it can search on the basis of meaning
For example, using the some of the search functions of the preferred embodiment of the present invention, the Examiner could conduct a search, and with no additional effort or time as presently invested, obtain search results relevant to patentability even if they did not contain a single word in common with the key words chosen by the Examiner. Therefore, the system would obtain results relevant to the Examiner's task that would not ordinarily be located by present systems because it can locate references on the basis of meaning
Also on the basis of meaning, it can exclude irrelevant references, even if they share a key word or words in common with the search request. In other words, one problem in prior art research is the problem of a false positive; results that the search engine “thought” were relevant merely because they had a key word in common, but that were in fact totally irrelevant because the key word, upon closer inspection in context, actually denoted or connoted an irrelevant idea. Therefore, the Examiner must search for the needle in the haystack, which is a waste of time.
In contrast, using some of the search functions of the preferred embodiment of the present invention, the density of relevant search results increases dramatically, because the system is “intelligent” enough to omit search results that, despite the common key words, are not relevant. Of course, it is not perfect in this respect any more than human beings are perfect in this respect. But, it is much more effective at screening irrelevant results than present systems, and in this respect resembles in function or in practice an intelligent research assistant than a mere keyword based search engine. Thus, using the system, the Examiner can complete a much better search in much less time. The specific mechanics of using the system this way, in one example, would work as follows:
Imagine the Examiner is assigned to examine an application directed to computer software for a more accurate method of interpreting magnetic resonance data and thereby generating more accurate diagnostic images. To search for relevant prior art using the search functions of the preferred embodiment of the present invention, the Examiner would:
a. Using the Create Entity wizard, create a “Topic” entity with the relevant categories in the various contexts in which “Magnetic Resonance Imaging” occurs. For example,
b. Name the new entity “Magnetic Resonance Imaging” and perhaps “imaging” and “diagnostic” or some variations and combinations of the same.
c. Drag and drop the “Magnetic Resonance Imaging” Topic entity to the Dossier (special agent or default knowledge request) icon in the desired profile (the profile is preferably configured to include the “Patent Database” knowledge community). This launches a new Dossier request/agent that displays each special agent (context template). Each special agent is displayed with the right default predicate as follows:
d. Alternatively, the request can be created by using the Create Request Wizard. To do this, select the Dossier context template and select the “Patent Database” knowledge community as the knowledge source for the request. Alternatively, you can configure the profile to include the “Patents Database” knowledge community and simply use the selected profile for the new request. Hit Next—the wizard intelligently suggests a name for the request based on the semantics of the request. The wizard also selects the right default predicates based on the semantics of the “Magnetic Resonance Imaging” “Topic” entity. Because the wizard knows the entity is a “Topic,” it selects the right entities that make sense in the right contexts. Hit Finish. The wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results.
In the foregoing example, the results could be drawn, ultimately, from any source. Preferably, some of the results would have originated on the Web, some on the PTO intranet, some on other perhaps proprietary extranets. Regardless of the scope or origin of the original documents, by use of the system they have been automatically processed, and automatically “read” and “understood” by the system, so that when the Examiner's query was initiated, and also “read” and “understood” semantically, and by context, the system locates all relevant, and only relevant results. Again, not perfectly, but radically more accurately than in any prior systems. Note also that the system does not depend on any manual tagging or categorization of the documents in advance. While that would also aid in accuracy, it is so labor intensive as to utterly eclipse the advantages of online research in the first place, and is perfectly impractical given the rate of increase of new documents.
In this scenario, the Examiner may also wish to use additional features of the preferred embodiment of the invention. For example, the Examiner may wish to consult experts within the PTO, or literature by experts outside the PTO, as follows (note that Experts in Magnetic Resonance Imaging would be included in the Dossier on Magnetic Resonance Imaging; however, the examiner might want to create a separate request for Experts in order to track it separately, save it as a “request document,” email it to colleagues, etc.). Find all Experts in Magnetic Resonance Imaging:
a. Follow steps 1-4 above.
b. Drag and drop the “Magnetic Resonance Imaging” entity to the Experts (special agent or default knowledge request) icon in the desired profile. This automatically launches a new request/agent appropriately titled “Experts in Magnetic Resonance Imaging.” The semantic browser selects the right default predicate “in” because it “knows” the entity is a “Topic” entity and the context template is a “People” template (Experts). As such, the default predicate is selected based on the intersection of these two arguments (“in”) since this is what makes sense.
2. BioTech Company Research Scenario
Biotech companies are research intensive, not only in laboratory research, but in research of the results of research by others, both within and outside of their own companies. Unfortunately, the research tools available to such companies have shortcomings. Proprietary services provide context-sensitive and useful results, but those services themselves have inferior tools, and thus rely heavily on indexing and human effort, and subscriptions to expensive specialized journals, and as consequence are very expensive and not as accurate as the present system. On the other hand, biotech researchers can search inexpensively using Google™□, but it shares all the key word based limitations described above.
In contrast, using the search features of the preferred embodiment of the present invention, a biotech researcher could more efficiently locate more relevant results. Specifically, the researcher might use the system as follows. For example, if some researchers wanted to Find Headlines on Genomics and Anatomy written by anyone in Marketing or Research, they would do that as follows:
a. Using the wizard, launch an information-type request/agent for distribution lists with the keywords “Marketing Research”.
b. Select the Marketing distribution list result and click “Save as Entity”—this saves the object as a “Team” entity (because the semantic browser “knows” the original object is a distribution list—as such, a “Team” entity makes sense in this context).
c. Select the Research distribution list result and click “Save as Entity”—this saves the object as a “Team” entity (because the semantic browser “knows” the original object is a distribution list).
d. Using the Create Entity Wizard, create a new “Team” entity and select the “Marketing” and “Research” team entities as members. Name the new entity “Marketing or Research”.
e. Using the Create Request Wizard, select the Headlines context template, and then select the “Marketing or Research” entity as a filter. Also, select the Genomics category and the Anatomy category. Next, select the “AND” operator. Hit Next—the wizard intelligently suggests a name for the request based on the semantics of the request. The wizard also selects the right default predicates based on the semantics of the “Marketing or Research” team entity (“by anyone in”). Because the wizard knows the entity is a “Team,” it selects “by anyone in” by default since this makes sense. Hit Finish. The wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results.
In addition, the researchers may wish to Find all Experts in Marketing or Research:
a. Follow steps 1-4 above.
b. Drag and drop the “Marketing or Research” entity to the Experts (special agent or default knowledge request) icon in the desired profile. This launches a new request/agent appropriately titled “Experts in Marketing or Research.” The semantic browser selects the right default predicate “in” because it “knows” the entity is a “Team” entity and the context template is a “People” template (Experts). As such, the default predicate is selected based on the intersection of these two arguments (“in”) since this is what makes sense.
If the researchers expect to need to return to this research, or to supplement it, or to later analyze the results, they may wish to Open a Dossier on Marketing or Research, as follows:
a. Follow steps 1-4 above.
b. Drag and drop the “Marketing or Research” entity to the Dossier (special agent or default knowledge request) icon in the desired profile. This launches a new Dossier request/agent that displays each special agent (context template). Each special agent is displayed with the right default predicate as follows:
The researchers may be interested in Finding “Breaking News on my Competitors”, and would do so as follows:
a. For each competitor, create a new “competitor” entity (under “companies”) using the Create Entity Wizard. Select the right filters as needed. For instance, a competitor with a well-known English name—like “Groove” should have an entity that includes categories in which the company does business and also the keyword.
b. Using the Create Entity Wizard, create a portfolio (entity collection) and add all the competitor entities you created in step a. Name the entity collection “My Competitors.”
c. Using the Create Request Wizard, select the Breaking News context template and add the portfolio (entity collection) you created in step b. as a filter. Keep the default predicate selection. Hit “Next”—the wizard intelligently suggests a name for the request using the default predicate (“Breaking News on My Competitors”). Hit Finish. The wizard launches a new request/agent named “Breaking News on My Competitors.”
In addition, the researchers may wish to be kept apprised. They could instruct the system to alert them on “Breaking News on our Competitors”, as follows:
a. Create the “Breaking News on My Competitors” request as described above.
b. Add the request to the request watch list. The semantic browser will now display a watch pane (e.g., a ticker) showing “Breaking News on My Competitors.” Using the Notification Manager (NM), you can also indicate that the semantic browser send alerts via email, instant messaging, text messaging, etc. when there are new results from the request/agent.
In addition, the researchers may wish to keep records of competitors for future reference, and to have them constantly updated. The system will create and update such records, by the researchers instructing the system to Show a collection of Dossiers on each of our competitors, as follows:
a. Create entities for each of your competitors as described in 4a. above.
b. For each competitor entity, create a new Dossier on that competitor by dragging the entity to the Dossier icon for the desired profile—this creates a Dossier on the competitor.
c. Using the Create Request Wizard, create a new request collection (blender) and add each of the Dossier requests created in step b. above to the collection (you can also drag and drop requests to the collection after it has been created in order to further populate the collection). Hit Next—the wizard intelligently suggests a name for the request collection. Hit Finish. The wizard launches a request collection that contains the individual Dossiers. You can then add the request collection as a favorite and open it everyday to get rich, contextual competitive intelligence.
The researchers may wish to review a particular dossier, and can do so by instructing the system to Show a Dossier on the CEO (e.g., named John Smith):
a. Using the wizard, launch an information-type request/agent for People with the keywords “John Smith”.
b. Select the result and click “Save as Entity”—this saves the object as a “Person” entity (because the semantic browser “knows” the original object is a person—as such, a “Person” entity makes sense in this context).
c. Using the Create Request Wizard, select the Dossier context template, and then select the “John Smith” entity as a filter. Hit Next—the wizard intelligently suggests a name for the request based on the semantics of the request. The wizard also selects the right default predicates based on the semantics of the “John Smith” person entity. Hit Finish. The wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results (as sub-queries/agents) as follows:
The foregoing scenarios illustrate the operation of the system. The system itself is described in greater detail below.
Several improvements, enhancements and variations have been developed since the filing of my co-pending parent application and prior provisional applications referenced above. Some of these are improvements on, or only clarifications of, features previously included in the parent application, and some are new features of the system altogether. These are listed and described below. They are not arranged in order of importance, or in any particular order. While the preferred embodiment of the present invention would allow the user to use any or all of these features and improvements described below, alone or in combination, no single feature is necessary to the practice of the invention, nor any particular combination of features.
Also, in this application, reference is made to the same terms as are defined in my parent application Ser. No. 10/179,651, and the Description throughout this application is intended to be read in conjunction with the definitions, terminology, nomenclature and Figures of my parent application except where the context of this application clearly indicates to the contrary.
1. Smart Selection Lens Overview
The Smart Selection Lens is similar to the Smart Lens feature of the Information Nervous System information medium. In this case, the user can select text within the object and the lens will be applied using the selected text as the object (dynamically generating new “images” as the selection changes). This way, the user can “lens” over a configurable subset of the object metadata, as opposed to being constrained to “lens” over either the entire object or nothing at all. This feature is similar to a selection cursor/verb overloaded with context. For example, the user can select a piece of text in the Presenter and hit the “Paste as Lens” icon over the object in which the text appears. The Presenter will then pass the text to the client runtime component (e.g., an ActiveX object) with a method call like:
This call then returns a temporary SRML buffer that encapsulates the argument text. The Presenter will then call a method like:
This method gets the SQML from the clipboard, takes the argument SRML for the object, and dynamically creates new SQML that includes the resource in the SRML as a link in the SQML (with the default predicate “relevant to”). The method then returns the new SQML. The Presenter then calls the method:
This method passes the generated lens SQML and then retrieves the number of items in the results and the SRML results, preferably asynchronously. For details on this call, see the specification “Information Nervous System Semantic Runtime OCX.” The Presenter then displays a preview window (or the equivalent, based on the current skin) with something like:
[Lens Agent Title]
Found 23 items
[PREVIEW OBJECT 1]
[PREVIEW WINDOW CONTROLS]
where the “Lens Agent Title” is the title of the agent on the clipboard. For details of the preview window (and the preview window controls), please refer to my parent application Ser. No. 10/179,651.
In the preferred embodiment, the preview window will:
The preferred embodiment also has the following features:
1. One selection range per object but multiple selections per results-set is the best option. Otherwise, the system would result in a confusing user experience and complex UI to show lens icons per selection per object (as opposed to per object).
2. Outstanding lens query requests (which are regular SQML queries, albeit with SQML dynamically generated consistent with the agent lens) should be cancelled when the Presenter no longer needs them (e.g. if the Presenter is navigating to a new page, or if we are requesting new lens info for an object). In any case, such cancellation is not critical from a performance (or bandwidth) standpoint because lens queries will likely only ask for a few objects at a time. Even if the queries are not cancelled, the Presenter can ignore the results. Regardless, because the Presenter also has to deal with stale results, dropping them on the floor—the Presenter will have to do this anyway (whether or not lens queries are also cancelled). There will be a window of delay between when the Presenter issues a cancel request and when the cancellation actually is complete. Because some results can trickle in during this time, they need to be discarded. Thus, the preferred embodiment has asynchronous cancellation implementations—the software component has been designed to always be prepared to ignore bad or stale results.
3. The Presenter preferably has both icons (indicating the current lens request state) and tool-tips: When the user hovers over or clicks on an object, the Presenter can put up a tool-tip with the words, “Requesting Lens Info” (or words to that effect). When the info comes back, hovering will show the “Found 23 Objects” tip and clicking will show the results. This interstitial tool tip can then be transitioned to the preview window if it is still up when the results arrive.
In addition, note that the smart selection lens, like the smart lens, can be applied to objects other than textual metadata. For instance, the Smart Selection Lens can be applied to images, video, a section of an audio stream, or other metadata. In these cases, the Presenter would return the appropriate SRML consistent with the data type and the “selection region.” This region could be an area of an image, or video, a time span in an audio stream, etc. The rest of the smart lens functionality would apply as described above, with the appropriate SQML being generated based on the SRML (which in turn is based on the schema for the data type under the lens).
2. Pasting Person Objects Overview
The Information Nervous System (which, again, is one of our current shorthand names for certain aspects of our presently preferred embodiments) also supports the drag and drop or copy and paste of ‘Person’ objects (People, Users, Customers, etc.). There are at least two scenarios to illustrate the operation of the preferred embodiment in this case:
1. Pasting a Person object on a smart request representing a Knowledge community (or Agency) from whence the Person came. In this case, the server's semantic query processor merely resolves the SQML from the client using the Person as the argument. For instance, if the user pastes (or drags and drops) a person ‘Joe’ on top of a smart request ‘Headlines on Reuters™,’ the client will create a new smart request using the additional argument. The Reuters™ Information Nervous System Web service will then resolve this request by returning all Headlines published or annotated by ‘Joe.’ In this case, the server will essentially apply the proper default predicate (‘published or annotated by’)—that makes sense for the scenario.
2. Pasting a Person object on a smart request representing a Knowledge community (or Agency) from whence the Person did not come. In this case, because the Person object is not in the semantic network of the destination Knowledge community (on its SMS), the server's semantic query processor would not be able to make sense of the Person argument. As such, the server must resolve the Person argument, in a different way, such as, for example, using the categories on which the person is an expert (in the preferred embodiment) or a newsmaker. For instance, taking the above example, if the user pastes (or drags and drops) a person ‘Joe’ on top of a smart request ‘Headlines on Reuters™’ and Joe is not a person on the Reuters™ Knowledge community, the Reuters™ Web service (in the preferred embodiment) must return Headlines that are “relevant to Joe's expertise.” This embodiment would then require that the client take a two-pass approach before sending the SQML to the destination Web service. First, it must ask the Knowledge community that the person belongs to for “representative data (SRML)” that represents the person's expertise. The Web service resolves this request by:
a. Querying the Knowledge community (e.g., Reuters™) on which the person object is pasted or dropped for that community's semantic domain information which comprises and/or represents that community's specifictaxonomy and ontology. Note that there could be several semantic domains.
b. Querying the Knowledge community from whence the person object came for that person object's semantic domain information.
c. If the semantic domains are identical or if there is at least one common semantic domain, the client queries the Knowledge community from whence the person came for the person's categories of expertise. The client then constructs SQML with these categories as arguments and passes this SQML to the Knowledge community on which the person was pasted or dropped.
If the semantic domains are not identical or there is not least one common semantic domain, the client queries the Knowledge community from whence the person came for several objects that belong to categories on which the person is an expert. In the preferred embodiment, the implementation should pick a high enough number of objects that accurately represent the categories of expertise (this number is preferably picked based on experimentation). The reason for picking objects in this case is that the destination Web service will not understand the categories of the Knowledge community from whence the person came and as such will not be able to map them to its own categories. Alternatively, a category mapper can be employed (via a centralized Web service on the Internet) that maps categories between different Knowledge Communities. In this case, the destination Knowledge community will always be passed categories as part of the SQML, even though it does not understand those categories—the Knowledge community will then map these categories to internal categories using the category mapper Web service. The category mapper Web service will have methods for resolving categories as well as methods for publishing category mappings.
3. Saving and Sharing Smart Requests Overview
Users of the Information Nervous System semantic browser (the Information Agent or Librarian) will also be able to save smart requests to disk, email them as an attachment, or share them via Instant Messenger (also as an attachment) or other means. The client application will expose methods to save a smart request as a sharable document. The client application will also expose methods to share a smart request document as an attachment in email or Instant Messenger.
A sharable smart request document is a binary document that encapsulates SQML (via a secure stream in the binary format). It provides a safe, serialized representation of a semantic query that, among other features, can protect the integrity and help protect the intellectual property of the specification. For example, the query itself may embody trade secrets of the researcher's employer, which, if exposed, could enable a competitor to reverse engineer critical competitive information to the detriment of the company. The protection can be accomplished in several ways, including by strongly encrypting the XML version of the semantic query (the SQML) or via a strong one-way hash. The sharable document has an extension (.REQ) that represents the request. An extension handler on the client operating system is installed to represent this extension. When a document with the extension is opened, the extension handler is invoked to open the document. The extension handler opens the document by extracting the SQML from the secure stream, and then creating a smart request in the semantic namespace with the SQML. The handler then opens the smart request in the semantic namespace.
When a smart request in the semantic namespace is saved or if the user wants to send it as an email attachment, the client serializes the SQML representing the smart request in the binary .REQ format and saves it at the requested directory path or opens the email client with the .REQ document as an attachment.
Saving and sharing entities—the same process applies as above except with a .ENT extension to represent an entity. When an entity document is invoked, the Nervana Librarian opens the entity SQML in the browser.
Extension Property Sheet—this will create a temporary smart request or entity (depending on the kind of document) in the semantic environment and display the property sheet for a smart request or entity.
Extension Tool tips—this will display a helpful tool tip when the user hovers over a librarian document (a request, .REQ or an entity, .ENT).
4. Saving and Sharing Smart Snapshots Overview
The Information Nervous System also supports the sharing of what the inventor calls “Smart Snapshots.” A smart snapshot is a smart request frozen in time. This will enable a scenario where the user wants to share a smart request but not have it be “live.” For instance, by default, if the user shares the smart request “Breaking News on Reuters™ related to this document” with a colleague, the colleague will see the live results of the smart request (based on the “current time”). However, if the user wants to share “[Current] Breaking News on Reuters™ related to this document,” a smart snapshot will be employed.
A smart snapshot is the same as a smart request (it is also represented by an SQML query document) except that the “attributes” section of the SQML document contains attributes marking it as a snapshot (the flag QUERYATTRIBUTES_SNAPSHOT). The creation date/time of the SQML document is also stored in the SQML (as before—the SQML schema contains a field for the creation date/time). When the user indicates that he/she wants to share the smart request, the user interface (the semantic browser, Information Agent, or Librarian) prompts him/her whether he/she wants to share the smart request (live) or a smart snapshot. If the user indicates s smart request, the process described above (in Part 3) is employed. If the user indicates a smart snapshot, the binary document is populated with the edited SQML (containing the snapshot attribute) and the remainder the process is followed as above.
When the recipient of the binary document receives it (by email, instant messaging, etc.), and opens it, the extension handler opens the document and adds an entry into the semantic namespace as a smart request (as described above). When the recipient opens the smart request, the client's semantic query processor will send the processed SQML to the server's XML web service (as previously described). The server's semantic query processor then processes the SQML and honors the snapshot attribute by invoking the semantic query relative to the SQML creation date/time. As such, results will be relative to the original date/time, thereby honoring the intent of the sender.
5. Virtual Knowledge Communities
Virtual Knowledge Communities (agencies) refer to a feature of the Information Nervous System that allows the publisher of a knowledge community to publish a group of servers to appear as though they were one server. For instance, Reuters™ could have per-industry Reuters™ Knowledge Communities (for pharmaceuticals, oil and gas, manufacturing, financial services, etc.) but might also choose to expose one ‘Reuters™’ knowledge community. To do this, Reuters™ will publish and announce the SQML for the virtual knowledge community (rather than the URL to the WSDL of the XML Web Service). The SQML will contain a blender (or collection) of the WSDLs of the actual Knowledge Communities. The semantic browser will then pick up the SQML and display an icon for the knowledge community (as though it were a single server). Any action on the knowledge community will be propagated to each server in the SQML. If the user does not have access for the action, the Web service call will fail accordingly, else the action will be performed (no different from if the user had manually created a blender containing the Knowledge Communities).
6. Implementing Time-Sensitive Semantic Queries
Semantic queries that are time-sensitive are preferably implemented in an intelligent fashion to account for the rate of knowledge generation at the knowledge community (agency) in question. For instance, ‘Breaking News’ on a server that receives 10 documents per second is not the same as ‘Breaking News’ on a server that receives 10 documents per month. As such, the server-side semantic query processor would preferably adjust its time-sensitive semantic query handling according to the rate at which information accumulates at the server. To implement this, general rules of thumb could be used, for instance:
The most recent N objects where N is adjusted based on the number of new objects per minute.
All objects received in the last N minutes with a cap on the number of objects (i.e., min (cap, all objects received in the last N minutes)).
N can also be adjusted based on whether the query is a Headline or Breaking News. In the preferred embodiment, newsmaker queries is preferably implemented with the same time-sensitivity parameters as Headlines.
7. Text-To-Speech Skins Overview
Text-to-speech is implemented at the object level and at the request level. At the object level, the object skin runs a script to take the SRML of the object, interprets the SRML, and then passes select pieces of text (in the SRML fields) to a text-to-speech engine (e.g., using the Microsoft™ Windows™ Speech SDK) that generates voice output.
1. Reading Email Message
2. Appropriate Delay
3. Message From Nosa Omoigui
4. Appropriate Delay
5. Message Sent to John Smith
6. Appropriate Delay
7. Message Copied To Joe Somebody
8. Appropriate Delay
9. Message Subject Is Web services are software building blocks used for distributed computing
10. Appropriate Delay
11. Message Summary is Web services
12. Appropriate Delay
13. [Optional] Message Body is Web services are software building blocks used for distributed computing
This example assumes a voice skin template as follows:
1. Reading Email Message
2. Appropriate Delay
3. Message From <message author name>
4. Appropriate Delay
5. Message Sent to <message to: recipient name>
6. Appropriate Delay
7. Message Copied To <message cc: recipient name>
8. Appropriate Delay
9. Message Subject Is <message subject text>
10. Appropriate Delay
11. Message Summary is <message body summary>
12. Appropriate Delay
13. [Optional] Message Body is <message body>
Other templates can also be used to render voice that is easily understandable and which conveys the semantics of the object type being rendered. Like the example shown above (which is for email), the implementation should use appropriate text-to-speech templates for all information object types, in order to capture the semantics of the object type.
At the request level, the semantic browser's presentation engine (the Presenter) loads a skin that takes the SRML for all the current objects being rendered (based on the user-selected cursor position) and then invokes the text-to-speech object skin for each object. This essentially repeats the text-to-speech action for each XML object being rendered, one after another.
Email Object (SRML)
Object Interpretation Engine (Object Skin)
From: Nosa Omoigui
To: John Smith
Cc: Joe Somebody
Subject: Web services
Summary: Web services are software building blocks used for distributed computing
Body: Web services . . .
Reading Email Message
Message From Nosa Omoigui
Message Sent To John Smith
Message Copied To Joe Somebody
Message Summary is Web services
Message Summary is Web services
From: Nosa Omoigui
To: John Smith
Cc: Joe Somebody
Subject: Web services
Summary: Web services are software building blocks used for distributed computing
Body: Web services . . .
Email Object 1
Object Skin (Object 1)
Email Object 2
Email Object 3
Email Object N
8. Language Translation Skins
Language translation skins are implemented similar to text-to-speech skins except that the transform is on the language axis. The XSLT skin (smart style) can invoke a software engine to automatically perform language translation in real-time and then generate XML that is encoded in Unicode (16 bits per character) in order to account for the universe of languages. The XSLT transform that generates the final presentation output then will render the output using the proper character set given the contents of the translated XML.
Language Agnostic Semantic Queries
Semantic queries can also be invoked in a language-agnostic fashion. This is implemented by having a translation layer (the SQML language translator) that translates the SQML that is generated by the semantic browser to a form that is suitable for interpretation by the KDS (or KBS) which in turn has a knowledge domain ontology seeded for one or more languages. The SQML language translator translates the objects referred to by the predicates (e.g., keywords, text, concepts, categories, etc.) and then sends that to the server-side semantic query processor for interpretation. The results are then translated back to the original language by the language translation skin.
9. Categories as First Class Objects in the User Experience
This refers to a feature by which categories of a knowledge community are exposed to the end user. The end user will be able to issue a query for a category as an information type—e.g., ‘Web services.’ The metadata will then be displayed in the semantic browser, as would be the case for any first-class information object type. Visualizations, dynamic links, context palettes, etc. will also be available using the category object as a pivot. This feature is useful in cases where the user wants to start with the category and then use that as a pivot for dynamic navigation, as opposed to starting off with a smart request (smart agent) that has the category as a parameter.
10. Categorized Annotations
Categorized annotations follow from categories being first-class objects. Users will be able to annotate a category directly—thereby simulating an email list that is mapped to a category. However, for cases where there are many categories (for instance, in pharmaceuticals), this is not recommended because information can belong to many categories and the user should not have to think about which category to annotate—the user should publish the annotation directly to the knowledge community (agency) where it will be automatically categorized or annotate an object like a document or email message that is more contextual than a category.
11. Additional Context Templates
1. Experts—The Experts feature was indicated as a special agent in my parent application Ser. No. 10/179,651. As should have also been understood from that application, the Experts feature can also operate in conjunction with the context templates section. Experts are a context template and as the name implies indicate people that have expertise on one or more subject matters or contexts (indicated by the PREDICATETYPEID_EXPERTON predicate).
2. Interest Group—this refers to a context template which as the name implies indicate people that have interest (but not necessarily expertise) on one or more subject matters or contexts (indicated by the PREDICATETYPEID_INTERESTIN predicate). This context template returns People that have shown interest in any semantic category in the semantic network. A very real-world scenario will have Experts returning people that have answers and Interest Group returning results of people that have questions (or answers). In the preferred embodiment, this is implemented by returning results of people who have authored information that in turn has been categorized in the semantic network, with the knowledge domains configured for the KIS. Essentially, this context template presents the user with dynamic, semantic communities of interest. It is a very powerful context template. Currently, most organizations use email distribution lists (or the like) to indicate communities of interest. However, these lists are hard to maintain and require that the administrator manually track (or guess) which people in the organization preferably belong to the list(s). With the Interest Group context template, however, the “lists” now become intelligent and semantic (akin to “smart distribution lists”). They are also contextual, a feature that manual email distribution lists lack.
Like with other context templates, the Interest Group context predicate in turn is interpreted by the server-side semantic query processor. This allows powerful queries like “Interest Group on XML” or “Interest Group on Bioinformatics.” Similarly, this would allow queries (via drag and drop and/or smart copy and paste) like “Interest Group on My Local Document” and “Interest Group on My Competitor (an entity).” The Interest Group context template also becomes a part of the Dossier (or Guide) context template (which displays all special agents for each context templates and loads them as sub-queries of the main agent/request).
In the preferred embodiment, the context template should have a time-limit for which it detects “areas of interest.” An example of this would be three months. The logic here is that if the user has not authored any information (most typically email) that is semantically relevant to the SQML filter (if available) in three months, the user either has no interest in that category (or categories) or had an interest but doesn't any longer.
3. Annotations of My Items—this is a context template that is a variant of Annotations but is further filtered with items that were published by the calling user. This will allow the user to monitor feedback specifically on items that he/she posted or annotated.
12. Importing and Exporting User State
The semantic browser will support the importation and exportation of user state. The user will be able to save his/her personal state to a document and export it to another machine or vice-versa. This state will include information (and metadata) on:
The semantic browser will show UI (likely a wizard) that will allow the user to select which of the user state types to import or export. The UI will also ask the user whether to include identity/logon information. When the UI is invoked, the semantic browser will serialize the user state into an XML document that has fields corresponding to the metadata of all the user state types. When the XML document is imported, the semantic browser will navigate the XML document nodes and add or set the user state types in the client environment corresponding to the nodes in the XML document.
13. Local Smart Requests
Local smart requests would allow the user to browse local information using categories from an knowledge community (agency). In the case of categorized local requests, the semantic client crawls the local hard drives, email stores, etc. extracts the metadata (including summaries) and stores the metadata in a local version of the semantic metadata store (SMS). The client sends the XML metadata (per object) to an knowledge community for categorization (via its XML Web Service). The knowledge community then responds with the category assignment metadata. The client then updates the local semantic network (via the local SMS) and responds to semantic queries just like the server would. Essentially, this feature can provide functionality equivalent to a local server without the need for one.
14. Integrated Navigation
Integrated Navigation allows the user to dynamically navigate from within the Presenter (in the main results pane on the right) and have the navigation be integrated with the shell extension navigation on the left. Essentially, this merges both stacks. In the preferred embodiment, this is accomplished via event signaling. When the Presenter wants to dynamically navigate to a new request, it sets some state off the GUID that identifies the current browser view. The GUID maps to a key in the registry that also has a field called ‘Navigation Event,’ ‘Next Namespace Object ID’ and ‘Next Path.’ The ‘Navigation Event’ field holds a DWORD value that points to an event handle that gets created by the current browser view when it is loaded. When the Presenter wants to navigate to a new request, it creates the request in the semantic environment and caches the returned ID of the request. It then dynamically gets the appropriate namespace path of the request (depending on the information/context type of the request) and caches that too. It then sets the two fields (‘Next Namespace Object ID’ and ‘Next Path’ with these two values). Next, it sets the ‘Navigation Event’ (in Windows™, this is done by calling a Win32 API named ‘SetEvent’).
To catch the navigation event, the browser view starts a worker thread when it first starts. This thread waits on the navigation event (and also simultaneously waits on a shutdown event that gets signaled when the browser view is being terminated—in Windows™, it does this via a Win32 API named ‘WaitForMultipleObjects’). If the navigation event is signaled, the ‘Wait’ API returns indicating that the navigation event was signaled. The worker thread then looks up the registry to retrieve the navigation state (the object id and the path). It then calls the shell browser to navigate to this object id and path (in Windows™, this is done by retrieving a ‘PIDL’ and then calling IShellBrowser::BrowseTo off the shell view instance that implements IShellView).
15. Hints for Visited Results
The Nervana semantic browser empowers the user to dynamically navigate a knowledge space at the speed of thought. The user could navigate along context, information or time axes. However, as the user navigates, he/she might be presented with redundant information. For instance, the user can navigate from a local document to ‘Breaking News’ and then from one of the ‘Breaking News’ result objects to ‘Headlines.’ However, semantically, some of the Headlines might overlap with the breaking news (especially if not enough time has elapsed). This is equivalent to browsing the Web and hitting the same pages over and over again from different ‘angles.’
The Nervana semantic browser handles this redundancy problem by having a local cache of recently presented results. The Presenter then indicates redundant results to the user by showing the results in a different color or some other UI mechanism. The local cache is aged (preferably after several hours or the measured time of a typical ‘browsing experience’). Old entries are purged and the cache is eventually reset after enough time might have elapsed.
Alternately, at the users option, the redundant results can be discarded and not presented at all. Specifically, the semantic browser will also handle duplicate results by removing duplicates before rendering them in the Presenter—for instance if objects with the same metadata appear on different Knowledge Communities (agencies). The semantic browser will detect this by performing metadata comparisons. For unstructured data like documents, email, etc., the semantic browser will compare the summaries—if the summaries are identical the documents are very likely to be identical (albeit this is not absolutely guaranteed, especially for very long documents).
16. Knowledge Federation
Client-Side Knowledge Federation
Server-Side Knowledge Federation
Server-Side Knowledge Federation is technology that allows external knowledge to be federated within the confines of a knowledge community. For instance, many companies rely on external content providers like Reuters™ to provide them with information. However, in the Information Nervous System, security and privacy issues arise—relating to annotations, personal publications, etc. Many enterprise customers will not want sensitive annotations to be stored on remote servers hosted and managed by external content providers.
To address this, external content providers will provide their content on a KIS metadata cache, which will be hosted and managed by the company. For instance, Reuters™ will provide their content to a customer like Intel™ but Intel™ will host and manage the KIS. The Intel™ KIS would crawl the Reuters™ KIS (thereby chaining KIS servers) or the Reuters™ DSA. This way, sensitive Intel™ annotations can be published as ‘Post-Its’ using Reuters™ content as context while Intel™ will still maintain control over its sensitive data.
Federated annotations is a very powerful feature that allows the user to annotate an object that comes from one agency/server (KIS) and annotate the object with comments (and/or attachment(s))—like “Post-Its” on another server. For example, a server (call it Server A) might not support annotations (this is configurable by the administrator and might be the common case for Internet-based servers that don't have a domain of trust and verifiable identity). A user might get a document (or any other semantic result) from Server A but might want to annotate that object on one or more agencies (KISes) that do support annotations (more typically Intranet or Extranet-based agencies that do have a domain of trust and verifiable identity). In such a case, the annotation email message would include the URI of the object to be annotated (the email message and its attachment(s) would contain the annotation itself). When the server crawls its System Inbox and picks up the email annotation, it scans the annotation's encoded To or Subject field and extracts the URI for the object to be annotated. If the URI refers to a different server, the server then invokes an XML Web Service call (if it has access) to that server to get the SRML metadata for the object. The server then adds the SRML metadata to its Semantic Metadata Store (SMS) and adds the appropriate semantic links from the email annotation to the SRML object. This is very powerful because it implies that users of the agency would then view the annotation and also be able to semantically navigate to the annotated object even though that object came from a different server.
If the destination server (for the annotation) does not have access to the server on which the object to be annotated resides, the destination server informs the client of this and the client then has to get the SRML from the server (on which the object resides) and send the complete SRML back to the destination server (for the annotation). This embodiment essentially implies that the client must first “de-reference” the URI and send the SRML to the destination server, rather than having the destination server attempt to “de-reference” the URI itself. This approach might also be superior for performance reasons as it spreads the CPU and I/O load across its clients (since they have to do the downloading and “de-referencing” of the URI to SRML).
Semantic Alerts for Federated Annotations
In the same manner that semantic browser would poll each KIS in the currently viewed user profile for “Breaking News” relevant to each currently viewed object on a regular basis (e.g., every minute), the same will be performed for annotations. Essentially, this resembles polling whether each object that is currently displayed “was just annotated.” For annotations that are not federated (i.e., annotations that have strong semantic links to the objects they annotate), this is a straightforward SQML call back to the KIS from whence the annotated object came. However, for federated annotations, the process is a bit more complicated because it is possible that a copy of object has been annotated on a different KIS even though the KIS from whence the object came doesn't support annotations or contain an annotation for the specific object.
In this case, for each object being displayed, the semantic browser would poll each KIS in the selected profile and pass the URI of the object to “ask” the KIS whether that object has been annotated on it. This way, semantic alerts will be generated even for federated annotations.
This refers to a feature where the KIS returns a context attribute indicating that an object has been annotated. This can be cached when the KIS detects an annotation (typically from the System Inbox) and is updating the semantic network. This context attribute then becomes a performance optimizer because for those objects with the attribute set, the client wouldn't have to query the KIS again to check if the object has been annotated. This amounts to caching the state of the object to avoid an extra (and unnecessary) roundtrip call to the KIS.
Another Perspective on Annotations
An interesting way to think of the Simple and Semantic Annotations feature of the Information Nervous System is that now every object/item/result in a user's knowledge universe will have its own contextual inbox. That way, if a user views the object, the inbox that is associated with the object's context is always available for viewing. In other words,
Category Naming and Identification (URIs) for Federated Knowledge Communities
This refers to how categories will be named on federated knowledge communities. For instance, a Reuters™ knowledge community (agency) deployed at Intel™ will be named Reuters@Intel with categories named like ‘Reuters@Intel/Information Technology/Wireless/80211’. In the preferred embodiment, every category will be qualified with at least the following properties:
The preferred embodiment, the categories knowledge domain id (and not the name) is preferably used in the category URI, because the category could be renamed as the knowledge domain evolves (but the identifier should remain the same). An example of a category URI in the preferred embodiment is:
In this example, the knowledge domain id is c9554bce-aedf-4564-81f7-48432bf8e5a0, the URI type is “category” and the category path is “Information Technology/Wireless/80211”.
17. Anonymous Annotations and Publications
The semantic browser will also allow users to anonymously annotate and publish to an knowledge community (agency). In this mode, the metadata is completely stored (with the user identity) but is flagged indicating that the publisher wishes to remain anonymous. This way, the Inference Engine can infer using the complete metadata but requests for the publisher will not reveal his/her identity. Alternately, the administrator will also be able to configure the knowledge community (agency) such that the inference engine cannot infer using anonymous annotations or publications.
18. Offline Support in the Semantic Browser
The semantic browser will also have offline support. The browser will have a cache for every remote call. The cache will contain entries to XML data. This could be SRML or could be any other data that gets returned from a call to the XML Web Service. Each call is given a unique signature by the semantic browser and this signature is used to hash into the XML data. For instance, a semantic query is hashed by its SQML. Other remote calls are hashed using a combination of the method name, the argument names and types, and the argument data.
For every call to the XML Web Service, the semantic runtime client will extract the signature of the call and then map this to an entry in the local cache. If the browser (or the system) is currently offline, the client will return the XML data in the cache (if it exists). If it does not exist, the client will return an error to the caller (likely the Presenter). If the browser is online, the client will retrieve the XML data from the XML Web Service and update the cache by overwriting the previous contents of the file entry with a file path indicated by the signature hash. This assumes that the remote call actually goes through—it might not even if the system/browser is online, due to network traffic and other conditions. In such a case, the cache does not get overwritten (it only gets overwritten when there is new data; it does not get cleared first).
19. Guaranteed Cross-Platform Support in the Semantic Browser
As discussed in my parent application (Ser. No. 10/179,651), the Information Nervous System can be implemented in a cross-platform manner. Standard protocols are preferably employed where possible and the Web service layer should use interoperable Web service standards and avoid proprietary implementations. Essentially, the test is that the semantic browser does not have to “know” whether the Knowledge community (or agency) Web service it is talking to is running on a particular platform over another. For example, the semantic browser need not know whether the Web service it is talking to is running on Microsoft's .NET™ platform or Sun's J2EE™ platform (to take 2 examples of proprietary application servers), a Linux or any other “open source” server. The Knowledge community Web service and the client-server protocol should employ Web service standards that are commonly supported by different Web service implementations like .NET™ and J2EE™.
In an ideal world, there will be a common set of standards that would be endorsed and properly implemented across Web service vendor implementations. However, this might not be the case in the real world, at least not yet. To handle a case where the semantic browser must handle unique functionality in different Web service implementations, the Knowledge community schema is preferably extended to include a field that indicates the Web service platform implementation. For instance, a .NET™ implementation of the Knowledge community is preferably published with a field that indicates that the platform is .NET™. The same applies to J2EE™. The semantic browser will then have access to this field when it retrieves the metadata for the Knowledge community (either directly via the WSDL URL to the Knowledge community, or by receiving announcements via multicast, the enterprise directory (e.g., LDAP), the Global Knowledge community Directory, etc.).
The semantic browser can then issue platform-specific calls depending on the platform that the Knowledge community is running on. This is not a recommended approach but if it is absolutely necessary to make platform-specific calls, this model is preferably employed in the preferred embodiment.
20. Knowledge Modeling
Knowledge Modeling refers to the recommended way enterprises will deploy an Information Nervous System. This involves deploying several KIS servers (per high-level knowledge domain) and one (or at most few) KDS (formerly KBS) servers that host the relevant ontology and taxonomy. KIS servers are preferably deployed per domain to strike a balance between being too narrow such that there is not enough knowledge sharing possibility of navigation and inference in the network and being too high that scalability (in storage and CPU horsepower needed by the database and/or the inference engine) becomes a problem. Of course, the specific point of balance will shift over time as the hardware and software technologies evolve, and the preferred embodiment does not depend on the particular balance struck. In addition, KIS servers are preferably deployed where access control becomes necessary at the server level (for higher-level security) as opposed to imposing access control at the group level with multiple groups sharing the same KIS. For instance, a large pharmaceutical company could have a knowledge community KIS for oncology for the entire company and another KIS for researchers working on cutting-edge R&D and applying for strategic patents. These two KIS' might crawl the same sources of information but the latter KIS would be more secure because it would provide access only to users from the R&D group. Also, optionally, these researchers' publications and annotations will not be viewable on the corporate KIS.
Knowledge Integration Server 1 (Oncology)
Knowledge Integration Server 2 (Pharmacology)
Knowledge Integration Server 3 (Biotechnology)
Knowledge Integration Server 4 (Cardiology)
Knowledge Domain Server (Pharmaceuticals)
21. KIS Housekeeping Rules
The Knowledge Integration Server (KIS) will allow the admin to set up ‘housekeeping’ rules to purge old or stale metadata. This will prevent the SMS on the KIS from growing infinitely large. These rules could be as simple as purging any metadata older than a certain age (between 2-5 years depending on the company's policies for keeping old data) and which does not have any annotations and that is not marked as a favorite (or rated).
22. Client Component Integration & Interaction Workflow
The client components of the system can be integrated in several different steps or sequences, as can the workflow interaction or usage patterns. In the presently preferred embodiment, the workflow and component integration would be as follows:
1) Shell: User implicitly creates a SQML query (i.e. an agent) via UI navigation or a wizard.
2) Shell: User opens an agent (via tree or folder view).
3) The query buffer is saved as a file, and a registry entry created is created for the agent.
a) Registry entry contains: Agent Name, Creation date, Agent (Request)-GUID, SQML path, Comments, Namespace object type (agency, agent, blender, etc), and attributes
4) Shell: The request is handed off to the presenter:
a) A registry request GUID entry is created containing (namespace path that generated the request, and SQML file URL).
b) Browser is initialized and opened with command line [http]://PresenterPage.html#RequestGUID [http]://presenterpage.html/. The Presenter loads default Chrome contained in the page.
c) Presenter page loads presenter binary behavior and Semantic Runtime OCX.
5) Presenter: Loads SQML and issues requests via the query manager.
a) Resolves request GUID to get SQML file path.
b) Loads SQML file into buffer, creates resource handler requests, passes them to resource handlers, waits for and gathers results. Summarization of local resources happens here. All summarization follows one of two paths: Summarize the doc indicated by this file path, or summarize this text (extracted from clipboard, Outlook™, Exchange™, etc.). Both paths produce a summary in the same form, suitable for inclusion in a request to the semantic server XML Web service.
c) Compiles SQML file into individual server request buffers, including any resource summary from above.
d) Initiates Server Requests by calling semantic runtime client Query Manager.
6) Query Manager: Monitors server requests and makes callback on data. It also signals an event on request completion or timeout. The callback is into the Presenter, which mean inter-process messaging to pass the XML.
7) Presenter: receives data and loads appropriate skin:
a) Receives SRML data in buffer; this will happen incrementally.
b) Determines if there is a preferred skin (smart style) associated with this agent, otherwise chooses default skin.
c) Transforms SRML into preferred skin format via XSLT. This is multistage, for the tree of results (root is list, then objects, then Deep/Lens/BN info) as results come in.
d) Display results in target DIV in page. The target is an argument to the behavior itself and is defined by the root page.
8) Presenter: Calls Semantic Runtime to fill context panels (per context template), deep info, smart copy and paste, and other semantic commands. The Presenter also loads the smart style, which then loads semantic images, motion, etc. consistent with the semantics of the request.
23. Categories Dialog Box User Interface Specification
The Categories Dialog Box allows the user to select one or more categories from a category folder (or taxonomy) belonging to a knowledge domain. While more or fewer can be deployed in certain situations, in the preferred embodiment, the dialog box has all of the following user interface controls:
1. Profile—this allows the user to select a profile with which to filter the category folders (or taxonomies) based on configured areas of interest. For instance, if a profile has areas of interest set to “Health and Medicine,” selecting that profile will display only those category folders that belong to the “Health and Medicine” area of interest (for instance, Pharmaceuticals, Healthcare, and Genes). This control allows the user to focus on the taxonomies that are relevant to his/her knowledge domain, without having to see taxonomies from other domains.
2. Area of Interest—this allows the user to select a specific area of interest. By default, this combo box is set to “My Areas of Interest” and the profile combo box is set to “All Profiles.” This way, the dialog box will display category folders for all areas of interest for all profiles. However, by using the “Area of Interest” combo box, the user can directly specify an area of interest with which to filter the category folders, regardless of the areas of interest in his/her profile(s).
3. Publisher Domain Zone/Name—this allows the user to select the domain zone and name of the taxonomy publisher. This is advantageous to distinguish publishers that might have name collisions. In the preferred embodiment, the Publisher Domain Name uses the DNS naming scheme (for instance, IEEE.org, Reuters.com™). The domain zone allows the user to select the scope of the domain name. In the preferred embodiment, the options are Internet, Intranet, and Extranet. The zone selection further distinguishes the published category folder (or taxonomy). A fairly common case would be where a department in a large enterprise has its own internal taxonomy. In this case, the department will be assigned the Intranet domain zone and will have its own domain name—for instance, Intranet\Marketing or Intranet\Sales.
4. Category Folder—this allows the user to select a category folder or taxonomy. When this selection is made, the categories for the selected category folder are displayed in the categories tree view.
5. Search categories—this allows the user to enter one or more keywords with which to filter the currently displayed categories. For instance, a Pharmaceuticals researcher could select the Pharmaceuticals taxonomy but then enter the keyword “anatomy” to display only the entries in the taxonomy that contain the keyword “anatomy.”
6. “Remember” check box—this allows the user to specify whether the dialog box should “remember” the last search when it exits. This is very helpful in cases where the user might want to perform many similar category-based searches/requests from the same category folder and with the same keyword filter(s).
7. Search Options—these controls allow the user to specify how the dialog box should interpret the keywords. The options allow the user to select whether the keywords should apply to the entire hierarchy of each entry in the taxonomy tree, or whether the keywords should apply to only the [end] names of the entries. For instance, the taxonomy entry “Anatomy\Cells\Chromaffin Cells” will be included in a hierarchy filter because the hierarchy includes the word “Anatomy.” However, it will be excluded from a names filter because the end-name (“Chromaffin Cells”) does not include the word “Anatomy.”
Also, the search options allow the user to select whether the dialog box should check for all keywords, for any keyword, or for the exact phrase.
8. Categories Tree View—the tree view displays the taxonomy hierarchy and allows the user to select one or more items to add to the Create Request Wizard or to open as a new Dossier (Guide) request/agent. The user interface breaks the category hierarchy into “category pages”—for performance reasons. The UI allows the user to navigate the pages via buttons and a slide control. There is also a “Deselect All” button that deselects all the currently selected taxonomy items.
9. Explore Button—this is the main invocation button of the dialog box. When the dialog box is launched from the Create Request Wizard, this button is renamed to “Add” and adds the selected items to the wizard “filters” property page. When the dialog box is launched directly from the application, the button is titled “Explore” and when clicked launches a Dossier request on the selected categories. If the user has multiple profiles or if multiple taxonomy categories are selected, the dialog box launches another dialog box, the “Explore Categories Options” dialog box that prompts the user to select the profile with which to launch the Dossier and/or the operator to use in applying the categories as filters to the Dossier (AND or OR).
24. Client-Assisted Server Data Consistency Checking
As the server (KIS) crawls knowledge sources, there will be times when the server's metadata cache is out of sync with the sources themselves. For instance, a web crawler on the KIS that periodically crawls the Web might add entries into the semantic metadata store (SMS) that become out of date. In this case, the client would get a 404 error when it tries to invoke the source URI. For data source adapters (DSAs) that have monitoring capabilities (for instance, for file-shares that can be monitored for changes), this wouldn't be much of an issue because the KIS is likely to be in sync with the knowledge source(s). However, for sources such as Web sites that don't have monitoring/change-notification services, this may present an issue of concern.
My parent application (Ser. No. 10/179,651) described how the KIS can use a consistency checker (CC) to periodically purge stale entries from the SMS. However, in some situations this approach might impair performance because the CC would have to periodically scan the entire SMS and confirm whether the indexed objects still exist. An alternative embodiment of this feature of the invention is to have the client (the semantic browser) notify the server if it gets a 404 error. To do this, the semantic browser would have to track when it gets a 404 error for each result that the user “opens.” For Web documents, the client can poll for the HTTP headers when it displays the results, even before the user opens the results. In this case, if the source web server reports a 404 error (object not found), the client should report this to the KIS.
When the KIS gets a “404 report” from the client, it then intelligently decides whether this means the object is no longer available. The KIS cannot arbitrarily delete the object because it is possible that the 404 error was due to an intermittent Web server failure (for instance, the directory on the Web server could have been temporarily disabled). The KIS should itself then attempt to asynchronously download the object (or at the very least, the HTTP headers in the case of a Web object) several times (e.g., 5 times). If each attempt fails, the KIS can then conclude that the object is no longer available and remove it from the SMS. If another client reports the 404 error for the same object while the KIS is processing the download, the KIS should ignore that report (since it is redundant).
This alternate technique could be roughly characterized as lazy consistency checking In some situations, it may be advantageous and preferred.
25. Client-Side Duplicate Detection
The server (KIS) performs duplicate detection by checking the source URIs before adding new objects into the semantic metadata store (SMS). However, for performance reasons, it is sometimes advantageous if the server does not perform strict duplicate-detection. In such cases, duplicate detection is best performed at the client. Furthermore, because the client federates results from several KISes, it is possible for the client to get duplicates from different KISes. As such, it is advantageous if the client also performs duplicate detection.
In the preferred embodiment, the client removes objects that are definitely duplicates and flags objects that are likely duplicates. Definite duplicates are objects that have the same URI, last modified time stamp, summary/concepts, and size. Likely duplicates are objects that have the same summary/concepts, but have different URIs, last modified times, or sizes. For objects for which summary extraction is difficult, it is recommended that the title also be used to check for likely duplicates (i.e., objects that have the same summary but different titles are not considered likely duplicates because the summary might not be a reliable indicator of the contents of the object). Also, if summary/concept extraction is difficult (in order to detect semantic overlap/redundancy), the semantic browser can limit the file-size check to plus or minus N % (e.g., 5%)—for instance, an object with the same summary/concepts and different URIs, last-modified times, and sizes might be disqualified as a likely duplicate if the file-size is within 5% of the file-size of the object it is being compared to for redundancy checking
26. Client-Side Virtual Results Cursor
The client (semantic browser) also provides the user with a seamless user experience when there are multiple knowledge communities (agencies) subscribed to a user profile. The semantic browser preferably presents the results as though they came from one source. Similarly, the browser preferably presents the user with one navigation cursor—as the user scrolls, the semantic browser re-queries the KISes to get more results. In the preferred embodiment, the semantic browser keeps a results cache big enough to prevent frequent re-querying—for instance, the cache can be initialized to handle enough results for between 5-10 scrolls (pages). The cache size are preferably capped based on memory considerations. As the cursor is advanced (or retreated), the browser checks if the current page generates a cache hit or miss. If it generates a cache hit, the browser presents the results from the cache, else if re-queries the KISes for additional results which it then adds to the cache.
The cache can be implemented to grow indefinitely or to be a sliding window. The former option has the advantage of simplicity of implementation with the disadvantage of potentially high memory consumption. The latter option, which is the preferred embodiment, has the advantage of lower memory consumption and higher cache consistency but with the cost of a more complex implementation. With the sliding window, the semantic browser will purge results from pages that do not fall within the window (e.g., the last N—e.g., 5-10—pages as opposed to all pages as with the other embodiment).
27. Virtual Single Sign-On
With virtual single sign-on, the user specifies his/her logon credentials to the semantic browser in a server (knowledge community)-independent fashion. The semantic browser stores the credentials in a Credential Cache Table (CCT). The CCT has columns as illustrated below:
When the user first attempts to subscribe to a knowledge community (or access the knowledge community in some other way—for instance, to get the properties of the community), the semantic browser prompts the user for his/her password and then tries to logon to the server using the supplied credentials. If a logon is successful, the semantic browser creates a new CCT entry (CCTE) with the supplied credentials and adds the KC to the Knowledge Community Entry List (KCEL) for the new CCT entry.
For each subsequent subscription attempt, the semantic browser checks the CCT to see if the KC the user is about to subscribe to is in the KCEL for any CCTE. If it is, the semantic browser retrieves the credentials for the CCTE and logs the user on with those credentials. This way, the user does not have to redundantly enter his/her logon credentials.
Note that the semantic browser also supports pass-through authentication when the operating system is already logged on to a domain. For instance, if a Windows™ machine is already logged on to an NT (or Active Directory™) domain, the client-side Web service proxy also includes the default credentials to attempt to logon to a KC. In the preferred embodiment, the additional credentials supplied by the user are preferably passed via SOAP security headers (via Web Services Security (WS-Security) or a similar scheme). For details of WS-Security and passing authentication-information in SOAP headers, see [http]://[www].oasis-open.org/committees/download.php/3281/WSS-SOAPMessageSecurity-17-082703-merged.pdf
The semantic browser exposes a property to allow the user to indicate whether the credentials for a CCTE are preferably purged when the KCEL for the CCTE is empty or whether the credentials should be saved. In the preferred embodiment, the credentials are preferably saved by default unless the user indicates otherwise. If the user wants the credentials purged, the semantic browser should remove a KC from a CCTE in which it exists when that KC is no longer subscribed to any profile in the browser. If after removing the KC from the CCTE's KCEL, the CCTE becomes empty, the CCTE is preferably deleted from the CCT.
The virtual single sign-on feature, like many of the features in this application, could be used in applications other than with my Information Nervous System or the Virtual Librarian. For example, it could be adapted for use by any computer user who must log into more than one domain.
28. Namespace Object Action Matrix
The table below shows the actions that the semantic browser invokes when namespace objects are copied and pasted onto other namespace objects.
29. Dynamic End-to-End Ontology/Taxonomy Updating and Synchronization
The Information Nervous System™ will support dynamic updates of ontologies and taxonomies. Knowledge domain plug-ins that are published by Nervana (or that are provided to Nervana by third-party ontology publishers) will be hosted on a central Web service (an ontology depot) on the Nervana Web domain (Nervana.com). Each KDS will then periodically poll the central Web service via a Web service call (for each of its knowledge domain plug-ins, referenced by the URI or a globally unique identifier of the plug-in) and will “ask” the Web service if the plug-in has been updated. The Web service will use the last-modified timestamp of the ontology file to determine whether the plug-in has been updated. If the plug-in has been updated, the Web service will return the new ontology file to the calling KDS. The KDS then replaces its ontology file.
If the KDS is running during the update, it will ordinarily temporarily stop the service before replacing the file, unless it supports file-change notifications and reloads the ontology (which is the recommended implementation).
Each KIS also has to poll each KDS it is connected to in order to “ask” the KDS if its ontology has changed. In the preferred embodiment, the KIS should poll the KDS and not the central Web service in case the KDS has a different version of the ontology. The KDS also uses the last modified time stamp of the knowledge domain plug-in (the ontology) to determine if the ontology has changed. It then indicates this to the KIS. If the ontology has changed, the KIS needs to update the semantic network accordingly. In the preferred embodiment, it does this by removing semantic links that refer to categories that are not in the new version of the ontology and adding/modifying semantic links based on the new version of the ontology. In an alternative embodiment, it purges the semantic network and re-indexes it.
The client then polls each KIS it is subscribed to in order to determine if the taxonomies it is subscribed to (directly via the central Web service or via the KISes) have changed. The KIS exposes a method via the XML Web service via which the client determines if the taxonomy has changed (via the last modified time stamp of the taxonomy/ontology plug-in file). If the taxonomy has changed, the client needs to update the Categories Dialog user interface (and other UI-based taxonomy dependents) to show the new taxonomy.
For taxonomies that are centrally published (e.g., via Nervana), the client should poll the central Web service to update the taxonomies.
With this model, the client, KIS, KDS, and central taxonomy/ontology depot will be kept synchronized.
30. Invoking Dossier (Guide) Queries
Dossier Semantic Query Processing
Dossier (Guide) queries are preferably invoked by the client-side semantic query processor by parsing the SQML of the request/agent and replacing the Dossier context predicate with each special agent (context template) context predicate—e.g., All Bets, Best Bets, Breaking News, Headlines, Random Bets, Newsmakers, etc. Each query (per context template) is then invoked via the query processor—just like an individual query. This way, the user operates at the level of the Dossier but the semantic browser maps the dossier to individual queries behind the scenes.
For example, the SQML for “Dossier on Category C” is parsed and new SQML queries are generated as follows:
All Bets on Category C
Best Bets on Category C
Breaking News on Category C
Headlines on Category C
Random Bets on Category C
Newsmakers on Category C
The client-side semantic query processor retains every other predicate except the context predicate. This way, the filters remain consistent as illustrated by the example above.
Dossier Smart Lens
Like other requests/agents in the Information Nervous System™, dossiers (guides) can be used as a Smart Lens (just like how they can be targets for drag and drop, smart copy and paste, etc.). In this case, the smart lens displays a “Dossier Preview Window” with sections/tabs/frames for each context template (special agent).
31. Knowledge Community (Agency) Semantics
The following describe the semantics of a knowledge community (agency) within the context of the semantic namespace/environment in the semantic browser:
1. Selecting a knowledge community—this opens a dossier request from that KC. Essentially, the Dossier becomes the equivalent of the KC's “home page.”
2. Drag and drop (document, text, entity, keywords, etc.) to a KC—this opens a Dossier request/agent on the object (using the default predicate) from the KC
3. Copy KC to the clipboard—this selects KC as the Smart Lens. When the user hovers over a result or entity, the semantic browser displays the Smart Lens by showing the KC name and the KC's profile name under the cursor and then opens a Dossier from the KC on the object underneath the lens in the lens preview pane
4. Subscribing to a KC—when a KC is subscribed for the first time, the semantic browser adds the KC's email address to the local email contacts (e.g., in Microsoft Outlook™ or Outlook Express™). This makes it easy for the user to publish knowledge to the KC by sending it email (via the integrated contacts list). Similarly, when the KC is unsubscribed from all profiles, the semantic browser prompts the user whether it should remove the KC from the local email contacts list.
32. Dynamic Ontology and Taxonomy Mapping
One of the challenges of using taxonomies and ontologies is how to map the semantics of one taxonomy/ontology onto another. The Information Nervous System™ accomplishes this by the following algorithm:
Each KDS will be responsible for ontology mapping (via an Ontology Mapper (OM)) and will periodically update the central Web service (the ontology depot) with an Ontology Mapping Table (OMT). The updates are bi-directional: the KDS will periodically update its ontologies and taxonomies from the central Web service and send updates of the OMT to the central Web service. Each OMT will be different but the central ontology depot will consolidate all OMTs into a Master OMT. The ontology mapper will create a consistent user experience because the user wouldn't have to select all items in the umbrella taxonomy that are relevant but overlapping. The semantic browser will automatically handle this. The KIS wouldn't have any concept of the mapper but will get mapped results from the KDS which it will then use to update the semantic network.
The KDS and KIS administrators would still be responsible for selecting the right KDS ontology plug-ins, however—based on the quality of each ontology/taxonomy (the ontology mapping doesn't improve ontologies; it merely maps them).
33. Semantic Alerts Optimizations
Semantic Alerts in the semantic browser can be optimized by employing the following rule (in order):
For a given filter (e.g., result, document, text, keywords, entity):
1. Check for Headlines first.
2. If there are Headlines, check for Breaking News and Newsmakers.
This is because in the preferred embodiment, Headlines are implemented similar to Breaking News except with a larger time window. As a consequence, if there are no Headlines (in the preferred embodiment), there is no Breaking News. Also, in the preferred embodiment, Newsmakers are implemented by returning the authors of Headlines. As such, if there are no Headlines, there are no Newsmakers.
34. Semantic “News” Images
Both Corbis™ ([http]://[www].corbis.com) and Getty Images™ ([http]://[www].gettyimages.com) have “News” images that are constantly kept fresh. The Information Nervous System™ can use these kinds of images for semantic images that are not only context-sensitive but also “fresh.” This can be advantageous in terms of keeping the user interface interesting and “new.” For instance, “Breaking News on SARS” can show not only pharmaceutical images but images showing doctors responding to recent SARS outbreaks, etc.
35. Dynamically Choosing Semantic Images
Semantic images can be dynamically and intelligently selected using the following rules:
1. If the currently displayed namespace object is a request, parse the SQML of the object for categories. If there are categories, send the categories to the central Web service (that hosts the semantic image cache) to get images that are relevant to the categories. Also, send the request type (e.g., knowledge types like All Bets and Headlines, or information types like Presentations) to the central Web service to return images consistent with the request type
2. If the namespace object is not a request, send the areas of interest for the current profile (if available) to the central Web service. The Web service then returns semantic images consistent with the profile's areas of interest. If the profile does not have configured areas of interest, send the areas of interest for the application (the semantic browser). If the application does not have configured areas of interest, send an empty string to the central Web service—in this case, the central Web service returns generic images (e.g., branded images).
36. Dynamic Knowledge Community (Agency) Contacts Membership
Knowledge communities (agencies) have members (users that have read, write, or read-write access to the community) and contacts. Contacts are users that are relevant to the community but are not necessarily members. For example, a departmental knowledge community (KC) in a large enterprise would likely have the members of the department as members of the KC but would likely have all the employees of the enterprise as contacts. Contacts are advantageous because they allow members of the KC to navigate users that are semantically relevant to the KC but might not be members. The KC might semantically index sent by contacts—the index in this case would include the contacts even though the contacts are not members of the KC.
Another way to think of this is that communities of knowledge in the real world tend to have core members and peripheral members. Core members are users that are very active in the community while peripheral members include “other” users such as knowledge hobbyists, occasional contributors, potential recruits, and even members of other relevant communities.
With dynamic KC contacts membership in the Information Nervous System™, the KIS will add users to its Contacts table in the semantic metadata store (SMS) and to the semantic network “when and as it sees them” (in other words, as it indexes email messages that have new users that are not members). This allows the community to dynamically expand its contacts, but in a way that distinguishes between Members and mere Contacts, and “understands” the importance of the distinction semantically when operating the system (e.g., executing searches and the like).
37. Integrated Full-Text Keyword and Phrase Indexing
The KIS also indexes concepts (key phrases) and keywords as first-class members of the semantic network. This can be done in a domain-independent fashion as follows:
For each new object (e.g., documents) to be added to the semantic network:
1. Extract concepts (key phrases) from the body of the object.
2. For each concept, add the concept to the semantic network with the object type id OBJECTTYPEID_CONCEPT. Add a semantic link with the predicate PREDICATETYPEID_CONTAINSCONCEPT to the “Semantic Links” table with the new object as subject and the new concept object as the subject.
3. For the current concept, extract the keywords from the concept key phrase and add each keyword to the semantic network with the object type id OBJECTTYPEID_KEYWORD. Also, add a semantic link with the predicate PREDICATETYPEID_CONTAINSKEYWORD to the “Semantic Links” table with the new object as subject and the new keyword object as the subject.
Repeat the steps above for the title of the object and other meta-tags as appropriate for the schema of the object.
While some embodiments do not require integrated full-text indexing, it is included in the presently preferred embodiment because it provides several useful advantages:
1. It allows a consistent model for implementing semantic filters (in SQML). The user can add categories, documents, entities, and keywords as filters and the filters are applied consistently to the semantic network (as sub-queries).
2. In particular, it supports the semantic query processing of entities. Entities can be defined with categories and can be further narrowed with keywords (to disambiguate the keywords in the case where the keywords could mean different things in different contexts). Integrated full-text indexing allows the KIS semantic query processor (SQP) to interpret entities seamlessly—by applying the necessary sub-queries with categories and keywords/concepts to the semantic network.
3. In general, integrated full-text indexing results in a seamless and consistent data and query model.
38. Semantic “Mark Object as Read”
In some cases, the KIS might not have the resources to store semantic links between People and objects on a per-object basis. In addition, semantic-based redundancy is not the same as per-object redundancy—as in email. To take an example, email clients allow users to select an email message as read or unread—this is typically implemented as a flag stored on the mail server with the email message. However, because email is not a semantic system, a semantically similar or identical message on the server would not be flagged as such—the user has to flag each message separately regardless of semantic redundancy.
In the Information Nervous System™, the user is able to flag an object as read not unlike in email. However, in this case, the semantic browser extracts the concepts from the object and informs all the KISes in the request profile that the “concepts” have been read. The KIS then dynamically maps the concepts to categories via the KDSes it is configured with and adds a flag to the objects belonging to those categories (in the preferred embodiment) and/or adds a flag to the semantic network with a semantic link with the predicate PREDICATETYPEID_VIEWEDCATEGORY between the categories corresponding to the concepts and all the objects that are linked to the categories. In the preferred embodiment, the KIS should only flag those categories over a link-strength threshold (for the source concepts). This ensures that only those objects (in the preferred embodiment) and/or categories that are semantically close to the original object will be flagged.
When the semantic browser flags the object via the KISes, the KISes should return a flag indicating whether the network was updated (it is possible that no changes would be made in the event that the object does not have any “strong” categories or if there are no other objects that share the same “strong” categories). If at least one KIS in the request profile indicates that the network was updated, the semantic browser should refresh the request/agent. The semantic browser can expose a property to allow the user to indicate whether he/she wants the KISes to return only unread objects or all objects (read or unread), in which case the browser should display unread objects differently (like how email clients display unread messages in a bold font). The presentation layer in the semantic browser should then display the read and unread objects with an appropriate font and/or color to provide a clear visual distinction.
39. Multi-Select Object Lens
Multi-select object lens is an alternative implementation of the object lens that was described in my parent application. In that embodiment, the object lens was invoked via smart copy and paste—pasting an object over another object would invoke the object lens with the appropriate default predicate. This has the benefit of allowing the user to copy objects across instances of the semantic browser, across profiles, and from other environments (like the file-system, word processors, email clients, etc.).
In the currently preferred embodiment, the object lens is a Dossier Lens (the context predicate is a Dossier, the filters are the source and target objects, and the profile is the profile in which the source object was displayed).
Multi-selection can also be used instead of copy and paste to invoke an object lens. The semantic browser will allow the user to select multiple objects (results). The user can then hit a button (or alternative user-interface object) to invoke the object lens on the selected objects. In this case, a Dossier Lens will be displayed (in a preview pane) with a Dossier context predicate, with the filters as the selected objects, and the current profile as the request profile.
40. Ontology-Based Filtering and Spam Management
The KIS (in the preferred embodiment) would only add objects to the Semantic Metadata Store (SMS) if those objects belong to at least one category from at least one of the knowledge domains the KIS is configured with (via one or more KDSes). This essentially means the KIS will not index objects it “does not understand.” The exception to this is that the KIS will index all objects from its System Inbox—because this contains at-times personal community-specific publications and annotations that might be relevant but not always semantically relevant.
A side-effect of this ontology-based filtering model is spam management—ontology-based indexing would be effective in preventing spam from being indexed and stored. If users use the semantic browser to access email, as opposed to their inboxes, only email that has been semantically filtered will get through.
41. Results Refinement
The results of a request/agent can be further refined via additional filters and predicates. For example, the request/agent Headlines on Bioinformatics could be further refined with keywords specific to certain areas of Bioinformatics. This way, the end-user can further narrow the result set using the request/agent as a base. In addition, for time-sensitive requests, the user can specify a time-window to override the default time-window. For example, the default Breaking News time-request could be set to 3 hours. The user should be able to override this for a specific request/agent (in addition to changing the defaults on a per-profile or application-wide basis) with an appropriate UI mechanism (e.g., a slider control that ranges from 1 hour to 24 hours). The same applies to Headlines and Newsmakers (e.g., a slider control that ranges from 1 day to 1 week).
When the user specifies a filter-override, the semantic browser invokes the XML Web Service call for each of the KISes in the request profile and passes the override arguments as part of the call. If override arguments are present, the Web service uses those values instead of the default filter values. The same applies to additional filters (e.g., keywords)—these will be passed as additional arguments to the Web service and the Web service will apply additional sub-queries appropriately to further filter the query that is specified in the agent/request SQML (in other words, the SQML is passed as always, but in addition, the filter overrides and additional filters are also passed).
A good case for filter-overrides will be for Best Bets. The default semantic relevance strength for Best Bets could be set to 90% (in the preferred embodiment). However, for a given request/agent, the user might want to see “bets” across a semantic relevance range. Exposing a relevance UI control (e.g., a slider control that ranges from 0% to 100%) will allow this. This essentially allows the user to change the Best Bets on the fly from “All Bets” (0%) all the way to “Perfect Bets” (100%).
A hybrid model should also be employed for embodiments of context template (special agent) implementations that involve multiple axes of filtering. For instance, Breaking News could also impose a relevance filter of 25% and Headlines and Newsmakers could impose a relevance filter of 50% (Breaking News has a lower relevance threshold because it has a higher time-sensitivity threshold; as such, the relevance threshold can be relaxed). In this case, the semantic browser should expose UI controls to allow the user to refine the special agents across both axes (a slider control for time-sensitivity and another slider control for relevance).
With dossiers, the semantic browser can display UI controls for each special agent displayed in the Dossier—the main Dossier pane can show all the UI controls (changing any UI control would then refresh the Dossier sub-request for that special agent). Also, if the Dossier has tabs for each special agent, each tab can have a UI control specific to the special agent for the tab.
42. Semantic Management of Information Stores
The Information Nervous System™ can also be used to manage information stores such as personal email inboxes, personal contact lists, personal event calendars, a desktop file-system (e.g., the Microsoft Windows Explorer™ file-management system for local and network-based files), and also other stores like file-shares, content management systems, and web sites.
For client-based stores (such as email inboxes and file-systems), the client runtime of the semantic browser should periodically poll the store via a programmatic interface to check for items that have become redundant, stale, or meaningless. This would address the problem today where email inboxes keep growing and growing with stale messages that might have “lost their meaning and relevance.” However, due to the sheer volume of information users are having to cope with, many computer users are losing the ability to manage their email inboxes themselves, resulting in a junk-heap of old and perhaps irrelevant messages that take up storage space and make it more difficult to find relevant messages and items.
The client runtime should enumerate the items in the user's information stores, extract the concepts from the items (e.g., from the body of email messages and from local documents) and send the concepts to the KISes in the user's profiles. In an alternative embodiment, only the default profile should be used. The client then essentially “asks” the user's subscribed KISes whether the items mean anything to them. In the preferred embodiment, the client should employ the following heuristics:
1. First, check for redundancy—by flagging (or deleting) duplicate email items, duplicate documents that share concepts and summaries (but perhaps with different titles or file-sizes). The client should either delete the duplicate items (user-configurable) or flag the items by moving them into a special folder (user-configurable) in the email client or desktop.
2. Next, for non-duplicate items, the client should check for meaninglessness or irrelevance. First, the client should only check items that are “older” than N days (e.g., 30 days) by examining the last-modified time of the email item, document, or other object. For items that qualify, extract the concepts and call the XML Web Service for each KIS in all the user's profiles (or the default profile in an alternative embodiment).
3. For very old items (e.g., older than 180 days), the client should specify a very low threshold of meaning to the XML Web Service (e.g., 25%) for preservation. Essentially, this is akin to deleting (or flagging) those items that are very old and weak in meaning.
4. For fairly old items (e.g., older than 90 days old but younger than 180 days old), the client should specify a very low threshold (e.g., 10%) for preservation. This is akin to deleting (or flagging) those items that are fairly old and very weak in meaning.
5. For old items (but not too old—e.g., older than 1 day old but younger than 30 days old), the client should specify a very low threshold (e.g., 0%) for preservation. This is akin to deleting (or flagging) those items that are old (but not too old) but are meaningless, based on the user's profile(s).
Essentially, the model for this aspect or feature of the preferred embodiment balances semantic sensitivity with time-sensitivity by imposing a higher semantic threshold on younger items (thereby preserving items that might be largely—albeit not totally—meaningless if they are fairly young. For example, fairly recent email threads might be very weak in meaning—the client should preserve them anyway because their “youth” is also a sign of relevance. As they “age,” however, the client can safely delete them (or flag them for deletion).
This model can also be applied to manage documents on local file-systems. The model can be extended to content-management systems, document repositories, etc. by configuring an Information Store Monitor (ISM) to monitor these systems (via calls to the Information Nervous System™ XML Web Services) and configuring the ISM with KISes that are configured with KDSes that have ontologies consistent with the domain of the repositories to be semantically managed. This feature will save storage space and storage/maintenance costs by semantically managing content management systems and ensuring that only relevant items get preserved on those systems over time.
43. Slide-Rule Filter User Interface
The refinement pane in the semantic browser allows the user to “search within results.” The user will be able to add additional keywords, specify date ranges, etc. The date-range control can be implemented like a slide-rule. Shifting one panel in the slide-rule would shift the lower date boundary while moving the other panel will shift the upper date boundary. Other panels can then be added for time boundaries—shifting both time and date panels will impose both date and time constraints. Panels can also be added for other filter axes.
This section describes a currently preferred embodiment of how the server-side semantic query processor (SQP) resolves SQML queries. On a given server, queries can be broken into several components:
a. Context (documents, keywords, entities, portfolios (or entity collections)).
b. Context/Knowledge Template (or Special Agent) or Information Template—this describes whether the request if for a knowledge type (e.g., Breaking News, Conversations, Newsmakers, or Popular Items) or for a particular information type (e.g., Documents, Email).
On the client, a semantic query is made up of the triangulation of context, request (or Agent) type, and the knowledge communities (or Agencies). The client sends the SQML that represents the semantic query to all the knowledge communities in the profile in which the request lives. The client asks for a few results at a time and then aggregates the results from one or more servers.
The server-side semantic query processor subdivides semantic queries into several sub-queries, which it then applies (via SQL inner joins or sub-queries in the preferred embodiment). These sub-queries are:
1. Request type sub-query—this represents a sub-query (semantic or non-semantic) depending on the request type. Examples are context (knowledge) types (e.g., All Bets, Best Bets, Headlines, Experts, etc.) and information types (like General Documents, Presentations, Web Pages, Spreadsheets, etc.).
2. Semantic context sub-query—this represents a semantic sub-query derived from the context (filter) passed from the client (an example of this is categories sent from the client or mapped from keywords/text via semantic stemming).
3. Non-semantic context sub-query—this represents a non-semantic sub-query derived from the context (filter) passed from the client (examples are keywords without semantic stemming—mapping to ontology-based categories).
4. Access-control sub-query—this represents a sub-query that filters out those items in the semantic metadata store (SMS) that the calling user does not have access to. For details, see the “Security” specification.
The foregoing steps are illustrated in
2. Semantic Relevance Score
The semantic relevance score defines the normalized score that the concept extraction engine returns. It maps a given term of “blob” of text to one or more categories for a given ontology. The score is added to the semantic network (in the “LinkStrength” field of the “SemanticLinks” table) when items are added to the Semantic Network.
3. Semantic Relevance Filter
The relevance filter is different from the relevance score (indeed, both will typically be combined). The relevance filter indicates how the SQP will semantically interpret context (note: in the currently preferred embodiment, the filtering is always semantic in this case). There are two relevance filters: High and Low. With the High relevance filter, the SQP will include a sub-query that is the intersection of categories and terms. For instance, context for the keyword “XML” will be interpreted as: Items that share the same categories as XML and also include the keyword “XML.” This is the highest level of ontology-based semantic filtering that can occur. However, it could lead to information loss in cases where there are objects in the Semantic Network (or Semantic Metadata Store (SMS)) that are semantically equivalent to the context but that do not share its keywords or terms. For instance, the query described above would miss items that share the same categories as XML but which include the term “Extensible Markup Language” instead. A Low relevance filter will only include objects that share the same categories as the context but unlike the High relevance filter, would not include the additional constraint of keyword equivalence.
For this reason, the relevance filter is preferably used only to create sub-query “buckets” that are then used for ordering results. For instance, the SQP might decide to prioritize a High relevance filter ahead of a Low relevance filter when filtering the semantic network but would still return both (with duplicates removed) in order to help guarantee that synonyms don't get rejected during the final semantic filtering process.
4. Time-Sensitivity Filter
The time-sensitivity filter determines how time-critical the semantic sub-query is. There are two levels: High and Low. A High filter is meant to be extremely time-critical. Default is 3 hours (this accounts for lunch breaks, time away from the office/desk, etc.). A Low filter is meant to be moderately time-critical. The default is 12 hours.
5. Knowledge Type Semantic Query Implementations
Throughout this application certain specific knowledge types are referred to by apt shorthand names, some of which the applicant uses or may use as trademarks. This section explains the nature and function of some of these in greater detail.
a. All Bets
For “All Bets” queries, the server simply returns all the items in the semantic metadata store. If the SQML has filters, the filters are imposed via an inner sub-query with no semantic link strength threshold. For instance, All Bets on Topic A will return all items that have anything (strongly or barely) to do with Topic A.
b. Random Bets
In the preferred embodiment, for “Random Bets” queries, the server simply returns all the items in the semantic metadata store (like in the case of “All Bets” queries) but orders the results randomly. If the SQML has filters, the filters are imposed via an inner sub-query with no semantic link strength threshold. For instance, Random Bets on Topic A will return all items (ordered randomly) that have anything (strongly or barely) to do with Topic A.
c. Breaking News
If the server has user-state, Breaking News can be implemented in a very intelligent way. The table below illustrates the currently preferred ranking and prioritization for Breaking News when the server tracks what items (and/or categories) the user has read:
In the preferred embodiment, the server processes SQML for Breaking News (via the Breaking News context predicate) as follows:
1. All breaking news is filtered with a sub-query that the returned news must be “younger” than N hours (or days, or months, configurable)—this imposes the key time-sensitivity constraint.
2. Breaking News is always semantic.
3. In the preferred embodiment, the Semantic Network Manager (SNM) should update the semantic network to indicate the “last read time” for each user to each category. This is then used in the sub-query to check whether news has been “read” or not (per category or per object—per category is the preferred embodiment because the latter will not scale).
4. Priority is given to news items that the user has not “read” (this is implemented by comparing the last read time in the SemanticLinks table with the semantic link type that links “User” to “Category”).
5. The implication of the semantic prioritization scheme is that the user could get “older” breaking news first because the news is more semantically relevant and “younger” breaking news “later” because the news is less semantically relevant. This results in a hybrid relevance-time sensitivity prioritization scheme.
6. The primary ordering axis (Creation Time) guarantees that results are filtered by freshness. The secondary ordering axis (Relevance Score) acts as a tiebreaker and guarantees that equally fresh results are distinguished primary based on relevance.
7. Breaking News Intrinsic Alerts can be implemented on the client by limiting the Breaking News priority to Priority 2 and by changing the Priority 1 and Priority time-sensitivity filters to high. This way, only very fresh Breaking Unread Semantic News (of both High and Low semantic relevance filters) will be returned. This is advantageous because the alert should have a higher disruption threshold than the Breaking News Request (or agent)—since it is implicit rather than explicit.
8. Unread Breaking News is higher priority than Read Breaking News because users are likely to be more interested in stuff they haven't seen yet.
9. Unread Breaking News has a lower time-sensitivity filter than Read Breaking News because users are likely to be more tolerant of older news that is new to them than younger news that is not.
In some cases, the server might not have user-state (and “read” information). In this case, a simple implementation of Breaking News is shown below:
1. By default (no filter), Breaking News should return only items younger than N hours (default is 3 hours).
2. If there is at least one filter in the SQML, Breaking News should apply the time-sensitivity filter (3 hours) to the outer sub-query and also apply a moderately strong relevance filter to the inner sub-query (off the SemanticLinks table). In the preferred embodiment, this should correspond to a relevance score (and link strength) of 50%. For instance, Breaking News on Topic A should return those items that have been posted in the last 3 hours and which belong to the category (or categories) represented by Topic A with at least a relevance score of 50%. This will avoid false positives like Breaking News items which are barely relevant to Topic A.
Ditto with Breaking News (except that time-sensitivity constraints are more relaxed—e.g., the High filter is 12 hours instead of 3 hours and the low filter is 1 day instead of 12 hours). In the simple implementation, the time-sensitivity constraint is 1 day. This can also be made 3-days on Mondays to dynamically handle weekends (making the number of days the “number of working days”).
Newsmakers are handled the same way as Headlines, except that the SQP returns the authors of the Headline items rather than the items themselves.
f. Best Bets
As described in my parent application (Ser. No. 10/179,651), Best Bets are implemented by imposing a filter on the strength of the semantic link with the “Belongs to Category” predicate. The preferred default is 90%, although the client (at the option of the user) can change this on the fly via an argument passed via the XML Web Service. Best Bets are implemented with a SQL inner join between the Objects table and the SemanticLinks table and joining only those rows in the SemanticLinks table that have the “Belongs to Category” predicate and a LinkStrength greater than 90% (default). When the SQML that is being processed contains filters (e.g., keywords, text, entities, etc.), the server-side semantic query processor must also invoke a sub-query, which is a SQL inner join that maps to the desired filters. In the preferred embodiment, this sub-query should also include a “Best Bets” filter.
In the preferred embodiment, it is advantageous and probably preferable for most users for the outer sub-query to be a Best Bet, and for the inner sub-query. To illustrate this, “Best Bets on Topic A” is semantically different from “Best Bets that are also relevant to Topic A.” In the first example, only Best Bets, which are Best Bets “ON” Topic A, will be returned (via applying the “Best Bets” semantic filter on the inner sub-query). In contrast, the second example will return Best Bets on anything that might have anything to do with Topic A. As such, the second example might return false positives because for example, a document, which is a Best Bet on Topic B but a “weak bet” on Topic B, will be returned and that is not consistent with the semantics of the query or the presumably desired results. Extending the “Best Bets” filter to not only the outer sub-query but also all inner sub-queries will prevent this from happening. Other query implementations can also follow this rule (with the right sub-queries applied based on the semantics of the main query) if the SQML contains filters.
g. Query Implementation for Other Knowledge Types
Other knowledge types are implemented in a similar fashion as above (via the right predicates). Several examples are described below.
Information Type Semantic Query Implementations
All information type semantic query implementations can follow, and preferably (but not necessarily) follow, the same pattern: the SQP returns only those objects that have the object type id that corresponds to the requested information type. An example is “Information Type\Presentations.” When the SQP parses the SQML received from the client, it extracts this attribute from the SQML and maps it to an object type id. It then invokes a SQL query with an added filter for the object type id. For special information types that could span several individual information types (such as “Information Type\All Documents”), the SQP maps the request to a set of object type ids and invokes a SQL query with this added filter.
Context Semantic Query Implementations
When the client sends SQML that contains concepts (extracted on the client from text or documents), the server-side SQP has to first semantically interpret the context before generating sub-queries that correspond to it. To do this, the server sends the concepts to all KDS'es (KBS'es) it is configured with (for the desired knowledge community or agency) for semantic categorization. When the server gets the categories back, it preferably determines which of those categories are “strong” enough to be used as filters before generating the appropriate sub-queries.
This “filter-strength” determination is advantageous because if the context is, for example, a fairly long document, that document could contain thousands of concepts and categories. As a result, the “representative semantics” of the document might be contained in only a subset of all the concepts/categories in the document. Mapping all the categories to sub-queries will return results that might be confusing to the user—the user would likely have a “sense” of what the document contains and if he/she sees results that are relevant to some weak concepts in the document, the user might not be able to reconcile the results with the document context. Therefore, in the preferred embodiment, the server-side SQP preferably chooses only “strong categories” to apply to the sub-queries. It is recommended that these be categories with a semantic strength of at least 50%. That way, only those categories that register strongly in the semantic context would be applied to the sub-query. The implementation of the sub-query would then follow the rules described above depending on whether the query contains a context predicate, is based on a knowledge type, information type, etc.
Semantic Stemming Implementation
As described in my parent application, the server-side semantic query processor performs semantic stemming to map keywords, text, and concepts to categories based on one or more domain ontologies. One way it does this by invoking an XML Web Service call to the KDS/KBS (or KDSes/KBSes) it is configured with in order to obtain the categories. It then maps the categories to its semantic network. This form of stemming is superior to regular stemming that is based on keyword variations (such as singular and plural variations, tense variations, etc.) because it also involves domain-specific semantic mapping that stems based on meaning rather than merely stemming based on keyword forms.
In the currently preferred embodiment, the KIS calls the KDS/KBS each time it receives SQML that requires further semantic interpretation. However, this could result in delays if the KDS/KBS resides on a different server, if the network connection is not fast, or if the KDS/KBS is busy processing many requests. In this case, the KIS can also implement a Semantic Stemming Cache. This cache maps keywords and concepts to categories that are fully qualified with URIs (making them globally unique). When the server-side semantic query processor receives SQML that contains keywords, text, or concepts (extracted from, say, documents on the client by the client-side semantic query processor), it first checks the cache to see if the keywords have already been semantically stemmed. If there is a cache hit, the SQP simply retrieves the categories from the cache and maps those categories to the semantic network via SQL queries. If there is a cache miss (i.e., if the context is not in the cache), it then calls the KDSes/KBSes to perform semantic categorization. It then takes the results, maps them to unique category URIs, and adds the entry to the cache (with the context as the hash code). Note that even if the context does not map to any category, the “lack of a category” is preferably cached. In other words, the context is added as a cache entry with no categories. This way, the server can also quickly determine that a given context does not have any categories, without having to call the KDSes/KBSes each time to find out.
The SQP can also manage the semantic stemming cache. It has to do this for two reasons: first, to keep the cache from growing uncontrollably and consuming too much system resources (particularly memory with a heap-based hash table); and, second, if the KIS configuration is changed (e.g., if knowledge domains are added/removed), the cache is preferably purged because the entries might now be stale. The first scenario can be handled by assigning a maximum number of entries to the cache. In the preferred embodiment, the SQP caches the current amount of memory consumed by the cache and the cache limit is dictated by memory usage. For example, the administrator might set the maximum cache size to 64 MB. To simplify the implementation, this can be mapped to an approximate count of items (e.g., by dividing the maximum memory usage by an estimate of the size of each cache entry).
For each new entry, if the cache limit has not been reached, the SQP simply adds the entry to the cache. However, if the cache limit has been reached, the SQP (in the preferred embodiment) should purge the least recently added items from the cache. In the preferred embodiment, this can be implemented by keeping a queue of items that is kept in sync with a hash table that implements the cache itself (for quick lookups using the context as a key). When the SQP needs to purge items from the cache to free up space, it de-queues an item from the least-recently-added queue and also removes the corresponding item from the hash table (using the context as key). This way, fresh items are more likely to result in a cache hit than older items. This will result in a faster user experience on the client because context for saved agents/requests/queries will end up being cached with quick-lookups each time the user opens the agent/request/query. The same goes for Dossier (Guide) queries which will have the same context (but with different knowledge types)—the client will request for each knowledge type for the same context and since the context will be cached, each sub-query will execute faster.
Extensible client-side user profiles allow the user of a semantic browser to have a different state for different job roles, knowledge sources, identities, personas, work styles, etc. This essentially allows the user to create different “knowledge worlds” for different scenarios. For instance, a Pharmaceuticals researcher might have a default profile that includes all sources of knowledge that are relevant to his/her work. As described in my parent application Ser. No. 10/179,651, the SRML from each of these sources will be merged on the client thereby allowing the user to seamlessly go through results as though they were coming from one source. However, the researcher might want to track patents separate from everything else. In such a case, the researcher would be able to create a separate “Patents” profile and also include those knowledge communities (agencies) that have to do with patents (e.g.,. the US Patent Office Database, the EU Patent Database, etc.)
To take another example, for instance, the user might create a profile for ‘Work’ and one for ‘Home.’ Many investment analysts track companies across a variety of industries. With the semantic browser, they would create profiles for each industry they track. Consultants move from project to project (and from industry to industry) and might want to save requests and entities created with each project. Profiles will be used to handle this scenario as well.
Profiles contain the following user state:
Name/Description—the descriptive name of the profile.
One or more knowledge communities (agencies) that indicate the source of knowledge (running on a KIS) at which requests (agents) will be invoked.
Identity Information—the user name (currently tagged with the user's email address) and password.
Areas of Interest or Favorite Categories—this is used to suggest information communities (agencies) to the user (by comparing against information communities with identical or similar categories) and as a default query filter for requests created with the profile.
Smart styles—the smart styles to be used by default for requests and entities created with the profile.
Default Flag—this indicates whether the profile is the default profile. The default profile is initiated by default when the user wishes to create requests and entities, browse information communities, etc. Unless the user explicitly selects a different profile, the default profile gets used.
Profiles can be created, deleted, modified, and renamed. However, in the preferred embodiment the default profile cannot be deleted because there has to be at least one profile in the system at all times. In alternate embodiments, a minimum profile would not be required.
Preferably, all objects in the semantic browser are opened within the context of a profile. For instance, a smart request is created in a profile and at runtime, the client semantic query processor will use the properties of the profile (specifically the subscribed knowledge communities (agencies) in that profile) to invoke the request. This allows a user to correlate or scope a request to a specific profile based on the knowledge characteristics of the request (more typically the sources of knowledge the user wants to use for the request).
In a preferred embodiment, the user is able to navigate his/her knowledge worlds via both profiles without interference.
1. Smart Styles Overview
A color theme and animation theme applied to a style theme yields a “smart style”. “Smart” in this context means the style is adaptive or responsive to the mood of its request, context panes, preview mode, handheld mode, live mode, slideshow mode, screensaver mode, blender/collection mode, accessibility, user settings recognition, and possibly other variables within the system (see below). There is an infinite number and kind or “Classes” of possible styles. The preferred embodiment comprises at least the following style Classes:
1. Subtle—for task-oriented productivity.
2. Moderate—for task-oriented productivity with some presentation effects.
3. Exciting—exciting effects (good for both primary and secondary machines, and for inactive Nervana Windows™—e.g., Nervana client Windows™ in the background or docked on the taskbar).
4. Super-exciting (great for smart screensavers with productivity—e.g., secondary machines—when the user is using his/her primary machine).
5 Sci-Fi (for Matrix fans, great for smart screensavers without specific need for productivity—e.g., when the user is away from his/her desk).
Style, Color & Animation Themes—Variable, unlimited—created by Nervana, and perhaps users and/or third party skin authors
2. Implicit and Dynamic Smart Style Properties
In a preferred embodiment, a Dialog Box can allow the user to browse smart styles by pivoting across style classes, style themes, color themes, and animation themes. A preview window shows the user a preview of the currently selected smart style.
Smart Request Watch refers to a feature of the Information Nervous System that allows users of the semantic browser (the Information Agent or the Librarian) to monitor (or “watch”) smart requests in parallel. This is a very advantageous feature in that it enhances productivity by allowing users to track several requests at the same time.
The feature is implemented in the client-side semantic runtime, the semantic browser, and skins that allow a configurable way of watching smart requests (via a mechanism similar to “Picture-In-Picture” (PIP) functionality in television sets). Preferably, one or more of the following software components are used:
1. The Request Watch List (RWL)
2. Request Watch Groups
3. The Notification Manager (NM)
4. Watch Group Monitors (WLM)
5. The Watch Pane
6. The Watch Window
2. Request Watch Lists (RWLs) and Groups (RWGs)
The Request Watch List is a list of smart requests (or smart agents) that the client runtime manages. This list essentially comprises the smart requests the user wishes to monitor. The Request Watch List comprises a list of entries, the Request Watch List Entry (RWLE) with the following data structure:
The Request Watch List (RWL) contains an array or vector of RWLE structures. The Request Watch List Manager manages the RWL. The semantic browser provides a user interface that allows the user to add smart requests to the RWL—the UI talks to the RWLM to add and remove RWLEs to/from the RWL. The RWL is stored (and persisted) centrally by the client-side semantic runtime (either as an XML file-based representation or in a store like the Windows™ registry).
The RWL can also be populated by means of Request Watch Groups (RWGs). A Request Watch Group provides a means for the user to monitor a collection of smart requests. It also provides a simple way for users to have the semantic browser automatically populate the RWL based on configurable criteria. There are at least two types of RWGs: Auto Request Watch Groups and the Manual Request Watch Group. Auto Request Watch Groups are groups that are dynamically populated by the semantic browser depending on the selected profile, the profile of the currently displayed request, etc. The Manual Request Watch Group allows the user to manually populate a group of smart requests (regular smart requests or blenders) to monitor as a collection. The Manual Request Watch Group also allows the user to add support context types (e.g., documents, categories, text, keywords, entities, etc.)—in this case, the system will dynamically generate the semantic query (SQML) from the filter(s) and add the resulting query to the Manual Request Watch Group. This saves the user from having to first create a time-sensitive request based on one or more filters before adding the filters to the Watch Group—the user can simply focus on the filters and the system will do the rest.
Users will be able to add the following types of Auto-RWGs:
1. Breaking News—this tells the semantic browser to automatically add a Breaking News smart request to the RWL (for the selected profile(s)).
2. Headlines—this tells the semantic browser to automatically add a Headlines smart request to the RWL (for the selected profile(s)).
3. Newsmakers—this tells the semantic browser to automatically add a Newsmakers smart request to the RWL (for the selected profile(s)).
4. Categorized Breaking News—this tells the semantic browser to automatically add Categorized Breaking News smart requests to the RWL (for the contextual profile). The semantic browser will dynamically add smart requests with category filters corresponding to each subcategory of the currently displayed smart request (and for the contextual or current profile)—if the currently displayed smart request has categories. For example, if the smart request “Breaking News” about Technology” is currently being displayed in a semantic browser instance, and if the category “Technology” has 5 sub-categories (e.g., Wireless, Semiconductors, Nanotechnology, Software, and Electronics), the following smart requests will be dynamically added to the RWL when the current smart request is loaded:
Also, the RWLEs for these entries will be initialized with the RequestViewInstanceID of the current semantic browser instance. If the user navigates to a new smart request, the categorized Breaking News for the previously loaded smart request will be removed from the RWL and a new list of categorized Breaking News will be added for the new smart request (if it has any categories)—and initialized with a new RequestViewInstanceID corresponding to the new smart request view. This creates a smart user experience wherein relevant categorized breaking news (for subcategories) will be dynamically displayed based on the currently displayed request. The user will then be able to monitor Categorized Breaking News smart requests as a watch group or collection.
5. Categorized Headlines—this tells the semantic browser to automatically add Categorized Headlines smart requests to the RWL (for the contextual profile). This is similar to Categorized Breaking News, except that Headlines are used in this case. The user will then be able to monitor Categorized Headlines smart requests as a watch group or collection.
6. Categorized Newsmakers—this tells the semantic browser to automatically add Categorized Newsmakers smart requests to the RWL (for the contextual profile). This is similar to Categorized Breaking News, except that Newsmakers are used in this case. The user will then be able to monitor Categorized Newsmakers smart requests as a watch group or collection.
7. My Favorite Requests—this tells the semantic browser to automatically add all favorite smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite smart requests as a group.
8. My Favorite Breaking News—this tells the semantic browser to automatically add all favorite breaking news smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite breaking news smart requests as a group.
9. My Favorite Headlines—this tells the semantic browser to automatically add all favorite headlines smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite headlines smart requests as a group.
10. My Favorite Newsmakers—this tells the semantic browser to automatically add all favorite newsmakers smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite newsmakers smart requests as a group.
Request Watch Group Manager User Interface
3. The Notification Manager (NM)
In the preferred embodiment the Notification Manager (NM) is a component of the semantic runtime client that monitors smart requests in the RWL. The NM has a thread that periodically invokes each smart request in the RWL (via the client semantic query processor) and updates the RWLE with the “results count” and the “last update time.” In the preferred embodiment the NM preferably invokes the smart requests every 5-30 seconds. The NM can intelligently adjust the periodicity or frequency of request checks depending on the size of the RWL (in order to minimize bandwidth usage and the scalability impact on the Web service).
For time-sensitive smart requests (like Breaking News, Headlines, and Newsmakers), the NM preferably invokes the smart request without any additional time filter. However, for non time-sensitive requests (like for information as opposed to context types or for non time-sensitive context templates like Favorites and Recommendations), the NM preferably invokes the query for the smart request with a time filter (e.g., the last 10 minutes).
4. Watch Group Monitors
In the preferred embodiment, the semantic runtime client manages what the inventor calls Watch Group Monitors (WGM). For each watch group the user has added to the watch group list, the client creates a watch group monitor. A watch group monitor tracks the number of new results in each request in its watch group. The watch group monitor creates a queue for the RWLEs in the watch group that have new results. The WGM manages the queue in order to maximize the freshness of the results. The WGM periodically polls the NM to see whether there are new results for each request in its watch group. If there are, it adds the request to the queue depending on the ‘last result time’ of the request. It does this in order to prioritize requests with the freshest results first. The currently displayed visual style (skin) running in the Presenter would then call the semantic runtime OCX to dequeue the requests in the WGM queue. This way, the request watch user interface will be consistent with the existence of new results and the freshness of the results. Once there are no more new results in the currently displayed request, the smart style will dequeue the next request from the WGM queue.
5. The Watch Pane
The Watch Pane (WP) refers to a panel that gets displayed in the Presenter (alongside the main results pane) and which holds visual representations of the user's watch groups. The WP allows the user to glance at each watch group to see whether there are new results in its requests. The WP also allows the user to change the current view with which each watch group's real-time status gets displayed. The following views are currently defined:
Tiled View—this displays the title of the watch group along with the total number of new results in all its smart requests.
Ticker View—this displays the total number of new results in all the watch group's smart requests but also shows an animation that sequentially displays the number of new results in each smart request (as a ticker).
Preview View—this is similar to the ticker view except that the most recent result per smart request is also displayed alongside the number of new results in the ticker.
Deep View—in this view, the WP displays the total number of new results in all the watch group's smart requests along with a ticker that shows the number of new results in each smart request and a slide-show of all the new results per smart request.
6. The Watch Window
The WP also allows the user to watch a watch group. The user will do this by selecting one of the watch groups in the WP and dragging it into the main results pane (or by a similar technique). This forms a Watch Window (WW). This WW resembles or can be analogized to TV's picture-in-picture functionality in appearance or layout, but differs in several ways, most noticeably in that in this case the displayed content is comprised of semantic requests and results as opposed to television channels are being “watched.” Of course, the underlying technology generating the content is also quite different. The WW can be displayed in any of the aforementioned views. When the WW is in Deep View however, the WW's view controls are displayed. The following controls are currently defined:
Pinning Requests—this allows the user to pin a particular request in the watch group. The WW will keep displaying the new results for only the pinned requests (in a cycle) and will not advance to other requests in the watch group for as long as the current request remains pinned.
Swapping Requests—this allows the user to swap the currently displayed request with the main request being shown in the semantic browser. The smart style will invoke a method on the OCX to create a temporary request with the swapped request (hashed by its SQML buffer) and then navigate to that request while also informing the Presenter to now display the main request in its place (in the WW).
Stop, Play, Seek, FF, RW, Speedup—these allow the user to stop, play, seek, fast-forward, rewind or speedup the “watch group request stream.” For instance, a fast-forward will advance to several requests ahead of the currently displayed one.
Results controls—this allows the user to control the results in each request in the watch group. Essentially, the results are a stream within a stream and this will also allow the user to control the results in the current request in the current watch group.
Auto-Display Mode—this will automatically hide the WW when there are no results to display and fade it in when there are new results. This way, the user can maximize the utility of his/her real estate on the screen knowing that watch windows will fade in when there are new semantic results. This feature also allows the user to manage his/her attention during information interaction in a personal and semantic way.
Docking, Closing, Minimizing, Maximizing—these features, as the names imply, allow the user to dock, close, minimize or maximize watch windows.
7. Watch List Addendum
In the User Interface, the Watch List can be named “News Watch.” The user will be asked to add/remove requests, objects, keywords, text, entities, etc. to/from the “News Watch.” The “News Watch” can be viewed with a Newsstand watch pane. This will provide a spatially-oriented view of the user's requests and dynamically-created requests (via objects added to the Watch List, and created dynamically by the runtime using those objects as filters)—not unlike the view of a news-magazine rack when one walks into a Library or Bookstore.
Entities are a very powerful feature of the preferred embodiment of the Information Nervous System. Entities allow the user to create a contextual definition that maps to how they work on a regular basis. Examples of entities include:
There are also industry-specific entities. For instance, in pharmaceuticals, entities could include drugs, drug interaction issues, patents, FDA clinical trials, etc. Essentially, an entity is a semantic envelope that is a smart contextual object. An entity can be dragged and dropped like any other smart object. However, an entity is represented by SQML and not SRML (i.e., it is a query-object because it has much richer semantics). An entity can be included as a parameter to a smart request.
The user creates entities based on his/her tasks. Entities in the preferred embodiment contain at least the following information (in alternate embodiments they could contain more or less information):
1. Name/Description—a friendly descriptive name for the entity.
2. The categories of the entity—based on standard cross-industry taxonomies or vertical/company-specific taxonomies.
3. Contextual resources—these could include keywords, local documents, Internet documents, or smart objects (such as people).
An entity can be opened in the semantic browser, can be used as a pivot for navigation, as a parameter for a smart request (e.g., Headlines on My Project), can be dragged and dropped, can be copied and pasted, can be used with the smart lens, can be visualized with a smart style, can be used as the basis for an intrinsic alert, can be saved as a .ENT document, can be emailed, shared, etc. In other words, an entity is a first-class smart object.
The semantic runtime client dynamically creates SQML by appending the rich metadata of the entity to the subject of the relational request to create a new rich SQML that refers to the entity.
Entities preferably also have other powerful characteristics:
1. Regarding topics, entities allow the user to create his/her private taxonomy (without being at the mercy of or restricted exclusively to a public taxonomy that is strictly defined and as such, might not map exactly to the user's specific context for a request). The problem with taxonomies is that no taxonomy can ever fit everybody's needs—even in the same organization. Context is very personal and entities allow the user to create a personal taxonomy. For instance, take the example of a dog (of the boxer breed) named Kashmir owned by a dog-owner Steve. To everyone else (but Steve), Kashmir can be expressed (taxonomically) as:
But to Steve, Kashmir is also:
To Steve's veterinary doctor, however, Kashmir is:
If taxonomies (standalone) were used to “define” Kashmir, none of the three taxonomies would satisfy the general public, Steve, and Steve's veterinary doctor. With entities on the other hand, Steve could create a “Kashmir” entity based on “what Kashmir means to him.” Everyone else could then do the same. And so can Steve's veterinary doctor. Entities therefore empower the user with the ability to create private topics that might be extensions of broad taxonomies.
To take another example, a Pharmaceuticals researcher in a large Pharmaceutical company might be working on a new top-secret project (named “Gene Project”) on Genomics. Because “Gene Project” is an internal project, it would likely not exist in a public taxonomy which could be used with the semantic browser of this the preferred embodiment of my invention. However, the researcher could create an entity named “Gene Project”, typed as a Project, and could then initialize the entity by scoping it to Genomics (which exists in broad taxonomies) and then also qualifying it with the keyword-phrase “Gene Project” (using the AND operator). Essentially, this is akin to defining “Gene Project” as anything on Genomics that has the phrase “Gene Project.” This will impose much stricter context than merely using the keywords “Gene Project” (which might return results that contain the word “Project” but have nothing to do with Genomics). By defining a personal topic, “Gene Project” that is scoped to Genomics but also extends “Gene Project” with a specific qualifier, the researcher now has much more precise and personal context. The entity can then be dragged and dropped, copied and pasted, etc. to create requests (e.g., “Experts on Gene Project.” At runtime, the server-side semantic query processor will interpret this (by mapping the SQML to the semantic network) as “Experts on any information that belongs to the category Genomics AND which also includes the phrase “Gene Project.”
2. Entities also allow the user to create a dynamic taxonomy—public taxonomies are very static and are not updated regularly. With entities, the user can “extend” his/her private taxonomy dynamically and at the speed of thought. Knowledge is transferred at the speed of thought. Entities allow the user to create context with the same speed and dynamism as his/her mind or thought flow. This is very significant. For instance, the user can create an entity for a newly scheduled meeting, a just-discovered conference, a new customer, a newly discovered competitor, etc.—ALL AT THE SPEED OF THOUGHT. Taxonomies don't allow this.
3. Taxonomies assume that topics are the only source of context. With entities, a user can create abstract contextual definitions that include—but are not limited to—topics. Examples include people, teams, events, companies, etc. Entities might eventually “evolve” into topics in a taxonomy (over time and as those entities gain “fame” or “notoriety”) but in the “short-term,” entities allow the user to create context that has not yet evolved (or might never evolve) into a full-blown taxonomic entry. For instance, Nervana (our company) was initially an entity (known only to itself and its few employees) but as we have grown and attracted public attention, as an entity we are evolving into a topic in a public taxonomy. With entities, users don't have to wait for context (like Nervana) to “eventually become” topics.
4. Entities allow the user to create what the inventor calls “compound context.” An example of this is a meeting. A meeting typically involves several participants with documents, presentation slides, and/or handouts relevant to the topic of discussion. With entities in the Information Nervous System, a user can create a “meeting” context that captures the semantics of the meeting. Using the Create Entity Wizard, the user can specify that the entity is a meeting, and then specify the semantic filters. Consider an example of a project meeting with five participants and 2 handed out documents, and one presentation slide. The Presenter of the meeting might want to create an entity in order to track knowledge specifically relevant to the meeting. For instance, he/she might want to do this to determine when to schedule a follow-up meeting or to track specific action items relating to the meeting. To create the entity, the user would add the email addresses of the participants, the handed out documents, and also the presentation to the entity filter definition. The user then saves the entity which is then created in the semantic namespace/environment. The user can then edit the entity with new or removed filters (and/or a new name/description) at a later date/time—for instance, if he/she has discovered new documents that would have been relevant to the meeting. When the user drags and drops the entity or includes it in a request/agent, the semantic browser then compiles the entity and includes it in a master SQML with the sub-queries also passed to the XML Web Service for interpretation. The server-side semantic query processor then processes the compound SQML by constructing a series of SQL sub-queries (or an equivalent) and by joining these queries with the entity sub-queries which in turn are generated using SQL sub-queries.
The user can use an AND or OR (or other) operator to indicate how the entity filters should be applied. For instance, the user can indicate that the meeting (semantically) is the participants of the meeting AND the documents/slides handed out during the meeting. When the entity is compiled at the client and the server, the SQML equivalent is used to interpret the entity (with the desired operator). This is very powerful. It means that the user can define an entity named “Project Meeting” and drag and drop that entity to the special agent named “Breaking News.” This then creates a request named “Breaking News on Project Meeting” (with the appropriate SQML referring to the identifier of the entity—which will then be compiled into sub-SQML before it is passed to the server(s) for interpretation. The server then applies default predicates to the entries in the entity (based on what “makes sense” for the object). In this particular example, because of the definition of the entity, the server will then only return:
Breaking News BY ALL the participants AND which is ALSO semantically relevant TO ALL the documents/slides
For instance, this will only return conversations/threads that involve all the participants of the meeting and which are semantically relevant to all the handouts given out during the meeting. This is precisely what the user desired (in this case) and the semantic browser would have empowered the user to essentially construct a rather complex query.
Even more complex queries are possible. Entities can include other entities to allow for compound entities. For instance, if an entire team of people were involved in the meeting, the Presenter might want to create an entity that includes an email distribution list of those people. In this case, the user might search the Information Nervous System for the distribution list and then save the result as an entity. The browser will allow the user to save results as entities and based on the result type, it will automatically create an entity with a default entity type that “makes sense.” For instance, if the user saves a document result as an entity, the semantic browser it will create a “Topic” entity. If the user saves a Person result as an entity, the semantic browser will create a “Person” entity. If the user saves an email distribution list as an entity, the semantic browser will create a “Team” entity.
In this example, the user can save a Person result as a Person entity and then drag and drop that entity into the Project Meeting entity. The Team entity that maps to the email distribution list of the meeting participants can be dragged and dropped to the Project Meeting entity. The user can then create a request called “Headlines on Project Meeting” that includes the entity. The semantic query processor will then return Headlines BY anyone in the email distribution list (using the right default predicate) and which is semantically relevant to ALL the handouts given out during the meeting. Similarly, a Dossier (Guide) on the Project Meeting will return All Bets on the meeting, Best Bets on meeting, Experts on the meeting, etc.
Note that such a compound entity that includes other entities gets checked by the client-side semantic consistency checker for referential integrity. In other words, if Entity A refers to Entity B and the user attempts to delete Entity B, the semantic browser will detect this and flag the user that Entity B has an outstanding reference. If the user deletes Entity B anyway, the reference in Entity A (and any other references to Entity B) will get removed. Alternately, in some embodiments, the user could be prohibited (whether informed or not) from deleting Entity B in the same situation, based on permissions of others within an organization associated with the entity. For example, employers could monitor activities of employees for risk management purposes, like as is done with email in some companies, only much potentially much more powerfully (Of course, appropriate policies and privacy considerations would have to be addressed). The same process applies to Request Collections (Blenders), Portfolios (Entity Collections—see below), and other compound items in the semantic namespace/environment (items that could refer to other items in the namespace/environment).
5. Popular entities can also be shared amongst members of a knowledge community. Like other items in the semantic browser (like requests or knowledge communities (agencies), entities can be saved as files (so the user can later open them or email them to colleagues, or save them on a central file share, etc.). A common scenario would be that the corporate Librarians at businesses would create entities that map to internal projects, meetings, seminars, tasks, and other important corporate knowledge items of interest. These entities would then be saved on a file-share or other sharing mechanism (like a portal or web-site) or on a knowledge community (agency). The knowledge workers in the organization would then be able to use the entities. As the entities get updated, in the preferred embodiment the Librarians can and will automatically edit their context and users will be able refresh or synchronize to the new entities. Entities could also and alternately be shared on a peer-to-peer basis by individual users. This is akin to a legal peer-to-peer file sharing for music, but instead of music, what is shared is context to facilitate meaning, or more meaningful communication.
2. Portfolios (or Entity Collections)
Portfolios are a special type of entity that contains a collection of entities. In the preferred embodiment, to minimize complexity and confusion (at least of nomenclature or terminology), while an entity can be of any size or composition, and portfolio can contain any kind or number of entities, a portfolio would not contain other portfolios. A portfolio allows the user to manage a group of entities as one unit. A portfolio is a first-class entity and as such has all the aforementioned features of an entity. When a portfolio is used as a parameter in a smart request, the OR qualifier is applied (by default) to its containing entities. In other words, if Portfolio P contains entities E1 and E2, a smart request titled ‘Headlines on P’ will be processed as ‘Headlines on E1 or E2.’ The user can change this setting on individual smart requests (to an AND qualifier).
3. Sample Scenarios
Again, in reviewing the scenarios below, it is helpful to recall that, conceptually, the system can gather more relevant information in part because it “knows” who is asking for it, and “understands” who that person or group is, and the kinds of information they are probably interested in. Of course, strictly speaking, the system is not cognitive or self aware in the full human sense, and the operative verbs in the preceding sentence are conceptual metaphors or similes. Still, in operation and results, it mimics understanding and knowledge to an unprecedented degree in part because of its underlying semantically-informed architecture and execution.
This point can be illustrated by a simplistic contrast: If two very different people entered the exact same search at the exact same time into a search engine such as Google™, they would get the exact same results. In contrast, with the preferred embodiment of the present system, if those same two people entered the same request via an Entity, each would get different results tailored to be relevant to each.
To appreciate some of the potential power of this feature, it is useful to note that while the system or Entities “know” who is posing the query, the Entities do not depend for that knowledge on the user informing them and keeping them constantly updated and informed (although user information can be supplied and considered at any time). If that were the case, the system could be too labor intensive to be efficient and useful in many situations; it would just be too much work. Instead, the Entities “know” who the requester is by inference and from semantics from characteristics sometimes supplied by others, sometimes derived or deduced, sometimes collected from other requests and the like, as explained throughout this application and its parent application.
Some example scenarios of Entities in operation:
1. A pharmaceuticals ‘patent’ entity could include the categories of the patent, relevant keywords, and relevant documents.
2. A CIA agent could create a ‘terrorist’ entity to track terrorists. This could include categories on terrorism, suspicious wire transfers, suspicious arms sales, classified documents, keywords, and terrorism experts in the information community.
3. Find All Breaking News on Yesterday's Meeting.
4. Find Headlines on any of my competitors (this is done by creating the competitor entities, and then creating a smart request with the entities as parameters using the OR qualifier with each predicate).
5. Find Experts on my investment portfolio companies (create the individual entities, create a portfolio containing these entities and then create a smart request that has the ‘Experts’ context template and that uses the portfolio as an argument).
6. Open a Dossier (Guide) on my competitors (create the individual competitor entities, create a portfolio containing these entities and then create a smart request that has the ‘Dossier’ (or ‘Guide’) context template and that uses the portfolio as an argument).
The Nervana semantic browser will allow the user to subscribe and unsubscribe to/from knowledge communities (agencies) for a given profile. These knowledge communities will be readily available to the user underneath the profile entry in the semantic environment. In addition, these knowledge communities will be queried by default for intrinsic alerts, context panels, and etc. whenever results are displayed for any request created using the same profile.
The semantic environment includes state indicating the subscribed knowledge communities for each profile. The client-side semantic query processor (SQP) uses this information for dynamic requests that start from results for requests of a given profile (the SQP will ask the semantic runtime client for the knowledge communities for the profile and then issue XML Web Service calls to those knowledge communities as appropriate).
In a preferred embodiment, a user interface for the knowledge community subscription and un-subscription can be used wherever the dialog box has combo boxes allowing the user to filter by profile, to view all, new, subscribed, suggested, and un-subscribed communities, by industry and area of interest, by keywords, by publishing point (all publishing points, the local area network, the enterprise directory, and the global knowledge community directory), and by creation time (anytime, today, yesterday, this week, and last week). The semantic runtime client queries the publishing point endpoint listeners (for each publishing point) using the filters. It then gathers the results and displays them in the results pane. The user is also able to view the categories of each knowledge community in the results pane via a combo box.
1. Semantic Query Markup Language (SQML) Overview
In the currently preferred embodiment, the Nervana Semantic DHTML Behavior is an Internet Explorer DHTML Behavior that, from the client's perspective, everything it understands as a query document. The client opens ‘query documents,’ in a manner resembling how a word processor opens ‘textual and compound documents.’ The Nervana client is primarily responsible for processing a Nervana semantic query document and rendering the results. A Nervana semantic query document is expressed and stored in form of the Nervana Semantic Query Markup Language (SQML). This is akin to a “semantic file format.”
In the preferred embodiment, the SQML semantic file format comprises of the following:
2. SQML Generation
Preferably, SQML is generated in any one or more of several possible ways:
By creating a smart request
By creating a local request
By creating an entity
By opening one or more local documents in the semantic browser
By the client (dynamically)—in response to a drag and drop, smart copy and paste, intrinsic alert, context panel/link invocation, etc.
3. SQML Parsing
In some embodiments in some situations, SQML that gets created on the client might not be ready (in real-time) for remote consumption—by the server's XML web service or at another machine site. This is especially likely to be the case when the SQML refers to local context such as documents, Entities, or Smart Requests (that are identified by unique identifiers in the semantic environment).1 In the preferred embodiment, the client generally creates SQML that is ready for remote consumption. Preferably, it does this by caching the metadata for all references in the metadata section of the document. This is preferable because in some cases, the resource or object to which the reference points might no longer exist when the query is invoked. For instance, a user might drag and drop a document from the Internet to a smart request in order to generate a new relational request. The client extracts the metadata (including the summary) from the link and inserts the metadata into the SQML. Because the resolution of the query uses only the metadata, the query is ready for consumption once the metadata is inserted into the SQML document. However, the link that the object refers to might not exist the day after the user found it. In such a case, even if the user invokes the relational request after the link might have ceased to exist, the request will still work because the metadata would already have been cached in the SQML. 1Blenders (or collections) contain references to smart requests.
The client SQML parser performs “lazy” updating of metadata in the SQML. When the request is invoked, it attempts to update the metadata of all parameters (resources, etc.) in the SQML to handle the case where the objects might have changed since they were used to create the relational request. If the object does not exist, the client uses the metadata it already has. Otherwise, it updates it and uses the updated metadata. That way, even if the object has been deleted, the user experience is not interrupted until the user actually tries to open the object from whence the metadata came.
1. Introducing the Nervana Semantic Runtime Control—Overview
In the preferred embodiment, the Nervana Semantic Runtime Control is an ActiveX control that exposes properties and methods for use in displaying semantic data using the Nervana semantic user experience. The control will be primarily called from XSLT skins that take XML data (using the SRML schema) and generate DHTML+TIME or SVG output, consistent with the requirements of the Nervana semantic user experience. Essentially, in this embodiment, the Nervana control encapsulates the “SDK” on top of which the XSLT skins sit in order to produce a semantic content-driven user experience. The APIs listed below illustrate the functionality that will be exposed or made available by the final API set in the preferred embodiment.
2. The Nervana Semantic Runtime Control API
The EnumObjectslnNamespacePath method returns the objects in a namespace path.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open a namespace path in order for the user to navigate the namespace from within the semantic browser.
The CompileSemanticQueryFromBuffer method opens an SQML buffer and compiles it into one or more execution-ready SQML buffers. For instance, an SQML file containing a blender will be compiled into SQML buffers representing each blender entry. If the blender contains blenders, the blenders will be unwrapped and an SQML buffer will be returned for each contained blender. A compiled or “execution-ready” SQML buffer is one that can be semantically processed by an agency. The implication is that a blender that has agents from multiple agencies will have its SQML compiled to buffers with the appropriate SQML from each agency.
Note: If the buffer is already compiled, the method returns S_FALSE and the return arguments are ignored.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to compile an SQML buffer and retrieve generated “compiled code” that is ready for execution. In typical scenarios, the application or skin will compile an SQML buffer and then prepare frame windows where it wants each individual SQML query to sit. It can then issue individual SQML semantic calls by calling OpenSemanticQueryFromBuffer and then have the results displayed in the individual frames.
The OpenSemanticQueryFromBuffer method opens an SQML buffer and asynchronously fires the XML results (in SRML) onto the DOM, from whence a Nervana skin can sink the event. Note that in this embodiment the SQML has to be “compiled” and ready for execution. If the SQML is not ready for execution, the call will fail. To compile an SQML buffer, call CompileSemanticQueryFromBuffer.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open a compiled SQML buffer.
The GetSemanticQueryBufferFromFile method opens an SQML file, and returns the buffer contents. The buffer can then be compiled and/or opened.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to convert an SQML file into a buffer before processing it.
The GetSemanticQueryBufferFromNamespace method opens a namespace object, and retrieves its SQML buffer.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open an SQML buffer when it already has access to the id and path of the namespace object.
The GetSemanticQueryBufferFromURL method wraps the URL in an SQML buffer, and returns the buffer.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to convert an URL of any type to SQML. This can include file paths, HTTP URLs, FTP URLs, Nervana agency object URLs (prefixed by “wsobject://”) or Nervana agency URLs (prefixed by “wsagency://”).
The GetSemanticQueryBufferFromClipboard method converts the clipboard contents to SQML, and returns the buffer.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to get a semantic query from the clipboard. The application can then load the query buffer.
The Stop method stops current open request.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to stop a load request is just issued.
The Refresh method refreshes the current open request.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to refresh the currently loaded request.
The CreateNamespaceObject method creates a namespace object and returns its GUID.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will typically call this method to create a temporary namespace object when a new query document has been opened.
The DeleteNamespaceObject method deletes a namespace object.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will typically call this method to delete a temporary namespace object.
The CopyObject method copies the semantic object to the clipboard as an SQML buffer using a proprietary SQML clipboard format. The object can then be “pasted” onto agents for relational semantic queries, or used as a lens over other objects or agents.
A Nervana skin will typically call the CopyObject method when the user clicks on the “Copy” menu option—off a popup menu on the object.
The CanObjectBeAnnotated method checks whether the given object can be annotated.
A Nervana skin will typically call the CanObjectBeAnnotated method to determine whether to show UI indicating the “Annotate” command.
The AnnotateObject method invokes the currently installed email client and initializes it to send an email annotation of the object to the email agent of the agency from whence the object came.
A Nervana skin will typically call the AnnotateObject method when the user clicks on the “Annotate” menu option—off a popup menu on the object.
The CanObjectBePublished method checks whether the given object can be published.
A Nervana skin will typically call the CanObjectBePublished method to determine whether to show UI indicating the “Publish” command.
The PublishObject method invokes the currently installed email client and initializes it to send an email publication of the object to the email agent of the agency from whence the object came.
A Nervana skin will typically call the PublishObject method when the user clicks on the “Publish” menu option—off a popup menu on the object.
The OpenObjectContents method opens the object using an appropriate viewer. For instance, an email object will be opened in the email client, a document will be opened in the browser, etc..
A Nervana skin will typically call the OpenObjectContents method when the user clicks on the “Open” menu option—off a popup menu on the object.
The SendEmailToObject method is called to send email to a person or customer object. The method opens the email client and initializes it with the email address of the person or customer object.
A Nervana skin will typically call the SendEmailToObject method when the user clicks on the “Send Email” menu option—off a popup menu on a person or customer object.
The GetObjectAnnotations method is called to get the annotations an object has on the agency from whence it came.
A Nervana skin will typically call the GetObjectAnnotations method when it wants to display the titles of the annotations an object has—for instance, in a popup menu or when it wants to display the annotations metadata in a window.
The IsObjectMarkedAsFavorite method is called to check whether an object is marked as a favorite on the agency from whence it came.
A Nervana skin will typically call the IsObjectMarkedAsFavorite method to determine what UI to show—either the “Mark as Favorite” or the “Unmark as Favorite” command. If the object cannot be marked as a favorite (for instance, if it did not originate on an agency), the error code E_INVALIDARG is returned.
The MarkObjectAsFavorite method is called to mark the object as a favorite on the agency from whence it came.
A Nervana skin will typically call the MarkObjectAsFavorite method when the user clicks on the “Mark as Favorite” command.
The UnmarkObjectAsFavorite method is called to unmark the object as a favorite on the agency from whence it came.
A Nervana skin will typically call the UnmarkObjectAsFavorite method when the user clicks on the “Unmark as Favorite” command.
The IsSmartAgentOnClipboard method is called to check whether a smart agent has been copied to the clipboard.
A Nervana skin will typically call the IsSmartAgentOnClipboard method when it wants to toggle the user interface to display the “Paste” icon or when the “Paste” command is invoked.
The GetSmartLensQueryBuffer method is called to get the query buffer of the smart lens. This returns the SQML of the query that represents the objects on the smart agent that is on the clipboard, and which are semantically relevant to a given object.
A Nervana skin will typically call the GetSmartLensQueryBuffer method when the user hits “Paste as Smart Lens” to invoke the smart lens off the smart agent that is on the clipboard.
The OpenObjectContents method opens the object using an appropriate viewer. For instance, an email object will be opened in the email client, a document will be opened in the browser, etc.
A Nervana skin will typically call the OpenObjectContents method when the user clicks on the “Open” menu option—off a popup menu on the object.
The Email_GetFromLinkObjects method is called to get the metadata for the “From” links on an email object from the agency from whence it came.
A Nervana skin will typically call the Email_GetFromLinkObjects method when it wants to navigate to the “From” list from an email object, or to display a popup menu with the name of the person in the “From” list.
The Email_GetFromLinkObjects method is called to get the metadata for the “To” links on an email object from the agency from whence it came.
A Nervana skin will typically call the Email_GetToLinkObjects method when it wants to navigate to the “To” list from an email object, or to display a popup menu with the name of the person in the “To” list.
The Email_GetCcLinkObjects method is called to get the metadata for the “CC” links on an email object from the agency from whence it came.
A Nervana skin will typically call the Email_GetCcLinkObjects method when it wants to navigate to the “CC” list from an email object, or to display a popup menu with the name of the person in the “CC” list.
The Email_GetBccLinkObjects method is called to get the metadata for the “BCC” links on an email object from the agency from whence it came.
A Nervana skin will typically call the Email_GetBccLinkObjects method when it wants to navigate to the “BCC” list from an email object, or to display a popup menu with the name of the person in the “BCC” list.
The Email_GetAttachmentLinkObjects method is called to get the metadata for the “Attachment” links on an email object from the agency from whence it came.
A Nervana skin will typically call the Email_GetAttachmentLinkObjects method when it wants to navigate to the “Attachments” link from an email object, or to display a popup menu with the titles of the attachments in the “Attachments” list.
The Person_GetDirectReports method is called to get the metadata for the “Direct Reports” links on a person object from the agency from whence it came.
A Nervana skin will typically call the Person_GetDirectReports method when it wants to navigate to the “Direct Reports” link from a person object, or to display a popup menu with the names of the direct reports in the “Direct Reports” list.
The Person_GetDistributionLists method is called to get the metadata for the “Member of Distribution Lists” links on a person object from the agency from whence it came.
A Nervana skin will typically call the Person_GetDistributionLists method when it wants to navigate to the “Member of Distribution Lists” link from a person object, or to display a popup menu with the names of the distribution lists of which the person is a member.
The Person_GetInfoAuthored method is called to get the metadata for the “Info Authored by Person” links on a person object from the agency from whence it came.
A Nervana skin will typically call the Person_GetInfoAuthored method when it wants to navigate to the “Info Authored by Person” link from a person object, or to display a preview window with time-critical or recent information that the person authored.
The Person_GetInfoAnnotated method is called to get the metadata for the “Info Annotated by Person” links on a person object from the agency from whence it came.
A Nervana skin will typically call the Person_GetInfoAnnotated method when it wants to navigate to the “Info Annotated by Person” link from a person object, or to display a preview window with time-critical or recent information that the person annotated.
The Person_GetAnnotationsPosted method is called to get the metadata for the “Annotations Posted by Person” links on a person object from the agency from whence it came.
A Nervana skin will typically call the Person_GetAnnotationsPosted method when it wants to navigate to the “Annotations Posted by Person” link from a person object, or to display a preview window with time-critical or recent annotations that the person posted.
The Person_SendEmailTo method is called to send email to a person or customer object. The method opens the email client and initializes it with the email address of the person or customer object.
A Nervana skin will typically call the Person_SendEmailTo method when the user clicks on the “Send Email” menu option—off a popup menu on a person or customer object.
a. Event: OnBeforeQuery
The OnBeforeQuery event is fired before the control issues a query to resources consistent with the current semantic request.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cancel a query or cache state before the query is issued.
b. Event: OnQueryBegin
The OnQueryBegin event is fired when the control issues the first query to a resource consistent with the current semantic request.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cache state or display status information when the query is in progress.
c. Event: OnQueryComplete
The OnQueryComplete event is fired before the control issues a query to resources consistent with the current semantic request.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cancel a query or cache state before the query is issued.
d. Event: OnQueryResultsAvailable
The OnQueryResultsAvailable event is fired when there are available results of an asynchronous method call. The event indicates the request GUID, via which the caller can uniquely identify the specific method call that generated the response.
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event to get responses to method calls on the control.
e. Appendix A
The ‘People’ DSA will be initialized with an LDAP Directory URL and Group Name. The ‘Users’ DSA will also be initialized with an LDAP Directory URL and Group Name. Typically, the ‘Users’ will be a subset of ‘People.’ For instance, a pharmaceuticals corporation might install a KIS for different large pharmaceutical categories (e.g., Biotechnology, Life Sciences, Pharmacology, etc). Each of these will have a group of users that are knowledgeable or interested in that category. However, the KIS will also have the ‘People’ group populated with all employees of the corporation. This will enable users of the KIS to navigate to members of the entire employee population even though those members are not users of the KIS. In addition, the inference engine will be able to infer expertise with semantic links off people that are in the corporation, not necessarily just users of the KIS.
This is also advantageous for access control at the KIS level—this complements or supplements access control provided by the application server at the Web service layer. The Users group will contain people that have access to the KIS knowledge. However, the People group will contain people that are relevant to the KIS knowledge, even though those people don't have access to the KIS.
Both People and Users DSA populate the People table in the Semantic Metadata Store (SMS) and indicate the object type id appropriately. Note that preferably the passwords are NOT stored in the People table in the SMS.
The Users DSA also populates the User Authentication Table (UAT). This is an in-memory hash table that maps the user names to passwords. The server's Web service will implement the IPasswordProvider interface or an equivalent. The implementation of the PasswordProvider object will return the password that maps to a particular user name. The C# example below illustrates this:
The following C# code shows how the Web service can retrieve the user information after the user has been authenticated:
The Nervana Web service can then go ahead and call the Server Semantic Runtime with the calling user name. The runtime then maps this to SQL and uses the appropriate filters to issue the semantic query.
For the Nervana ASP.NET application, the following entry is added as a child of the parent configuration element in the Web.config file:
a. Client-Side Authorization Request
In order to create a UsernameToken for the request, the Nervana client has to pass the username and password as part of the SOAP request. The Nervana client can pass multiple tokens as part of the request—this is preferable for cases where the user's identity is federated across multiple authentication providers. The Nervana client will gather all the user account information the user has supplied (including user name and password information), convert these to WS-Security tokens, and then issue the SOAP request. The client code will look like the following (reference: [http]://[www].msdn.microsoft.com):
b. Validating the UsernameToken on the Server
Although the WSDK verifies the Security header syntax and checks the password hash against the password from the Password Provider, there is some extra verification that is preferably be performed on the request. For instance, the WSDK will not call the Password Provider if a UsernameToken is received that does not include a password element. If there is no password to check, there is no reason to call the password provider. This means we need to verify the format of the UsernameToken ourselves.
Another possibility is that there is more than one UsernameToken element included with the request. WS-Security provides support for including any number of tokens with a request that may be used for different purposes.
The code above can be modified for the Nervana Web method to verify that the UsernameToken includes a hashed password and to only accept incoming requests with a single UsernameToken. The modified code is listed below.
2. People Groups
The KIS will include metadata for people groups. These are not unlike user groups in modern operating systems. The People Group will be a Nervana first-class object (i.e., it will inherit from the Object class). In addition, the People Group schema will be as follows:
In most cases, people groups will map to user groups in directory systems (like LDAP). For instance, the KIS server admin will have the KIS crawl a configurable set of user groups. There will be a People DSA that will crawl the user groups and populate the People Groups and Users tables in the SMS. The People DSA will perform the following actions:
3. Identity Metadata Federation
Identity Metadata Federation (IMF) refers to a feature wherein an Information Community (agency) is deployed over the Internet but is used to service corporate or personal customers. For instance, Reuters™ could set up an information community for all its corporate customers that depend on its proprietary content. In such a case where multiple customers share an information community (likely in the same industry), Reuters™ will have a group on the SMS for each customer. However, each of these customers would have to have its corporate directory mirrored on Reuters™ in order for people metadata to be available. This would cause problems, particularly from a security and privacy standpoint. Corporations will probably not be comfortable with having external content providers obtaining access to the metadata of their employees. IMF addresses this problem by having the Internet-hosted information community (agency) host only enough metadata for authentication of the