CA2267451C - Method for characterizing and visualizing patterns of usage of a web site by network users - Google Patents
Method for characterizing and visualizing patterns of usage of a web site by network users Download PDFInfo
- Publication number
- CA2267451C CA2267451C CA002267451A CA2267451A CA2267451C CA 2267451 C CA2267451 C CA 2267451C CA 002267451 A CA002267451 A CA 002267451A CA 2267451 A CA2267451 A CA 2267451A CA 2267451 C CA2267451 C CA 2267451C
- Authority
- CA
- Canada
- Prior art keywords
- web
- hit
- directory
- information
- record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/323—Visualisation of programs or trace data
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99935—Query augmenting and refining, e.g. inexact access
Abstract
A system is disclosed for displaying information pertaining to the usage of Web pages. The system comprises first and second Web sites. The first Web site comprises plural Web-component files, each having a name in a Web-site directory.
The second Web site comprises plural statistics files, each containing usage information about a corresponding Web-component file or sub-directory of Web-component files. The system further comprises a computing device that has a display screen, is operable by a user, and is in communication with the first and second Web sites. The computing device is operated under the control of Web-browser software effective for displaying, on the screen, Web components of the respective Web sites. Significantly, the computing device is effective for requesting and retrieving, from either of the Web sites, data that correspond to user-designated Web components, and it is effective for directing a data request to either of the Web sites in response to user-designation of a Web component from the other Web site.
The second Web site comprises plural statistics files, each containing usage information about a corresponding Web-component file or sub-directory of Web-component files. The system further comprises a computing device that has a display screen, is operable by a user, and is in communication with the first and second Web sites. The computing device is operated under the control of Web-browser software effective for displaying, on the screen, Web components of the respective Web sites. Significantly, the computing device is effective for requesting and retrieving, from either of the Web sites, data that correspond to user-designated Web components, and it is effective for directing a data request to either of the Web sites in response to user-designation of a Web component from the other Web site.
Description
i METHOD FOR CHARACTERIZING AND VISUALIZING PATTERNS OF
USAGE OF A WEB SITE BY NETWORK USERS
Field of the Invention The invention relates to tools, for use, e.g., by a content provider for a Web site, for summarizing and displaying information descriptive of usage patterns exhibited by visitors to the Web site.
Art Background The World Wide Web is currently a subject of intense and rapidly growing interest.
The World Wide Web is composed of interconnected data sources that are accessible to computer users through data-communication networks such as the Internet. The data available on the World Wide Web have been assembled by private individuals, commercial companies, government agencies, and special interest ~o organizations. Much of this assembled information is organized into Web pages. A
Web site is a collection of Web pages (and possibly other data which, together with Web pages, are generically referred to as Web components) offered by a sponsoring entity, herein referred to as the site owner.
Large Web sites are typically organized hierarchically. For example, corporate Web sites often consist of smaller Web sites, each providing information about a business unit of the parent company.
The Web site itself resides on one or more server hosts. Web components stored on the server host are offered to users of the World Wide Web through a software program known as a Web server. A network user downloads data from a 2o Web site through a browser, a software program running on the client host.
The browser establishes contact with theWeb server and issues a request for data stored on the server host. This results in data from the server host being downloaded into the browser. This data is typically a HyperText document specifying information required by the browser to display the Web page (i.e., formatting information _2_ specifying the structure of the page, or URLs of images that are to be placed on the page), embedded client software programs which run inside the browser (e.g., Java bytecode), and other content to be downloaded to the client computer or displayable through client software programs that add to the browser's functionality (sometimes referred to as "browser plug-ins").
A visit to a Web site is defined as a series of downloads, from a specified Web server by a fixed client browser, that are contiguous in time. Each request for a Web component made by a client browser during the course of a visit is referrred to as a hit. (In at least some cases, it may be useful for distinguishing separate visits to to consider the dormancy period between successive hits by a given client browser. A
dormancy period exceeding a threshold of, e.g., fifteen minutes, may be taken to indicate the end of a visit.) Commercial Web servers have the option of recording client requests in a logfile, generating a separate entry for each hit. In many cases, the logfile resides (at least temporarily) on the server host. The information collected in this file can include the hostname or host address of the visiting client, the time of the hit, and the name of the requested data file. An illustrative record of a client request is given below:
147.atlanta-02.ga.dial-access.att.net hostname - - userid and authentication (not shown here) [30/Nov/1997:00:03:09 -0500] date and time GET request method /work/work.html name of page requested HTTP/1.0 protocol used 200 return code 9391 number of bytes transferred http://biz.yahoo.com/lucent.html referral page Mozilla/2.02E (Macintosh; U; 68K) agent used (browser) From a visitor's point of view, it is clear that a visit to a given Web site begins with an initial request to the Web server (the entry point), consists of a number of consecutive downloads, and ends when the visitor either: (i) begins to request pages from a different Web site, or (ii) stops browsing altogether.
The visitor's final request is referred to as the exit point.
The server host, on the other hand, experiences hits from many users simultaneously, and it records all requests chronologically. Consequently, the server host mixes visit information from different clients in the logfile. Because of this, it is not immediately evident, from an examination of the raw logfile, which hits correspond to which visit. Even the length of a given visit is not immediately evident. The lack of this information prevents the compilation of higher-level summaries of usage patterns.
Some software tools are commercially available for summarizing and displaying data describing Web-site usage Often, these packages require the running of a special client software program in order to view the usage data.
One drawback of such a tool is that only those users who have installed the client software will have access to the Web site's usage information, even though many geographically separated people may have a legitimate interest in this information. This group may include, e.g., content providers, Web designers, and even visitors.
A second drawback is that such a specialized client approach can become impractical because of the cost to install and maintain the client program for each interested party.
A third drawback is that the presentation of the reported information is divorced from the immediate context of the Web site itself. Thus, although it may be convenient for the user to move quickly from a statistic about some Web page to the page itself, it is much less convenient for the user to move from any desired page or a feature of such a page to a corresponding statistic.
Other software tools provide reports, in the form of HyperText documents, on the usage of selected (such as the most popular) pages. Information from these reports can be displayed via the user-side browser, and links are provided for viewing the selected Web pages. However, these software tools also fail to provide convenient access from a Web page to the statistics that pertain to it.
Summary of the Invention We have provided a mechanism for rapid and convenient access from any selected Web page to the usage information that pertains to it, and from any selected display of usage information to the Web page or pages to which it pertains.
Respective displays of Web-site content and of usage information can coexist on the 1o screen of, e.g., the user's personal computer. Designation of an item of interest (by, e.g., clicking a mouse) in one of the respective displays results in the updating of information in the other display to correspond to the designated item.
Moreover, our mechanism makes it readily achievable to synchronize one of the respective displays with the other. That is, as the user browses through one of the displays, the 15 information in the other is automatically updated to correspond to that in the first display.
Thus, in a broad aspect, our invention involves a system for displaying information pertaining to the usage of Web pages. The system comprises first and second Web sites. The first Web site comprises plural Web-component files, each 2o having a name in a Web-site directory. The second Web site comprises plural statistics files, each containing usage information about a corresponding Web-component file or sub-directory of Web-component files. The system further comprises a computing device that has a display screen, is operable by a user, and is in communication with the first and second Web sites. The computing device is 25 operated under the control of Web-browser software effective for displaying, on the screen, Web components of the respective Web sites. Significantly, the computing device is effective for requesting and retrieving, from either of the Web sites, data that correspond to user-designated Web components, and it is effective for directing -S-a data request to either of the Web sites in response to user-designation of a Web component from the other Web site.
Our preferred access mechanism involves a relationship between the Web site and the database in which the usage information is stored. As is well known, each Web component (i.e., Web page or one of its basic data building blocks) resides in a file, accessible through its URL. According to our access mechanism, the database that contains the usage information is organized such that each record is indexed by, and thus is retrievable under, the name of the corresponding file in the Web site directory.
1o Thus, in specific embodiments of the invention, each statistics file is indexed by the name of the corresponding Web file or sub-directory of Web files, and the computing device uses a common name of a file or file directory when it directs a data request to one Web site in response to user-designation of a Web component from the other Web site.
In further embodiments of the invention, we additionally provide a mechanism for distinguishing, by respective visit, visit information from a Web-server logfile, and for extracting informative usage statistics from such information.
Glossary of Terms Browser: a software program that runs on a client host and is used to request Web pages and other data from server hosts. These data can be downloaded to the client's 2o disk or displayed on the screen by the browser.
Client host: a computer that requests Web pages from server hosts, and generally communicates through a browser program.
Content provider: a person responsible for providing the information that makes up a collection of Web pages.
Embedded client software programs: software programs that comprise part of a Web site and that get downloaded into, and executed by, the browser.
Hit: the event of a browser requesting a single Web component.
Host: a computer that is connected to a network such as the Internet. Every host has a hostname(e.g., mypc.mycompany.com) and a numeric 1P address (e.g., 123.104.35.12).
HTML (HyperText Markup Language): the language used to author Web Pages.
In its raw form, HTML looks like normal text, interspersed with formatting commands. A browser's primary function is to read and render HTML.
1o HTTP (HyperText Transfer Protocol): protocol used between a browser and a Web server to exchange Web pages and other data over the Internet.
HyperText: text annotated with links to other Web pages (e.g., I-ITML).
IP (Internet Protocol): the communication protocol governing the Internet.
Logfile: a file residing on the Web site in which the Web server logs information ~5 about browsers requesting Web components. The logfile typically contains one line per hit.
Pageview: the event of a browser downloading some or all of the Web components that make up a Web page and displaying the Web page. A pageview often consists of several hits.
Referral page: the URL of the Web page containing the HyperText link that led a visitor to the data currently being viewed. In most commercial browsers, the BACK
button returns the visitor to this referral page.
Server host: a computer on the Internet that hands out Web pages through a Web server program.
URL (Uniform Resource Locator): the address of a Web component or other data.
The URL identifies the protocol used to communicate with the server host, the IP
address of the server host, and the location of the requested data on the server host.
For example, "http://www.lucent.com/work.html" specifies an HTTP connection 1o with the server host www.lucent.com, from which is requested the Web page (HTML file) work.html.
UWU server: in connection with the present invention, a special Web server in charge of distributing statistics describing Web traffic.
Visit: a series of requests to a fixed Web server by a single person (through a browser), occurnng contiguously in time.
Visitor: a person operating a browser and through it, visiting a Web site.
Web component: a basic data building block that makes up a Web page. A Web component may contain text, HyperText, images, embedded client software programs, or other data displayable by a browser (such as, for example, Quicklime 2o videos).
Web designer: a person, typically one skilled in graphical design, who has charge of designing Web pages.
_g_ Web master.: the (typically, technically trained) person in charge of keeping a host server and Web server program running.
Web page: a canonical piece of multimedia information on a Web site. A Web page is typically an HTML document comprising other Web components, such as images.
Web server: a software program running on a server host, for handing out Web pages.
Web site: a collection of Web pages residing on one or multiple server hosts and accessible through the same hostname (such as, for example, www.lucent.com).
In accordance with one aspect of the present invention there is provided a method for building a database of information about visits to a Web site, the Web site comprising Web components organized in a file directory structure having directories and sub-directories, the method comprising: a) retrieving raw hit records, each raw hit record pertaining to a particular Web component of the Web l~5 site; b) extracting selected information from said raw hit records, thereby to create, from each raw hit record, a filtered hit record, the selected information including information identifying the Web component to which the respective hit record pertains; c) distinguishing respective filtered hit records according to the visit to which each of them belongs, and associating a particular visit index with each :?0 filtered hit record; d) storing the filtered hit records in a data storage device, said storing step comprising the further step of indexing said records such that each record is retrievable by reference to the filename of the corresponding Web component; c;) proceeding hierarchically from lowest sub-directories to highest directories, compiling for each sub-directory and directory a respective record of :?5 collective usage information pertaining to all Web components associated with such sub-dirc;ctory or directory; f) storing said collective-hit-information records in the filtered hit record data storage device, said storing step comprising the further step of indexing each said record such that it is retrievable by reference to the -8a-name of the corresponding sub-directory or directory; and h) organizing said filtered hit re~~ords and said collective-hit records as a further Web site.
Brief Descr~tion of the Drawin s FIG. 1 is a schematic diagram of a system for retrieving and displaying Web site usage information, in accordance with the invention in an exemplary embodiment.
FIG. lA is a detail of an exemplary display for window 30 of FIG. 1.
FIG. 1B is a detail of an exemplary display for window 35 of FIG. 1.
FIG. lC is a detail of an exemplary display for window 40 of FIG. 1.
FIG. :? is a flowchart of an exemplary procedure for assembling hit information into visit records, in accordance with the invention.
FIG. :3 is a flowchart of an exemplary procedure for building a shadow-directory dat~~base in accordance with the invention.
FIG. ~I is a flowchart ~of an alternative procedure for building a shadow-l~ 5 directory dat~~base.
FIG. :5 is a flowchart .of an exemplary procedure for further processing of the shadow-directory database in accordance with the invention in certain embodiment:..
FIG. ~5 is a flowchart of an exemplary procedure for compiling directory-~0 level usage information, in accordance with the invention in certain embodiments.
FIG. 7 is a tree diagram of an illustrative Web site and its corresponding shadow-directory database. Included in the shadow-directory tree structure are extra files that contain directory-level usage information.
Detailed Description 1. Viewing Web Usage Information With reference to FIG. 1 (and with further reference to FIGs. lA-1C), client 10 is a Web browser, such as Netscape Navigator, Microsoft Internet Explorer, or another such readily available standard browser. Client 10 must be specially configured to function as a data browser for accessing the usage statistics that are stored in storage device 15 of UWU server 20. This configuration is carried to out by requesting an initializing page from UWU server 20. This page includes embedded client programs that coordinate the ordinary browsing of Web pages from Web site 25 (for simplicity of presentation, reference numeral 25 also denotes the server corresponding to this Web site) with the retrieval and display of usage statistics stored on UWU server 20. In our current implementation, JavaScript 1.2 is t5 used to manage this interaction.
In use, web pages (stored in storage device 27) that belong to Web site 25 are displayed in browser window 30 on the screen of, e.g. the usei s personal computer (pc). Usage data collected from visits to site 25 are requested from server 20 and displayed in browser windows 35 and 40. In addition, statistical information from 20 the UWU server 20 is superimposed directly onto window 30 through a sequence of layers 42.
To achieve a complete correspondence between the components of a Web site and usage information, the displays in data browser 10 must be synchronized.
For example, when a link on page 45 (shown as being displayed in the browser 25 window 30) is selected, the requested UItL is downloaded into the browser window 30, while at the same time the corresponding usage information for the requested URL is displayed in windows 35 and 40, and layers 42. In our current implementation, JavaScript 1.2 is used to intercept "click" events occurring in the browser window 30 and then dispatch requests to both servers 20 and 25. These requests are indicated by the directed communication links 50, 55 and 60. The action of updating windows 35 and 40 and layers 42 in response to a event "click" in window 30 is indicated by the arrows 65 in the figure.
Within the client 10, we enable two other types of synchronized interaction.
In the first (indicated by arrow 70 in the figure), the user "clicks" on HyperText links displayed in browser window 35, triggering the loading of the associated URL
into window 30. For example, window 35 might contain a list of the most frequent referral pages followed by those visitors requesting the page displayed in browser window 30. This type of usage information is presented as a list of HyperText links that, when "clicked,"directs the browser window 30 to the selected refernng page (whether or not it resides on the Web site 25).
In the second coordinated interaction, designated by communication branches 75 and 80, statistical displays (such as scatter plots, simple tables and tree-or graph-based representations) are used to direct the browser window 42 and window 30. For example, our current implementation can display a summary of all the visits that access the page displayed in window 30. This summary is viewed in the auxiliary browser window 40.
By way of illustration, window 35 might show a graph of usage versus time (measured in days), in which the quantity plotted is the total number of visits (each day) in the course of which a particular page is requested. By clicking on a particular portion of the graph, the user could make a selection of a particular day.
In response, auxiliary window (or windows) 40 might display a summary list of all visits that took place on the selected day. By using an auxiliary window in this manner, the information is displayed without effacing the original graph.
Pursuing this illustrative example further, the links displayed in window 40 are effective for retrieving, from the data base embodied in storage device 15, the actual requests issued by a visitor during a visit. Each element of the tabulation displayed in window 40 is an HTML link that directs the browser window 30 to the 3o indicated page.
In our current implementation, the statistical displays are written in Java, and communicate with the other browser windows through JavaScript 1.2 protocols.
In addition to separate displays (in, e.g., windows 30, 35, and 40, as shown in the figure), the data browser can directly impose statistical information onto, e.g., window 30 in the visible form of overlays 42, as noted above.
One exemplary use of these overlays is to provide an indication of the respective popularities of selected links followed by visitors requesting page 30. For example, links may be labeled, in an overlay, by respective images each of whose sizes represents the relative popularity of the corresponding link. One image readily 1o sized in that manner is a simple bullseye.
By way of another example, the population requesting a given link may be analyzed into classes, such as "customers" and "competitors" of a corporate content provider. A graphical image such as a bar chart, indicating the relative distribution of requests among these various classes is readily superimposed on the relevant link 15 by means of an overlay.
In yet another example, an overlay contains symbols that designate respective links as principally "content-providing" or principally "aiding navigation." In yet another example, an overlay provides a miniature, pop-up view of the Web page corresponding to a given link.
2o In our current implementation, JavaScript 1.2 layers and coordinate information contained in JavaScript 1.2 link objects are used to position the overlays 42.
Thus, the use of overlays 42 and windows 30, 35, and 40 as described above enables the user to acquire information about visitors in a natural manner, and to 25 directly visualize the usage data in their proper context, namely, the actual Web pages of Web site 25.
It should be noted in this regard that JavaScript 1.2 layers and HTML frames have the feature that information from different servers can be displayed on the same page. In order to do so, what is needed is simply to provide each relevant URL
30 to the browser through an SRC tag. Thus, as those skilled in the art will appreciate, the display of Web pages simultaneously with statistical information about their patterns of usage as described here is readily implementable using commercially available software.
A significant advantage of our invention is that information about visitors can be readily obtained for every page of Web site 25. This is in contrast to those approaches of the prior art that compile information only about the most popular pages (and typically ignore smaller components such as sub-pages). In particular, this limited type of reporting may be disadvantageous for large corporate sites that attract many thousands, and even millions, of visitors each week. Despite such large 1o traffic, sub-sites corresponding to separate organizations within the company might only entertain a few hundred visitors per week. Under a conventional, limited reporting approach, such sites would typically be forced to go without usage information, or to use specialized reporting software that would have to be regenerated each time the Web site was altered.
USAGE OF A WEB SITE BY NETWORK USERS
Field of the Invention The invention relates to tools, for use, e.g., by a content provider for a Web site, for summarizing and displaying information descriptive of usage patterns exhibited by visitors to the Web site.
Art Background The World Wide Web is currently a subject of intense and rapidly growing interest.
The World Wide Web is composed of interconnected data sources that are accessible to computer users through data-communication networks such as the Internet. The data available on the World Wide Web have been assembled by private individuals, commercial companies, government agencies, and special interest ~o organizations. Much of this assembled information is organized into Web pages. A
Web site is a collection of Web pages (and possibly other data which, together with Web pages, are generically referred to as Web components) offered by a sponsoring entity, herein referred to as the site owner.
Large Web sites are typically organized hierarchically. For example, corporate Web sites often consist of smaller Web sites, each providing information about a business unit of the parent company.
The Web site itself resides on one or more server hosts. Web components stored on the server host are offered to users of the World Wide Web through a software program known as a Web server. A network user downloads data from a 2o Web site through a browser, a software program running on the client host.
The browser establishes contact with theWeb server and issues a request for data stored on the server host. This results in data from the server host being downloaded into the browser. This data is typically a HyperText document specifying information required by the browser to display the Web page (i.e., formatting information _2_ specifying the structure of the page, or URLs of images that are to be placed on the page), embedded client software programs which run inside the browser (e.g., Java bytecode), and other content to be downloaded to the client computer or displayable through client software programs that add to the browser's functionality (sometimes referred to as "browser plug-ins").
A visit to a Web site is defined as a series of downloads, from a specified Web server by a fixed client browser, that are contiguous in time. Each request for a Web component made by a client browser during the course of a visit is referrred to as a hit. (In at least some cases, it may be useful for distinguishing separate visits to to consider the dormancy period between successive hits by a given client browser. A
dormancy period exceeding a threshold of, e.g., fifteen minutes, may be taken to indicate the end of a visit.) Commercial Web servers have the option of recording client requests in a logfile, generating a separate entry for each hit. In many cases, the logfile resides (at least temporarily) on the server host. The information collected in this file can include the hostname or host address of the visiting client, the time of the hit, and the name of the requested data file. An illustrative record of a client request is given below:
147.atlanta-02.ga.dial-access.att.net hostname - - userid and authentication (not shown here) [30/Nov/1997:00:03:09 -0500] date and time GET request method /work/work.html name of page requested HTTP/1.0 protocol used 200 return code 9391 number of bytes transferred http://biz.yahoo.com/lucent.html referral page Mozilla/2.02E (Macintosh; U; 68K) agent used (browser) From a visitor's point of view, it is clear that a visit to a given Web site begins with an initial request to the Web server (the entry point), consists of a number of consecutive downloads, and ends when the visitor either: (i) begins to request pages from a different Web site, or (ii) stops browsing altogether.
The visitor's final request is referred to as the exit point.
The server host, on the other hand, experiences hits from many users simultaneously, and it records all requests chronologically. Consequently, the server host mixes visit information from different clients in the logfile. Because of this, it is not immediately evident, from an examination of the raw logfile, which hits correspond to which visit. Even the length of a given visit is not immediately evident. The lack of this information prevents the compilation of higher-level summaries of usage patterns.
Some software tools are commercially available for summarizing and displaying data describing Web-site usage Often, these packages require the running of a special client software program in order to view the usage data.
One drawback of such a tool is that only those users who have installed the client software will have access to the Web site's usage information, even though many geographically separated people may have a legitimate interest in this information. This group may include, e.g., content providers, Web designers, and even visitors.
A second drawback is that such a specialized client approach can become impractical because of the cost to install and maintain the client program for each interested party.
A third drawback is that the presentation of the reported information is divorced from the immediate context of the Web site itself. Thus, although it may be convenient for the user to move quickly from a statistic about some Web page to the page itself, it is much less convenient for the user to move from any desired page or a feature of such a page to a corresponding statistic.
Other software tools provide reports, in the form of HyperText documents, on the usage of selected (such as the most popular) pages. Information from these reports can be displayed via the user-side browser, and links are provided for viewing the selected Web pages. However, these software tools also fail to provide convenient access from a Web page to the statistics that pertain to it.
Summary of the Invention We have provided a mechanism for rapid and convenient access from any selected Web page to the usage information that pertains to it, and from any selected display of usage information to the Web page or pages to which it pertains.
Respective displays of Web-site content and of usage information can coexist on the 1o screen of, e.g., the user's personal computer. Designation of an item of interest (by, e.g., clicking a mouse) in one of the respective displays results in the updating of information in the other display to correspond to the designated item.
Moreover, our mechanism makes it readily achievable to synchronize one of the respective displays with the other. That is, as the user browses through one of the displays, the 15 information in the other is automatically updated to correspond to that in the first display.
Thus, in a broad aspect, our invention involves a system for displaying information pertaining to the usage of Web pages. The system comprises first and second Web sites. The first Web site comprises plural Web-component files, each 2o having a name in a Web-site directory. The second Web site comprises plural statistics files, each containing usage information about a corresponding Web-component file or sub-directory of Web-component files. The system further comprises a computing device that has a display screen, is operable by a user, and is in communication with the first and second Web sites. The computing device is 25 operated under the control of Web-browser software effective for displaying, on the screen, Web components of the respective Web sites. Significantly, the computing device is effective for requesting and retrieving, from either of the Web sites, data that correspond to user-designated Web components, and it is effective for directing -S-a data request to either of the Web sites in response to user-designation of a Web component from the other Web site.
Our preferred access mechanism involves a relationship between the Web site and the database in which the usage information is stored. As is well known, each Web component (i.e., Web page or one of its basic data building blocks) resides in a file, accessible through its URL. According to our access mechanism, the database that contains the usage information is organized such that each record is indexed by, and thus is retrievable under, the name of the corresponding file in the Web site directory.
1o Thus, in specific embodiments of the invention, each statistics file is indexed by the name of the corresponding Web file or sub-directory of Web files, and the computing device uses a common name of a file or file directory when it directs a data request to one Web site in response to user-designation of a Web component from the other Web site.
In further embodiments of the invention, we additionally provide a mechanism for distinguishing, by respective visit, visit information from a Web-server logfile, and for extracting informative usage statistics from such information.
Glossary of Terms Browser: a software program that runs on a client host and is used to request Web pages and other data from server hosts. These data can be downloaded to the client's 2o disk or displayed on the screen by the browser.
Client host: a computer that requests Web pages from server hosts, and generally communicates through a browser program.
Content provider: a person responsible for providing the information that makes up a collection of Web pages.
Embedded client software programs: software programs that comprise part of a Web site and that get downloaded into, and executed by, the browser.
Hit: the event of a browser requesting a single Web component.
Host: a computer that is connected to a network such as the Internet. Every host has a hostname(e.g., mypc.mycompany.com) and a numeric 1P address (e.g., 123.104.35.12).
HTML (HyperText Markup Language): the language used to author Web Pages.
In its raw form, HTML looks like normal text, interspersed with formatting commands. A browser's primary function is to read and render HTML.
1o HTTP (HyperText Transfer Protocol): protocol used between a browser and a Web server to exchange Web pages and other data over the Internet.
HyperText: text annotated with links to other Web pages (e.g., I-ITML).
IP (Internet Protocol): the communication protocol governing the Internet.
Logfile: a file residing on the Web site in which the Web server logs information ~5 about browsers requesting Web components. The logfile typically contains one line per hit.
Pageview: the event of a browser downloading some or all of the Web components that make up a Web page and displaying the Web page. A pageview often consists of several hits.
Referral page: the URL of the Web page containing the HyperText link that led a visitor to the data currently being viewed. In most commercial browsers, the BACK
button returns the visitor to this referral page.
Server host: a computer on the Internet that hands out Web pages through a Web server program.
URL (Uniform Resource Locator): the address of a Web component or other data.
The URL identifies the protocol used to communicate with the server host, the IP
address of the server host, and the location of the requested data on the server host.
For example, "http://www.lucent.com/work.html" specifies an HTTP connection 1o with the server host www.lucent.com, from which is requested the Web page (HTML file) work.html.
UWU server: in connection with the present invention, a special Web server in charge of distributing statistics describing Web traffic.
Visit: a series of requests to a fixed Web server by a single person (through a browser), occurnng contiguously in time.
Visitor: a person operating a browser and through it, visiting a Web site.
Web component: a basic data building block that makes up a Web page. A Web component may contain text, HyperText, images, embedded client software programs, or other data displayable by a browser (such as, for example, Quicklime 2o videos).
Web designer: a person, typically one skilled in graphical design, who has charge of designing Web pages.
_g_ Web master.: the (typically, technically trained) person in charge of keeping a host server and Web server program running.
Web page: a canonical piece of multimedia information on a Web site. A Web page is typically an HTML document comprising other Web components, such as images.
Web server: a software program running on a server host, for handing out Web pages.
Web site: a collection of Web pages residing on one or multiple server hosts and accessible through the same hostname (such as, for example, www.lucent.com).
In accordance with one aspect of the present invention there is provided a method for building a database of information about visits to a Web site, the Web site comprising Web components organized in a file directory structure having directories and sub-directories, the method comprising: a) retrieving raw hit records, each raw hit record pertaining to a particular Web component of the Web l~5 site; b) extracting selected information from said raw hit records, thereby to create, from each raw hit record, a filtered hit record, the selected information including information identifying the Web component to which the respective hit record pertains; c) distinguishing respective filtered hit records according to the visit to which each of them belongs, and associating a particular visit index with each :?0 filtered hit record; d) storing the filtered hit records in a data storage device, said storing step comprising the further step of indexing said records such that each record is retrievable by reference to the filename of the corresponding Web component; c;) proceeding hierarchically from lowest sub-directories to highest directories, compiling for each sub-directory and directory a respective record of :?5 collective usage information pertaining to all Web components associated with such sub-dirc;ctory or directory; f) storing said collective-hit-information records in the filtered hit record data storage device, said storing step comprising the further step of indexing each said record such that it is retrievable by reference to the -8a-name of the corresponding sub-directory or directory; and h) organizing said filtered hit re~~ords and said collective-hit records as a further Web site.
Brief Descr~tion of the Drawin s FIG. 1 is a schematic diagram of a system for retrieving and displaying Web site usage information, in accordance with the invention in an exemplary embodiment.
FIG. lA is a detail of an exemplary display for window 30 of FIG. 1.
FIG. 1B is a detail of an exemplary display for window 35 of FIG. 1.
FIG. lC is a detail of an exemplary display for window 40 of FIG. 1.
FIG. :? is a flowchart of an exemplary procedure for assembling hit information into visit records, in accordance with the invention.
FIG. :3 is a flowchart of an exemplary procedure for building a shadow-directory dat~~base in accordance with the invention.
FIG. ~I is a flowchart ~of an alternative procedure for building a shadow-l~ 5 directory dat~~base.
FIG. :5 is a flowchart .of an exemplary procedure for further processing of the shadow-directory database in accordance with the invention in certain embodiment:..
FIG. ~5 is a flowchart of an exemplary procedure for compiling directory-~0 level usage information, in accordance with the invention in certain embodiments.
FIG. 7 is a tree diagram of an illustrative Web site and its corresponding shadow-directory database. Included in the shadow-directory tree structure are extra files that contain directory-level usage information.
Detailed Description 1. Viewing Web Usage Information With reference to FIG. 1 (and with further reference to FIGs. lA-1C), client 10 is a Web browser, such as Netscape Navigator, Microsoft Internet Explorer, or another such readily available standard browser. Client 10 must be specially configured to function as a data browser for accessing the usage statistics that are stored in storage device 15 of UWU server 20. This configuration is carried to out by requesting an initializing page from UWU server 20. This page includes embedded client programs that coordinate the ordinary browsing of Web pages from Web site 25 (for simplicity of presentation, reference numeral 25 also denotes the server corresponding to this Web site) with the retrieval and display of usage statistics stored on UWU server 20. In our current implementation, JavaScript 1.2 is t5 used to manage this interaction.
In use, web pages (stored in storage device 27) that belong to Web site 25 are displayed in browser window 30 on the screen of, e.g. the usei s personal computer (pc). Usage data collected from visits to site 25 are requested from server 20 and displayed in browser windows 35 and 40. In addition, statistical information from 20 the UWU server 20 is superimposed directly onto window 30 through a sequence of layers 42.
To achieve a complete correspondence between the components of a Web site and usage information, the displays in data browser 10 must be synchronized.
For example, when a link on page 45 (shown as being displayed in the browser 25 window 30) is selected, the requested UItL is downloaded into the browser window 30, while at the same time the corresponding usage information for the requested URL is displayed in windows 35 and 40, and layers 42. In our current implementation, JavaScript 1.2 is used to intercept "click" events occurring in the browser window 30 and then dispatch requests to both servers 20 and 25. These requests are indicated by the directed communication links 50, 55 and 60. The action of updating windows 35 and 40 and layers 42 in response to a event "click" in window 30 is indicated by the arrows 65 in the figure.
Within the client 10, we enable two other types of synchronized interaction.
In the first (indicated by arrow 70 in the figure), the user "clicks" on HyperText links displayed in browser window 35, triggering the loading of the associated URL
into window 30. For example, window 35 might contain a list of the most frequent referral pages followed by those visitors requesting the page displayed in browser window 30. This type of usage information is presented as a list of HyperText links that, when "clicked,"directs the browser window 30 to the selected refernng page (whether or not it resides on the Web site 25).
In the second coordinated interaction, designated by communication branches 75 and 80, statistical displays (such as scatter plots, simple tables and tree-or graph-based representations) are used to direct the browser window 42 and window 30. For example, our current implementation can display a summary of all the visits that access the page displayed in window 30. This summary is viewed in the auxiliary browser window 40.
By way of illustration, window 35 might show a graph of usage versus time (measured in days), in which the quantity plotted is the total number of visits (each day) in the course of which a particular page is requested. By clicking on a particular portion of the graph, the user could make a selection of a particular day.
In response, auxiliary window (or windows) 40 might display a summary list of all visits that took place on the selected day. By using an auxiliary window in this manner, the information is displayed without effacing the original graph.
Pursuing this illustrative example further, the links displayed in window 40 are effective for retrieving, from the data base embodied in storage device 15, the actual requests issued by a visitor during a visit. Each element of the tabulation displayed in window 40 is an HTML link that directs the browser window 30 to the 3o indicated page.
In our current implementation, the statistical displays are written in Java, and communicate with the other browser windows through JavaScript 1.2 protocols.
In addition to separate displays (in, e.g., windows 30, 35, and 40, as shown in the figure), the data browser can directly impose statistical information onto, e.g., window 30 in the visible form of overlays 42, as noted above.
One exemplary use of these overlays is to provide an indication of the respective popularities of selected links followed by visitors requesting page 30. For example, links may be labeled, in an overlay, by respective images each of whose sizes represents the relative popularity of the corresponding link. One image readily 1o sized in that manner is a simple bullseye.
By way of another example, the population requesting a given link may be analyzed into classes, such as "customers" and "competitors" of a corporate content provider. A graphical image such as a bar chart, indicating the relative distribution of requests among these various classes is readily superimposed on the relevant link 15 by means of an overlay.
In yet another example, an overlay contains symbols that designate respective links as principally "content-providing" or principally "aiding navigation." In yet another example, an overlay provides a miniature, pop-up view of the Web page corresponding to a given link.
2o In our current implementation, JavaScript 1.2 layers and coordinate information contained in JavaScript 1.2 link objects are used to position the overlays 42.
Thus, the use of overlays 42 and windows 30, 35, and 40 as described above enables the user to acquire information about visitors in a natural manner, and to 25 directly visualize the usage data in their proper context, namely, the actual Web pages of Web site 25.
It should be noted in this regard that JavaScript 1.2 layers and HTML frames have the feature that information from different servers can be displayed on the same page. In order to do so, what is needed is simply to provide each relevant URL
30 to the browser through an SRC tag. Thus, as those skilled in the art will appreciate, the display of Web pages simultaneously with statistical information about their patterns of usage as described here is readily implementable using commercially available software.
A significant advantage of our invention is that information about visitors can be readily obtained for every page of Web site 25. This is in contrast to those approaches of the prior art that compile information only about the most popular pages (and typically ignore smaller components such as sub-pages). In particular, this limited type of reporting may be disadvantageous for large corporate sites that attract many thousands, and even millions, of visitors each week. Despite such large 1o traffic, sub-sites corresponding to separate organizations within the company might only entertain a few hundred visitors per week. Under a conventional, limited reporting approach, such sites would typically be forced to go without usage information, or to use specialized reporting software that would have to be regenerated each time the Web site was altered.
2. Preprocessing With further reference to FIG. 1, a preprocessing stage 85 is advantageously applied for compiling, from raw logfiles obtained from storage device 90, the file resident on storage device 15 that is needed for the on-line browsing of usage statistics.
2o In the figure, arrow 95 indicates the operation of handing raw data from device 90 to preprocessor 85, and arrow 100 indicates the operation of handing processed records to device 15. Operations 95 and 100 are advantageously performed periodically. A typical repetition frequency for these operations is once per day. However, such a frequency is not essential to the invention, and in fact the system designer has broad discretion in determining this frequency. In fact, processing and updating of statistics files can even be done on a continuous basis.
In an illustrative implementation, the preprocessing is carried out in the following four stages:
(1) filtering and cleaning of the log data;
(2) defining visits;
2o In the figure, arrow 95 indicates the operation of handing raw data from device 90 to preprocessor 85, and arrow 100 indicates the operation of handing processed records to device 15. Operations 95 and 100 are advantageously performed periodically. A typical repetition frequency for these operations is once per day. However, such a frequency is not essential to the invention, and in fact the system designer has broad discretion in determining this frequency. In fact, processing and updating of statistics files can even be done on a continuous basis.
In an illustrative implementation, the preprocessing is carried out in the following four stages:
(1) filtering and cleaning of the log data;
(2) defining visits;
(3) building the shadow directory database; and (4) hierarchical accounting of visit information.
Each of these stages will now be briefly described.
2.1. Filtering. As mentioned above, every hit is typically logged by the Web server. However, it is often advantageous to retain only selected lines of the log for further processing. (In particular, this reduces the volume of data that is subsequently manipulated and stored, and thus improves economy and speed.) Such selection can be carried out on any of the fields in the log. As is well known to those skilled in the art, such selection is readily specified in a configuration file. It is 1o not critical where this filtering is carried out. In our current implementation, by way of example, the filtering is carried out by the UWU server.
By way of example, some or all of the following fields may be of interest for treatment in the preprocessing stage:
hostname: The user who is accessing usage data may wish to filter out his own accesses to the Web site, because they might otherwise skew the statistics.
Moreover, filtering on this field may be desirable in order to focus specifically on internal or on external visitors.
request method: Ordinary Web browsers typically use the GET request method for downloading Web pages, whereas automated programs such as Web 2o robots (or Web spiders), that download pages for subsequent indexing, often use the HEAD request instead. To exclude accesses by such programs, the user may filter out HEAD requests.
return code: This field can be used, e.g., to retain only those downloads that were successful (as indicated, typically, by a return code of 200).
file type: A typical Web page consists of an HTML document combined with other Web components such as images or sound. If only pageviews are of interest, this field can be used to filter out hits to image and sound files (which are generally identifiable through their file extension, such as ".gig' for images).
2.2. Identifying visits. Several mechanisms can be used for grouping hits into visits. One possibility is to use so-called "cookies". When a visitor first comes to a Web site, the Web server will hand a unique "cookie" to the browser. During subsequent hits the browser hands this cookie back to the Web server. Thus, hits originated from a fixed visitor can be grouped by cookie. This mechanism leads to a reliable method for defining visits. However, Web users are becoming increasingly concerned about the use and misuse of cookies, and current browsers allow the user to disable the cookie mechanism. Therefore, this mechanism is of only limited use for the present purpose.
A method for tracking visitors to a Web site without using cookies, involves serving each Web component through the so-called cgi (Common Gateway Interface) mechanism. That is, the first time a visitor comes through the site, a unique number is assigned to that user. (This first time can be identified by the fact that the request does not include a visitor number mechanism, while all subsequent requests will). The requested page is now served through the cgi mechanism, which changes all the internal links on that page to the appropriate cgi links including the unique visitor number. Consequently, all subsequent requests will go through the cgi mechanism, and all the pages served will have their links correspondingly altered.
The visitor number now allows robust tracking of visitors.
2o Another mechanism, which we use in our current implementation, infers visit sequences from only the information in the log file. Hits are combined into a particular visit sequence if they are separated by less than a selectable time interval T (typically, ten minutes, although there is wide latitude for setting this interval) and they originate from the same host into a visit.
We have found that this scheme often works well. However, it can produce an erroneous result if two or more users, acting through the same host, request pages from the same Web site at the same time. This typically happens when user requests are grouped by a so-called "proxy" machine. The individual users request pages only from the proxy, while the proxy caches pages and requests new ones from the actual Web server. Because the Web server sees requests only from the proxy, this method may be unable to distinguish different visitors using the same proxy. We have found, however, that this situation tends to occur infrequently.
More advanced methods for grouping hits into visits can also be used. For example, if a hit has a referral field from a Web page of a different Web site, this can be taken as an indication that a new visit has started. Also, if two visitors using the same proxy visit the same Web site at the same time, they may still be viewing different parts of the Web site, and thus they can be distinguished by the actual web components they download.
Further methods can take advantage of additional information that the 1o browser communicates to the server. For example, the HTTP protocol allows for browsers to provide, among other kinds of information:
( 1 ) the type of browser used, which version, and what kind of platform;
(2) the screen resolution; and (3) which types of web component the browser is able to display.
Thus, users who differ in any of these fields can be distinguished.
In the following discussion, it is assumed that the information to be used for distinguishing users is combined in a user tag. With reference to FIG. 2, we now describe an exemplary procedure for assembling hits (indexed by the variable "hitnr") into respective visits (indexed by the variable "visitnr").
2o Lines are retrieved sequentially from the log (box 105). The new line so retrieved is the current hit. For each new hit, a visitor tag is composed (box 110). A
determination is then made, whether this visitor tag belongs to a new visit (box 115). That is, all of the currently open visits are recorded in working memory. The visit is new if the visitor tag does not belong to any of the currently open visits.
If it is determined that the visit is new, a new visitor-record data structure is created in working memory, and the current hit and hitnr are added to the data structure (box 120). If it is determined that the visit is not new, the hit and hitnr are added to the currently open visitor record identified by the visitor tag (box 125). (It should be noted in this regard that a former visitor who initiates a new visit is treated as a new visitor.) When each new visitor-record data structure is created, a sequential index number (visitnr) is assigned to the corresponding visit (box 130).
Next, a determination is made whether any of the currently open visits have expired. For each of the currently open visit records, the time of the last hit in that record is subtracted from the time of the current hit (box 135). The visit is deemed to have expired if the difference is greater than the threshold T. If the visit has expired, the record of that visit is added to the visit database on disk, and removed from working memory (box 140). The visitnr is included with the data that are sent to disk storage.
to Then, the next hit is retrieved, and the process is repeated. Hits with the same visit tag are added in chronological order so long as they are separated by less than the time interval T.
2.3. Building the shadow directory. The visit database is next processed into a new database resident on storage device 15, which we refer to as the shadow-directory database.
It is a general characteristic of files containing Web components that they are organized in a hierarchical file structure. A correspondence to this hierarchical file structure is built into the shadow-directory database, such that its records are 2o retrievable by the names of the corresponding files on the Web server.
By way of example, this correspondence can be achieved by using the same filenames, in the shadow-directory database, as the corresponding Web-page files.
Alternatively, as will be appreciated by those skilled in the art, such a correspondence is readily achieved using well-known data-base technologies to relate files of the shadow-directory database to corresponding Web-page files.
As a consequence, each file in the Web-site directory will have a unique counterpart in the shadow-directory database. The correspondence described above is useful for facilitating on-line retrieval of information from the shadow-directory database by operating it as a parallel Web site.
Each file in the shadow-directory database contains the compiled information pertaining to the hits made to the corresponding web component.
With reference to FIG. 3, we next describe how this information is compiled.
Each visit record is retrieved, in turn (box 145). For each of these visit records, we record information for each of the hits that it contains.
Specifically, we record the filename of the downloaded Web component (box 150) and other information concerning the hit, together with the hitnr index, the visitnr index, the filenames of the next and previous hits, and the time interval between the current hit and the next hit (box 155).
1o Once built, the shadow-directory database affords immediate access to all the hits to a particular Web component, by simply accessing the corresponding file.
The process described above for building the shadow directory may in some cases be undesirably slow because it calls for shadow-directory files to be opened and closed many times. For this reason, it will in at least some cases be advantageous to first annotate the hits with hitnr, visitnr, and "previous"
and "next"
fields, and then sort the hits per filename (i.e., according to the name of the requested Web file). This way each file of the shadow directory only needs to be opened and closed once. Then, all of the records associated with each given Web file can be loaded, en masse, into the corresponding shadow-directory file.
2o Thus, as shown, e.g., in FIG. 4, the shadow-directory data base is advantageously built by retrieving, in turn, each of these collections of sorted records (box 160). The filename of the respective Web component is recorded (box 165). The hit information is then recorded (box 170) for each of the annotated hit records in the collection.
Below is one illustrative line, with explanatory annotations, in the hypothetical shadow-directory file named "shadow/card.html":
poo1044-maxi.sc-ca-us.dialup.earthlink.net hostname - userid [not shown here]
1/05/1998 23:43:09 date and time /card.html file accessed 200 return code 14176 bytes transferred to /work.html previous file /contact/contact.html next file 78 time interval in seconds It should be noted that it is sufficient, e.g. for alternate embodiments of the invention, for the shadow directory data base simply to store visitnr indices, rather than to store complete visitor records. The complete records are instead readily retrievable, by visitnr, from the visit data base resident on the UWU server.
This scheme avoids duplication, and thus helps to conserve disk space.
2.4. Hierarchical Accounting. Further preprocessing is advantageously performed after the shadow directory has been built. In a first stage, as illustrated in FIG. 5, each shadow-directory file is processed in turn. For each of these files, the number of hits over time, the distribution of "previous" and "next" hits, and the distribution of time spent are computed and stored (box 175). It is also advantageous to store the index visitnr for each visit that contains a hit to the corresponding Web file. It should be noted in this regard that the visits to a file are typically fewer than the hits to a file, because a visitor may download a web component multiple times within a single visit.
In a second stage, with reference to FIG. 6, directory-level information is computed (step 176) for each directory in the file tree. This is done hierarchically, beginning with the sub-directories at the lowest levels of the tree. For each directory, there is recorded the total number of hits over time to all the files in the directory or any of its subdirectories. It is also advantageous to count the number of distinct visitors to the files in each directory by merging the visitnr indices of the visits to the files in the directory.
Since no such directory-level record specifically relates to any one file, an extra file is created (step 177) in each pertinent directory to contain this information.
(In our current implementation, we have found it convenient to designate each of these files by the name "UWU.info".) Thus, each UWU.info file contains, in particular, hit counts and visit counts accumulated over a period of time.
Thus, with reference to FIG. 7, directory-level information is provided in a file 190 for each directory (180, 180.1-180.3) in the Web file tree. In the figure, the corresponding directories of the shadow-directory database are numbered 180', 180.1'-180.3', respectively. The sub-directories at the lowest levels of the tree shown are numbered 180.3, 180.3'. To each Web file 185, there corresponds a file 185' in the shadow-directory database.
As a conceptual aide in the figure, the symbol that represents each file 190 is drawn as containing the symbols of any lower-level files 190 whose information it (wholly or partly) includes as a component element.
Each of these stages will now be briefly described.
2.1. Filtering. As mentioned above, every hit is typically logged by the Web server. However, it is often advantageous to retain only selected lines of the log for further processing. (In particular, this reduces the volume of data that is subsequently manipulated and stored, and thus improves economy and speed.) Such selection can be carried out on any of the fields in the log. As is well known to those skilled in the art, such selection is readily specified in a configuration file. It is 1o not critical where this filtering is carried out. In our current implementation, by way of example, the filtering is carried out by the UWU server.
By way of example, some or all of the following fields may be of interest for treatment in the preprocessing stage:
hostname: The user who is accessing usage data may wish to filter out his own accesses to the Web site, because they might otherwise skew the statistics.
Moreover, filtering on this field may be desirable in order to focus specifically on internal or on external visitors.
request method: Ordinary Web browsers typically use the GET request method for downloading Web pages, whereas automated programs such as Web 2o robots (or Web spiders), that download pages for subsequent indexing, often use the HEAD request instead. To exclude accesses by such programs, the user may filter out HEAD requests.
return code: This field can be used, e.g., to retain only those downloads that were successful (as indicated, typically, by a return code of 200).
file type: A typical Web page consists of an HTML document combined with other Web components such as images or sound. If only pageviews are of interest, this field can be used to filter out hits to image and sound files (which are generally identifiable through their file extension, such as ".gig' for images).
2.2. Identifying visits. Several mechanisms can be used for grouping hits into visits. One possibility is to use so-called "cookies". When a visitor first comes to a Web site, the Web server will hand a unique "cookie" to the browser. During subsequent hits the browser hands this cookie back to the Web server. Thus, hits originated from a fixed visitor can be grouped by cookie. This mechanism leads to a reliable method for defining visits. However, Web users are becoming increasingly concerned about the use and misuse of cookies, and current browsers allow the user to disable the cookie mechanism. Therefore, this mechanism is of only limited use for the present purpose.
A method for tracking visitors to a Web site without using cookies, involves serving each Web component through the so-called cgi (Common Gateway Interface) mechanism. That is, the first time a visitor comes through the site, a unique number is assigned to that user. (This first time can be identified by the fact that the request does not include a visitor number mechanism, while all subsequent requests will). The requested page is now served through the cgi mechanism, which changes all the internal links on that page to the appropriate cgi links including the unique visitor number. Consequently, all subsequent requests will go through the cgi mechanism, and all the pages served will have their links correspondingly altered.
The visitor number now allows robust tracking of visitors.
2o Another mechanism, which we use in our current implementation, infers visit sequences from only the information in the log file. Hits are combined into a particular visit sequence if they are separated by less than a selectable time interval T (typically, ten minutes, although there is wide latitude for setting this interval) and they originate from the same host into a visit.
We have found that this scheme often works well. However, it can produce an erroneous result if two or more users, acting through the same host, request pages from the same Web site at the same time. This typically happens when user requests are grouped by a so-called "proxy" machine. The individual users request pages only from the proxy, while the proxy caches pages and requests new ones from the actual Web server. Because the Web server sees requests only from the proxy, this method may be unable to distinguish different visitors using the same proxy. We have found, however, that this situation tends to occur infrequently.
More advanced methods for grouping hits into visits can also be used. For example, if a hit has a referral field from a Web page of a different Web site, this can be taken as an indication that a new visit has started. Also, if two visitors using the same proxy visit the same Web site at the same time, they may still be viewing different parts of the Web site, and thus they can be distinguished by the actual web components they download.
Further methods can take advantage of additional information that the 1o browser communicates to the server. For example, the HTTP protocol allows for browsers to provide, among other kinds of information:
( 1 ) the type of browser used, which version, and what kind of platform;
(2) the screen resolution; and (3) which types of web component the browser is able to display.
Thus, users who differ in any of these fields can be distinguished.
In the following discussion, it is assumed that the information to be used for distinguishing users is combined in a user tag. With reference to FIG. 2, we now describe an exemplary procedure for assembling hits (indexed by the variable "hitnr") into respective visits (indexed by the variable "visitnr").
2o Lines are retrieved sequentially from the log (box 105). The new line so retrieved is the current hit. For each new hit, a visitor tag is composed (box 110). A
determination is then made, whether this visitor tag belongs to a new visit (box 115). That is, all of the currently open visits are recorded in working memory. The visit is new if the visitor tag does not belong to any of the currently open visits.
If it is determined that the visit is new, a new visitor-record data structure is created in working memory, and the current hit and hitnr are added to the data structure (box 120). If it is determined that the visit is not new, the hit and hitnr are added to the currently open visitor record identified by the visitor tag (box 125). (It should be noted in this regard that a former visitor who initiates a new visit is treated as a new visitor.) When each new visitor-record data structure is created, a sequential index number (visitnr) is assigned to the corresponding visit (box 130).
Next, a determination is made whether any of the currently open visits have expired. For each of the currently open visit records, the time of the last hit in that record is subtracted from the time of the current hit (box 135). The visit is deemed to have expired if the difference is greater than the threshold T. If the visit has expired, the record of that visit is added to the visit database on disk, and removed from working memory (box 140). The visitnr is included with the data that are sent to disk storage.
to Then, the next hit is retrieved, and the process is repeated. Hits with the same visit tag are added in chronological order so long as they are separated by less than the time interval T.
2.3. Building the shadow directory. The visit database is next processed into a new database resident on storage device 15, which we refer to as the shadow-directory database.
It is a general characteristic of files containing Web components that they are organized in a hierarchical file structure. A correspondence to this hierarchical file structure is built into the shadow-directory database, such that its records are 2o retrievable by the names of the corresponding files on the Web server.
By way of example, this correspondence can be achieved by using the same filenames, in the shadow-directory database, as the corresponding Web-page files.
Alternatively, as will be appreciated by those skilled in the art, such a correspondence is readily achieved using well-known data-base technologies to relate files of the shadow-directory database to corresponding Web-page files.
As a consequence, each file in the Web-site directory will have a unique counterpart in the shadow-directory database. The correspondence described above is useful for facilitating on-line retrieval of information from the shadow-directory database by operating it as a parallel Web site.
Each file in the shadow-directory database contains the compiled information pertaining to the hits made to the corresponding web component.
With reference to FIG. 3, we next describe how this information is compiled.
Each visit record is retrieved, in turn (box 145). For each of these visit records, we record information for each of the hits that it contains.
Specifically, we record the filename of the downloaded Web component (box 150) and other information concerning the hit, together with the hitnr index, the visitnr index, the filenames of the next and previous hits, and the time interval between the current hit and the next hit (box 155).
1o Once built, the shadow-directory database affords immediate access to all the hits to a particular Web component, by simply accessing the corresponding file.
The process described above for building the shadow directory may in some cases be undesirably slow because it calls for shadow-directory files to be opened and closed many times. For this reason, it will in at least some cases be advantageous to first annotate the hits with hitnr, visitnr, and "previous"
and "next"
fields, and then sort the hits per filename (i.e., according to the name of the requested Web file). This way each file of the shadow directory only needs to be opened and closed once. Then, all of the records associated with each given Web file can be loaded, en masse, into the corresponding shadow-directory file.
2o Thus, as shown, e.g., in FIG. 4, the shadow-directory data base is advantageously built by retrieving, in turn, each of these collections of sorted records (box 160). The filename of the respective Web component is recorded (box 165). The hit information is then recorded (box 170) for each of the annotated hit records in the collection.
Below is one illustrative line, with explanatory annotations, in the hypothetical shadow-directory file named "shadow/card.html":
poo1044-maxi.sc-ca-us.dialup.earthlink.net hostname - userid [not shown here]
1/05/1998 23:43:09 date and time /card.html file accessed 200 return code 14176 bytes transferred to /work.html previous file /contact/contact.html next file 78 time interval in seconds It should be noted that it is sufficient, e.g. for alternate embodiments of the invention, for the shadow directory data base simply to store visitnr indices, rather than to store complete visitor records. The complete records are instead readily retrievable, by visitnr, from the visit data base resident on the UWU server.
This scheme avoids duplication, and thus helps to conserve disk space.
2.4. Hierarchical Accounting. Further preprocessing is advantageously performed after the shadow directory has been built. In a first stage, as illustrated in FIG. 5, each shadow-directory file is processed in turn. For each of these files, the number of hits over time, the distribution of "previous" and "next" hits, and the distribution of time spent are computed and stored (box 175). It is also advantageous to store the index visitnr for each visit that contains a hit to the corresponding Web file. It should be noted in this regard that the visits to a file are typically fewer than the hits to a file, because a visitor may download a web component multiple times within a single visit.
In a second stage, with reference to FIG. 6, directory-level information is computed (step 176) for each directory in the file tree. This is done hierarchically, beginning with the sub-directories at the lowest levels of the tree. For each directory, there is recorded the total number of hits over time to all the files in the directory or any of its subdirectories. It is also advantageous to count the number of distinct visitors to the files in each directory by merging the visitnr indices of the visits to the files in the directory.
Since no such directory-level record specifically relates to any one file, an extra file is created (step 177) in each pertinent directory to contain this information.
(In our current implementation, we have found it convenient to designate each of these files by the name "UWU.info".) Thus, each UWU.info file contains, in particular, hit counts and visit counts accumulated over a period of time.
Thus, with reference to FIG. 7, directory-level information is provided in a file 190 for each directory (180, 180.1-180.3) in the Web file tree. In the figure, the corresponding directories of the shadow-directory database are numbered 180', 180.1'-180.3', respectively. The sub-directories at the lowest levels of the tree shown are numbered 180.3, 180.3'. To each Web file 185, there corresponds a file 185' in the shadow-directory database.
As a conceptual aide in the figure, the symbol that represents each file 190 is drawn as containing the symbols of any lower-level files 190 whose information it (wholly or partly) includes as a component element.
Claims
1. A method for building a database of information about visits to a Web site, the Web site comprising Web components organized in a file directory structure having directories and sub-directories, the method comprising:
a) retrieving raw hit records, each raw hit record pertaining to a particular Web component of the Web site;
b) extracting selected information from said raw hit records, thereby to create, from each raw hit record, a filtered hit record, the selected information including information identifying the Web component to which the respective hit record pertains;
c) distinguishing respective filtered hit records according to the visit to which each of them belongs, and associating a particular visit index with each filtered hit record;
d) storing the filtered hit records in a data storage device, said storing step comprising the further step of indexing said records such that each record is retrievable by reference to the filename of the corresponding Web component;
e) proceeding hierarchically from lowest sub-directories to highest directories, compiling for each sub-directory and directory a respective record of collective usage information pertaining to all Web components associated with such sub-directory or directory;
f) storing said collective-hit-information records in the filtered hit record data storage device, said storing step comprising the further step of indexing each said record such that it is retrievable by reference to the name of the corresponding sub-directory or directory; and h) organizing said filtered hit records and said collective-hit records as a further Web site.
a) retrieving raw hit records, each raw hit record pertaining to a particular Web component of the Web site;
b) extracting selected information from said raw hit records, thereby to create, from each raw hit record, a filtered hit record, the selected information including information identifying the Web component to which the respective hit record pertains;
c) distinguishing respective filtered hit records according to the visit to which each of them belongs, and associating a particular visit index with each filtered hit record;
d) storing the filtered hit records in a data storage device, said storing step comprising the further step of indexing said records such that each record is retrievable by reference to the filename of the corresponding Web component;
e) proceeding hierarchically from lowest sub-directories to highest directories, compiling for each sub-directory and directory a respective record of collective usage information pertaining to all Web components associated with such sub-directory or directory;
f) storing said collective-hit-information records in the filtered hit record data storage device, said storing step comprising the further step of indexing each said record such that it is retrievable by reference to the name of the corresponding sub-directory or directory; and h) organizing said filtered hit records and said collective-hit records as a further Web site.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA002349466A CA2349466C (en) | 1998-05-21 | 1999-03-29 | Method for characterizing and visualizing patterns of usage of a web site by network users |
| CA002349420A CA2349420C (en) | 1998-05-21 | 1999-03-29 | Method for characterizing and visualizing patterns of usage of a web site by network users |
| CA002349471A CA2349471C (en) | 1998-05-21 | 1999-03-29 | Method for characterizing and visualizing patterns of usage of a web site by network users |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US09/082,792 US6182097B1 (en) | 1998-05-21 | 1998-05-21 | Method for characterizing and visualizing patterns of usage of a web site by network users |
| US09/082,792 | 1998-05-21 |
Related Child Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002349466A Division CA2349466C (en) | 1998-05-21 | 1999-03-29 | Method for characterizing and visualizing patterns of usage of a web site by network users |
| CA002349420A Division CA2349420C (en) | 1998-05-21 | 1999-03-29 | Method for characterizing and visualizing patterns of usage of a web site by network users |
| CA002349471A Division CA2349471C (en) | 1998-05-21 | 1999-03-29 | Method for characterizing and visualizing patterns of usage of a web site by network users |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2267451A1 CA2267451A1 (en) | 1999-11-21 |
| CA2267451C true CA2267451C (en) | 2002-04-30 |
Family
ID=22173497
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002267451A Expired - Fee Related CA2267451C (en) | 1998-05-21 | 1999-03-29 | Method for characterizing and visualizing patterns of usage of a web site by network users |
Country Status (4)
| Country | Link |
|---|---|
| US (2) | US6182097B1 (en) |
| EP (1) | EP0959419A3 (en) |
| JP (1) | JP4024426B2 (en) |
| CA (1) | CA2267451C (en) |
Families Citing this family (126)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6108637A (en) * | 1996-09-03 | 2000-08-22 | Nielsen Media Research, Inc. | Content display monitor |
| US7523191B1 (en) * | 2000-06-02 | 2009-04-21 | Yahoo! Inc. | System and method for monitoring user interaction with web pages |
| JP3485253B2 (en) * | 1999-06-18 | 2004-01-13 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Information processing method, information terminal support server, storage medium for storing information processing program |
| AUPQ206399A0 (en) | 1999-08-06 | 1999-08-26 | Imr Worldwide Pty Ltd. | Network user measurement system and method |
| US7035926B1 (en) * | 1999-11-30 | 2006-04-25 | International Business Machines Corporation | Real-time monitoring of web activities |
| US6864904B1 (en) * | 1999-12-06 | 2005-03-08 | Girafa.Com Inc. | Framework for providing visual context to www hyperlinks |
| FR2807604B1 (en) * | 2000-04-07 | 2003-08-01 | Netvalue | METHOD FOR MEASURING ACTIVITY OF USERS OF WAP TERMINALS |
| US6996612B1 (en) * | 1999-12-30 | 2006-02-07 | Vignette Corporation | Method of providing information related to activity of a user and a data processing system program product |
| EP1113381A1 (en) * | 1999-12-30 | 2001-07-04 | Alexander Dresen | Method and system for tracking web usage and internet applications |
| US6691104B1 (en) * | 2000-01-12 | 2004-02-10 | International Business Machines Corporation | System and method for personalizing and applying a post processing tool system |
| AU2001217524A1 (en) | 2000-01-12 | 2001-07-24 | Jupiter Media Metrix, Inc. | System and method for estimating prevalence of digital content on the world-wide-web |
| US6665658B1 (en) * | 2000-01-13 | 2003-12-16 | International Business Machines Corporation | System and method for automatically gathering dynamic content and resources on the world wide web by stimulating user interaction and managing session information |
| US20020004749A1 (en) | 2000-02-09 | 2002-01-10 | Froseth Barrie R. | Customized food selection, ordering and distribution system and method |
| AU2001245481A1 (en) * | 2000-03-07 | 2001-09-17 | Hotlens.Com Inc. | Server-side web browsing and multiple lens system, method and apparatus |
| WO2001071523A1 (en) * | 2000-03-20 | 2001-09-27 | Hotlens.Com Inc. | Transparent user and session management for web applications |
| US7260774B2 (en) * | 2000-04-28 | 2007-08-21 | Inceptor, Inc. | Method & system for enhanced web page delivery |
| US8086697B2 (en) * | 2005-06-28 | 2011-12-27 | Claria Innovations, Llc | Techniques for displaying impressions in documents delivered over a computer network |
| US7475404B2 (en) | 2000-05-18 | 2009-01-06 | Maquis Techtrix Llc | System and method for implementing click-through for browser executed software including ad proxy and proxy cookie caching |
| US7962603B1 (en) | 2000-06-06 | 2011-06-14 | Nobuyoshi Morimoto | System and method for identifying individual users accessing a web site |
| US6882977B1 (en) * | 2000-07-31 | 2005-04-19 | Hewlett-Packard Development Company, L.P. | Method and facility for displaying customer activity and value |
| US6904461B1 (en) * | 2000-10-19 | 2005-06-07 | Unitel Technologies, Inc. | Method for making time-sensitive determinations of traffic intensity for a visitable site |
| GB0027280D0 (en) | 2000-11-08 | 2000-12-27 | Malcolm Peter | An information management system |
| US20020062223A1 (en) * | 2000-11-21 | 2002-05-23 | Netiq Corporation | System and method for adding network traffic data to a database of network traffic data |
| US6928474B2 (en) * | 2000-12-14 | 2005-08-09 | Honeywell International, Inc. | Using a probability associative matrix algorithm to modify web pages |
| US20060288212A1 (en) * | 2001-03-20 | 2006-12-21 | Gutenberg Printing Llc | Transparent user and session management for web applications |
| US20020186237A1 (en) * | 2001-05-16 | 2002-12-12 | Michael Bradley | Method and system for displaying analytics about a website and its contents |
| US7139972B2 (en) * | 2001-07-26 | 2006-11-21 | International Business Machines Corporation | Preemptive downloading of web pages with terms associated with user interest keywords |
| US6963874B2 (en) * | 2002-01-09 | 2005-11-08 | Digital River, Inc. | Web-site performance analysis system and method utilizing web-site traversal counters and histograms |
| US6973492B2 (en) * | 2001-09-07 | 2005-12-06 | International Business Machines Corporation | Method and apparatus for collecting page load abandons in click stream data |
| US7376730B2 (en) * | 2001-10-10 | 2008-05-20 | International Business Machines Corporation | Method for characterizing and directing real-time website usage |
| US20030101160A1 (en) * | 2001-11-26 | 2003-05-29 | International Business Machines Corporation | Method for safely accessing shared storage |
| US7216056B2 (en) * | 2001-12-06 | 2007-05-08 | C-Live, Inc. | Access log analyzer and access log analyzing method |
| US20030131106A1 (en) * | 2002-01-09 | 2003-07-10 | Stephane Kasriel | Web-page performance toolbar |
| US20030131097A1 (en) * | 2002-01-09 | 2003-07-10 | Stephane Kasriel | Interactive path analysis |
| US7631035B2 (en) * | 2002-01-09 | 2009-12-08 | Digital River, Inc. | Path-analysis toolbar |
| US20030128231A1 (en) * | 2002-01-09 | 2003-07-10 | Stephane Kasriel | Dynamic path analysis |
| US20030149567A1 (en) * | 2002-02-04 | 2003-08-07 | Tony Schmitz | Method and system for using natural language in computer resource utilization analysis via a communications network |
| US20030202009A1 (en) * | 2002-04-24 | 2003-10-30 | Stephane Kasriel | Integration toolbar |
| US7069515B1 (en) * | 2002-05-21 | 2006-06-27 | Claria Corporation | Method and apparatus for displaying messages in computer systems |
| US7792844B2 (en) | 2002-06-28 | 2010-09-07 | Adobe Systems Incorporated | Capturing and presenting site visitation path data |
| US20040098229A1 (en) * | 2002-06-28 | 2004-05-20 | Brett Error | Efficient click-stream data collection |
| US8271778B1 (en) | 2002-07-24 | 2012-09-18 | The Nielsen Company (Us), Llc | System and method for monitoring secure data on a network |
| SE522178C2 (en) | 2002-09-12 | 2004-01-20 | Admeta Ab | Device for indicating the download of a data record |
| US7853684B2 (en) * | 2002-10-15 | 2010-12-14 | Sas Institute Inc. | System and method for processing web activity data |
| US7603341B2 (en) | 2002-11-05 | 2009-10-13 | Claria Corporation | Updating the content of a presentation vehicle in a computer network |
| US20040122939A1 (en) * | 2002-12-19 | 2004-06-24 | Russell Perkins | Method of obtaining economic data based on web site visitor data |
| US7441195B2 (en) * | 2003-03-04 | 2008-10-21 | Omniture, Inc. | Associating website clicks with links on a web page |
| US7603373B2 (en) * | 2003-03-04 | 2009-10-13 | Omniture, Inc. | Assigning value to elements contributing to business success |
| US20050216844A1 (en) * | 2004-03-03 | 2005-09-29 | Error Brett M | Delayed transmission of website usage data |
| US7519704B2 (en) * | 2003-04-28 | 2009-04-14 | International Business Machines Corporation | Filtering non-invasive data collection |
| US7480716B2 (en) * | 2003-04-28 | 2009-01-20 | International Business Machines Corporation | Non-invasive collection of data |
| US7991732B2 (en) | 2005-06-03 | 2011-08-02 | Adobe Systems Incorporated | Incrementally adding segmentation criteria to a data set |
| US9081863B2 (en) | 2005-06-03 | 2015-07-14 | Adobe Systems Incorporated | One-click segmentation definition |
| US8255514B2 (en) | 2003-11-04 | 2012-08-28 | Covenant Eyes, Inc. | Internet use monitoring system and method |
| US8170912B2 (en) | 2003-11-25 | 2012-05-01 | Carhamm Ltd., Llc | Database structure and front end |
| US20050198315A1 (en) * | 2004-02-13 | 2005-09-08 | Wesley Christopher W. | Techniques for modifying the behavior of documents delivered over a computer network |
| US7584435B2 (en) * | 2004-03-03 | 2009-09-01 | Omniture, Inc. | Web usage overlays for third-party web plug-in content |
| US7707265B2 (en) * | 2004-05-15 | 2010-04-27 | International Business Machines Corporation | System, method, and service for interactively presenting a summary of a web site |
| US8255413B2 (en) * | 2004-08-19 | 2012-08-28 | Carhamm Ltd., Llc | Method and apparatus for responding to request for information-personalization |
| US8078602B2 (en) * | 2004-12-17 | 2011-12-13 | Claria Innovations, Llc | Search engine for a computer network |
| US7693863B2 (en) * | 2004-12-20 | 2010-04-06 | Claria Corporation | Method and device for publishing cross-network user behavioral data |
| US20060235965A1 (en) * | 2005-03-07 | 2006-10-19 | Claria Corporation | Method for quantifying the propensity to respond to an advertisement |
| US8645941B2 (en) | 2005-03-07 | 2014-02-04 | Carhamm Ltd., Llc | Method for attributing and allocating revenue related to embedded software |
| US8073866B2 (en) * | 2005-03-17 | 2011-12-06 | Claria Innovations, Llc | Method for providing content to an internet user based on the user's demonstrated content preferences |
| US8538969B2 (en) * | 2005-06-03 | 2013-09-17 | Adobe Systems Incorporated | Data format for website traffic statistics |
| US7941394B2 (en) | 2005-06-03 | 2011-05-10 | Adobe Systems Incorporated | User interface providing summary information or a status pane in a web analytics tool |
| US8578041B2 (en) * | 2005-06-03 | 2013-11-05 | Adobe Systems Incorporated | Variable sampling rates for website visitation analysis |
| US7761457B2 (en) * | 2005-06-06 | 2010-07-20 | Adobe Systems Incorporated | Creation of segmentation definitions |
| US7908254B2 (en) * | 2005-06-10 | 2011-03-15 | Hewlett-Packard Development Company, L.P. | Identifying characteristics in sets of organized items |
| US20070061869A1 (en) * | 2005-07-25 | 2007-03-15 | Dehaas Ronald J | Access of Internet use for a selected user |
| US7383334B2 (en) * | 2005-10-28 | 2008-06-03 | Omniture, Inc. | Comparison of website visitation data sets generated from using different navigation tools |
| US20070156923A1 (en) * | 2005-12-29 | 2007-07-05 | Webex Communications, Inc. | Methods and apparatuses for tracking progress of an invited participant |
| US7941525B1 (en) | 2006-04-01 | 2011-05-10 | ClickTale, Ltd. | Method and system for monitoring an activity of a user |
| US8775919B2 (en) * | 2006-04-25 | 2014-07-08 | Adobe Systems Incorporated | Independent actionscript analytics tools and techniques |
| US7992135B1 (en) * | 2006-06-26 | 2011-08-02 | Adobe Systems Incorporated | Certification of server-side partner plug-ins for analytics and privacy protection |
| US8352917B2 (en) * | 2006-06-26 | 2013-01-08 | Adobe Systems Incorporated | Web-beacon plug-ins and their certification |
| US8583772B2 (en) | 2008-08-14 | 2013-11-12 | International Business Machines Corporation | Dynamically configurable session agent |
| US8949406B2 (en) * | 2008-08-14 | 2015-02-03 | International Business Machines Corporation | Method and system for communication between a client system and a server system |
| US8868533B2 (en) | 2006-06-30 | 2014-10-21 | International Business Machines Corporation | Method and apparatus for intelligent capture of document object model events |
| US8127000B2 (en) | 2006-06-30 | 2012-02-28 | Tealeaf Technology, Inc. | Method and apparatus for monitoring and synchronizing user interface events with network data |
| US7644375B1 (en) | 2006-09-18 | 2010-01-05 | Adobe Systems Incorporated | Dynamic path flow reports |
| US8620952B2 (en) | 2007-01-03 | 2013-12-31 | Carhamm Ltd., Llc | System for database reporting |
| US9405830B2 (en) | 2007-02-28 | 2016-08-02 | Aol Inc. | Personalization techniques using image clouds |
| US10169781B1 (en) | 2007-03-07 | 2019-01-01 | The Nielsen Company (Us), Llc | Method and system for generating information about portable device advertising |
| US20080299895A1 (en) * | 2007-05-31 | 2008-12-04 | University Of Georgia Research Foundation, Inc. | Systems and Methods for Providing Information to Exhibition Visitors |
| US8042055B2 (en) | 2007-08-31 | 2011-10-18 | Tealeaf Technology, Inc. | Replaying captured network interactions |
| US9946722B2 (en) | 2007-11-30 | 2018-04-17 | Red Hat, Inc. | Generating file usage information |
| US7890511B2 (en) * | 2008-02-05 | 2011-02-15 | Blue Coat Systems, Inc. | System and method for conducting network analytics |
| US20090327913A1 (en) * | 2008-06-27 | 2009-12-31 | Microsoft Corporation | Using web revisitation patterns to support web interaction |
| US8234582B1 (en) | 2009-02-03 | 2012-07-31 | Amazon Technologies, Inc. | Visualizing object behavior |
| US8341540B1 (en) | 2009-02-03 | 2012-12-25 | Amazon Technologies, Inc. | Visualizing object behavior |
| US8250473B1 (en) * | 2009-02-03 | 2012-08-21 | Amazon Technoloies, Inc. | Visualizing object behavior |
| US8930818B2 (en) * | 2009-03-31 | 2015-01-06 | International Business Machines Corporation | Visualization of website analytics |
| US9934320B2 (en) | 2009-03-31 | 2018-04-03 | International Business Machines Corporation | Method and apparatus for using proxy objects on webpage overlays to provide alternative webpage actions |
| US8914736B2 (en) | 2010-03-30 | 2014-12-16 | International Business Machines Corporation | On-page manipulation and real-time replacement of content |
| US8214390B2 (en) * | 2009-06-03 | 2012-07-03 | Yahoo! Inc. | Binary interest vector for better audience targeting |
| US8533532B2 (en) | 2010-06-23 | 2013-09-10 | International Business Machines Corporation | System identifying and inferring web session events |
| US9031863B2 (en) * | 2010-08-19 | 2015-05-12 | Yahoo! Inc. | Contextual advertising with user features |
| US9058323B2 (en) | 2010-12-30 | 2015-06-16 | Ss8 Networks, Inc. | System for accessing a set of communication and transaction data associated with a user of interest sourced from multiple different network carriers and for enabling multiple analysts to independently and confidentially access the set of communication and transaction data |
| US8938534B2 (en) | 2010-12-30 | 2015-01-20 | Ss8 Networks, Inc. | Automatic provisioning of new users of interest for capture on a communication network |
| US8972612B2 (en) | 2011-04-05 | 2015-03-03 | SSB Networks, Inc. | Collecting asymmetric data and proxy data on a communication network |
| US9563714B2 (en) | 2011-06-16 | 2017-02-07 | Microsoft Technology Licensing Llc. | Mapping selections between a browser and the original file fetched from a web server |
| US9753699B2 (en) | 2011-06-16 | 2017-09-05 | Microsoft Technology Licensing, Llc | Live browser tooling in an integrated development environment |
| US9460224B2 (en) * | 2011-06-16 | 2016-10-04 | Microsoft Technology Licensing Llc. | Selection mapping between fetched files and source files |
| US10061860B2 (en) * | 2011-07-29 | 2018-08-28 | Oath Inc. | Method and system for personalizing web page layout |
| US20130091214A1 (en) * | 2011-10-08 | 2013-04-11 | Broadcom Corporation | Media social network |
| US9350762B2 (en) | 2012-09-25 | 2016-05-24 | Ss8 Networks, Inc. | Intelligent feedback loop to iteratively reduce incoming network data for analysis |
| US9635094B2 (en) | 2012-10-15 | 2017-04-25 | International Business Machines Corporation | Capturing and replaying application sessions using resource files |
| US9536108B2 (en) | 2012-10-23 | 2017-01-03 | International Business Machines Corporation | Method and apparatus for generating privacy profiles |
| US8793236B2 (en) | 2012-11-01 | 2014-07-29 | Adobe Systems Incorporated | Method and apparatus using historical influence for success attribution in network site activity |
| US9535720B2 (en) | 2012-11-13 | 2017-01-03 | International Business Machines Corporation | System for capturing and replaying screen gestures |
| US10474735B2 (en) | 2012-11-19 | 2019-11-12 | Acoustic, L.P. | Dynamic zooming of content with overlays |
| US9185435B2 (en) | 2013-06-25 | 2015-11-10 | The Nielsen Company (Us), Llc | Methods and apparatus to characterize households with media meter data |
| US20150067472A1 (en) * | 2013-08-28 | 2015-03-05 | F5 Networks, Inc. | Web browser fingerprinting |
| US9277265B2 (en) | 2014-02-11 | 2016-03-01 | The Nielsen Company (Us), Llc | Methods and apparatus to calculate video-on-demand and dynamically inserted advertisement viewing probability |
| US9830593B2 (en) | 2014-04-26 | 2017-11-28 | Ss8 Networks, Inc. | Cryptographic currency user directory data and enhanced peer-verification ledger synthesis through multi-modal cryptographic key-address mapping |
| US9516001B2 (en) | 2014-09-30 | 2016-12-06 | The Nielsen Company (Us), Llc | Methods and apparatus to identify media distributed via a network |
| US10219039B2 (en) | 2015-03-09 | 2019-02-26 | The Nielsen Company (Us), Llc | Methods and apparatus to assign viewers to media meter data |
| US9848224B2 (en) | 2015-08-27 | 2017-12-19 | The Nielsen Company(Us), Llc | Methods and apparatus to estimate demographics of a household |
| US10546036B2 (en) * | 2016-02-26 | 2020-01-28 | International Business Machines Corporation | Web browser having improved navigational functionality |
| US10476790B2 (en) | 2016-12-19 | 2019-11-12 | Cisco Technology, Inc. | Service chaining at a network device |
| US10594577B2 (en) | 2016-12-19 | 2020-03-17 | Cisco Technology, Inc. | Sorting and displaying network traffic analytics |
| US10791355B2 (en) | 2016-12-20 | 2020-09-29 | The Nielsen Company (Us), Llc | Methods and apparatus to determine probabilistic media viewing metrics |
| US11082312B2 (en) | 2017-10-04 | 2021-08-03 | Cisco Technology, Inc. | Service chaining segmentation analytics |
| US10965598B1 (en) | 2017-10-04 | 2021-03-30 | Cisco Technology, Inc. | Load balancing in a service chain |
| US10965596B2 (en) | 2017-10-04 | 2021-03-30 | Cisco Technology, Inc. | Hybrid services insertion |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0991308A (en) * | 1995-09-28 | 1997-04-04 | Nippon Telegr & Teleph Corp <Ntt> | Information search system |
| JPH09153059A (en) * | 1995-11-30 | 1997-06-10 | Matsushita Electric Ind Co Ltd | History display method |
| US5751956A (en) * | 1996-02-21 | 1998-05-12 | Infoseek Corporation | Method and apparatus for redirection of server external hyper-link references |
| JPH09282210A (en) * | 1996-04-15 | 1997-10-31 | Hitachi Ltd | Method and system for monitoring data base |
| JP3437373B2 (en) * | 1996-04-16 | 2003-08-18 | 株式会社野村総合研究所 | Information usage status grasping method and information providing system using the method |
| US5935207A (en) * | 1996-06-03 | 1999-08-10 | Webtv Networks, Inc. | Method and apparatus for providing remote site administrators with user hits on mirrored web sites |
| JP3996673B2 (en) * | 1996-08-08 | 2007-10-24 | 義宇 江 | Information collection method and information collection system on the Internet |
| US5958008A (en) * | 1996-10-15 | 1999-09-28 | Mercury Interactive Corporation | Software system and associated methods for scanning and mapping dynamically-generated web documents |
| US5974572A (en) * | 1996-10-15 | 1999-10-26 | Mercury Interactive Corporation | Software system and methods for generating a load test using a server access log |
| US5870559A (en) * | 1996-10-15 | 1999-02-09 | Mercury Interactive | Software system and associated methods for facilitating the analysis and management of web sites |
| US5875446A (en) * | 1997-02-24 | 1999-02-23 | International Business Machines Corporation | System and method for hierarchically grouping and ranking a set of objects in a query context based on one or more relationships |
| JPH1185654A (en) * | 1997-09-12 | 1999-03-30 | Matsushita Electric Ind Co Ltd | Virtual www server device and camera controllable www server device |
-
1998
- 1998-05-21 US US09/082,792 patent/US6182097B1/en not_active Expired - Lifetime
-
1999
- 1999-03-29 CA CA002267451A patent/CA2267451C/en not_active Expired - Fee Related
- 1999-05-10 EP EP99303604A patent/EP0959419A3/en not_active Ceased
- 1999-05-21 JP JP14088899A patent/JP4024426B2/en not_active Expired - Fee Related
-
2000
- 2000-06-06 US US09/588,488 patent/US6449604B1/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| JP2000040046A (en) | 2000-02-08 |
| CA2267451A1 (en) | 1999-11-21 |
| JP4024426B2 (en) | 2007-12-19 |
| US6449604B1 (en) | 2002-09-10 |
| EP0959419A2 (en) | 1999-11-24 |
| US6182097B1 (en) | 2001-01-30 |
| EP0959419A3 (en) | 2003-01-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2267451C (en) | Method for characterizing and visualizing patterns of usage of a web site by network users | |
| US7596533B2 (en) | Personalized multi-service computer environment | |
| KR100612711B1 (en) | Gathering enriched web server activity data of cached web content | |
| CA2530565C (en) | Server architecture and methods for persistently storing and serving event data | |
| Wu et al. | Speedtracer: A web usage mining and analysis tool | |
| US6401118B1 (en) | Method and computer program product for an online monitoring search engine | |
| US6983320B1 (en) | System, method and computer program product for analyzing e-commerce competition of an entity by utilizing predetermined entity-specific metrics and analyzed statistics from web pages | |
| US7676574B2 (en) | Internet website traffic flow analysis | |
| JP4025379B2 (en) | Search system | |
| US7013323B1 (en) | System and method for developing and interpreting e-commerce metrics by utilizing a list of rules wherein each rule contain at least one of entity-specific criteria | |
| Abd Wahab et al. | Data pre-processing on web server logs for generalized association rules mining algorithm | |
| US8095644B2 (en) | System and method for analyzing web paths | |
| EP1716508A1 (en) | Method and system for recording search trails across one or more search engines in a communications network | |
| CA2349471C (en) | Method for characterizing and visualizing patterns of usage of a web site by network users | |
| AU2004313991B2 (en) | Method and system for recording search trails across one or more search engines in a communications network | |
| Meo et al. | Employing a Data Mining Query Language to the Discovery of Interesting Patterns in WEB Logs | |
| Wright | Constituencies for users: How to develop them by interpreting logs of web site access | |
| Ivory et al. | Usability Testing Methods | |
| Biswal | Website Optimization through Mining User Navigational Pattern | |
| EP1510934A1 (en) | Inter-host web log reporting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |
Effective date: 20180329 |