US20010037374A1 - Internet server system test and measurement - Google Patents

Internet server system test and measurement Download PDF

Info

Publication number
US20010037374A1
US20010037374A1 US09/825,403 US82540301A US2001037374A1 US 20010037374 A1 US20010037374 A1 US 20010037374A1 US 82540301 A US82540301 A US 82540301A US 2001037374 A1 US2001037374 A1 US 2001037374A1
Authority
US
United States
Prior art keywords
data
server
servers
database
business
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US09/825,403
Inventor
Kenneth Shrum
Thomas Bartz
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Agilent Technologies Inc
Original Assignee
Agilent Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Agilent Technologies Inc filed Critical Agilent Technologies Inc
Priority to US09/825,403 priority Critical patent/US20010037374A1/en
Assigned to AGILENT TECHNOLOGIES, INC. reassignment AGILENT TECHNOLOGIES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BARTZ, THOMAS G., SHRUM, KENNETH W.
Publication of US20010037374A1 publication Critical patent/US20010037374A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/22—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/2294—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing by remote test
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G06F11/3414—Workload generation, e.g. scripts, playback
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G06F11/3495—Performance evaluation by tracing or monitoring for systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/50—Testing arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0227—Filtering policies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0281—Proxies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/14—Session management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/329—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40—Network security protocols
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G06F11/3476—Data logging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/805—Real-time
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/81—Threshold
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/875—Monitoring of systems including the internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/04—Processing captured monitoring data, e.g. for logfile generation
    • H04L43/045—Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/06—Generation of reports
    • H04L43/067—Generation of reports using time frame reporting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0823—Errors, e.g. transmission errors
    • H04L43/0829—Packet loss
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/16—Threshold monitoring

Definitions

  • Internet server systems are now a critical component to many successful businesses. Many Internet server systems are configured to function as e-commerce web sites where computer users can purchase goods and services. The efficient and reliable operation of the e-commerce web site is vital to many businesses.
  • test systems In response to the need for efficient and reliable e-commerce web sites, test systems have been developed to ensure that the web site is operating within tolerable thresholds. These test systems perform automated tests using transactions that were previously recorded. To the web site, the test transaction appears like another customer. These test user transactions are able to determine how long a typical transaction takes and whether or not the e-commerce web site is responding at all.
  • test systems treat the Internet server system that provides the e-commerce web site as a black box—meaning that the test system sends in stimulus and measures response. Test systems do not effectively correlate user test results with internal performance measurements from the Internet server system. If there is a problem, the test system does not effectively isolate the responsible component within the Internet server system.
  • the invention addresses the above problems with a software solution that delivers key business and financial measures in near real time, providing information on current (“live”) business operations.
  • An e-commerce installation is typically constructed from several tiers. These tiers may or may not be physically realized as separate systems.
  • An Internet server system would typically comprises a web servers tier, a session servers tier, a transaction servers tier and a database servers tier. Each of these tiers potentially generates information detailing operations as they are performed.
  • the invention monitors in near real time the business and financial performance of the e-commerce business.
  • the invention reads the log files from each of the tiers, makes queries to databases, and interacts with the e-commerce infrastructure. This information is presented as measurements varying overtime, where the measurements are comprised of key business and financial measures such as the number of new orders per minute, the dollar volume of completed orders per minute, the number of abandoned “shopping carts” per minute, lost sales due to system errors, etc.
  • the invention assesses and reports key business and financial measurements.
  • the system of the invention delivers these measurements in near-real time allowing e-commerce businesses to quickly determine current financial and business performance. This in turn allows quick detection and response to variances.
  • FIG. 1 is a diagram illustrating a four tier construction of an Internet server system.
  • An e-commerce installation is typically constructed from several tiers.
  • FIG. 1 shows a four tier construction of an Internet server system. This construction typically comprises a web servers tier 101 , a session servers tier 102 , a transaction servers tier 103 and a database servers tier 104 .
  • the web servers 101 are responsible for handling direct customer interaction with the e-commerce web pages and delivering content back to the customer.
  • a session server 102 keeps track of individual customers and what actions those customers are performing.
  • the e-commerce transactions are supported by the transaction servers 103 and the database servers 104 manage the e-commerce database. Each of these tiers potentially generates information detailing operations as they are performed. Also shown in FIG.
  • the agents 105 are shown connected to each of the elements in the four tiers. For purposes of this example, an agent 105 is shown connected to each server in each tier, but it is understood that agents 105 are connected to those servers in the tiers 101 , 102 , 103 , 104 as desired by a user.
  • the agents 105 are software programs that run on each desired server. Agent programs 105 are termed “tests”.
  • the invention monitors in near real time the business and financial performance of the e-commerce business.
  • Applications and computer systems or subsystems generate log files in their normal course of operation.
  • a web server in the web server tier 101 generates a log file containing hit rates, client/server errors, error rates, etc. which can be used to indicate health and performance of the server.
  • log files generated by any subsystem in the tiers 101 , 102 , 103 , 104 track what occurs in that portion of the system. Accordingly, log files will contain data regarding events, state, transactions and anything of interest. It is understood that a log file may contain any type or class of data dependent upon programming.
  • Agents 105 associated with the web servers tier 101 are web server log readers. Examples of web servers 106 are Netscape, Apache, MicroSoft/IIS, America On-Line, etc., and include generic web servers. Events in a web server 106 log file include but are not limited to hit rate, hosts served, data volume, error rates, log file size and system load. System load includes but is not limited to CPU utilization memory usage and I/O transfers.
  • Agents 105 associated with the session servers tier 102 are session server application tests and log readers. Examples of sessions servers 107 are session management process tests. Events in a session server 107 log file include but are not limited to users served, interaction manager (IM) restarts, IM status, IM load database connectivity, error rates and system load.
  • IM interaction manager
  • Agents 105 associated with the transaction servers tier 103 are transaction server application tests and log readers.
  • Transaction servers 108 process information on a variety of levels, including system level performance to transaction process count and availability, sales and orders, shopping statistics, payment service and order fulfillment (shipping).
  • Events in a transaction server 108 log file may include but are not limited to inventory status, system load, open and discarded cart rates, shoppers in store, authorized, cancelled and declined data and payment service errors.
  • Transaction servers 108 also compile and maintain databases.
  • the agents 105 associated with transaction servers 108 query the transaction server databases using standard SQL database queries. It is understood that any type of database query capable of obtaining data from a database can be used. Examples of data in a transaction server 108 data base may include but are not limited to sales per hour, items sold per hour, new order rates, processed order rates, fulfilled order rates orders awaiting authorization and orders awaiting shipping.
  • Agents 105 associated with the database servers tier 104 are database server tests.
  • Database servers 109 process information and create and maintain databases.
  • the agents 105 associated with database servers 109 query the database server databases using standard SQL database queries. It is understood that any type of database query capable of obtaining data from a database can be used. While any data contained in the database can be retrieved, for the examples used in accordance with the invention, the pertinent data to query is health information such as number of the transactions that have occurred, the number of users, sales volume and other application or business health indicators of the e-commerce website which the database may contain.
  • each agent 105 is shown as being associated with a particular server.
  • the agents are software programs, or tests, that are resident on the servers due to the need for direct access to the relevant log files or databases. It is understood, that a particular agent 105 will be programmed to read the format, or formats, of log files and/or databases resident on a particular server. For example, if the agent 105 is interested in events relating to sale amount, it knows that column four (4) of the log file contains sale amount data. The agent 105 retrieves the appropriate data from the log files or databases and sends that information to a user site.
  • the user site is an Internet service management solution such as the Firehunter® products offered by Agilent Technologies that enables a user to measure and monitor an e-commerce solution's performance. It is understood that the invention is not limited to use with Firehuntere and can be used in any manner capable of displaying data.
  • the data from the agents 105 is sent to a user site via means capable of transferring data.
  • the data is transmitted via the Internet or an intranet. It is understood that any network means can be used to transmit data from the agents 105 to a user site. It is also considered that a direct connection be used to transmit data from the agents to the user site.
  • the invention reads the log files from each of the tiers 101 , 102 , 103 , 104 in which there is an agent, makes queries to the database, and interacts with the e-commerce infrastructure.
  • This information is presented as measurements varying overtime, where the measurements are comprised of system infrastructure (such as load or error information) and key business and financial measures such as the number of new orders per minute, the dollar volume of completed orders per minute, the number of abandoned “shopping carts” per minute, etc. Gathering these infrastructural and business measurements together into a system like Firehunter® then enables correlated results, such as lost sales due to system errors and the impact of system load on the rate and number of business transactions.
  • the invention assesses and reports key business and financial measurements.
  • the system of the invention delivers these measurements in real time allowing e-commerce businesses to quickly determine current financial and business performance. This in turn allows quick detection and response to variances.

Abstract

A software product for monitoring the performance of an e-commerce business by reading log files from a plurality of tiers, making queries to a database, and interacting with an e-commerce infrastructure. Resulting information is presented as measurements varying over time, where the measurements are business and financial measures.

Description

    RELATED APPLICATIONS
  • This application is a non-provisional application based on a provisional application, Ser. No. 60/200,295, filed Apr. 28, 2000.[0001]
    • BACKGROUND OF THE INVENTION
  • Internet server systems are now a critical component to many successful businesses. Many Internet server systems are configured to function as e-commerce web sites where computer users can purchase goods and services. The efficient and reliable operation of the e-commerce web site is vital to many businesses. [0002]
  • In response to the need for efficient and reliable e-commerce web sites, test systems have been developed to ensure that the web site is operating within tolerable thresholds. These test systems perform automated tests using transactions that were previously recorded. To the web site, the test transaction appears like another customer. These test user transactions are able to determine how long a typical transaction takes and whether or not the e-commerce web site is responding at all. [0003]
  • Unfortunately, current test systems treat the Internet server system that provides the e-commerce web site as a black box—meaning that the test system sends in stimulus and measures response. Test systems do not effectively correlate user test results with internal performance measurements from the Internet server system. If there is a problem, the test system does not effectively isolate the responsible component within the Internet server system. [0004]
  • Current test systems also fail to correlate system testing and performance data with business performance data. Business performance data may only be produced in weekly or monthly reports. If the web site operator receives an alarm from a test system, another system must be used to assess the financial damage due to the system error. The use of multiple systems is complex and time consuming. [0005]
    • SUMMARY
  • The invention addresses the above problems with a software solution that delivers key business and financial measures in near real time, providing information on current (“live”) business operations. [0006]
  • An e-commerce installation is typically constructed from several tiers. These tiers may or may not be physically realized as separate systems. An Internet server system would typically comprises a web servers tier, a session servers tier, a transaction servers tier and a database servers tier. Each of these tiers potentially generates information detailing operations as they are performed. [0007]
  • The invention monitors in near real time the business and financial performance of the e-commerce business. The invention reads the log files from each of the tiers, makes queries to databases, and interacts with the e-commerce infrastructure. This information is presented as measurements varying overtime, where the measurements are comprised of key business and financial measures such as the number of new orders per minute, the dollar volume of completed orders per minute, the number of abandoned “shopping carts” per minute, lost sales due to system errors, etc. [0008]
  • The invention assesses and reports key business and financial measurements. The system of the invention delivers these measurements in near-real time allowing e-commerce businesses to quickly determine current financial and business performance. This in turn allows quick detection and response to variances.[0009]
    • DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a diagram illustrating a four tier construction of an Internet server system.[0010]
    • DETAILED DESCRIPTION OF THE INVENTION
  • An e-commerce installation is typically constructed from several tiers. [0011]
  • These tiers may or may not be physically realized as separate systems. A preferred embodiment of the invention is shown in FIG. 1. FIG. 1 shows a four tier construction of an Internet server system. This construction typically comprises a [0012] web servers tier 101, a session servers tier 102, a transaction servers tier 103 and a database servers tier 104. The web servers 101 are responsible for handling direct customer interaction with the e-commerce web pages and delivering content back to the customer. A session server 102 keeps track of individual customers and what actions those customers are performing. The e-commerce transactions are supported by the transaction servers 103 and the database servers 104 manage the e-commerce database. Each of these tiers potentially generates information detailing operations as they are performed. Also shown in FIG. 1 is a plurality of agents 105. The agents 105 are shown connected to each of the elements in the four tiers. For purposes of this example, an agent 105 is shown connected to each server in each tier, but it is understood that agents 105 are connected to those servers in the tiers 101,102,103,104 as desired by a user. The agents 105 are software programs that run on each desired server. Agent programs 105 are termed “tests”.
  • The invention monitors in near real time the business and financial performance of the e-commerce business. Applications and computer systems or subsystems generate log files in their normal course of operation. For example, a web server in the [0013] web server tier 101 generates a log file containing hit rates, client/server errors, error rates, etc. which can be used to indicate health and performance of the server. Generally, log files generated by any subsystem in the tiers 101,102,103,104 track what occurs in that portion of the system. Accordingly, log files will contain data regarding events, state, transactions and anything of interest. It is understood that a log file may contain any type or class of data dependent upon programming.
  • [0014] Agents 105 associated with the web servers tier 101 are web server log readers. Examples of web servers 106 are Netscape, Apache, MicroSoft/IIS, America On-Line, etc., and include generic web servers. Events in a web server 106 log file include but are not limited to hit rate, hosts served, data volume, error rates, log file size and system load. System load includes but is not limited to CPU utilization memory usage and I/O transfers.
  • [0015] Agents 105 associated with the session servers tier 102 are session server application tests and log readers. Examples of sessions servers 107 are session management process tests. Events in a session server 107 log file include but are not limited to users served, interaction manager (IM) restarts, IM status, IM load database connectivity, error rates and system load.
  • [0016] Agents 105 associated with the transaction servers tier 103 are transaction server application tests and log readers. Transaction servers 108 process information on a variety of levels, including system level performance to transaction process count and availability, sales and orders, shopping statistics, payment service and order fulfillment (shipping). Events in a transaction server 108 log file may include but are not limited to inventory status, system load, open and discarded cart rates, shoppers in store, authorized, cancelled and declined data and payment service errors. Transaction servers 108 also compile and maintain databases. The agents 105 associated with transaction servers 108 query the transaction server databases using standard SQL database queries. It is understood that any type of database query capable of obtaining data from a database can be used. Examples of data in a transaction server 108 data base may include but are not limited to sales per hour, items sold per hour, new order rates, processed order rates, fulfilled order rates orders awaiting authorization and orders awaiting shipping.
  • [0017] Agents 105 associated with the database servers tier 104 are database server tests. Database servers 109 process information and create and maintain databases. The agents 105 associated with database servers 109 query the database server databases using standard SQL database queries. It is understood that any type of database query capable of obtaining data from a database can be used. While any data contained in the database can be retrieved, for the examples used in accordance with the invention, the pertinent data to query is health information such as number of the transactions that have occurred, the number of users, sales volume and other application or business health indicators of the e-commerce website which the database may contain.
  • In the preferred embodiment of the invention shown in FIG. 1, each [0018] agent 105 is shown as being associated with a particular server. The agents are software programs, or tests, that are resident on the servers due to the need for direct access to the relevant log files or databases. It is understood, that a particular agent 105 will be programmed to read the format, or formats, of log files and/or databases resident on a particular server. For example, if the agent 105 is interested in events relating to sale amount, it knows that column four (4) of the log file contains sale amount data. The agent 105 retrieves the appropriate data from the log files or databases and sends that information to a user site. The user site is an Internet service management solution such as the Firehunter® products offered by Agilent Technologies that enables a user to measure and monitor an e-commerce solution's performance. It is understood that the invention is not limited to use with Firehuntere and can be used in any manner capable of displaying data.
  • The data from the [0019] agents 105 is sent to a user site via means capable of transferring data. Preferably, the data is transmitted via the Internet or an intranet. It is understood that any network means can be used to transmit data from the agents 105 to a user site. It is also considered that a direct connection be used to transmit data from the agents to the user site.
  • The invention reads the log files from each of the [0020] tiers 101,102,103,104 in which there is an agent, makes queries to the database, and interacts with the e-commerce infrastructure. This information is presented as measurements varying overtime, where the measurements are comprised of system infrastructure (such as load or error information) and key business and financial measures such as the number of new orders per minute, the dollar volume of completed orders per minute, the number of abandoned “shopping carts” per minute, etc. Gathering these infrastructural and business measurements together into a system like Firehunter® then enables correlated results, such as lost sales due to system errors and the impact of system load on the rate and number of business transactions.
  • The invention assesses and reports key business and financial measurements. The system of the invention delivers these measurements in real time allowing e-commerce businesses to quickly determine current financial and business performance. This in turn allows quick detection and response to variances. [0021]
  • It is understood that the invention is not limited to the specific examples and illustrations discussed above, but only by the following claims and their equivalents. Accordingly, the invention is applicable to a broad variety of log files and databases that contain information that has been logged by an application. [0022]

Claims (13)

What is claimed is:
1. A method for monitoring an e-commerce installation, said comprising a plurality of tiers and said tiers generating log files and databases, wherein said method comprises:
reading the log files and querying the databases to obtain information stored therein, said information comprising performance measurements of the installation; and
graphically presenting said measurements.
2. The method of
claim 1
, wherein said plurality of tiers comprises a web serverstier, a session servers tier, a transaction servers tier and a database servers tier.
3. The method of
claim 2
, wherein;
said web servers tier comprises a plurality of web servers with each of said plurality of web servers generating log files, said log files comprising hit rate data, hosts served data, data volume data, error rates data, log file size monitor data, system load data and generic data,
said session servers tier comprises a plurality of session servers with each of said plurality of session servers generating a plurality of log files, said plurality of log files comprising users served data, restarts data, IM status data, IM load data, database connectivity data, error rates data and system load data,
said transaction servers tier comprises a plurality of transaction servers with each of said plurality of transaction servers generating a plurality of log files and a plurality of databases, said log files comprising open and discarded cart rates data, shoppers in store data, authorized, cancelled, declined data, payment service errors data, inventory status data, system load data and said databases comprising sales per hour data, items sold per hour data, new order rates data, processed order rates data, fulfilled order rates data shipping network connectivity data, orders awaiting authorization data and orders awaiting shipment data; and
said database servers tier comprises a plurality of database servers with each of said plurality of database servers generating a plurality of databases, said plurality of databases comprising database access performance data, database size monitor data, database connectivity error rate data and system load data.
4. The method of
claim 3
, wherein said user site is a fully functional web-based business application or e-commerce monitoring solution.
5. A software product for testing and monitoring a web-based business application or e-commerce installation comprising at least one web server, at least one session server, at least one transaction server, and at least one database server, the software product comprising:
test instructions configured to direct a processor to configure and execute user transaction tests and report user transaction test results;
system performance instructions configured to direct the processor to measure system performance data for each of the web server, the session server, the transaction server, and the database server; business performance instructions configured to direct the processor to measure
business performance data comprising monetary volume transacted by the e-commerce installation during a time period; and
a first storage media configured to store the user test instructions, the system performance instructions, and the business performance instructions.
6. The software product of
claim 5
, wherein the business performance data further comprises new orders transacted by the e-commerce installation during a time period.
7. The software product of
claim 5
, wherein the business performance data further comprises sales volume transacted by the e-commerce installation during a time period.
8. The software product of
claim 5
, wherein the business performance dta further comprises lost sales due to system errors during a time period.
9. The software product of
claim 5
, wherein the business performance data further comprises abandoned shopping carts during a time period.
10. A method of operating a computer system to test and monitor an Internet server system comprising a web server, session server, transaction server, and database server, the method comprising:
configuring and executing user transaction tests and reporting user transaction test results;
measuring system performance data for each of the web server, the session server, the transaction server, and the database server; and
measuring business performance data comprising monetary volume transacted by the e-commerce installation during a time period.
11. The method of
claim 10
, wherein the business performance data further comprises financial data transacted by the e-commerce installation during a time period.
12. The method of
claim 10
, further comprising:
processing the system performance data to generate system graphics illustrating system performance measured against system performance baselines and system performance thresholds; and
processing the business performance data to generate business graphics illustrating billing performance measured against business performance baselines and business performance thresholds.
13. The method of
claim 10
, further comprising:
collecting the system performance data and the business performance data from the e-commerce installation; and
transferring the system performance data and the business performance data to the computer system.
US09/825,403 2000-04-28 2001-04-03 Internet server system test and measurement Abandoned US20010037374A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US09/825,403 US20010037374A1 (en) 2000-04-28 2001-04-03 Internet server system test and measurement

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US20029500P 2000-04-28 2000-04-28
US09/825,403 US20010037374A1 (en) 2000-04-28 2001-04-03 Internet server system test and measurement

Publications (1)

Publication Number Publication Date
US20010037374A1 true US20010037374A1 (en) 2001-11-01

Family

ID=22741108

Family Applications (4)

Application Number Title Priority Date Filing Date
US09/822,397 Expired - Lifetime US7072935B2 (en) 2000-04-28 2001-03-30 Filtering web proxy for recording web-based transactions that supports secure HTTP steps
US09/821,917 Expired - Lifetime US7043546B2 (en) 2000-04-28 2001-03-30 System for recording, editing and playing back web-based transactions using a web browser and HTML
US09/822,124 Expired - Lifetime US7120676B2 (en) 2000-04-28 2001-03-30 Transaction configuration system and method for transaction-based automated testing
US09/825,403 Abandoned US20010037374A1 (en) 2000-04-28 2001-04-03 Internet server system test and measurement

Family Applications Before (3)

Application Number Title Priority Date Filing Date
US09/822,397 Expired - Lifetime US7072935B2 (en) 2000-04-28 2001-03-30 Filtering web proxy for recording web-based transactions that supports secure HTTP steps
US09/821,917 Expired - Lifetime US7043546B2 (en) 2000-04-28 2001-03-30 System for recording, editing and playing back web-based transactions using a web browser and HTML
US09/822,124 Expired - Lifetime US7120676B2 (en) 2000-04-28 2001-03-30 Transaction configuration system and method for transaction-based automated testing

Country Status (4)

Country Link
US (4) US7072935B2 (en)
EP (4) EP1277126A4 (en)
AU (4) AU2001253377A1 (en)
WO (4) WO2001084335A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050102580A1 (en) * 2001-08-24 2005-05-12 House Richard W. Test configuration and data management system and associated method for enterprise test operations
US20050268171A1 (en) * 2001-08-24 2005-12-01 House Richard W Enterprise test data management system utilizing hierarchical test data models and related methods
US20080005317A1 (en) * 2006-06-30 2008-01-03 International Business Machines Corporation Method and apparatus for cross-tier management in multi-tier computing system architecture
US7568019B1 (en) * 2002-02-15 2009-07-28 Entrust, Inc. Enterprise management system for normalization, integration and correlation of business measurements with application and infrastructure measurements
TWI477963B (en) * 2012-09-07 2015-03-21 Inventec Corp Method for testing client apparatus
US11061799B1 (en) * 2017-12-28 2021-07-13 Cerner Innovation, Inc. Log analysis application
EP4142257A1 (en) * 2021-08-23 2023-03-01 Shopify Inc. Systems and methods for server load balancing
TWI828506B (en) * 2023-01-03 2024-01-01 中華電信股份有限公司 System and method for assessing security standard
US11880874B2 (en) 2021-08-23 2024-01-23 Shopify Inc. Systems and methods for server load balancing based on correlated events

Families Citing this family (107)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2402437C (en) * 2000-04-07 2004-10-26 James Edward Appenzeller Methods and systems for monitoring quality assurance
US7072935B2 (en) * 2000-04-28 2006-07-04 Agilent Technologies, Inc. Filtering web proxy for recording web-based transactions that supports secure HTTP steps
WO2001095208A1 (en) * 2000-06-02 2001-12-13 Iprint.Com, Inc. Integrated electronic shopping cart system and method
US20010054044A1 (en) * 2000-06-14 2001-12-20 Liu Yi Bo Method for monitoring and browsing database of test system
WO2001098916A1 (en) * 2000-06-21 2001-12-27 Concord Communications, Inc. Liveexception system
US7287071B2 (en) 2000-09-28 2007-10-23 Vignette Corporation Transaction management system
US7231606B2 (en) * 2000-10-31 2007-06-12 Software Research, Inc. Method and system for testing websites
US7533336B2 (en) * 2001-03-01 2009-05-12 International Business Machines Corporation Method and apparatus for facilitating construction of e-commerce web pages having featured products and the like
US7065566B2 (en) * 2001-03-30 2006-06-20 Tonic Software, Inc. System and method for business systems transactions and infrastructure management
US6785845B2 (en) * 2001-04-10 2004-08-31 Hewlett-Packard Development Company, L.P. POS terminal test system and method
BR0210885A (en) * 2001-07-06 2004-06-22 Computer Ass Think Inc Method and system for monitoring interaction associated with an application, and, computer readable storage medium
US7113301B2 (en) * 2001-10-31 2006-09-26 Hewlett-Packard Development Company, L.P. System and method for automated access of a network page
US7363368B2 (en) * 2001-12-24 2008-04-22 International Business Machines Corporation System and method for transaction recording and playback
US10296919B2 (en) 2002-03-07 2019-05-21 Comscore, Inc. System and method of a click event data collection platform
US9092788B2 (en) * 2002-03-07 2015-07-28 Compete, Inc. System and method of collecting and analyzing clickstream data
US8095589B2 (en) 2002-03-07 2012-01-10 Compete, Inc. Clickstream analysis methods and systems
US20080189408A1 (en) 2002-10-09 2008-08-07 David Cancel Presenting web site analytics
US9129032B2 (en) * 2002-03-07 2015-09-08 Compete, Inc. System and method for processing a clickstream in a parallel processing architecture
US20070055937A1 (en) * 2005-08-10 2007-03-08 David Cancel Presentation of media segments
US7937471B2 (en) 2002-06-03 2011-05-03 Inpro Network Facility, Llc Creating a public identity for an entity on a network
US8234358B2 (en) * 2002-08-30 2012-07-31 Inpro Network Facility, Llc Communicating with an entity inside a private network using an existing connection to initiate communication
US20040054715A1 (en) * 2002-09-16 2004-03-18 Paul Cesario Capturing and replaying internet application transactions using an embedded browser
US7890451B2 (en) * 2002-10-09 2011-02-15 Compete, Inc. Computer program product and method for refining an estimate of internet traffic
US20040172253A1 (en) * 2003-02-28 2004-09-02 Sun Microsystems, Inc., A Delaware Corporation Capture and playback web automation tool
US20040249973A1 (en) * 2003-03-31 2004-12-09 Alkhatib Hasan S. Group agent
US7949785B2 (en) * 2003-03-31 2011-05-24 Inpro Network Facility, Llc Secure virtual community network system
US20040249974A1 (en) * 2003-03-31 2004-12-09 Alkhatib Hasan S. Secure virtual address realm
WO2004091141A1 (en) * 2003-04-14 2004-10-21 Bsafe Information Systems (1983) Ltd. Computer system to record, process and analyse information concerning communications transmitted through a telecommunications network managed by at least one computer and a method to implement such computer system
US7543051B2 (en) 2003-05-30 2009-06-02 Borland Software Corporation Method of non-intrusive analysis of secure and non-secure web application traffic in real-time
US7398291B2 (en) * 2003-06-26 2008-07-08 International Business Machines Corporation Method, system and program product for providing a status of a transaction with an application on a server
US20050010651A1 (en) * 2003-07-10 2005-01-13 Jie Xu Communication system supporting communication between executable applications
US7107186B2 (en) * 2003-10-31 2006-09-12 Abb Research Ltd. Transformer testing
JP4311194B2 (en) * 2003-12-24 2009-08-12 ソニー株式会社 Image reproducing apparatus and image reproducing method
US20050177613A1 (en) * 2004-02-11 2005-08-11 Scott Dresden Statistical and vouyeristic link behavioral tracking and presentation tools
US20050188080A1 (en) * 2004-02-24 2005-08-25 Covelight Systems, Inc. Methods, systems and computer program products for monitoring user access for a server application
US20050187934A1 (en) * 2004-02-24 2005-08-25 Covelight Systems, Inc. Methods, systems and computer program products for geography and time monitoring of a server application user
US20050188222A1 (en) * 2004-02-24 2005-08-25 Covelight Systems, Inc. Methods, systems and computer program products for monitoring user login activity for a server application
US7373524B2 (en) * 2004-02-24 2008-05-13 Covelight Systems, Inc. Methods, systems and computer program products for monitoring user behavior for a server application
US20050188221A1 (en) * 2004-02-24 2005-08-25 Covelight Systems, Inc. Methods, systems and computer program products for monitoring a server application
US8065410B1 (en) 2004-03-31 2011-11-22 Compuware Corporation Methods and apparatus for collecting performance metrics from a web site
US7685273B1 (en) * 2004-03-31 2010-03-23 Compuware Corporation Methods and apparatus for collecting and displaying performance metrics from a web site
US8245201B2 (en) * 2004-04-30 2012-08-14 International Business Machines Corporation Method and system for recording and replaying service interactions
US7627821B2 (en) * 2004-06-15 2009-12-01 Microsoft Corporation Recording/playback tools for UI-based applications
US7653896B2 (en) * 2004-06-30 2010-01-26 Microsoft Corporation Smart UI recording and playback framework
US20060101404A1 (en) * 2004-10-22 2006-05-11 Microsoft Corporation Automated system for tresting a web application
US7536280B2 (en) * 2005-08-03 2009-05-19 Agilent Technologies, Inc. Multisided synchronization of execution in a wireless test environment
US9105028B2 (en) 2005-08-10 2015-08-11 Compete, Inc. Monitoring clickstream behavior of viewers of online advertisements and search results
JP2007264921A (en) * 2006-03-28 2007-10-11 Fujitsu Ltd Performance information extraction program and apparatus
US7941525B1 (en) 2006-04-01 2011-05-10 ClickTale, Ltd. Method and system for monitoring an activity of a user
US7805510B2 (en) * 2006-05-11 2010-09-28 Computer Associates Think, Inc. Hierarchy for characterizing interactions with an application
US20070288473A1 (en) * 2006-06-08 2007-12-13 Rajat Mukherjee Refining search engine data based on client requests
US20080228675A1 (en) * 2006-10-13 2008-09-18 Move, Inc. Multi-tiered cascading crawling system
US7793154B2 (en) 2006-11-30 2010-09-07 International Business Machines Corporation Method and implementation for automating processes using data driven pre-recorded transactions
US20080141221A1 (en) * 2006-12-08 2008-06-12 Microsoft Corporation Using recorder technology to find defects in software applications
US7783469B2 (en) * 2007-01-05 2010-08-24 International Business Machines Corporation Methods and computer program products for benchmarking multiple collaborative services provided by enterprise software
US8504348B2 (en) * 2007-01-31 2013-08-06 Adobe Systems Incorporated User simulation for viewing web analytics data
US20080183858A1 (en) * 2007-01-31 2008-07-31 Error Christopher R Retrieval Mechanism for Web Visit Simulator
US7970857B2 (en) * 2007-05-18 2011-06-28 Red Hat, Inc. Method and an apparatus to record web transactions using a proxy server
US8392890B2 (en) 2007-10-15 2013-03-05 Software Research, Inc. Method and system for testing websites
KR20100051633A (en) * 2007-06-28 2010-05-17 누스타, 인크. Monitoring web service transactions
US8561136B2 (en) * 2007-10-10 2013-10-15 R. Brent Johnson System to audit, monitor and control access to computers
US9558097B2 (en) * 2007-11-13 2017-01-31 Red Hat, Inc. Automated recording and playback of application interactions
US8234393B2 (en) * 2007-11-16 2012-07-31 Red Hat, Inc. Generic network protocol scripting
US8849944B2 (en) * 2007-11-27 2014-09-30 Red Hat, Inc. Multi-use application proxy
US20090265719A1 (en) * 2008-04-18 2009-10-22 Microsoft Corporation Application macro recording utilizing method interception
US20090287964A1 (en) * 2008-05-17 2009-11-19 Sunrise Telecom Incorporated Internet accessible test system
US8719838B1 (en) * 2008-05-28 2014-05-06 United Services Automobile Association (Usaa) Systems and methods for generating scripts to interact with web sites
US8656478B1 (en) * 2008-07-15 2014-02-18 Zscaler, Inc. String based detection of proxy communications
US8055767B1 (en) * 2008-07-15 2011-11-08 Zscaler, Inc. Proxy communication string data
US8589810B2 (en) * 2008-11-18 2013-11-19 At&T Intellectual Property I, L.P. Methods, systems, and products for recording browser navigations
US9934320B2 (en) 2009-03-31 2018-04-03 International Business Machines Corporation Method and apparatus for using proxy objects on webpage overlays to provide alternative webpage actions
US20100325615A1 (en) * 2009-06-23 2010-12-23 Myspace Inc. Method and system for capturing web-page information through web-browser plugin
US9350817B2 (en) * 2009-07-22 2016-05-24 Cisco Technology, Inc. Recording a hyper text transfer protocol (HTTP) session for playback
US9785722B2 (en) * 2010-04-01 2017-10-10 Forsee Results, Inc. Systems and methods for remote replay of user interaction with a webpage
US8977739B2 (en) * 2010-05-03 2015-03-10 Salesforce.Com, Inc. Configurable frame work for testing and analysis of client-side web browser page performance
US11562013B2 (en) 2010-05-26 2023-01-24 Userzoom Technologies, Inc. Systems and methods for improvements to user experience testing
US11934475B2 (en) 2010-05-26 2024-03-19 Userzoom Technologies, Inc. Advanced analysis of online user experience studies
US11544135B2 (en) 2010-05-26 2023-01-03 Userzoom Technologies, Inc. Systems and methods for the analysis of user experience testing with AI acceleration
US10691583B2 (en) 2010-05-26 2020-06-23 Userzoom Technologies, Inc. System and method for unmoderated remote user testing and card sorting
US11348148B2 (en) 2010-05-26 2022-05-31 Userzoom Technologies, Inc. Systems and methods for an intelligent sourcing engine for study participants
US11494793B2 (en) 2010-05-26 2022-11-08 Userzoom Technologies, Inc. Systems and methods for the generation, administration and analysis of click testing
US11068374B2 (en) 2010-05-26 2021-07-20 Userzoom Technologies, Inc. Generation, administration and analysis of user experience testing
US8458757B2 (en) 2010-08-31 2013-06-04 At&T Intellectual Property I, L.P. Method and system for region-based monitoring of video assets
US8966446B1 (en) * 2010-09-29 2015-02-24 A9.Com, Inc. Systems and methods of live experimentation on content provided by a web site
US8924395B2 (en) 2010-10-06 2014-12-30 Planet Data Solutions System and method for indexing electronic discovery data
CN102508773B (en) * 2011-10-18 2014-12-31 深圳市易聆科信息技术有限公司 Method and device for monitoring WEB service system simulation based on Internet explorer (IE) kernel
US9900395B2 (en) 2012-01-27 2018-02-20 Comscore, Inc. Dynamic normalization of internet traffic
US8954580B2 (en) 2012-01-27 2015-02-10 Compete, Inc. Hybrid internet traffic measurement using site-centric and panel data
WO2014048851A2 (en) * 2012-09-25 2014-04-03 Telefonica, S.A. A method and a system to surveil applications of online services
US9536108B2 (en) * 2012-10-23 2017-01-03 International Business Machines Corporation Method and apparatus for generating privacy profiles
US9009544B2 (en) * 2013-03-28 2015-04-14 Oracle International Corporation User operation history for web application diagnostics
US9706004B2 (en) * 2013-04-06 2017-07-11 Citrix Systems, Inc. Systems and methods for exporting client and server timing information for webpage and embedded object access
CN103258005B (en) * 2013-04-12 2017-02-08 百度在线网络技术（北京）有限公司 Processing method and device for search results
CN105808417B (en) * 2014-12-29 2019-02-26 平安科技（深圳）有限公司 Automated testing method and proxy server
US9225625B1 (en) * 2015-03-26 2015-12-29 Linkedin Corporation Detecting and alerting performance degradation during features ramp-up
US10432490B2 (en) * 2015-07-31 2019-10-01 Cisco Technology, Inc. Monitoring single content page application transitions
US10320948B2 (en) 2015-11-30 2019-06-11 Successfactors, Inc. Application footprint recorder and synchronizer
US9444703B1 (en) * 2015-11-30 2016-09-13 International Business Machines Corporation Interconnecting electronic devices for reporting device status
US10339564B2 (en) 2016-02-16 2019-07-02 Oracle International Corporation System and method for providing an adaptively ordered presentation of objects
CN106452942B (en) * 2016-08-29 2019-08-27 中车株洲电力机车研究所有限公司 The vehicle-mounted Ethernet exchanging software performance test system and method for track transportation industry
IL297393A (en) * 2016-09-01 2022-12-01 Verint Americas Inc System and computer-implemented method for in-page reporting of user feedback on a website or mobile app
US10474563B1 (en) * 2016-12-28 2019-11-12 Wells Fargo Bank, N.A. System testing from production transactions
CN108445847A (en) * 2018-03-22 2018-08-24 上海精鲲计算机科技有限公司 A kind of visualization computer room management system
KR102180592B1 (en) * 2018-12-14 2020-11-18 주식회사 엘지씨엔에스 Method and system for testing it system
US10754762B2 (en) 2019-01-30 2020-08-25 Walmart Apollo, Llc Automatic tracking of feature usage of webpages on a website
US11909100B2 (en) 2019-01-31 2024-02-20 Userzoom Technologies, Inc. Systems and methods for the analysis of user experience testing with AI acceleration
US10884907B1 (en) * 2019-08-26 2021-01-05 Capital One Services, Llc Methods and systems for automated testing using browser extension

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5210687A (en) * 1987-04-16 1993-05-11 L & C Family Partnership Business transaction and investment growth monitoring data processing system
US6701342B1 (en) * 1999-12-21 2004-03-02 Agilent Technologies, Inc. Method and apparatus for processing quality of service measurement data to assess a degree of compliance of internet services with service level agreements

Family Cites Families (75)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5432940A (en) * 1992-11-02 1995-07-11 Borland International, Inc. System and methods for improved computer-based training
US5715393A (en) * 1993-08-16 1998-02-03 Motorola, Inc. Method for remote system process monitoring
US5440710A (en) 1994-03-08 1995-08-08 Exponential Technology, Inc. Emulation of segment bounds checking using paging with sub-page validity
US6167534A (en) * 1995-11-24 2000-12-26 Rational Software Corporation Load test system and method
US5848396A (en) * 1996-04-26 1998-12-08 Freedom Of Information, Inc. Method and apparatus for determining behavioral profile of a computer user
US5812668A (en) * 1996-06-17 1998-09-22 Verifone, Inc. System, method and article of manufacture for verifying the operation of a remote transaction clearance system utilizing a multichannel, extensible, flexible architecture
US6134583A (en) * 1996-07-01 2000-10-17 Sun Microsystems, Inc. Method, system, apparatus and article of manufacture for providing identity-based caching services to a plurality of computer systems (#16)
FR2751448B1 (en) * 1996-07-17 1999-01-15 Bull Sa METHOD FOR REAL-TIME MONITORING OF A COMPUTER SYSTEM FOR ITS ADMINISTRATION AND ASSISTANCE IN MAINTAINING IT IN OPERATION
US5809247A (en) * 1996-07-22 1998-09-15 Intel Corporation Method and apparatus for guided touring of internet/intranet websites
US5961594A (en) 1996-09-26 1999-10-05 International Business Machines Corporation Remote node maintenance and management method and system in communication networks using multiprotocol agents
US5949976A (en) * 1996-09-30 1999-09-07 Mci Communications Corporation Computer performance monitoring and graphing tool
US5974572A (en) * 1996-10-15 1999-10-26 Mercury Interactive Corporation Software system and methods for generating a load test using a server access log
US6115742A (en) * 1996-12-11 2000-09-05 At&T Corporation Method and apparatus for secure and auditable metering over a communications network
US5938737A (en) * 1997-02-14 1999-08-17 Stanford Telecommunications, Inc. Internet upstream request compression
US5903721A (en) * 1997-03-13 1999-05-11 cha|Technologies Services, Inc. Method and system for secure online transaction processing
US6041041A (en) * 1997-04-15 2000-03-21 Ramanathan; Srinivas Method and system for managing data service systems
US6006260A (en) 1997-06-03 1999-12-21 Keynote Systems, Inc. Method and apparatus for evalutating service to a user over the internet
DE19727036A1 (en) 1997-06-25 1999-01-07 Ibm Performance measurement method for processing steps of programs
JP2000501542A (en) 1997-07-01 2000-02-08 プログレス ソフトウェア コーポレイション Test and debug tools for network applications
US6119167A (en) 1997-07-11 2000-09-12 Phone.Com, Inc. Pushing and pulling data in networks
US5991705A (en) * 1997-07-23 1999-11-23 Candle Distributed Solutions, Inc. End-to-end response time measurement for computer programs using starting and ending queues
US6108700A (en) * 1997-08-01 2000-08-22 International Business Machines Corporation Application end-to-end response time measurement and decomposition
US6411998B1 (en) * 1997-09-08 2002-06-25 International Business Machines Corporation World wide web internet delay monitor
EP0907132A1 (en) * 1997-09-30 1999-04-07 Hitachi, Ltd. Page transition system, device to implement the same, and medium having a program and data recorded thereon
US6002871A (en) * 1997-10-27 1999-12-14 Unisys Corporation Multi-user application program testing tool
US6134594A (en) * 1997-10-28 2000-10-17 Microsoft Corporation Multi-user, multiple tier distributed application architecture with single-user access control of middle tier objects
US6064982A (en) * 1997-11-12 2000-05-16 Netscape Communication Corporation Smart configurator
US5889520A (en) * 1997-11-13 1999-03-30 International Business Machines Corporation Topological view of a multi-tier network
US6044398A (en) 1997-11-21 2000-03-28 International Business Machines Corporation Virtual dynamic browsing system and method for automated web server and testing
US6157940A (en) * 1997-11-21 2000-12-05 International Business Machines Corporation Automated client-based web server stress tool simulating simultaneous multiple user server accesses
US6620204B1 (en) * 1997-11-25 2003-09-16 International Business Machines Corporation Automated HTML test page and test data generation for javascript objects
US6014692A (en) * 1997-11-25 2000-01-11 International Business Machines Corporation Web browser file system attachment
US6446120B1 (en) * 1997-11-26 2002-09-03 International Business Machines Corporation Configurable stresser for a web server
US6237035B1 (en) * 1997-12-18 2001-05-22 International Business Machines Corporation System and method for preventing duplicate transactions in an internet browser/internet server environment
US6044389A (en) * 1997-12-29 2000-03-28 Quantum Corporation System for computing the multiplicative inverse of a field element for galois fields without using tables
US6195679B1 (en) 1998-01-06 2001-02-27 Netscape Communications Corporation Browsing session recording playback and editing system for generating user defined paths and allowing users to mark the priority of items in the paths
JP3561139B2 (en) * 1998-01-27 2004-09-02 シャープ株式会社 File object relay method, computer-readable recording medium storing program of file object relay method, and gateway computer
US6223221B1 (en) 1998-02-05 2001-04-24 International Business Machines Corporation System and method for calculating the transfer rate across a communication medium using a downloaded test program and transferring data accordingly
US6161125A (en) * 1998-05-14 2000-12-12 Sun Microsystems, Inc. Generic schema for storing configuration information on a client computer
EP1076871A1 (en) * 1998-05-15 2001-02-21 Unicast Communications Corporation A technique for implementing browser-initiated network-distributed advertising and for interstitially displaying an advertisement
US6253326B1 (en) * 1998-05-29 2001-06-26 Palm, Inc. Method and system for secure communications
US6338096B1 (en) * 1998-06-10 2002-01-08 International Business Machines Corporation System uses kernals of micro web server for supporting HTML web browser in providing HTML data format and HTTP protocol from variety of data sources
US6278966B1 (en) * 1998-06-18 2001-08-21 International Business Machines Corporation Method and system for emulating web site traffic to identify web site usage patterns
US6360332B1 (en) * 1998-06-22 2002-03-19 Mercury Interactive Corporation Software system and methods for testing the functionality of a transactional server
US6151599A (en) * 1998-07-17 2000-11-21 International Business Machines Corporation Web client scripting test architecture for web server-based authentication
US6212546B1 (en) * 1998-10-01 2001-04-03 Unisys Corporation Providing a modular gateway architecture which isolates attributes of the client and server systems into independent components
US6138157A (en) * 1998-10-12 2000-10-24 Freshwater Software, Inc. Method and apparatus for testing web sites
US6297819B1 (en) * 1998-11-16 2001-10-02 Essential Surfing Gear, Inc. Parallel web sites
US6662217B1 (en) * 1999-01-19 2003-12-09 Microsoft Corporation Distributed and automated test administration system for administering automated tests on server computers over the internet
US6507908B1 (en) * 1999-03-04 2003-01-14 Sun Microsystems, Inc. Secure communication with mobile hosts
US6393479B1 (en) * 1999-06-04 2002-05-21 Webside Story, Inc. Internet website traffic flow analysis
US6418544B1 (en) * 1999-06-22 2002-07-09 International Business Machines Corporation Use of a client meta-cache for realistic high-level web server stress testing with minimal client footprint
US6449739B1 (en) * 1999-09-01 2002-09-10 Mercury Interactive Corporation Post-deployment monitoring of server performance
US6553515B1 (en) 1999-09-10 2003-04-22 Comdial Corporation System, method and computer program product for diagnostic supervision of internet connections
US6347305B1 (en) * 1999-10-29 2002-02-12 E-Duction, Inc. Method for selecting and processing a payroll deduction plan as a payment option during electronic commerce
US6772107B1 (en) * 1999-11-08 2004-08-03 J.D. Edwards World Source Company System and method for simulating activity on a computer network
US6522995B1 (en) * 1999-12-28 2003-02-18 International Business Machines Corporation Method and apparatus for web-based control of a web-based workload simulation
US6763380B1 (en) 2000-01-07 2004-07-13 Netiq Corporation Methods, systems and computer program products for tracking network device performance
US6477483B1 (en) * 2000-01-17 2002-11-05 Mercury Interactive Corporation Service for load testing a transactional server over the internet
US7319847B2 (en) * 2000-03-20 2008-01-15 Nielsen Mobile, Inc. Bitwise monitoring of network performance
US20030084098A1 (en) 2000-04-13 2003-05-01 Daniel Lavin Navigation server for use with, for example, a wireless web access device having a navigation control unit
US7072935B2 (en) * 2000-04-28 2006-07-04 Agilent Technologies, Inc. Filtering web proxy for recording web-based transactions that supports secure HTTP steps
US6754701B1 (en) * 2000-05-05 2004-06-22 Mercury Interactive Corporation Use of a single thread to support multiple network connections for server load testing
US20020029259A1 (en) * 2000-07-26 2002-03-07 Nec Corporation Remote operation system and remote operation method thereof
US6829638B1 (en) * 2000-08-03 2004-12-07 International Business Machines Corporation System and method for managing multiple proxy servers
US20020073124A1 (en) * 2000-12-11 2002-06-13 Takahiro Masuda Network browser
US20020112049A1 (en) * 2000-12-14 2002-08-15 International Business Machines Corporation Measuring response time for a computer accessing information from a network
US6654699B2 (en) * 2000-12-29 2003-11-25 Microsoft Corporation Computer network testing system and method using client playback of edited network information
US20020138226A1 (en) * 2001-03-26 2002-09-26 Donald Doane Software load tester
US6721686B2 (en) * 2001-10-10 2004-04-13 Redline Networks, Inc. Server load testing and measurement system
US6840908B2 (en) * 2001-10-12 2005-01-11 Sound Id System and method for remotely administered, interactive hearing tests
US20030182408A1 (en) * 2002-02-15 2003-09-25 Qinglong Hu Load test system for a server and method of use
US20040003033A1 (en) 2002-06-27 2004-01-01 Yury Kamen Method and system for generating a web service interface
US20040133635A1 (en) 2002-11-26 2004-07-08 Axel Spriestersbach Transformation of web description documents
US7194531B2 (en) * 2002-12-02 2007-03-20 International Business Machines Corporation System and method for determining the availability of a web page

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5210687A (en) * 1987-04-16 1993-05-11 L & C Family Partnership Business transaction and investment growth monitoring data processing system
US6701342B1 (en) * 1999-12-21 2004-03-02 Agilent Technologies, Inc. Method and apparatus for processing quality of service measurement data to assess a degree of compliance of internet services with service level agreements

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7539591B2 (en) 2001-08-24 2009-05-26 Vi Technology, Inc. Enterprise test data management system utilizing hierarchical test data models and related methods
US20050268171A1 (en) * 2001-08-24 2005-12-01 House Richard W Enterprise test data management system utilizing hierarchical test data models and related methods
US7113883B1 (en) 2001-08-24 2006-09-26 Vi Technology, Inc. Test configuration and data management system and associated method for enterprise test operations
US20050102580A1 (en) * 2001-08-24 2005-05-12 House Richard W. Test configuration and data management system and associated method for enterprise test operations
US7568019B1 (en) * 2002-02-15 2009-07-28 Entrust, Inc. Enterprise management system for normalization, integration and correlation of business measurements with application and infrastructure measurements
US20080005317A1 (en) * 2006-06-30 2008-01-03 International Business Machines Corporation Method and apparatus for cross-tier management in multi-tier computing system architecture
US20080215729A1 (en) * 2006-06-30 2008-09-04 International Business Machines Corporation Method and Apparatus for Cross-Tier Management in Multi-Tier Computing System Architecture
US8473603B2 (en) * 2006-06-30 2013-06-25 International Business Machines Corporation Cross-tier management in multi-tier computing system architecture
TWI477963B (en) * 2012-09-07 2015-03-21 Inventec Corp Method for testing client apparatus
US11061799B1 (en) * 2017-12-28 2021-07-13 Cerner Innovation, Inc. Log analysis application
US11256600B2 (en) * 2017-12-28 2022-02-22 Cerner Innovation, Inc. Log analysis application
EP4142257A1 (en) * 2021-08-23 2023-03-01 Shopify Inc. Systems and methods for server load balancing
US11880874B2 (en) 2021-08-23 2024-01-23 Shopify Inc. Systems and methods for server load balancing based on correlated events
US11893614B2 (en) 2021-08-23 2024-02-06 Shopify Inc. Systems and methods for balancing online stores across servers
TWI828506B (en) * 2023-01-03 2024-01-01 中華電信股份有限公司 System and method for assessing security standard

Also Published As

Publication number Publication date
AU2001257375A1 (en) 2001-11-12
EP1277126A1 (en) 2003-01-22
WO2001084273A3 (en) 2002-05-23
US7072935B2 (en) 2006-07-04
AU2001253377A1 (en) 2001-11-12
WO2001084432A1 (en) 2001-11-08
US20020046363A1 (en) 2002-04-18
EP1277127A4 (en) 2007-04-25
EP1277126A4 (en) 2007-04-25
US20020035498A1 (en) 2002-03-21
EP1277156A4 (en) 2007-04-25
US20020143931A1 (en) 2002-10-03
WO2001084337A1 (en) 2001-11-08
WO2001084335A1 (en) 2001-11-08
EP1277156A1 (en) 2003-01-22
EP1277310A4 (en) 2007-04-25
US7120676B2 (en) 2006-10-10
EP1277310A2 (en) 2003-01-22
US7043546B2 (en) 2006-05-09
EP1277127A1 (en) 2003-01-22
AU2001261069A1 (en) 2001-11-12
WO2001084273A2 (en) 2001-11-08
AU2001259229A1 (en) 2001-11-12

Similar Documents

Publication Publication Date Title
US20010037374A1 (en) Internet server system test and measurement
US8290838B1 (en) Indicating irregularities in online financial transactions
US7627572B2 (en) Rule-based dry run methodology in an information management system
JP2884115B2 (en) Method and apparatus for filtering point of sale data
US8612348B1 (en) Systems and methods for interfacing merchants with third-party service providers
US20070260735A1 (en) Methods for linking performance and availability of information technology (IT) resources to customer satisfaction and reducing the number of support center calls
KR20070085242A (en) Method for performing retail sales analysis
CA2504476A1 (en) Method and system for monitoring electronic transactions
JP2005521973A (en) Product recall using customer purchase history data
US20050015300A1 (en) System and method for managing incentive offers
Chu et al. Quantifying the economic value of warranties in the US server market
US20050261993A1 (en) Method and/or system for customer/vendor data management and analysis
US20090033980A1 (en) Hardware controlled one-time write for fiscal printers
KR100736798B1 (en) Method and system for managing report on goods advertisement
US20060074948A1 (en) Management server, system, method and program
JP3922104B2 (en) Point management device, point management system, point management method, and point management program
US6792394B1 (en) Method and apparatus for determining the retail performance metric of entry identification time
US20060047575A1 (en) System and method for processing orders for multiple multilevel marketing business models
US11200102B1 (en) System for tracking transaction data across applications
JP2003150874A (en) Settlement of account method of electronic commerce and settlement of account system thereof
US20050171807A1 (en) Transaction processing engine
JP2003203152A (en) Marketing information providing method, program, and device
JP6046593B2 (en) Planning effect analysis system, planning effect analysis method, and planning effect analysis program
JP2002183562A (en) Lottery reservation purchase system, lottery reservation purchase method, and storage medium
KR100853271B1 (en) The tax refund system

Legal Events

Date Code Title Description
AS Assignment

Owner name: AGILENT TECHNOLOGIES, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHRUM, KENNETH W.;BARTZ, THOMAS G.;REEL/FRAME:011774/0512;SIGNING DATES FROM 20010420 TO 20010424

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION