CA2581719A1 - Performance management system - Google Patents

Abstract

An outsourcing command center collects and processes outsourcing data for multiple engagements and multiple outsourcing offerings. The command center may apply a metric set to the outsourcing data that is broadly applicable across multiple outsourcing offerings, but also tailored to specific outsourcing offerings, and extended by engagements to meet their particular metric reporting criteria. The command center provides a portal through which an engagement may access processed metric information for outsourced functions, including metric reporting screens with convenient drill-down link sets to display progressively more detailed views of metrics processed for the engagement.

Description

2 PCT/EP2005/010821 PERFORMANCE MANAGEMENT SYSTEM

INVENTORS
John B. Baughn Raymond Goff Miranda L. Mason Barry A. Gleichenhaus Karen M. Bobear BACKGROUND
1. Technical Field [001] This invention relates to processing systems for collecting, processing, and presenting data. In particular, this invention relates to data processing systems for collecting engagement data across multiple outsourcing offerings, applying analysis metrics to the engagement data, and presenting analysis results.

2. Background Information [002] Outsourcing has emerged as a widely applicable and popular procedure that businesses employ to reduce costs, increase profitability, and focus the business on its core operations. A business may outsource nearly any aspect of it operations. As a few examples, businesses may outsource human resource operations, payroll operations, and supply chain operations.

[003] Outsourcing providers have responded to the desire to outsource business functions. For example, many outsourcing providers setup independent outsourcing centers to implement specific outsourcing offerings.
Each outsourcing provider may determine and report low level business performance measures. However, in most cases, the low level measures were not useful to key business decision makers because the metrics were not directly indicative of business value or were not correlated to progress against desired business outcomes.

[004] In addition, because so many aspects of a business may be outsourced, a single business may find their outsourced functions logically or physically distributed among multiple outsourcing centers operated by a single outsourcing provider or possibly multiple outsourcing providers. When outsourcing functions are distributed, it becomes time and cost intensive for the business to monitor not only the performance of the individual outsourcing functions, but also to gain an understanding of the performance impact on the business of its outsourced functions as a whole.

[005] Furthermore, because a business may outsource multiple services, each group providing the outsourced services may process and report outsourcing statistics in a different manner. As a result a business may not have a consistent reference point for comparing the performance of different outsourced functions. In addition, when outsourced functions are distributed among multiple outsourcing centers, it becomes difficult even for the outsourcing provider to obtain convenient access to, and a common reference point for, all of the outsourcing offerings implemented by the outsourcing provider.

[006] There is a need for addressing the problems noted above and other previously experienced.

BRIEF SUMMARY

[007] An outsourcing command center collects and processes outsourcing data that may span multiple engagements and multiple outsourcing offerings. The command center may generate high level business value delivered information based on underlying low level metrics.
The command center may include a metric set that is broadly applicable across multiple outsourcing offerings, tailored metric sets applicable to particular outsourcing offerings, and engagement extensions to the metrics sets. The command center provides a portal through which an engagement may access processed metric information for all of their outsourced functions, and which provides a flexible access hierarchy defining permissions to the processed metric information.

[008] In one implementation, a data processing system implements a command center data that may include an upper tier metric set applicable to multiple outsourcing offerings. As examples, the outsourcing offerings may be a Business Process Outsourcing (BPO) offering, a Technology Infrastructure Outsourcing (TIO) offering, or an Application Outsourcing (AO) offering. The upper tier metric set may be applicable to any other outsourcing offerings.

[009] In addition, the data processing system may store engagement extension metrics. For example, a TIO engagement extended metric set may include the upper tier metric set and a TIO engagement extension. Similarly, a BPO extended metric set may include the upper tier metric set and a BPO
engagement extension.

[010] A database in the data processing system stores engagement operations data obtained from engagement outsourcing operation sites that may be logically or physically distinct. The data processing system applies the engagement extended metric sets to the engagement operations data to generate processed metrics for each engagement outsourcing offering.
Accordingly, the data processing system may generate TIO processed metrics and BPO processed metrics for the engagement.

[011] A user interface generated by the data processing system may include a metric reporting interface. The metric reporting interface reports the processed metrics externally to the engagement or internally for review and analysis. The metric reporting interface may include a drill-down link set to present the processed metrics in increasingly detailed or increasingly specific form.

BRIEF DESCRIPTION OF THE DRAWINGS

[012] Figure 1 illustrates a data processing system that may implement an outsourcing command center.

[013] Figure 2 shows a detailed block diagram of the data processing system of Figure 1.

[014] Figure 3 shows a reporting rule that may be established in the outsourcing command center as part of a reporting ruleset.

[015] Figure 4 shows a drill-down link set to progressively more detailed views of outsourcing metrics.

[016] Figure 5 shows an engagement metric reporting screen with drill-down links to detailed metric reporting screens.

[017] Figure 6 shows an engagement metric reporting screen with detailed metric reporting elements.

[018] Figure 7 shows a geographic location screen for reporting metrics.

[019] Figure 8 shows a geographic location screen for presenting detail metric reports.

[020] Figure 9 shows a service agreement rule that may be established in the outsourcing command center.

[021] Figure 10 illustrates an access hierarchy with access entries that may authorize or deny access to data in the outsourcing command center.

[022] Figure 11 shows a command center portal into metrics processed according to a multiple level metrics hierarchy.

[023] Figure 12 shows a metric reporting screen with detailed metric reporting elements for outsourcing categories shown in Figure 11.

[024] Figure 13 shows a metric reporting screen with detailed metric reporting elements for a service excellence metric category.

[025] Figure 14 shows a metric reporting screen for service excellence.

[026] Figure 15 shows the acts that may be taken by the command center shown in the Figures to display processed metric data.

[027] Figure 16 shows the acts that may be taken by the command center shown in the Figures to process engagement outsourcing data.

DETAILED DESCRIPTION

[028] The elements illustrated in the Figures interoperate as explained in more detail below. Before setting forth the detailed explanation, however, it is noted that all of the discussion below, regardless of the particular implementation being described, is exemplary in nature, rather than limiting.

For example, although selected aspects, features, or components of the implementations are depicted as stored in program, data, or multipurpose system memories, all or part of systems and methods consistent with the outsourcing command center technology may be stored on or read from other machine-readable media, for example, secondary storage devices such as hard disks, floppy disks, and CD-ROMs; electromagnetic signals; or other forms of machine readable media either currently known or later developed.

[029] Furthermore, although this specification describes specific components of a command center data processing system, methods, systems, and articles of manufacture consistent with the command center technology may include additional or different components. For example, a processor may be implemented as a microprocessor, microcontroller, application specific integrated circuit (ASIC), discrete logic, or a combination of other types of circuits acting as explained above. Databases, tables, and other data structures may be separately stored and managed, incorporated into a single memory or database, or generally logically and physically organized in many different ways. The programs discussed below may be parts of a single program, separate programs, or distributed across several memories and processors.

[030] Figure 1 shows a data processing system that may implement an outsourcing command center 100. The command center 100 includes a processor 102, a memory 104, and a display 106. In addition, a network interface 108 is present.

[031] The memory 104 stores a metric processing program 110, and a metrics set 112. The metrics set 112 includes upper tier metrics 114, middle tier metrics 116, and engagement extension metrics 118. Engagement operations data 120 may also be present in the memory 104.

[032] Any of the metrics 112 - 118 may be inward facing metrics, outward facing metrics, or both. Inward facing metrics may be metrics that are determined, but that are not reported to an engagement. Instead, the inward facing metrics may be reviewed and analyzed for internal management of the command center 100, an engagement, or for other reasons. Outward facing metrics may be metrics that are reported to an engagement.

[033] The processor 102 generates a user interface 122 on the display 106. As will be explained in more detail below, the user interface 122 may provide a metric reporting interface that presents processed metrics to an engagement or internal reviewer logged into the command center 100. To that end, the metric reporting interface may include drill-down links distributed over one or more user interfaces screens that present the processed metrics in increasingly detailed or increasingly specific form.

[034] The network interface 108 may include a network interface card or other network connection device to connect the command center 100 to internal or external networks 126. The networks 126 connect, in turn, to one or more outsourcing operations sites 128, one or more engagement login sites 130, and one or more internal review sites 132. As examples, the outsourcing operations sites 128, engagement login sites 130, and internal review sites 132 may include computers communicating over a dial-up modem, DSL, T1, or other network connection with the command center 100.

[035] The outsourcing operations sites 128 may represent local or remote locations where outsourcing operations are performed for an engagement, or where outsourcing data relevant to an engagement outsourcing operation is present. The engagement login sites 130 may represent local or remote locations from which an engagement may connect to the command center 100 to review statistics relating to their outsourced business functions.
Similarly, the internal review sites 132 may represent local or remote locations from which administrators, outsourcing offering or marketing executives, or other internal individuals who provide, configure, or run the command center 100 may access the command center 100 to investigate outsourcing performance for their engagements or to investigate the performance of the command center 100 itself.

[036] The networks 126 may adhere to one or more network topologies and technologies. For example, the networks 126 may be Ethernet networks, but in other implementations may be implemented with a Fiber Distributed Data Interconnect (FDDI) network, Copper Distributed Data Interface (CDDI) network, or another network technology.

[037] The network interface 108 has one or more network addresses 124.
The network address 124 may be a packet switched network identifier such as a Transmission Control Protocol / Internet Protocol (TCP/IP) address (optionally including port numbers), or any other communication protocol address. Accordingly, the command center 100 may communicate with local or remote outsourcing operations sites 128, engagement login sites 130, and internal review sites 132.

[038] In one implementation, the networks 126 may be IP packet switched networks, employing addressed packet communication between the command center 100 and the outsourcing operations sites 128. Different or additional communication protocols may be employed to support communication between the command center 100 and individual outsourcing operation sites. Thus, the networks 126 may represent a transport mechanism or interconnection of multiple transport mechanisms for data exchange between the command center 100 and the outsourcing operations sites 128 and engagement login sites 130.

[039] Figure 2 expands upon the view of the command center 100 shown in Figure 1. Figure 2 shows an engagement operations database 202, a processed metrics database 204, and one or more ruiesets. As examples, the rulesets may include a reporting ruleset 206 such as an aggregation reporting ruleset, a business value delivered ruleset 207, and a service agreement ruieset 208. Display templates 210 for the user interface 122 are also present.

[040] The user interface 122 may include an internal investigation portal 212 and an engagement investigation portal 214. The portals 212 and 214 may accept a login and password that authorize access to the command center 100. Each portal is explained in more detail below. The internal investigation portal 212 may provide links 216, 218 to one or more metric reporting screens such as the screens 220 and 222. Similarly, the engagement investigation portal 214 may provide links 224, 226 to one or more metric reporting screens such as the screens 228 and 230.

[041] The links 216, 218, 224, and 226 may provide access to or between the screens 220-222 and 228-230. The links 216, 218, 224, 226 may be html hyperlinks, buttons, or other user interface elements that connect to additional screens, graphics, or other data in the command center 100. One or more links may form one or more drill-down link sets to or between metric reporting screens. The drill-down link sets may then provide reporting on engagement outsourcing metrics in a progressively more detailed manner.

[042] The command center 100 communicates through the networks 126 with local or remote engagement outsourcing operations sites. As examples, Figure 2 shows the outsourcing operation sites 232, 234, 236, and 238. Each outsourcing operation site 232-238 may perform one or more aspects of one or more outsourcing offerings for any engagement.

[043] By way of example only, the outsourcing operations site 232 may provide Business Process Outsourcing (BPO) functions for engagements A
and B. The outsourcing operations site 232 may provide additional or different business processing outsourcing functions for engagements B and C, as well as application outsourcing functions for engagements B and C.
The outsourcing operations site 236 may provide Application Outsourcing (AO) functions for engagements A and C. The outsourcing operations site 238 may provide Information Technology Outsourcing (TIO) for engagements A, B, and C.

[044] Figure 2 shows that the metrics set 112 may include the upper tier metrics 114, the middle tier metrics 116, and the engagement extension metrics 118. Individual metrics may repeat between or may overlap between the upper tier metrics 114, middle tier metrics 116, and engagement extension metrics 118. The upper tier metrics 114 may include one or more outsourcing metrics applicable to multiple outsourcing offerings. For example, upper tier metrics apply to BPO, AO, and TIO outsourcing. The middle tier metrics 116 may include one or more outsourcing metrics applicable to a particular outsourcing offering. The middle tier metrics may include BPO
metrics 240, TIO metrics 242, and AO metrics 244.

[045] Each engagement may specify additions or extensions to any of the upper tier metrics 114 or middle tier metrics 240-244. Figure 2 shows that engagement A BPO extension metrics 246, engagement B BPO extension metrics 248, and engagement C BPO extension metrics 250 established in the command center 100. Also established are engagement A TIO extension metrics 252, engagement B TIO extension metrics 254, and engagement C
TIO extension metrics 256 established in the command center 100.
Engagement A AO extension metrics 258, engagement B AO extension metrics 260, and engagement C AO extension metrics 262 are also established in the command center 100.

[046] Examples of upper tier metrics 114 are shown below in Table 1.
Examples of middle tier BPO metrics 240 are shown below in Table 2.
Examples of middle tier AO metrics 244 are shown below in Table 3.
Examples of engagement extension metrics are shown below in Table 4.

Table 1- Upper Tier Metrics Business Value Delivered Category Engagement Margin Workforce Mix Cash Flow Service Excellence Category Engagement Satisfaction Percentage of SLAs Met Percentage of Critical SLAs Met Percentage of SLAs Missed More than Once Fees at Risk Process Excellence Category CQMA Process Compliance Organizational Excellence Category Average Training Hours Per Employee Employee Satisfaction Rating Percentage of Unplanned Employee Attrition Headcount Variance Table 2 - Middle Tier Metrics BPO Metrics Business Value Category Revenue Per FTE
Total Revenue Gross Margin Gross Margin %
EVA Charges Controllable Income Controllable Income %
Payroll % of Revenue Service Excellence Category No. High Impact Exceptions & Service Issues CQMA Engagement Satisfaction Process Excellence Category Overheads % of Revenue Key Accounts Not Fully Reconciled %
Organization Excellence Category Total Headcount (FTEs) Ave. Cost Per Seat % Days Absence Overtime %

Table 3 - Middle Tier Metrics AO Metrics Business Value Category Return on Investment Average Cost per Service Request Speed to Market Average Cost Per Hour % Offshore Resources Service Excellence Category End User Satisfaction Rating CQMA Engagement Satisfaction SLA Compliance Percentage Performance to Budget Performance to Schedule Response Time by Priority Resolution Time by Priority Backlog Volumes by Priority for Support Backlog Volumes by Priority for Development Average Age of Backlog by Priority for Support Average Age of Backlog by Priority for Development Process Excellence Category Fault Rate Defect Rate Percentage of Rework Peer Review Execution Rate Percentage of KPAs Achieved (CMM Level n) Thousand Lines of Code (KLOC) or Function Point per Employee for Development Thousand Lines of Code (KLOC) or Function Point per Employee for Support Organization Excellence Category Employee Utilization and Chargeability Percentage of Planned Employee Attrition Table 4 - Engagement Extension Metrics BPO Metric Example TIO Metric Example AO Metric Example Percentage error rate of Percentage of The percent of existing accounts payable, based network downtime resource capacity spent on the number of correct, during the period on incorrect, and total working maintenance number of invoices requests traced to all processed. defects and faults.

[047] An engagement may span multiple types of outsourcing. For example, an engagement may include both AO and BPO. The command center 100 may apply metrics from one or more tiers of metrics to the engagement. In the example given above, the command center 100 may apply metrics from both Table 2 and Table 3 to the engagement.

[048] In addition, the command center 100 may display or otherwise make available any of the metrics for internal review, for example by personnel employed by the outsourcing provider. The command center 100 may also display or otherwise make available any of the internal metrics for review by personnel associated with the engagement itself. Accordingly, the command center 100 may communicate metrics that the engagement may employ to determine operational and service level performance of the outsourcing provider handling their outsourced operations.

[049] The outsourcing operations sites 232-238 collect engagement outsourcing data for each outsourcing function performed on behalf of each engagement. For example, the operations site 238 may collect TIO data such as the number of service requests handled, average response time, the nature of each service request, the location of each requestor, and other TIO
data. Outsourcing operations sites that perform AO functions may collect AO
data including application name and version outsourced, amount of downtime per hour, day, month, or other time frame, performance of the outsourced application, number of simultaneous executions of the outsourced application, and other AO data. Similarly, outsourcing operations sites that perform BPO
functions may collect BPO data including accounts payable data, fixed asset accounting data, billing data, general accounting data, capital and expense projection accounting data, reimbursement data, cash application data, tax accounting data, credit and collection data, time and expense data, finance center management data, and vendor payment data, and other BPO data.

[050] In one implementation, the command center 100 may automatically schedule transfer of the engagement outsourcing data from each operations site 232-238 to the command center 100. For example, the command center 100 may periodically execute file transfer programs that retrieve files of engagement outsourcing data and store them in the engagement operations data database 202. The command center 100 also may collect outsourcing data through manual data input via a keyboard, mouse, or other data entry interfaces, through manual data transfer on a floppy disk or other medium, or through any other data entry or data transfer mechanism.

[051] The command center 100 also may perform pre-processing operations on the engagement outsourcing data. For example, the command center may process the engagement outsourcing data for storage according to a common data model for a data warehouse, including dimension tables and fact tables, in the engagement operations database 202. The command center 100 may thereby consistently store, retrieve, and process the data across all engagements during subsequent processing operations.

[052] Instances of dimension tables and fact tables in the engagement operations database 202 may store metric information. The dimension tables may provide underlying numerical measures and keys that relate facts to the dimension tables. The fact tables may store numeric or other fields that may represent metric data or other data applicable at the intersection of the dimension keys in the fact table.

[053] As one example, a measurement dimension table may be defined to store fundamental descriptors of a metric. Additional measurement dimension tables may be established to setup temporal dimensions such as a yearly, quarterly, monthly, weekly, daily, or time dimension. The engagement operations database 202 also establish fact tables for market unit or operating group, 'outsourcing capability offering, and delivery locations or geographic areas on a yearly, quarterly, or monthly basis that store data according to the established dimensions.

[054] The processed metrics database 204 may also include dimension tables and fact tables in a data warehouse model for the processed metrics.
The metrics processing programs 110 may access the processed metrics database 204 as one step in generating the reporting elements discussed below. The processed metrics database 204 may include a metric dimension, monthly dimension, and a comment dimension. A fact table for metric values may then organize metric values according to monthly key and a metric key.

[055] The processed metric database 204 may further organize metric data in one or more multi-level hierarchies. For example, the processed metrics database 204 may define hierarchical fact tables to organize the processed metric data. The hierarchical fact tables may establish one or more levels of geographic hierarchy, customer hierarchy, or service hierarchy, as examples.

[056] In operation, the processor 102 the command center 100 executes metrics processing programs 110 at any scheduled period, frequency, or time slot, or according to any other execution schedule, optionally selected by each engagement. The metrics processing program 110 retrieves the engagement outsourcing data from the database 202 and applies the metrics appropriate for each engagement to the engagement outsourcing data. Processed engagement metrics result and may be stored in the processed metrics database 204.

[057] As an example, the command center may process BPO metrics for Engagement C. Accordingly, the metrics processing program 110 may retrieve an engagement extended metric set for the BPO offering. For example, the engagement extended metric set may include the upper tier metrics 114, the middle tier BPO metrics 240, and the engagement C BPO
extension metrics 250. The metrics processing program 110 may also retrieve the engagement C outsourcing data from the database 202.

[058] The metrics processing program 110 applies the engagement extended metric set to the engagement C outsourcing data. Processed engagement C BPO metrics result, and may be stored in the processed metrics database 204. The metrics processing program 110 may also prepare reporting elements for the user interface 122 to display.

[059] To that end, the metrics processing programs 110 may also include graphical, textual, or audible reporting tools. For example, the metrics processing programs 110 may generate scorecards, dashboards, line charts, bar charts, pie charts, or any other graphical reporting element that presents processed metrics. Examples of suitable metrics processing programs 110 including programs for service level management, application availability, network management, database monitoring, or systems management include Sharepoint, Microsoft Reporting Services, and SqlServer products available from Microsoft, Help Desk/ Call Center Management Tools available from SupportWorks, Remedy, Siebel, and Peregrine, ERP Systems/ Workflow Tools available from SAP, Oracle, Mercury ITG, EAI Tools such as Microsoft BizTalk, Tibco, and SeeBeyond, Business Service Management/
Infrastructure Monitoring Tools, such as BMC SIM (formerly Mastercell), Proxima Centauri, Managed Objects Formula, Smarts SAM Solution, HP
Overview and Add-ons, and Mercury Interactive Topaz, Process Management Tools such as Nimbus Control ES, Documentum, Crystal Reports software, Crystal Enterprise software, Crystal Analysis software, Performance Manager, and/or Dashboard Manager, software available from Business Objects of San Jose, California. Other programs may be employed in addition to or as alternatives to those noted above.

[060] The command center 100 displays processed metrics through the user interface 122. More specifically, the command center 100 may accept an engagement login at the engagement information portal 214 or an internal login at the internal information portal 212. Both portals 212, 214 may be combined into a single portal. In one implementation, the command center 100 may employ Sharepoint Portal Server (TM) software available from Microsoft of Redmond Washington to provide secure login and access to the command center 100.

[061] Once an engagement has logged in, the engagement may access the metric reporting screens appropriate for that engagement. Continuing the example above, engagement C may access BPO reporting screens that report the results of the metrics processing discussed above. While the reporting screens may include graphical or textual data, they may also include multi-media content such as text messaging, email, and/or audible reporting.

[062] The command center 100 builds one or more metric reporting screens appropriate for each engagement. Accordingly, the command center 100 may retrieve the display templates 210 established for engagement C.
The display templates 210 may take the form of user interface screen definitions, including graphical element layout, positioning, color, size, font, window or frame size and positions, and other graphical specifiers, links between screens, and any other user interface element or object that may appear on a screen.

[063] A different display template may be established for each engagement, for each outsourcing offering, or for each engagement and for each outsourcing offering, as examples. Alternatively, each engagement may share one or more portions of a display template 210 to define common elements on the screens for each engagement. The display templates 210 may define htmi, xml, or other web-based documents, suitable for display in a browser window through which the engagement interacts with the command center 100.

[064] The reporting rulesets 206 may include rules that specify what form, shape, color, or other aspect a reporting element may take, based on the results of the metrics processing. The reporting rulesets 206 may be established for each engagement, for each outsourcing offering, for each reporting element, or for any combination of those three. Figure 3 shows a high level example of a reporting rule 300 for a Sales & Marketing reporting element (e.g., a stoplight that may display a Red light, a Yellow light, or a Green light).

[065] The reporting rule 300 includes a Green condition 302, a Yellow condition 304, and a Red condition 306. In addition, the reporting rule 300 includes an engagement identifier 308 and a metric identifier 310. The engagement identifier 308 provides a numeric, string, or other data type entry that associates the reporting rule 300 with a particular engagement.
Similarly, the metric identifier 310 provides a numeric, string, or other data type entry that associates the reporting rule with a metric, in this case a Selling, General, and Administrative (SG&A) metric.

[066] The green condition 302 specifies that when SG&A expenditures are below a lower threshold (e.g., $350,000), the command center 100 should display the SG&A reporting element as a Green light. Similarly, the yellow condition 304 specifies that when SG&A expenditures are between the lower threshold and an upper threshold (e.g., $500,000) the command center 100 will display the reporting element as a Yellow light. The red condition 306 specifies that when SG&A expenditures are above the upper threshold, the command center 100 will display the reporting element as a Red light.

[067] Note that the metrics may be associated with outsourcing offerings in general, but may also be categorized within an outsourcing offering. As one example, a Return on Investment (ROI) metric may be part of a Business Results Achieved category of BPO offering metrics. The category may include additional metrics, such as an Average Cost per Service Request metric. Detailed examples of BPO metrics, categories, metric calculations and Red / Yellow / Green display reporting rules are given below in Tables 5-8. Note that any given metric need be not limited to a single outsourcing offering. Instead, a metric may be applied to multiple outsourcing offerings.

[068] Taking an example from Table 5, the ROI metric may be calculated as a percentage by subtracting program costs from program benefits, then dividing by program costs, then multiplying by 100. The user interface 122 may display the ROI metric as a percentage, as a decimal, using a graphic, or using any other reporting element. A Target value may be selected for ROI.
When ROI is greater than the Target, the user interface 122 may display the reporting element in Green; when the ROI is in jeopardy of not meeting the Target, the user interface 122 may display the reporting element in Yellow;
when the ROI is less than the Target, then the user interface 122 may display the reporting element in Red.

Table 5 - Business Value Delivered Category Sub-Category Metric Description Calculation Example Business Return on This metric measures the ((Program Benefits -Results Investment ROI business results achieved Program Costs) /

Achieved by comparing development Program Costs) * 100 costs against realized benefits based on actuals.

Business Average Cost This metric measures the Total Cost of Results per Service business results achieved Maintenance / Total Achieved Request by measuring the cost per Number of Service service request adjusted for Requests volume of requests.

Cycle Time Speed to Market This metric compares the Time to Complete Project Improvements number of work days / Time to Complete required to complete a Project of Similar Size project to other projects of similar size.

Financial Engagement This metric measures the (Net Revenue / Total Excellence Margin margin percentage for the Program Costs) * 100 engagement.

Financial Average Cost This metric measures the Total Cost of Services /
Excellence per Hour average cost of services Total Hours delivered by the IT
Organization.
Financial Percentage of This metric measures the (Number of Offshore Excellence Offshore percentages of offshore Personnel / Total Number Resources personnel of Personnel )* 100 Financial Workforce These metrics measure the (Total Number of Excellence Percentages makeup of the workforce: Consulting Personnel /
Consulting, Services, Total Number of Solutions Workforce, and Personnel) * 100 Contractor.
(Total Number of Services personnel /
Total Number of Personnel) * 100 (Total Number of Solutions Workforce personnel / Total Number of Personnel) * 100 (Total Number of Contractors / Total Number of Personnel) *

Table 5 continued - Business Value Delivered Metrics Metric Field Descriptions Green Yellow Red Return on Program Benefits may be > Target Slightly Significantly Investment (ROI) the Total Benefits (profit) below below Target from a Program. Target Program Costs may be the Total Cost of Program.
The user interface may display this metric as a percentage.

Average Cost per Total Cost of maintenance < Target Slightly Significantly Service Request may be the Direct Labor + above above Target Indirect Labor + Non-labor Target Costs.
Number of service requests may be the Total number of requests over a pre-selected time period.

The user interface may display this metric as a dollar value.

Speed to Market Time to Complete Project < Target Slightly Significantly may be the Number of above above Target Months to Complete Target Projects.
Time to Complete Projects of Similar Size may be the Average number of months to complete projects of similar size.

The user interface may display this metric as a comparison of months.

Engagement Net Revenue may be the > Target Slightly Significantly Margin Total Revenue from below below Target Services. Target Total Cost may be the Total Cost of Services Delivered.
The user interface may display this metric as a percentage.

Average Cost per Total Cost of Services may Target Slightly Significantly Hour be the Total Dollar cost of above above Target services. Target Total Hours to Deliver Service may be the Total Number of Hours needed to deliver requested services.
The user interface may display this metric as a dollar value.

Onshore: Total Number of Offshore > Target Slightly Significantly Offshore Personnel may be the Total below below Target Percentages number of personnel Target categorized as Offshore.
Total Number of Personnel may be the Total number of offshore personnel + Total Number of Onshore Personnel.

The user interface may display this metric as a percentage of total personnel.

Workforce Total Number of > Target Slightly Significantly Percentages Consulting, Services, SWF, below below Target Contractor Personnel may Target be the Total number of each type of personnel.

Total Number of Personnel < Target Slightly Significantly may be the Total number of based on above above Target personnel in IT Targets set Target based on organization. for each Targets set for type of each type of The user interface may Personnel Employee display this metric as percentages of types of personnel Table 6 - Process Excellence Category Sub-Category Metric Description Calculation Example Quality/Reliability Fault Rate This metric may measure Total Number of Faults the number of faults per Reported / Total thousand lines of code Delivered FPs or KLOCs (KLOC), or per function point or feature point or FPs detected after the software has been Total Number of Faults delivered to production. Reported / Total number of Earned Hours Engagements with size data available may measure based on FPs or KLOCs.
Engagements without size data may measure based on Effort or Earned Hours.

Quality/Reliability Defect Rate This metric may measure Total Number of Defects the number of defects per Reported / Total KLOC (or FPs) detected Delivered FPs or KLOCs after the software has been delivered to the next or phase(s) of development.
Number of Defects Detected after software is delivered to next phase / Actual Original Development Effort Engagements with size data available may measure based on FPs or KLOCs.
Engagements without size data may measure based on Effort or Earned Hours.

Quality/Reliability Percentage of This metric may measure (Rework Hours /
Total Rework the amount of effort spent Actual Original performing rework on Development Hours) *
deliverables from previous 100 phases.
Process Peer Review This metric may measure (Total Number of Peer Compliance Execution Rate how well Peer Reviews Reviews Conducted /
are being executed. The Number of Peer peer reviews examine the Reviews Planned) * 100 content in scope deliverables.
Process SQA Execution This metric may measure (Total number of SQA
Compliance Rate how consistently Software reviews Conducted /
Quality Assurance (SQA) Number of SQA
Reviews are being Reviews Planned) * 100 executed.

Process SQA Process This metric may measure (Total Number of SQA
Compliance Compliance the percentage of time Process Review defects standard IT processes / Total SQA Process were followed. Each Review O ortunities *

project having an SQA 100 Review may use a standard SQA Plan. A
Standard SQA plan may ask a series of questions to determine if the appropriate processes were followed. A defect may be any time there is a No-Response to a SQA
question.

Certification Percentage of This metric may measure (Number of KPAs Met /
Targets Achieved KPAs Achieved the organization's ability to Total Number of KPAs (CMM Level n) achieve required Key assessed) * 100 Process Areas (KPAs).

Productivity KLOC/ Function This metric may measure Total Number of KLOCs Levels Point per Full the development (or FPs) Delivered /
(Development) Time Equivalent productivity of the Total Number of FTEs (FTE) organization by KLOC or FP.
Productivity KLOC/Function This metric may measure Total Number of KLOCs Levels Point per FTE the maintenance (or FPs) Supported /
(Maintenance) productivity of the Total Number of FTEs organization by KLOC or FP.

Table 6 continued - Process Excellence Metrics Metric Field Descriptions T Green Yellow Red Fault Rate Total Number of Faults Reported Target Slightly Significantly may be the Total number of Faults above above Target (defects found post production) Target 4 reported in a selected timeframe.
Total Number of Delivered FPs or KLOCs may be the Function points or KLOCs that account for the engagement's application software (newly developed and/or reused), the execution architecture, and the development architecture.

Total number of Earned Hours for Release may be the Total number of hours earned for a release within a specific period of time.

The user interface may display this metric as Fault Rate per Size or Effort.

This metric may also be tracked as an SLA metric.

Defect Rate Total Number of Defects Reported < Target Slightly Significantly may be the Defect Rates for all above above Target phases completed period-to-date, Target averaged. Initially, each project may be weighted equally.
However, relative weighting (in terms of project size) may be incorporated based on engagement specifications.

Total Number of Delivered FPs or KLOCs may be the Function points or KLOCs developed that are delivered to the next development phase.

Actual Development Effort may be the Total Number of original development hours.

The user interface may display this metric as Defect rate per Size or Effort.

This metric may also be tracked as an SLA metric.

Percentage of Rework Hours may be the Total < Target Slightly Significantly Rework number of hours spent on post above above Target implementation rework, such as Target fixing deliverables from prior phases.

Total Development Hours may be the Total number of development hours (direct hours) for developing deliverables.

The user interface may display this metric as a percentage.

This metric may also be tracked as an SLA metric.

Peer Review Number of Peer Reviews conducted > Target Slightly Significantly Execution Rate may be the Total number of reviews below below Target conducted during review phase. Target Number of Peer Reviews Planned may be the Total number of reviews planned during review phase.

The user interface may display this metric as a percentage.

SQA Execution Number of SQA reviews conducted > Target Slightly Significantly Rate may be the Total number of SQA below below Target reviews conducted for in scope Target projects.
Number of SQA Reviews planned may be the Total number of reviews planned for in scope projects.

The user interface may display this metric as a percentage.

SQA Process Total Number of SQA Process < Target Slightly Significantly Compliance Review Defects may be the number above above Target of "No" responses to an SQA Target Process question.

Total SQA Process Review Opportunities may be the Total applicable questions asked (total "Yes" + "No" responses).

The user interface may display this metric as a percentage.

Percentage of Number of KPAs met may be the > Target Slightly Significantly KPAs Achieved Total number of KPAs that the below below Target (CMM Level n) organization meets during an Target assessment.
Total Number of KPAs assessed may be the Total Number of KPAs being assessed during an assessment.

The user interface may display this metric as a percentage.

KLOC/ Total Number of KLOCs (or FPs) > Target Slightly Significantly Function Point Delivered may be the Total number below below Target per Full Time of KLOCs or FPs developed during Target Equivalent the development effort.
(FTE) Total Number of FTEs may be the Total number of FTEs included in Development Effort.

The user interface may display this metric as rate of KLOC or FP per FTE.

KLOC/Function Total Number of KLOCs (or FPs) Target Slightly Significantly Point per FTE Supported may be the Total number below below Target of KLOCs or FPs in an application Target portfolio.
Total Number of FTEs may be the Total number of FTEs providing maintenance and support.

The user interface may display this metric as KLOC or FP per FTE.

Table 7 - Service Excellence Category Sub-Category Metric Description Calculation Example Engagement Engagement This metric measures the (Total number Satisfaction Satisfaction percentage of Executives Executives 'satisfied' /
Rating who are 'satisfied' with Total Number of overall IT Service Executives Surveyed) *
delivered. 100 Engagement End User This metric measures the (Total number End Satisfaction Satisfaction percentage of End-Users Users 'satisfied' / Total Rating who are 'satisfied' with the Number of End Users IT service they received. Surveyed) * 100 SLA Compliance SLA This metric measures the (Total Number of SLAs Compliance organization's ability to met / Total Number of Percentage meet SLA requirements. SLAs) * 100 Outcome Performance to This metric measures the (Number of Projects Predictability Budget percentage of Projects completed on or under completed on or under budget / Number of Budget. Projects completed YTD) * 100 Outcome Performance to This metric measures the (Number of Projects Predictability Schedule percentage of Project completed on time or Start and End Dates early / Number of which occur "on time". Projects completed YTD) * 100 Response/ Response Time This metric measures the (Total Number of Calls Turnaround Time by Priority percentage of calls responded to within responded to within established Response established parameters. Time by Priority / Total Number of Calls) * 100 Response/ Resolution This metric measures the (Total Number of Calls Turnaround Time Time by Priority percentage of calls Resolved within resolved within established Resolution established parameters. Time by Priority / Total Number of Calls) * 100 Throughput Backlog This metric measures the Total Number in Backlog Volume by size of the backlog by by Priority Priority priority and gives the organization a sense of throughput ability.

Throughput Average Age of This metric measures the (Sum of all active Backlog by average age of the requests (Current Date -Priority backlog by priority and Request Start Date)) /
gives the organization a Total number of Active sense of their throughput Requests ability.
Table 7 continued - Service Excellence Metrics Metric Field Descriptions Green Yellow Red Engagement Number of Executives who are > Target Slightly Significantly Satisfaction 'satisfied' may be the Total Number below below Target Rating of Executives who are'satisfied' Target (e.g., selected a numerical designation (e.g., '4') for Satisfied on a survey) or'Very Satisfied' (e.g., selected a numerical designation (e.g., 'S') for Very Satisfied on a survey).

Total number of Executives surveyed may be the Total Number of Executives who responded to the Survey.

The user interface may display this metric as a percentage.

End User Number of End Users who are > Target Slightly Significantly Satisfaction 'satisfied' may be the Total Number below below Target Rating of End Users who are 'satisfied' (4) Target or 'Very Satisfied' (5).
Total number of End Users surveyed may be the Total Number of End Users who responded to the Survey.

The user interface may display this metric as a percentage.

SLA Total number of SLAs met may be > Target Slightly Significantly Compliance the Total number of SLAs met (for a below below Target Percentage specific time period). Target Total Number of SLAs may be the Total number of SLAs being tracked.

The user interface may display this metric as a percentage.

Performance Number of Projects completed on or Target Slightly Significantly to Budget under budget may be determined below below Target as being within +/- x% of Budget Target calculated as (Planned Effort -Actual Effort) / Planned Effort) *
100).

Number of Projects completed YTD
may be the Total number of projects completed YTD.

The user interface may display this metric as a percentage.

Performance Number of Projects completed on > Target Slightly Significantly to Schedule time or early may be determined as below below Target being within 'x' number of days of Target Schedule Completion calculated as (Planned Date - Actual Date) /
Planned Date) * 100).

Total Number of Projects completed YTD may be the Total number of projects completed within the YTD.
The user interface may display this metric as a percentage.

Response Total Number of calls responded to Target Slightly Significantly Time by within established Response Time below below Target Priority by Priority may be the Total number Target of calls reported, response time starts from the time the problem is reported and may be segmented by severity of business impact (Priority 1, 2, 3+).

Total Number of calls may be the Total number of calls generally segmented by severity of business impact (Priority 1, 2, 3+).

The user interface may display this metric as a percentage.

Resolution Total number of calls resolved > Target Slightly Significantly Time by within established Resolution Time below below Target Priority by Priority may be the Total number Target of calls resolved, resolution time starts from the point the problem is reported and may be segmented by business impact (Priority 1, 2, 3+).
Total number of calls may be the Total number of calls generally segmented by severity of business impact (Priority 1, 2, 3+).

The user interface may display this metric as a percentage.

Backlog Open Problems Previous Month < Target Slightly Significantly Volume by End. This metric may include the above above Target Priority Number of Problems that had open Target status on the last business day of the previous month (and may or may not have been closed since).
New Problems Current Month. This metric may also include the Number of New problems that have an opened or submit date in the current month.

Closed Problems Current Month.
This metric may also include the Number of Problems where status was changed to Closed in the current month Calculations that include this metric may omit problems that have an On Hold status.

Average Age Sum of all active requests may be < Target Slightly Significantly of Backlog by the Total age of all active requests. above above Target Priority Target Current Date may be the Date the backlog was queried.

Request Start Date may be the Original assigned start date.
Total Number of Active Requests may be the Sum of all active requests.

The user interface may display this metric as Average Age in days.

Table 8 - Organizational Excellence Category Sub-Category Metric Name Description Calculation Example Training Training Hours This metric measures the Total Number of training Effectiveness per Employee annualized hours per hours YTD / Total employee spent on Number of IT
process and technical Employees / 12 Months training. per Year Employee Employee This metric measures the (Total number IT
Satisfaction Satisfaction percentage of IT Employees who are Rating employees who are 'satisfied' / Total Number 'satisfied' with their jobs. of IT Employees Surveyed) * 100 Attrition Unplanned This metric measures the (Number of Unplanned Employee percentage of the Departures / Total Attrition workforce that leaves Number of IT
voluntarily (unmanaged). Employees) * 100 Attrition Planned This metric measures the (Number of Planned Employee percentage of the Departures / Total Attrition workforce that leaves as Number of IT
planned (managed). Employees) * 100 Utilization Employee This metric measures the Annualized Calculation:
Utilization/Char percentage of hours spent geability on IT activities. Total number of Direct hours / Total number of Planned Hours, where Planned Hours may be:
(Standard hours per FTE per year * Total Number of employees *
Number of Months / 12 Months per Year) And Monthly calculation 1- (Non-Direct Hours in Reporting Month / (Total Hours in Reporting Month)) x 100%

Both monthly and annualized data may be collected. Unit can choose one or both, both may be aggregated and reported.

Table 8 continued - Organizational Excellence Metrics Category Field Descriptions Green Yellow Red Training Hours Total number of training at Target Slightly Significantly per Employee hours YTD may be the Total above above Target number of hours categorized Target as training hours YTD. or or Total Number of IT Significantly employees may be the Total Slightly below Target number of IT employees. below Target Number of Months may be the Total number of months YTD.

The user interface may dis la this metric as training hours.

Employee Total number of IT > Target Slightly Significantly Satisfaction Employees who are'satisfied' below below Target Rating may be the Total number of Target IT Employees who respond as 'satisfied' or 'very satisfied' according to a survey.

Total number of IT
Employees surveyed may be the Total Number of IT
Employees who responded to the Survey.

The user interface may display this metric as a percentage.

Unplanned Total number of unplanned < Target Slightly Significantly Employee departures may be the above above Target Attrition Employees who leave the Target organization in an unplanned manner.

Total number of IT employees may be the Total number of IT Employees.

The user interface may display this metric as a percentage.

Planned Total number of planned at Target Slightly Significantly Employee departures may be the above above Target Attrition Employees who leave the Target organization in a planned or manner. or Significantly Total number of IT employees Slightly below Target may be Total number of IT below Employees. Target The user interface may display this metric as a percentage.

Employee Total hours spent on IT at Target Slightly Significantly Utilization/Charge activities categorized as above above Target ability productive YTD (directly Target related to support and/or or development) may be the or Total hours spent on activities Significantly categorized as IT YTD. Slightly below Target below Non-Direct Hours may be Target those categorized as non-direct (administration, general training, management, career development, general meetings) Standard hours may be the Standard hours per FTE per year (e.g., 2080 hours) reduced by anticipated holidays, training, vacation time, sick time, and jury duty.
Total Number of IT
employees may be the Total number of IT employees.
Number of Months may be the Total number of months YTD.

The user interface may display this metric as percentage of hours.

[069] Each exemplary category shown in Tables 5-8 may be represented on the user interface using any graphical, textual, audible or other reporting element. In some implementations, the reporting element for a category may represent a summary, average, or other consolidated view of multiple underlying metrics in the category. The category presentation may thereby represent a"rolled-up" view of one or more metrics into fewer display elements.

[070] Links, buttons, menus, or other user interface elements may then implement a drill-down sequence through a set of progressively more detailed views of the underlying metrics. For example, a reporting element may provide a rolled-up summary of the Business Results Achieved category of BPO metrics. The user interface 122 may then include a link associated with the rolled-up summary to a display of the individual underlying metrics, as examples, the ROI and Average Cost per Service Request metrics.

[071] Figure 4 shows one example of metrics reporting screens 400 linked by a drill-down link set to progressively more detailed views of outsourcing metrics. In Figure 4, an overall view screen 402 and a geographical detail view screen 404 are present. The Overall view screen 402 includes a rolled-up reporting pie chart 406 for the business value metric, a rolled-up reporting pie chart 408 for the service excellence metric, a rolled-up reporting pie chart 410 for the process excellence metric, and a rolled-up reporting pie chart 412 for the organizational excellence metric. The pie charts 406-412 may provide a view, at the highest level, of the performance of the corresponding metrics. For example, the pie chart 406 indicates that 56%
of metrics are performing poorly (shown as red in the pie chart 406) and that 44% of metrics are performing adequately (shown as yellow in the pie chart 406).

[072] The overall view may roll up metrics across multiple dimensions such as geography, operating group, and engagement. The overall view screen 402 also includes drill-down links to more detailed metric reporting screens. As shown in Figure 4, the overall view screen 402 includes a geography link 414, an operating group link 416, and an engagement link 418.
The command center 100 responds to a click on a link 414-418 by presenting a more detailed metric reporting screen according to the particular link clicked.

[073] For example, when the command center 100 determines that the geography link 414 is activated, the command center 100 may display the geographical detail screen 404. The geographical detail screen 404 provides rolled-up summaries 420, 422, and 424 of the metrics by geography, in this case by the Americas, Asia Pacific, and Europe.

[074] The rolled-up pie chart 406 may represent a summary of the rolled-up pie charts for business value across each geographical region 420, 422, and 424. Similarly, the rolled-up pie charts 408, 410, and 412 may also represent summaries of the rolled-up pie charts for service excellence, process excellence, and organizational excellence across the geographical regions 420, 422, and 424. Additional drill down links may be provided to reporting screens with further details underlying the pie charts shown in Figure 4, including links all the way down to screens that display the actual metrics and the value of the metrics.

[075] The metrics may be consolidated according to any criteria. For example, all BPO metrics may be consolidated into fewer (e.g., one) overall reporting element that represents an overall status of a BPO offering. As another example, multiple BPO metrics may be rolled-up into multiple sub-categories, the sub-categories may be rolled-up into higher level categories, and the higher level categories may be rolled-up into an overall BPO reporting element. Due to their customized nature, the command center 100 may optionally refrain from rolling-up the engagement extension metrics 246-262 into a summary view. Instead, the engagement extension metrics 246-262 may be reported separately from a rolled-up view of other metrics, such as standardized upper tier metrics 114 or middle tier metrics 116.

[076] To summarize or roll-up underlying metrics, the command center 100 may apply any summary function to the metrics. For example, the command center 100 may apply an average or weighted average function, or may apply any other function that summarizes multiple inputs into fewer outputs. For example, as shown in Figure 4, the command center 100 has summarized in one pie chart 406 that 56 percent of the metrics underlying business value are performing poorly and that 44 percent of the metrics underlying business value are performing only adequately.

[077] Alternatively or additionally, one or more metric in each outsourcing offering may be summarized in a rolled-up outsourcing offering display element for each outsourcing offering, one or more of which may be rolled-up in to a summary across all outsourcing offerings, or sets of outsourcing offerings. The user interface 122 may then provide a drill-down link set to progressively more detailed views of the underlying rolled-up summaries, down to the individual underlying metrics, parameters, and target values that determine the metrics.

[078] Figure 5 shows an example of an engagement metric reporting screen 500 with drill-down links to underlying metric reporting screens.
Additional examples are shown in Figures 11-13 and described below. In Figure 5, the reporting screen 500 is defined for the engagement Acme Pharmaceuticals Inc, according, for example, to the display templates 210.
The display templates 210 may specify which metrics are displayed for the engagement on which screen, the location of each reporting element for each metric, the graphical reporting element to be used for each metrics, sizes, shapes, colors, orientations, or any other information relating to the content of the reporting screen 500. The reporting screen 500 includes a business performance dashboard 502 and a service performance dashboard 504.

[079] The business performance dashboard 502 presents a rolled-up view or summary of the status for selected aspects of a BPO offering. The dashboard 502 includes a delivery locations summary 506 in addition to a business value delivered summary 508 and a service performance summary 510. The reporting elements for the summaries 506-510 are stoplights, although other graphics, text representations, multi-media representations, or other representations may alternatively or additionally be employed as reporting elements.

[080] The summaries 506-510 may include links to views of their underlying metrics. For example, the business value delivered summary 508 may include a link 512 to the summary reporting screen 600 shown in Figure 6. The reporting screen 600 provides reporting elements 602, 604, 606, 608, 610, and 612 in the form of dashboards for the underlying components of the business value delivered summary 508. Each reporting element 602-612 may provide a drill-down link to additional detail on each underlying metric.

[081] The reporting elements 602-612 provide additional information on the metrics underlying the business value delivered summary 508. As shown in Figure 6, the reporting element 602 provides a summary view of a Reduction in Risk metric, the reporting element 604 provides a summary view of a Reduction in Operating Costs metric, and the reporting element 606 provides a summary view of a Reduction in Time to Market metric. The reporting element 608 provides a summary view of an Increase in Revenue Metric, the reporting element 610 provides a summary view of a Reduction in Working Capital metric, and the reporting element 612 provides a summary view of the business value delivered summary 508.

[082] The reduction in risk report 602 includes a graphical reporting element 614, a textual metric reporting element 616, and a trend indicator element 618. The graphical reporting element 614 is shown as a pie chart, but may take any other graphical form to convey information about one or more metrics to the engagement. Similarly, the textual metric reporting element 616 may set forth parameter values, metric names, or other textual elements that convey information about one or more metrics to the engagement. The trend indicator element 618 may show an up-trend, flat-trend, down-trend, or other trend information on a pre-selected time frame or interval with respect to the metrics. Each report 602-612 may include more, fewer, alternate or additional reporting elements to form the report.

[083] The command center 100 provides business value delivered information for review and evaluation by key business decision makers.
Although shown as a stoplight, the business value delivered summary 508 may be presented in many other ways and in many other forms. More generally, the command center 100 translates underlying low-level metrics to higher level business value delivered metrics. An example in the context of a Communications Service provider that supports Digital Subscriber Lines (DSL) is given below.

[084] The DSL service provider is assumed to have outsourced Order to Cash (BPO) and supporting application systems (AO). Take-up of new product by customers of the DSL service is critical because of fierce competition for that customer, because take-up has a significant impact on revenue cash flow and operating costs, and because industry analysts respond to take-up in their analyses. The DSL business engages in a multi-step order process that consumes valuable time before a new customer may be billed. The time consumed before billing may occur is sometimes referred to as the cycle time.

[085] The order process may include sales and customer contract activities, through which the DSL business identifies a new customer and generates a contract for the new customer; ordering activities, through which the DSL business enters contract and customer data in their processing systems; engineering activities, through which technical personnel determine whether the customer can be connected, or assign technicians to setup the service; service provision activities, through which the technicians take actions to install or start the service; service activation and test activities, through which the DSL business turns on the service and verifies its operation; and billing, at which point the DSL service can being earning revenue from the customer. Each activity may have underlying activities. For example, the ordering activities may include order entry, order qualification, credit verification, and meeting network requirements.

[086] The outsourced functions may be characterized by many underlying metrics. As examples, the metrics may include the amount of time spent in each phase of the order process by product, geography, or other factor; the numbers and types of customers pre-sold, but then canceling; the numbers, types, and amounts of orders that require 'rework', the number of 'escalations' employed to expedite installation and delivery of the service; the number of passed and filed pre-qualifications; the number of activated services that match services subsequently billed (revenue leakage); the volumes of orders handled by agents and engineering (efficiency); and the volumes and lengths of calls from customers checking on order status or service status. Additional or different metrics may also be gathered.

[087] The command center 100 may apply the business value delivered rules 207 to one or more detailed operational metrics to generate business value delivered metrics. The business rules 207 thereby translate detailed underlying metrics into higher level business value outcomes. The business value outcomes may vary widely between engagements. Six examples are shown in Figure 6: Reduction in Risk, which may depend on a customer commitments met on time metric; Increase in Revenue, which may depend on revenue leakage and/or revenue recognition metrics; Reduction in Operating Costs, which may depend on operating costs metrics; Reduction in Working Capital; Reduction in Time to Market; and Business Value Delivered, which may depend on a customer satisfaction metric and/or a percentage of customer commitments met on time metric. One or more of the business value outcomes may be included in an Economic Value Added (EVA) calculation and reported.

[088] Applying the business value delivered rules 207 to the underlying metrics yields the business value outcome metrics. A business value delivered rule 207 may be a function with low level metric input parameters and a high level business value output. As one example, a business value delivered rule 207 may be established to produce the Increase in Revenue business value outcome according to: Every x% improvement in or reduced ratio of customers leaving after a sale, but before order activation, yields y%
of Increase in Revenue. As another example, every a% improvement or reduced number of days cycle time yields b% of Increase in Revenue through earlier revenue recognition.

[089] In some implementations, the command center 100 may present metric summaries organized by geographical location. With regard to Figure 7, for example, a geographical location screen 700 is shown. The screen 700 may include a map 702 and pinpoint locations on the map 702. The Prague pinpoint location 704 and the Minneapolis pinpoint location 706 are labeled in Figure 7. An expansion window 708 is also present in Figure 7.

[090] In general, the map 702 may display a pinpoint location anywhere services are offered or performed for an engagement. Each pinpoint location may include a reporting element (e.g., a stoplight) that provides a rolled-up summary of the status, performance, or other aspect of an outsourcing offering occurring at that location. In addition, each pinpoint location may include a link or hotspot that upon activation (e.g., a mouse click or a mouse-over event) causes the command center 100 to show an expansion window for that pinpoint location.

[091] The command center 100 displays the expansion window 708 when the Prague pinpoint location 704 is active. The expansion window 708 may display one or more rolled-up metrics 710, individual metrics, or other conditions that affect the status of the reporting element shown for the Prague pinpoint location 704 on the map 702. As shown in Figure 7, the expansion window 708 lists underlying metrics for outsourcing operations in Prague.

[092] The command center 100 may respond to a click or other activation of a pinpoint location to display a reporting screen with additional detail underlying the status of the location. For example, Figure 8 shows a reporting screen 800 that reports additional detail for the Prague pinpoint location 704.
The additional detail may be the same or different than that presented in the expansion window 708, for example.

[093] As shown in Figure 8, the reporting screen 800 displays a detail panel 802 with the map 702. The detail panel 802 includes a more detailed display of reporting elements for metrics or other conditions underlying the overall status of the Prague pinpoint location 708. For example, the reporting element 804 is a stoplight that may assume a Green, Yellow, or Red color.

[094] In addition, the detail panel may provide a type indicator 806 and a description 808 for each reporting element. The type indicator may specify whether the associated reporting element represents a single metric, a set of metrics, an outsourcing site, or another underlying parameter. For example, the reporting element 804 is a Site type reporting element associated with an outsourcing site in Prague for ACME Pharmaceuticals.

[095] Each reporting element may include a link to additional more detailed reporting elements. For example, the reporting element 804 may provide a link to individual metrics, metric summaries, or other parameters underlying the Prague ACME Pharmaceuticals site. In other words, the reporting element may inciude a link or links from rolled-up metrics to more detailed underlying rolled-up metrics or individual metrics.

[096] The location screen 700 may display pinpoint locations relevant to a particular engagement. For example, the delivery locations summary 506 in Figure 5 may include a link to a location screen that provides additional detail underlying the status of the delivery locations summary 506. The additional detail may be one or more pinpoint locations with an associated rolled-up summary of outsourcing operations at that location.

[097] Alternatively or additionally, location screens may be employed for internal review of one or more outsourcing offerings through the internal investigation portal 212. Accordingly, the location screen may reflect rolled-up summaries of outsourcing offerings on a local or global scale, across one or more engagements. As will be explained in more detail below, specific internal logins may be permitted access to selected sets of metrics, subsets of metrics, processed or unprocessed outsourcing data, or other data. The baseline metrics, subsets of baseline metrics, and other metrics may be geographically organized on a location screen such as that shown in Figure 7.

[098] Returning to Figure 2, the service agreement rulesets 208 establish rules that may govern overall performance of outsourcing offerings. In one implementation, the service agreement ruleset 208 implements conditions included in Service Level Agreements (SLAs) or any other agreement setting forth expectations of performance, availability, or other characteristics of the outsourcing offering itself.

[099] Figure 9 shows an example of a service agreement rule 900 for Availability of an ITO offering. The service agreement rule 900 includes an engagement identifier 902, an offering identifier 904, and an agreement characteristic identifier 906. The service agreement rule 900 also includes a Green condition 908, a Yellow condition 910, and a Red condition 912.

[0100] As shown, the engagement identifier 902 associates the rule 900 with engagement C. The offering identifier 904 associates the rule 900 with the BPO offering (for engagement C), while the characteristic identifier 908 associates the rule 900 with Availability of the BPO functions. The Green condition 906 specifies that when the BPO functions are available for more than 99% of the time, the command center will display the availability reporting element as Green. The Yellow condition 910 specifies that availability between 80% and 99% results in a Yellow availability reporting element, while availability under 80% results in a Red availability reporting element according to the Red condition 912.

[0101] The conditions, characteristics and rule formats are not limited to those discussed above. Rather, the rulesets 206-208 may include additional, conjunctive, disjunctive, or other logical connections of one or more rules and may specify other tests, other results, or other characteristics. The rulesets 206-208 may include default level rules applicable to one or more engagements, and may be extended or tailored as specified by each individual engagement for each outsourcing offering. The individual rules may be formed from equations or logical tests in an Excel (TM) spreadsheet, from conditional statements in a program, from rules supported and defined using the metric processing programs 110, or in any other manner.

[0102] Figure 10 illustrates one implementation of an access hierarchy 1000 that may authorize or deny access to data in the outsourcing command center 100. The access hierarchy discussed below is exemplary in nature only, and may vary widely between implementations. The access hierarchy may include access entries 1002, 1004, 1006, 1008, 1010, a login field 1012, and data access fields such as a metric access field 1014, and an engagement access field 1016. Additional or alternate access fields may be provided, for example that permit or deny access to specific types of reporting elements. The access hierarchy 1000 may be implemented by any permissions manager, such as a Lightweight Directory Access Protocol (LDAP) service, login, password, and access control provided by the metric processing programs 110 noted above, an operating system, or through another permissions control programs.

[0103] The login field 1012 specifies the command center login for which the access entry will apply. The metric access field 1014 specifies individual metrics or sets of inetrics for which the login has or does not have access.
The engagement access field 1016 specifies engagements or sets of engagements for which the login has or does not have access.

[0104] The access entries 1002-1008 may represent internal logins to the command center 100 for the company that provides the outsourcing offerings.
For example, the access entry 1002 applies to a Chief Outsourcing Executive login. According to the access entry 1002, the Chief Outsourcing Executive has access to all metric data for all engagements. According to the access entry 1004, an Outsourcing Assistant login 1004 permits access to BPO and TIO metrics for all engagements. In other words, the Outsourcing Assistant has access to a subset of metrics within the command center 100.

[0105] Similarly, the access entry 1006 permits the BPO Executive login to view BPO metrics for engagements A and C. Thus, the access hierarchy 1000 may provide access to a subset of inetrics and well as a subset of engagements. As another example, the access entry 1008 permits the TIO
Executive login to view all TIO metrics for engagement C.

[0106] The access hierarchy 1000 may also specify the logins for individual engagements. For example, the access entry 1010 may specify permissions for Engagement C. In particular, as shown in Figure 10, the access entry 1010 specifies that Engagement C has access to all of its own metrics.

[0107] The metric evaluation program 110 applies the metric set 112 to the engagement outsourcing data to generate the processed metrics data. For internal review, the user interface 122 and internal investigation portal 212 may display processed metrics in accordance with the access hierarchy 1000.
For example, the metric reporting screens (such as those shown in Figures 4-8) may include reporting elements for all metrics and all engagements for access by the Chief Outsourcing Executive. Optionally, the command center 100 may summarize or roll-up multiple processed metrics into one or more baseline metrics.

[0108] As specified by the access hierarchy 1000, the metric reporting screens in the internal investigation portal 212 may omit metric reporting elements for certain metrics or certain engagements. For example, the outsourcing assistant login 1004 may view BPO and TIO metrics for all engagements. Accordingly, the metric reporting screens for the assistant login 1004 may omit metrics other than BPO and T{O metrics so that the outsourcing assistant login views a subset of all metrics.

[0109] Figure 11 shows an example of a command center portal 1100 that may form part of the internal investigation portal 212. In this example, the portal 1100 includes an introduction and explanatory information for the portal 1100, as well as a summary reporting element 1102 of Finance Solutions offerings. The summary reporting element 1102 includes a Process Excellence section 1104, an Organizational Excellence section 1106, a Service Excellence section 1108, and a Business Value section 1110. In addition, the reporting element 1102 includes an overall scorecard section 1112. The sections 1104-1110 may represent categories of metrics that measure or quantify performance for a finance solutions outsourcing offering, for example.

[0110] The overall scorecard section 1112 provides a link to a performance summary view of the finance solution offerings. The link drills down from the high level reporting element 1102 to more detailed views of underlying metrics. For example, the drill down link may lead to the balanced scorecard details screen 1200 shown in Figure 12.

[0111] The interface screen 1200 includes detailed tabular reporting elements for each of the metric categories underlying the financial solutions outsourcing offering. As shown in Figure 12, the interface screen includes a Process Excellence table 1202, an Organizational Excellence table 1204, a Service Excellence table 1206, and a Business Value table 1208. Each table 1202-1208 includes detailed reporting elements for each metric in each category.

[0112] As shown in Figure 12, the tables 1202-1208 include a status column, a frequency column (e.g., annual or monthly metric), and a metric name column. In addition, each table 1202-1208 includes actual and overall actual metric value columns, target value and overall target values columns, and a trend indicator column. Stoplight reporting elements are present in the status column and give a visual indication of how well the associated metric is performing. Other columns, reporting elements, or visual representations may be employed, however.

[0113] Each table 1202-1208 may include additional drill down links to more detailed views of metrics or additional information about the metrics.
For example, the title bar of the Process Excellence table 1202 may include a link to a user interface screen with additional detail for the Process Excellence category of metrics. As another example, a navigation bar may provide links to additional interface screens. The navigation bar may include a region selector, a deliver center selector, and a client name selector. Other selectors or links may be provided, however.

[0114] The region selector may provide a drop down list of geographical regions (e.g., Europe, Asia, or North America) that an internal reviewer may select. In response, the command center 100 updates the user interface to report metrics measured on data received from the selected geographical region. The delivery center selector provides a drop down list of delivery center locations (e.g., Prague, Chicago, or Munich). The command center 100 responds to a delivery center location by reporting metrics measured on data received from the selected delivery center. The engagement name selector may provide a drop down list of engagement names for which the viewer has access to. The engagement names may represent engagements for which an internal reviewer has access permission to review engagement data. The command center 100 responds to an engagement selection by displaying metrics for the selected engagement.

[0115] Figures 13 and 14 show an example of a metric reporting screen 1300 reachable by a drill down link in the interface screen 1200 for the service excellence category of metrics. The reporting screen 1300 shows the Service Excellence table 1206, and detailed reporting elements for metrics in the service excellence category. In particular, Figures 13 and 14 show a vertical bar graph 1302 for the Number of High Impact Exceptions metric, a vertical bar graph 1306 for the CQMA Client Satisfaction metric, and a vertical bar graph 1308 for the CQMA compliance metric. For each metric, the reporting screen 1300 may show the red, yellow, and green stoplight conditions 1304 established for the metric. Additional or alternative reporting elements may be employed to deliver the detailed information underlying each metric.

[0116] Figure 15 shows an example of the acts 1500 that may be taken by the command center 100 to display processed metric data. The command center accepts and processes a login (Act 1502). The login may be an engagement login, an internal login, or another type of login. Based on the login, the command center 100 may determine access permissions based on the access hierarchy 1000, for example.

[0117] As described above with reference to Figures 4-8, as examples, the command center 100 displays metric reporting screens (Act 1504). The metric reporting screens may be reached through an internal investigation portal 212, an engagement investigation portal 214, or another interface. The command center 100 may display the metric reporting screens in accordance with the display templates 210 that specify presentation characteristics such as reporting elements, colors, positions, sizes, orientations, window sizes, window positions, window content, or other characteristics for one or more engagements, one or more metrics, one or more sets of metrics, or for other sets or subsets of elements.

[0118] The metric reporting screens may include links to other screens.
The other screens may set forth additional detail for one or more metrics. In that regard, the command center 100 may respond to link activations by displaying metric reporting screens that may include additional detail underlying rolled-up or summarized metrics on prior screens (Act 1506). The links included on the metric reporting screens may thereby form a drill-down link set that may assist the engagement or internal reviewer with accessing progressively more detailed views of initially rolled-up metrics.

[0119] When the engagement or internal reviewer is finished interacting with the command center 100, the engagement or internal reviewer may then log out. The command center 100 processes the logout (Act 1508).

[0120] Figure 16 shows one example of the acts 1600 that may be taken by the command center 100 to process engagement outsourcing data. The command center 100 retrieves and stores engagement outsourcing data in the database 202 (Act 1602). For example, the command center 100 may automatically connect to the operations sites 232-238 and transfer engagement outsourcing data to the command center 100. Alternatively, the command center 100 may accept manual input or semi-automated input of engagement outsourcing data through input mechanisms including keyboards, scanners, user interface screens, and other input mechanisms.

[0121] The command center 100 also establishes engagement metrics 112 (Act 1604). The engagement metrics 112 may include an upper tier metric set 114, a middle tier metric set 116, and engagement extension metrics 118.
The upper tier metric set 114 may include metrics that are applicable across multiple outsourcing offerings. The middle tier metric set 116 may include metrics that are generally applicable to a particular outsourcing offering such as BPO or TIO. The engagement extension metrics 118 may include metrics for any particular outsourcing offering chosen, extended, or modified by a particular engagement. Any of the metrics 112 may be stored in the command center 100 for application to engagement outsourcing data for one or more engagements.

[0122] The command center 100 may apply the metrics 112 to the engagement outsourcing data to generate processed engagement metrics (Act 1606). The command center 100 may thereby apply standardized metrics in the form of upper tier metrics 114 and middle tier metrics 116 to the engagement outsourcing data. The command center 100 may also apply the engagement extension metrics to generate processed engagement metrics according to criteria, if any, specified by an engagement. The processed engagement metrics may then be stored, for example, in the processed metrics database 204 (Act 1608).

[0123] The command center 100 may help centralize the review of outsourcing functions, internally and externally. Accordingly, a business may realize time and cost savings for monitoring the performance of multiple outsourced functions. In addition, a business may more readily gain an understanding of the performance impact on the business of its outsourced functions as a whole. Furthermore, because the command center 100 employs a regular metric set application to outsourced functions, the business may achieve a consistent reference for comparing the performance of different outsourced functions. The command center 100 may also provide convenient access to, and a common reference point for, internal review of outsourcing offerings.

[0124] It is therefore intended that the foregoing detailed description be regarded as illustrative rather than limiting, and that it be understood that it is the following claims, including all equivalents, that are intended to define the spirit and scope of this invention.

Claims (31)

1. A data processing system for engagement review of multiple outsourcing offerings, the data processing system comprising:

an upper tier metric set applicable to a first outsourcing offering and a different second outsourcing offering;
a first engagement extended metric set for the first outsourcing offering comprising the upper tier metric set and a first engagement extension to the upper tier metric set;
a second engagement extended metric set for the second outsourcing offering comprising the upper tier metric set and a second engagement extension to the upper tier metric set;

a database comprising engagement operations data from engagement outsourcing operation sites;

a metric evaluation program operable to apply the first engagement extended metric set to the engagement operations data to generate first processed metrics for the first outsourcing offering, and operable to apply the second engagement extended metric set to the engagement operations data to generate second processed metrics for the second outsourcing offering;

a user interface comprising a metric reporting interface for reporting the first and second processed metrics; and a processor that executes the metric evaluation program and displays the user interface.

2. The data processing system of claim 1, further comprising a business value delivered ruleset, and where the metric evaluation program is further operable to apply the business value delivered ruleset to generate business value delivered metrics.

3. The data processing system of claim 2, where the user interface further comprises a business value delivered reporting element.

4. The data processing system of claim 1, where the metric reporting interface comprises drill-down links to increasingly detailed reports of the first or second processed metrics.

5. The data processing system of claim 1, where the first outsourcing offering is business process outsourcing (BPO), application outsourcing (AO), or information technology outsourcing (ITO).

6. The data processing system of claim 1, where the first outsourcing offering is business process outsourcing (BPO), application outsourcing (AO), or information technology outsourcing (ITO), and where the second outsourcing offering is BPO, AO, or ITO.

7. The data processing system of claim 2, where the business value delivered ruleset includes an Increase in Revenue rule.

8. The data processing system of claim 1, further comprising:
a first pre-selected middle tier metric set tailored to the first outsourcing offering;
a second pre-selected middle tier metric set tailored to the second outsourcing offering; and where the first engagement extended metric set further comprises the first pre-selected middle tier metric set and where the second engagement extended metric set further comprises the second pre-selected middle tier metric set.

9. A method for engagement review of multiple outsourcing offerings, the method comprising the acts of:
storing engagement operations data in a memory;
defining an upper tier metric set applicable to a first outsourcing offering and a different second outsourcing offering;
extending the upper tier metric set with a first engagement extension to define a first engagement extended metric set applicable to the first outsourcing offering;
extending the upper tier metric set with a second engagement extension to define a second engagement extended metric set applicable to the second outsourcing offering;
applying the first engagement extended metric set to at least a portion of the engagement operations data to generate first processed metrics for the first outsourcing offering;
applying the second engagement extended metric set to at least a portion of the engagement operations data to generate second processed metrics for the second outsourcing offering; and generating a user interface comprising a metric reporting interface for reporting the first and second processed metrics.

10. The method of claim 9, further comprising the act of collecting at least a portion of the engagement operations data from a remote engagement operations site.

11. The method of claim 9, where the act of extending the upper tier metric set with a first engagement extension comprises:
extending the upper tier metric set with a first engagement extension and with a first pre-selected middle tier metric set tailored to the first outsourcing offering.

12. The method of claim 11, where the act of extending the upper tier metric set with a second engagement extension comprises:
extending the upper tier metric set with a second engagement extension and with a second pre-selected middle tier metric set tailored to the second outsourcing offering.

13. The method of claim 9, further comprising the act of responding to drill-down links in the metric reporting interface to provide increasingly detailed reports of the first or second processed metrics.

14. The method of claim 9, where the act of defining an upper tier metric set comprises:
defining an upper tier metric set applicable to a business process outsourcing (BPO) offering, an application outsourcing (AO) offering, or an information technology outsourcing (ITO) offering.

15. The method of claim 9, where the act of defining an upper tier metric set comprises defining an engagement satisfaction metric.

16. The method of claim 15, where the act of defining an upper tier metric set further comprises defining an employee satisfaction metric.

17. A machine readable medium encoded with instructions that cause a data processing system to perform a method comprising the acts of:
defining an upper tier metric set applicable to a first outsourcing offering and a different second outsourcing offering;
defining a first engagement extended metric set comprising the upper tier metric set and a first engagement extension;
defining a second engagement extended metric set comprising the upper tier metric set and a second engagement extension;

applying the first engagement extended metric set to engagement operations data to generate first processed metrics for the first outsourcing offering;
applying the second engagement extended metric set to the engagement operations data to generate second processed metrics for the second outsourcing offering; and generating a user interface comprising a metric reporting interface for reporting the first and second processed metrics.

18. The machine readable medium of claim 17, further comprising the act of collecting at least a portion of the engagement operations data from a remote engagement operations site.

19. The machine readable medium of claim 17, where the act of extending the upper tier metric set with a first engagement extension comprises:
extending the upper tier metric set with a first engagement extension and with a first pre-selected middle tier metric set tailored to the first outsourcing offering.

20. The machine readable medium of claim 19, where the act of extending the upper tier metric set with a second engagement extension comprises:
extending the upper tier metric set with a second engagement extension and with a second pre-selected middle tier metric set tailored to the second outsourcing offering.

21. The machine readable medium of claim 17, further comprising the act of expanding a rolled-up summary of the first or second processed metrics to provide increasingly detailed reports of the first or second processed metrics.

22. A data processing system for reviewing outsourcing operations across multiple engagements and multiple outsourcing offerings, the data processing system comprising:

a database comprising engagement operations data spanning multiple engagements and multiple outsourcing offerings;
an upper tier metric set applicable to each of the outsourcing operations;

a metric evaluation program operable to apply the upper tier metric set to the engagement operations data to generate baseline metrics across the multiple outsourcing offerings;

an access hierarchy comprising a first access entry authorizing access for a first login to the baseline metrics, and a second access entry authorizing access to a first subset of the baseline metrics;

a user interface comprising a metric reporting interface for displaying, according to the access hierarchy, the baseline metrics and the first subset of the baseline metrics; and a processor that executes the metric evaluation program and that displays the user interface.

23. The data processing system of claim 22, further comprising:
a first middle tier metric set tailored to a first outsourcing offering of the multiple outsourcing offerings;
a second middle tier metric set tailored to a second outsourcing offering of the multiple outsourcing offerings;

where the metric evaluation program is further operable to apply the first middle tier metric set to the engagement operations data to generate first processed metrics for the first outsourcing offering and operable to apply the second middle tier metric set to the engagement operations data to generate second processed metrics for the second outsourcing offering;

and where the first access entry authorizes access by the first login to the first and second processed metrics, and denies access by the second login to the first or second processed metrics.

24. The data processing system of claim 23, where the metric reporting screens comprise a rolled-up summary of the first and second processed metrics.

25. The data processing system of claim 23, where the metric reporting interface comprises drill-down links to increasingly detailed reports of the first or second processed metrics.

26. The data processing system of claim 22, further comprising:
a first engagement extended metric set for a first outsourcing offering of the multiple outsourcing offerings;
a second engagement extended metric set for a second outsourcing offering of the multiple outsourcing offerings;

where the metric evaluation program is further operable to apply the first engagement extended metric set to the engagement operations data to generate first processed metrics for the first outsourcing offering and operable to apply the second engagement extended metric set to the engagement operations data to generate second processed metrics for the second outsourcing offering;

and where the first access entry authorizes access by the first login to the first and second processed metrics, and denies access by the second login to the first or second processed metrics.

27. A machine readable medium encoded with instructions that cause a data processing system to perform a method comprising the acts of:

defining an upper tier metric set applicable to multiple outsourcing offerings;
applying the upper tier metric set to the engagement operations data to generate baseline metrics across the multiple outsourcing offerings;
querying an access hierarchy comprising a first access entry authorizing access for a first login to the baseline metrics, and a second access entry authorizing access to a first subset of the baseline metrics; and generating a user interface comprising a metric reporting interface for displaying, according to the access hierarchy, the baseline metrics and the first subset of the baseline metrics.

28. The machine readable medium of claim 27, where the method further comprises the acts of:
defining a first middle tier metric set tailored to a first outsourcing offering of the multiple outsourcing offerings;
defining a second middle tier metric set tailored to a second outsourcing offering of the multiple outsourcing offerings;

applying the first middle tier metric set to the engagement operations data to generate first processed metrics for the first outsourcing offering;
applying the second middle tier metric set to the engagement operations data to generate second processed metrics for the second outsourcing offering;

and where the first access entry authorizes access by the first login to the first and second processed metrics, and denies access by the second login to the first or second processed metrics.

29. The machine readable medium of claim 28, where generating comprises the act of generating drill-down links to increasingly detailed reports of the first or second processed metrics.

30. The machine readable medium of claim 27, where the method further comprises the acts of:
defining a first engagement extended metric set for a first outsourcing offering of the multiple outsourcing offerings;
defining a second engagement extended metric set for a second outsourcing offering of the multiple outsourcing offerings;

applying the first engagement extended metric set to the engagement operations data to generate first processed metrics for the first outsourcing offering;
applying the second engagement extended metric set to the engagement operations data to generate second processed metrics for the second outsourcing offering;

and where the first access entry authorizes access by the first login to the first and second processed metrics, and denies access by the second login to the first or second processed metrics.

31. The machine readable medium of claim 30, where generating comprises the act of generating drill-down links to increasingly detailed reports of the first or second processed metrics.