CA2519693A1 - Performing predictive pricing based on historical data - Google Patents

Abstract

Techniques are described for using predictive pricing information for items to assist in evaluating buying and/or selling decisions in various ways, such as on behalf of end-user item acquirers and/or intermediate item providers. The predictive pricing for an item may be based on an analysis of historical pricing information for that item and/or related items, and can be used to make predictions about future pricing information for the item. Such predictions may then be provided to users in various ways to enable comparison of current prices to predicted future prices. In some situations, predictive pricing information is used to assist customers when purchasing airline tickets and/or to assist travel agents when selling airline tickets. This abstract is provided to comply with rules requiring an abstract, and it is submitted with the intention that it will not be used to interpret or limit the scope or meaning of the claims.

Description

PERFORMING PREDICTIVE PRICING BASED ON HISTORICAL DATA
CROSS REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of provisional U.S. Patent Application No.
60/458,321, filed March 27, 2003 and entitled "Mining Historical Pricing Data To Provide Guidance For Current Purchases," which is hereby incorporated by reference in its entirety.
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH
The University of Washington and the University of Southern California have granted a royalty-free non-exclusive license to the U.S. government pursuant to 35 U.S.C. Section 202(c)(4) for any patent claiming an invention subject to 35 U.S.C.
Section 201.
TECHNICAL FIELD
The following disclosure relates generally to the use of techniques for predicting future pricing information for items based on analysis of prior pricing information for the items, and more particularly to using such predicted future pricing information in a variety of ways, such as to assist users in making better buying and/or selling decisions.
BACKGROUND
In many situations, potential buyers or other acquirers of various types of items (such as products and/or services) are faced with difficult decisions when attempting to determine whether acquiring a particular item of interest under current conditions is desirable or optimal based on their goals, or whether instead delaying the acquisition would be preferable. For example, when the potential acquirer desires to obtain the item at the lowest price possible before some future date, and the item is currently offered by a seller for a current price, the potential acquirer needs to evaluate whether accepting the current price is more advantageous than the potential benefits and costs associated with waiting to see if the item will continue to be available and will be later offered at a lower price before the future date. Such potential acquisitions can include a variety of types of transactions (e.g., fixed-price purchase, auction-based purchase, reverse auction purchase, name-your-price purchase, rent, lease, license, trade, evaluation, sampling, etc.), and can be performed in a variety of ways (e.g., by online shopping using a computing device, such as via the World Wide Web or other computer network).
The difficulty of evaluating a potential current item acquisition is exacerbated in environments in which the prices of the items frequently change, such as when sellers or other suppliers of the items frequently modify item prices (e.g., in an attempt to perform yield management and maximize overall profits). In such environments, the likelihood of future price changes may be high or even a certainty, but it may be difficult or impossible for the potential acquirer to determine whether the future price changes are likely to be increases or drops, let alone a likely magnitude and timing of such changes. A large number of types of items may have such frequent price changes, such as airline tickets, car rentals, hotel rentals, gasoline, food products, jewelry, various types of services, etc. Moreover, a potential acquirer may in some situations need to evaluate not only a current price of an item of interest from a single seller or other provider, but may future need to consider prices offered by other providers and/or prices for other items that are sufficiently similar to be potential substitutes for the item of interest (e.g., airline flights with the same route that leave within a determined period of time, whether from the same airline or from competitor airlines).
In a similar manner, some sellers or other providers of items may similarly face difficulties in determining an advantageous strategy related to the providing of the items, such as for intermediary sellers that must acquire an item from a third-party supplier (e.g., an original supplier of the item or other intermediary seller) before providing it to a customer. For example, it may be difficult in at least some situations for such intermediary sellers to know what price to offer to customers in order to maximize profit, as well as whether to immediately acquire from a third-party supplier an item purchased by a customer or to instead delay such an acquisition in an attempt to later acquire the item at a lower price. In the context of the airline industry, for example, such intermediary sellers may include various types of travel agents, including travel agents that typically buy only single airline tickets in response to explicit current instructions from a customer, consolidators that buy large numbers of airline tickets in advance for later resale, tour package operators that buy large numbers of airline tickets for bundling with other tickets andfor services, etc.
Thus, it would be beneficial to be able to predict future pricing information for items, such as likely future directions in price changes and/or likely specific future item prices, as doing so would enable buyers and/or intermediate sellers to make better acquisition-related decisions.
BRIEF DESCRIPTION OF THE DRAWINGS
Figures 1A-10 provide examples illustrating the use of predictive pricing techniques in a variety of situations.
Figure 2 is a block diagram illustrating an embodiment of a computing system suitable for providing and using disclosed techniques related to predictive pricing.
Figure 3 is a flow diagram of an embodiment of a Predictive Pricing Determiner routine.
Figure 4 is a flow diagram of an embodiment of a Predictive Pricing Provider routine.
Figure 5 is a flow diagram of an embodiment of a Predictive Pricing Seller routine.
Figure 6 is a flow diagram of an embodiment of a Predictive Pricing Advisor routine.
Figure 7 is a flow diagram of an embodiment of a Predictive Pricing Buyer routine.
Figure 8 is a flow diagram of an embodiment of a Predictive Pricing Analyzer routine.
DETAILED DESCRIPTION
A software facility is described below that uses predictive pricing information for items in order to assist in evaluating decisions related to the items in various ways, such as to assist end-user item acquirers in evaluating purchasing decisions related to the items and/or to assist intermediate providers of the items in evaluating selling decisions related to the items.
In some embodiments, the predictive pricing techniques are used for items whose prices are dynamically changed by suppliers of the items to reflect various factors and goals, such as to use yield, management and differential pricing techniques in an attempt to maximize profits - moreover, in some such embodiments the factors and/or any underlying pricing algorithms that are used by the item suppliers may further be unknown when performing the predictive pricing activities.
Furthermore, in some embodiments the predictive pricing techniques may be applied to items that are "perishable" so as to have an expiration or other use date after which the item has a different value (e.g., a lower value, including no value), such as performances/events/occurrences and/or services that occur on or near the expiration or use date, or information whose value is based at least in part on its timeliness or recency - in such embodiments, a supplier of such an item may alter prices or other conditions for the item as its expiration or use date approaches, such as to discount a price for the item or alternatively to raise such a price.
Any such actions by suppliers based on expiration or use dates for items may in some embodiments be performed by the suppliers in a purely formulaic and repeatable manner (e.g., as an automated process), while in other embodiments some subjective variability may be included with respect to such actions (e.g., based on manual input for or oversight of the actions). When discussed herein, a supplier of an item includes an original supplier of an item and/or any other party involved in providing of the item who has control over or influence on the setting of a current price for the item before it becomes available to an acquirer (whether an intermediate seller or an end-user customer), and may further in some situations include multiple such parties (e.g., multiple parties in a supply chain).
In particular, in some embodiments the predictive pricing for an item is based on an analysis of historical pricing information for that item and/or related items.
Such historical pricing information analysis may in some embodiments automatically identify patterns in the prices, such as patterns of price increases or drops.
In addition, in some such embodiments the analysis further associates the prices and/or patterns with one or more related factors (e.g., factors that have a causal effect on the prices or are otherwise correlated with the prices), such as factors that are automatically identified in one or more of a variety of ways during the analysis.
Furthermore, in some embodiments predictive pricing policies are also automatically developed based on other automatically identified predictive pricing information, such as to enable specific price-related predictions for a particular item given specific current factors. In addition, in some embodiments the historical pricing information may reflect prices for items that were previously offered by item suppliers to others, while in other embodiments the historical pricing information may reflect prices at which the items were actually previously acquired/provided.
Specific mechanisms for performing such predictive pricing analysis are'discussed in greater detail below.
When such predictive pricing information is available for an item, that information can then be used to assist acquirers (e.g., buyers) and/or intermediate providers (e.g., sellers) of an item in making better decisions related to acquiring and/or providing of the item. In particular, given information about current factors that are associated with the pricing information for the item, the predictive pricing information for the item can be used to make predictions about future pricing information for the item. Such future predicted pricing information can take various forms in various embodiments, including a likely direction of any future price changes, a likely timing of when any future price changes will occur, a likely magnitude of any future price changes, likely particular future prices, etc.
In addition, in some embodiments the future predicted pricing information may further include predictions of the specific likelihood of one or more of such types of future pricing information. Moreover, in some embodiments and/or situations the predictive pricing information and/or assistance/functionality provided based on that information may be performed for a fee.
As one example, the facility may in some embodiments use predictive pricing information for items to advise potential buyers of those items in various ways. For example, when providing pricing information for an item to a current customer, a notification may in some embodiments be automatically provided to the customer to advise the customer in a manner based on predicted future pricing information for the item, such as whether the current price is generally a "good buy" price 'given those predicted future prices, or more specifically whether to buy immediately due to predicted future price increases or to delay buying due to predicted future price drops. Such advice could further in some embodiments provide specific reasons for the provided advice, such as based on information about specific predicted future pricing information (e.g., a specific predicted direction, time and/or magnitude of a future price change, a specific predicted future price, etc.), as well as additional details related to the advice (e.g., specific future conditions under which to make an acquisition, such as a specific amount of delay to wait and/or a specific future price to wait for). In situations in which the potential buyer does not need to immediately obtain the item and the predicted future pricing information indicates that the price is likely to drop, for example, the potential buyer can use that information to determine to delay a purchase.
In other situations, advice may be automatically provided to a user in other ways, such as by proactively alerting a potential buyer regarding a current and/or predicted future price for an item (e.g., an item in which the customer has previously expressed an interest), such as when a current price for the item reflects a good buy for the customer. In addition, in some embodiments the facility may further act as an agent on behalf of a customer in order to automatically acquire an item for the customer, such as based on prior general instructions from the customer related to purchasing an item or type of item when it is a good buy. Thus, in some embodiments such advice may be provided to users as part of an interactive response to a request, while in other situations the providing of the advice may instead be automatically ~ initiated. In addition, in some embodiments and/or situations the providing of such advice and/or related functionality to a potential buyer or other acquirer may be performed for a fee, such as a fee charged to that acquirer.
In other embodiments, the facility may act on behalf of an intermediate seller of the item in order to provide advice to the seller. For example, in some situations in which a predicted future price for an item is lower than a current price, the intermediate seller may determine based on such advice to offer a price to a customer that is lower than the current price available to the intermediate seller (e.g., but above the lower predicted future price) - if so, and if the customer indicates to purchase the item, the intermediate seller may further delay purchasing the item from its supplier in order to attempt to acquire the item at a lower future price. More generally, when a predicted future price for an item is tower than a current price being offered to an intermediate seller, that knowledge about the lower predicted price may enable the intermediate seller to currently use the potential cost savings based on acquisitions at the later lower price in a variety of ways, including by passing some or all of the price difference on to customers, by retaining some or all of the price difference as profit, andlor by sharing some or all of the price difference with the supplier of the item to the intermediate seller (e.g., in exchange for the supplier immediately lowering their price to the intermediate seller).
In addition, in situations in which the future price is predicted not to drop, the intermediate seller may choose based on such advice to offer price protection to a customer (e.g., for a fee to the customer) based on that prediction, such as to provide additional benefit to the customer if the future actual price were to drop below the current price or some other specified price (e.g., a refund of the difference). In other embodiments, the facility may assist the intermediate seller to offer items to customers using a variety of other sales models, such as to allow customers to name a price at which the customer will purchase an item of interest (whether identified as a particular item or a category of items that is specified at any of a variety of levels of details) and to then assist the intermediate seller in determining whether to accept such an offer based on a comparison of the named price to a predicted future price for the item. In addition, in some embodiments and/or situations the providing of such advice and/or related functionality to intermediate sellers and other providers may be performed for a fee, such as a fee charged to that provider.
In addition, the facility may in some embodiments further assist buyers that purchase items in bulk (e.g., by aggregating numerous individual purchases into one), such as customers that themselves buy large numbers of the items (e.g., large corporations) and/or intermediate sellers (e.g., item consolidators) that are purchasing items from other suppliers. In such situations, the bulk purchaser may take a variety of types of steps to accomplish desired goals of the purchaser, such as to hedge or otherwise limit exposure to loss based on predicted future prices (e.g., by purchasing some but not all of multiple items at a current price even when the predicted future price is lower in order to minimize the risk of the actual future prices being higher than the current price). Alternatively, the bulk purchaser may be able to use information about predicted future prices to manually negotiate better current prices with a seller. In other embodiments, information about such predicted future prices can assist other types of users, including suppliers of items when they have such information about similar items offered by competitors, such as to provide first mover advantage for price decreases that are likely to occur in the future by the competitors. The information provided by the analysis may further assist in some embodiments in more generally identifying and predicting trends in prices over time for specific items and/or groups of related items, such as to enable a user to immediately take action in such a manner as to benefit from such trends. In addition, in some embodiments and/or situations the providing of such advice and/or related functionality to bulk purchasers may be performed for a fee, such as a fee charged to that bulk purchaser.
In some embodiments, the facility further assists users in analyzing historical purchase data, such as for bulk purchasers. For example, a large corporation may want to analyze their prior item purchases over a specified period of time in order to determine whether the purchase prices for the items were advantageous and/or optimal in light of later available prices for those items. In some situations, the actual prior purchase prices for items could be compared against alternative prior purchase prices for those items that would have been obtained based on following predictive pricing information for those items that would have been provided at the time of actual purchase (e.g., for advice related to delaying a purchase, determining a difference between the actual prior purchase price and a later actually available price at which the item would likely have been acquired based on the advice).
This allows a determination to be made of the benefits that would have been obtained by using predictive pricing in those situations. In addition, in some embodiments the actual prior purchase prices could further be compared to optimal purchase prices for those items within a relevant time period before the item was needed, such as to see the differential (if any) between the actual purchase price and the lowest possible purchase price given perfect hindsight knowledge. In addition, in some embodiments and/or situations the providing of such advice and/or related functionality for analyzing historical purchase data may be performed for a fee, such as a fee charged to a customer to whom the historical purchase data corresponds.
Moreover, such analysis of prior purchase decisions provides information not only about the benefits of using the predictive pricing techniques, but also to assist in further refining the predictive pricing techniques (e.g., in an automated manner, such as based on a learning mechanism), such as based on identifying situations in which the predictive pricing techniques did not provide the best prediction available.
In addition, in various embodiments the predictive pricing information and/or related functionality is used to generate revenue and/or produce savings in a variety of ways, including through service fees, license fees, by maximizing profit for sellers and/or savings for acquirers, through related advertising, etc. Such revenue can be based on any or all of the various example types of functionality discussed above and in greater detail below.
For illustrative purposes, some embodiments of the software facility are described below in which particular predictive pricing techniques are used for particular types of items, and in which available predictive pricing information is used to assist buyers and/or sellers in various ways. However, those skilled in the art will appreciate that the techniques of the invention can be used in a wide variety of other situations, and that the invention is not limited to the illustrated types of items or predictive pricing techniques or uses of predictive pricing information. For example, some such items with which the illustrated predictive pricing techniques and/or uses of predictive pricing information include car rentals, hotel rentals, vacation packages, vacation rentals (e.g., homes, condominiums, timeshares, etc.), cruises, transportation (e.g., train, boat, etc.), gasoline, food products, jewelry, consumer electronics (e.g., digital and non-digital still and video cameras, cell phones, music players and recorders, video players and recorders, video game players, PDAs and other computing systems/devices, etc.), books, CDs, DVDs, video tapes, software, apparel, toys, electronic and board games, automobiles, furniture, tickets for movies and other types of performances, various other types of services, etc.
Furthermore, the disclosed techniques could further be used with respect to item-related information other than prices, whether instead of or in addition to price information.
Figures 1A-10 provide examples illustrating the use of predictive pricing techniques in a variety of situations. In these examples, the predictive pricing techniques are applied to airline ticket information and are used by a provider of information about airline ticket prices, such as an intermediate seller travel agency.
However, such techniques can also be used for other types of items and in other manners, as discussed elsewhere.

In particular, Figure 1A illustrates a table 100 that provides examples of historical pricing information that may be gathered for airline flights and then analyzed to produce various types of predictive pricing information. In this example, the table includes entries 111-116 that each correspond to a different instance of actual price information for a particular flight, such as a ticket price offered for a flight at a particular time. The table also includes columns 102-104 that store information about the specific offer instance for the flight number indicated in column 101 and the departure date indicated in column 105. In addition, in the illustrated embodiment the table further includes columns 106-108 with additional information about the flight, although in other embodiments such information may be stored separately. Moreover, in some embodiments the table could store additional information about other factors that may have an effect on price changes, such as one or more sell-out factors related to whetherlwhen a flight may sell out (e.g., based in part on a number of remaining available seats in column 109, although in the illustrated embodiment that information is not currently available).
While the flight prices reflect one-way tickets for specific flights of specific airlines in this example, related information could similarly be gathered and/or aggregated in various other ways, whether instead of or in addition to one-way tickets, including for round-trip and/or multi-segment tickets, and so as to enable predictive pricing for flights on a particular route between a departure airport and an arrival airport, for flights on one or more routes between a particular pair of cities or regions (e.g., when the departure and/or arrival citieslregions include multiple airports), for flights into and/or out of an airport hub for one or more airlines, for flights on a route with an associated time (e.g., a specified departure time or departure time range, a specified arrival time or arrival time range, a specified interval of time for the travel, etc.), for some or all flights from a particular airline, for some or all flights into or out of an airport and/or city/region, for some or all flights that depart and/or arrive at a specified airport/city/region within a specified time frame, etc.
Figure 1 B illustrates a chart 131 that provides an example of historical offer price information over time for a particular airline flight, such as a flight that departs in this example on January 7, and is represented with a particular flight number from a particular airline (not shown). As one example, the illustrated price information to may correspond to a group of historical information that includes entries 111 and 115-116 of fiable 100 in Figure 1A. In other embodiments, information could instead be analyzed for airline flights in other ways, such as by aggregating information for a particular airline route over multiple days and/or by aggregating information for multiple airline flights of a particular airline or that are otherwise similar. In this example, the price data generally shows three tiers of relatively stable prices, although there is an additional small price fluctuation in the first price tier around the dates of 12/17 and 12/18 (e.g., based on a reaction of the airline to a temporary price increase by a competitor on a flight for the same route and date).
Figure 1 C illustrates a chart 132 that provides an example of historical price information for the same flight except with a different departure date that is five days earlier, which in this example results in a departure date of January 2 rather than January 7, Despite the similarities between the two flights (the same airline, flight number, route and close departure date), the price of this airline flight with the earlier departure date fluctuates much more than that of the flight discussed with respect to Figure 1 B, such as based at least in part on the increased demand for travel near the New Year holiday.
Figure 1 D illustrates an example of a chart 133 that provides an example of historical price information for a different airline flight, such as on a different route and/or from a different airline. In this example, there are two primary price tiers, but there is also a large drop in price in the middle of the second price tier (near the date of 1/5 in this example). Thus, customers who express interest in the item near the beginning of the second price tier (e.g., around the dates of 12/30-1/3) might pay a large price if they purchased immediately, but could significantly benefit by waiting to purchase the ticket during the later price drop. Figure 1 E illustrates yet another example chart 134 with example historical price information, which in this example shows a more gradual increase in price over time as the departure date approaches.
A variety of other types of price change behavior could similarly occur.
As illustrated in these examples, prices for airline tickets can change in various ways and based on various factors. For example, information other than an amount of time before departure can be a factor that affects price changes in at least some embodiments, and can thus be used as part of a later predictive pricing determination in those situations. For example, Figure 1 F illustrates an example m chart 135 for the same flight previously discussed with respect to Figure 1 B, although in this example the price is shown as it varies based on factor of the availability of remaining seats on the flight. In other embodiments, however, such flight availability factor information may not be available and/or other additional factors could similarly be considered.
In a similar manner, the example chart 136 illustrated in Figure 1 G
illustrates that information other than price may be tracked, analyzed and used in some embodiments, such as in this example displaying historical flight availability information for one or more flights (e.g., flights that depart on a particular day or instead on any of a group of similarly situated days). Such availability information may then be used in some embodiments to assist in a determination of whether a current price is currently a good buy, such as based on considering the likelihood of the flight selling out in the future. However, in embodiments where the price of an item typically already varies based on remaining availability, such as for airline tickets, availability information may instead be considered implicitly based merely on the price factor (e.g., if the availability does not independently affect a decision).
Thus, as previously indicated, historical price information for airline ticket prices can be illustrated in a graphical manner to show changes over time as the departure time nears. In addition, such historical data can also be analyzed in a variety of ways to provide other types of information that can assist in later performing predictive pricing. For example, with respect to Figure 1 H, example information is shown in table 142 that indicates a historical minimum price, maximum price, and maximum price change for particular routes. Similarly, Figure 11 illustrates in table 144 an average number of price changes for particular routes. A
variety of other types of analyses can similarly be performed related to price changes if the resulting information assists in predictive pricing and/or in providing advice based on current prices.
Once predictive pricing information is available, it can be used in a variety of manners to assist potential acquirers and/or providers of items. For example, Figure 1 J illustrates example information 159 that provides flight alternative information to a potential customer, such as via a Web page provided to the customer for display from an online travel agent. In this example, the provider of information is referred to as "Hamlet". In particular, in this example four alternative flight options 150 are displayed to the customer, such as by including them in search results in response to a prior request from the customer, The alternatives are listed in order from lowest price to highest price in this example, with the lowest price being $499 for alternative 150a. However, in this example the low price for alternative 150a corresponds to a special fare that is being offered to the customer by the travel agent, as is indicated to the customer in this example via nofiification 155 and is explained more fully to the customer if they select the control 156. In particular, alternative 150a corresponds to the same flight as that for alternative 150c, but the indicated price of $649 for alternative 150c reflects the actual price currently offered by the original supplier of this flight (in this example, Alaska Airlines).
The special fare offered in this example for alternative 150a is instead based on predictive pricing for this flight, which in this example has provided an indication that a future price for this flight will be lower (e.g., as low as $499 if the travel agency plans to offer the full potential savings to the customer, lower than $499 if the travel agency plans to retain some of the potential savings, somewhat above $499 if the travel agency is willing to offer an additional discount to this customer and/or in this situation, etc.).-Thus, in this example the travel agency has elected to provide at least $150 worth of potential savings to the customer if the customer purchases now by offering a price to the customer that is lower than the price currently offered from the original supplier. If the customer then proceeds to purchase the flight for alternative 150a at the special fare, such as by selecting control 157, the travel agency may nonetheless wait until later to actually purchase a ticket for the customer on this flight, such as a later time when the actual price offered by the item supplier is lower.
In this example, the customer is selecting a round-trip flight, and thus after selecting the control 157 for alternative 150a the customer will be prompted to provide information related to a return flight. If the customer was instead selecting a one-way trip or this was the last selection of a trip with multiple legs or segments, selection of a corresponding control by the customer could instead prompt the online travel agency to provide confirmation to the customer of their purchase having been completed at the indicated price of $499, even if the travel agency delays a purchase of tickets for the flight until later.

In other embodiments, explicit notification that the alternative 150a is a special fare might instead not be provided, such as by not showing alternative 150c with the actual current price offered by the original supplier airline, and instead merely listing a price for alternative 150a that is selected as satisfying one or more goals of the travel agency, such as to maximize profit (e.g., a price that is between the lowest predicted future price and the lowest actual current price offered by a supplier for one of the alternatives, such as in this example to be less than the $598 price for alternative 150b). Conversely, in some embodiments additional aspects related to the special fare may be conveyed to the customer, such as if any special fares selected by the customers for purchase are contingent on the travel agency acquiring the ticket under specified conditions (e.g., at or below a specified price and/or within a specified amount of time). If so, the travel agency may not confirm purchase to the customer until after the ticket is actually acquired from a supplier.
Figure 1 K illustrates example information 169 that provides return flight information to the customer after selection of control 157 in Figure 1J, including information 162 about that previously selected flight. The information includes two alternative return flights 160, with both of the alternatives in this example including a notification 165 that the indicated prices are special fares from the travel agent. In some embodiments, such notifications may further explicitly indicate to a customer that a special fare is based on predictive pricing, while in other embodiments such information may not be provided to the customer.
Figure 1 L illustrates example information 179 that provides alternative return flight information to the customer if the customer instead selected alternative 150b in Figure 1J for the initial segment of the trip, including information 172 about that previously selected flight. In this example, three alternative return flights 170 are included, and an additional notification 177 is provided to the customer to remind them that a lower price trip is available based on the special fare offered by the travel agency. In addition, in this example the travel agency provides various other types of notifications based on the use of predictive pricing. For example, as indicated in information 178 and with notification 175, the current price of $598 round trip for f4igl-~t alternative 170a is indicated in this example to be a good buy that may justify immediate purchase, such as due to the price being unlikely to fall but may possibly rise in the near future. In other embodiments, customers may instead be referred to an agent or a supplier from whom they can immediately acpuire such good buy tickets, such as in exchange for a referral fee. In addition, in this example the travel agency further provides an option 176 to the customer that is also based on predictive pricing information - in particular, since in this example the predictive pricing indicates that the price is not likely to drop, the travel agency is willing to offer price protection insurance to the customer for a small fee, such that if the actual offered price drops after purchase the customer would then receive an additional benefit (e.g., a discount on their purchased price so as to reduce it to the lowest actual price that was offered). While the price protection insurance is offered to the customer for an additional fee in this example, in other embodiments such price protection insurance may not be offered or instead may be offered to a customer without additional explicit cost to the customer.
Figure 1 M illustrates example information 189a that provides return flight information to the customer if the customer instead selected alternative 150d in Figure 1J for the initial segment of the trip, including information 182 about that previously selected flight. In particular, in this example three alternative return flights 180 are provided to the user, and predictive pricing information allows the travel agency to determine whether some or all of the alternatives are good fares or are otherwise good buys. However, in this example such additional information based on predictive pricing is available only to registered customers, and thus the information 189a includes indications 181 a-181 c to the user that they can obtain such notification information after they register (e.g., via selection of the control in section 183), such as based on a fee charged to the customer (e.g., a one-time fee or an ongoing subscription), or instead based on other benefits to the travel agency of such registration (e.g., obtaining additional information about the customers for use in better serving them and/or tailoring advertising or other information that will be displayed or otherwise provided to them). Alternatively, the initial registration may be free and may provide a basic level of information to a customer, while an upgrade to one or more premium fee-based registration services with additional information and/or functionality (e.g., to provide details and/or reasons about notifications, to provide alerting functionality, etc.) may additionally be available.
Dififerent types of services could also be used for different types of customers, such as individuals is purchasing on their own behalf versus users acting on behalf of others (e.g., travel agents, corporate travel managers, etc.).
As noted above, in some embodiments and situations revenue may be derived through various types of advertising to users, such as advertising supplied interactively to users along with other supplied information (e.g., as banner or pop-up ads, sponsored listings in search results, paid inclusion for search or other results, etc.), advertising supplied or otherwise made available to users in a non-interactive manner (e.g., permission-based or other email or other forms of notification) such as based on demographic and/or personal preference information for the users, etc. Similarly, in some embodiments and situations revenue may be derived through other uses of information about users themselves and/or about purchase-related activities of such users, including selling or otherwise providing such information to third-parties (e.g., with permission of the users).
Figure 1 N illustrates information 189b that is similar to that displayed with respect to Figure 1 M, but which includes alternative types of notifications to the customer for the return flight alternatives. For example, these alternative notifications may be provided to the customer after they complete the registration process with respect to Figure 1 M. In particular, example notification 184a provides additional information to a user for a particular flight, such as to buy the flight at the current price now because the price is not likely to drop and the flight may soon sell out. Conversely, notification 184b indicates to the customer to hold off on purchasing the indicated flight at its current price, as the price of that flight is likely to drop in the future. Notification 184c indicates for its alternative flight that the price is not likely to rise or drop, and thus advice on whether to purchase immediately cannot be made based purely on price information. In other alternatives, yet other types of information could be provided, such as by including information in alternative 184b that further indicates to the customer a length of time that the customer should wait before purchasing and/or a price or price range for which the customer should wait before completing the purchase.
While not illustrated here, advice could also be provided to customers in a variety of ways other than as part of an interactive response to the customer.
For example, various types of alerts could instead be provided to a customer in a manner initiated by the travel agency or a related system with access to the airline price information, such as for alternative 150a in Figure 1 J if the customer had previously requested information on special fares for this flight or on fares below $500 for any flights between Seattle and Boston. Such alerts could take a variety of forms, including e-mail, instant message, a phone call, fax, etc. In addition, in other alternatives the travel agent and/or an independent agent acting on behalf of the customer could automatically purchase a flight when it met certain criteria for the customer, including if the flight is determined to be a good buy.
Figure 10 illustrates example information 199 showing another alternative for using predictive pricing information to assist customers and/or sellers. In particular, in this example information is shown that is similar to that illustrated in Figure 1J, but with only two alternatives illustrated to the customer. The top alternative in this example corresponds to the same flight that was previously indicated to be alternative 150a in Figure 1J, but in this example a specific special fare is not offered to the customer based on the predictive pricing. Instead, as indicated by the customer-selectable control 193, the customer is in this example offered the opportunity to offer a named price for the particular flight shown. In addition, the displayed information to the customer further includes an indication of a second alternative flight, which in this example does not include the name-your-price functionality, although the specific offered price does provide context to the customer of other current prices offered for competitive flights - in other embodiments, such additional information may instead not be provided. In this example, if the customer selects the control 193 and offers a price above $499 (the special fare in Figure 1J for this flight), the travel agency may accept that offer even though it is below the current price offered for the flight of $649. In other alternatives, customers could name prices for flights at varying degrees of specificity, such as any flight that is sufficiently similar to previously indicated search criteria by the customer, flights on a specified airline but not limited to a particular flight, etc. In addition, customers could similarly purchase items using other purchase models that similarly use predicted price information, such as based on various auction-related purchase models.
Thus, Figures 1 J-1 O provide examples of specific types of functionality that may be provided to customers by intermediate sellers based on the use of predictive pricing information, although in other embodiments such predictive pricing information could be used in other ways. Also, as was shown in these examples, the predictive pricing information allows different types of functionality to be offered to different types or categories of customers. For example, the special fares and general notifications of whether a flight is a good buy may be of interest to bargain and value shoppers. Similarly, the name-your-price model may allow such customers to save money, while also being able to specify flights at a much more detailed level than is currently provided in the marketplace (e.g., by Priceline), which provides less uncertainty and less restrictions for the customers. Conversely, frequent travelers may prefer to obtain additional information related to predictive pricing, such as details and/or reasons related to why a flight is a good buy, or specific recommendations on how to obtain potential savings when the future price may drop - if so, such additional information may be available to them for an additional fee, such as based on a premium registration service. In addition, professionals that represent other travelers (e.g., travel agents, in-house corporate travel managers, etc.) may want even more information and/or the ability to obtain predictive pricing information in high volume and/or in bulk, such as for additional fees.
Figure 2 illustrates a server computing system 200 suitable for executing embodiments of one or more software systems/modules that perform analyses related to predictive pricing information. The example server computing system includes a CPU 205, various I/O devices 210, storage 220, and memory 230. The I/O devices include a display 211, a network connection 212, a computer-readable media drive 213, and other I/O devices 215.
A Predictive Pricing ("PP") Determiner system facility 240 is executing in memory 230 in this example in order to analyze historical price data and determine predictive pricing information. Similarly, a PP Provider system facility 241 is executing in memory 230 in order to provide predictive pricing information relative to current items on request, such as to users (e.g., buyers and/or sellers) and/or to other system facilities that use that information to provide various services to users.
As the PP Determiner system executes in memory 230, it analyzes various historical item price information, such as that available in a database 221 of storage 220 or instead as obtained from another executing system or remote storage location. After analyzing the historical price information, such as at the request of a i8 user or instead on a scheduled basis, the PP Determiner system determines various predictive pricing information related to the historical item prices (e.g., underlying factors that affect price changes, various patterns or other information about price changes relative to the factors, policies related to responding to current factors, etc.). The system then in the illustrated embodiment stores the determined information in a database 223 on storage, although in other embodiments the system could provide the information interactively to a user or other executing system. In some embodiments and/or situations, the PP Determiner system could also obtain historical price information for use in its analysis by repeatedly querying an external supplier of such information to obtain then-current information, and could then analyze the obtained information, whether dynamically as it is obtained or instead later after a sufficient amount of historical price information has been gathered or on a periodic basis. Such external information sources could be accessed in a variety of ways, such as via one or more server computers 270 over a network 280 (e.g., to retrieve stored information 273 directly and/or via interaction with an application 279 executing in memory).
When predictive pricing information is available, whether via previously stored information in database 223 or in response to a query to the PP Determiner system, the PP Provider system facility 241 executing in memory 230 can obtain and provide predictive pricing information (e.g., for a specified item or group of items), such as in response to a request from a user or other executing system facility. In this illustrated embodiment, example system facilities 243-249 are executing in memory 230 to provide functionality based on predictive pricing information, and thus may provide such requests to the PP Provider system, although in other embodiments some or all of those additional system facilities may instead be executing remotely or may not be present. In this illustrated embodiment, the PP Provider system provides predictive pricing information for a request by obtaining information about predicted future prices for the item as discussed above, by analyzing and modifying the obtained information if needed, and providing information about those predicted future prices. In some embodiments, the system 241 could further obtain, use and provide current pricing information for the items, such as from a current item information database 225 on storage 220, while in other embodiments the PP
Provider system may instead obtain and provide predictive pricing information based merely on various current factors for an item, such as those supplied in the request or instead otherwise obtained by the PP Provider system (e.g., from the database 225).
In particular, as one example of a system facility that can obtain and use predictive pricing information, the PP Advisor system facility 243 is executing in memory 230. In response to an indication to provide advice, such as based on an interactive request from a customer or instead based on a scheduled indication to determine whether to provide an alert to a customer based on a previously received request, the PP Advisor system obtains predictive pricing information for one or more items, such as by interacting with the PP Provider system. The PP Advisor system also obtains current price information for those items, and then determines one or more types of advice to provide to an appropriate customer based on that information. In some embodiments, the advice is provided via notifications interactively displayed to the customer that indicate information about current item prices to advise the customer. In other embodiments, the advice may be provided in other forms, such as via an alert sent to a registered customer. Various information about customers may be stored and used when providing advice, such as in a customer database 227 on storage 220, in other to determine when, whether, and how to provide notification to a customer in accordance with their preferences and interests.
As another example of a system facility that uses predictive pricing information, the illustrated embodiment further includes a PP Seller system facility 245 executing in memory 230. The PP Seller system obtains predictive pricing information far one or more items, such as from the PP Provider system, as well as current price information for the items. The PP Seller system then assists a seller (e.g., an intermediate seller) in using the predictive pricing information in one or more of a variety of ways, such as to determine whether and when to offer prices to customers that are lower than prices currently offered by suppliers of items, to accept bids or offers from customers that are lower than prices currently offered by item suppliers but higher than predicted lower future prices for the items, to delay an actual purchase of one or more items from item suppliers that have been purchased from the seller by customers, etc.

The PP Buyer system facility 247 is another example of a system facility executing in memory 230 in the illustrated embodiment that can obtain and use predictive pricing information in order to enable better buying decisions, in this situation by directly assisting buyers (e.g., bulk buyers). In particular, the PP Buyer system obtains predictive pricing information for one or more items, such as from the PP Provider system, as well as current price information for those items. The PP
Buyer system facility then assists the buyer in determining whether and when to make purchasing decisions, such as to delay purchases based on predicted future price drops and/or to aggregate multiple purchases together to provide additional benefits, to hedge against such delays by purchasing some items immediately and delaying others, to negotiate with an intermediate seller or item supplier for lower prices based on predicted future price drops, to immediately purchase items that are not otherwise immediately needed based on predicted future price increases, etc.
The PP Analyzer system facility 249 is another example system executing in memory 230 that uses predictive pricing information to provide benefits to customers or other users. The PP Analyzer system analyzes prior purchase information, such as that stored in database 229 on storage 220 or instead as interactively supplied by a user making a request, in order to determine whether the prior purchasing decisions were made effectively. In particular, the PP Analyzer system obtains information about pricing information that would have been predicted for those items at the time of purchase, such as from the PP Provider system, and then compares the actual purchase decisions made to the decisions that would have been advised based on use of the predictive pricing information. In some embodiments, the PP
Analyzer system further may obtain historical price information for the purchase items (e.g., from the database 221 ) that corresponds to offered prices after the purchase date but before a date that the item is needed, such as to determine whether the actual purchase prices were higher than an optimal purchase price that was available. The PP Analyzer system can then provide information about the analysis performed to assist in better future buying decisions.
Those skilled in the art will appreciate that computing systems and devices 200, 250 and 270 are merely illustrative and are not intended to limit the scope of the present invention. Computing system 200 may be connected to other devices that are not illustrated, including through one or more networks such as the Internet (e.g., via the World Wide Web ("Web")) or other computer network. More generally, a "client" or "server" may comprise any combination of hardware or software that can interact in the indicated manner, including computers, network devices, Internet appliances, PDAs, wireless phones, pagers, electronic organizers, television-based systems and various other consumer products that include inter-communication capabilities. In addition, the functionality provided by the various system components may in some embodiments be combined in fewer components or distributed in additional components. Similarly, in some embodiments the functionality of some of the illustrated components may not be provided and/or other additional functionality may be available.
Those skilled in the art will also appreciate that, while various items are illustrated as being stored in memory or on storage while being used, these items or portions of them can be transferred between memory and other storage devices for purposes of memory management and data integrity. Alternatively, in other embodiments some or all of the software components may execute in memory on another device and communicate with the illustrated computing device via inter-computer communication. Some or all of the system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-readable medium, such as a hard disk, a memory, a network, or a portable article to be read by an appropriate drive. The system components and data structures can also be transmitted as generated data signals (e.g., as part of a carrier wave) on a variety of computer-readable transmission mediums, including wireless-based and wired/cable-based mediums. Accordingly, the present invention may be practiced with other computer system configurations.
Figure 3 is a flow diagram of an embodiment of a Predictive Pricing Determiner routine 300. The routine begins at step 305, where historical pricing information is obtained for one or more items, and continues to step 310 to analyze the data to determine predictive pricing information based on the historical data. In step 315, the routine then stores or updates previously stored predictive pricing information from the analysis in step 310. After step 315, the routine continues to step 395 to determine whether to continue. if so, the routine returns to step 305, and if not the routine continues to step 399 and ends.

Figure 4 is a filow diagram of an embodiment of a Predictive Pricing Provider routine 400. While illustrated here as a routine that is separate from the Predictive Pricing Determiner routine, as well as from other later-discussed routines that use the provided determined information, the routine could instead in other embodiments be incorporated together with one or more such other routines.
The routine begins in step 405, where a request is received for predictive pricing information for one or more specified items and/or a specified situation. The routine continues to step 410 to determine whether the requestor is authorized to receive the requested information, such as for a registered customer (whether directly or via another system facility acting as an intermediary on behalf of that customer). If so, the routine continues to step 415 to obtain corresponding predictive pricing information, such as by retrieving stored information or instead by interactively requesting the PP Determiner to provide the infiormation. After step 415, the routine continues to step 420 to determine predictive pricing specifiic to the request based on the retrieved information, such as one or more specific predicted fiuture prices, a predicted future price pattern, a predicted future direction, predictions about specific times in the future corresponding to predictive prices, etc.
While not illustrated here, in some embodiments the routine may further obtain information about current prices for the items, such as to assist in the predictive pricing (e.g., to determine a future price relative to the current price) and/or to enable comparison between the current and predicted future prices. After step 420, the routine continues to step 425 to provide the determined information to the requestor.
After step 425, or if it was instead determined in step 410 that the requestor was not authorized, the routine continues to step 495 to determine whether to continue. If so, the routine returns to step 405, and if not the routine continues instead to step 499 and ends.
Figure 5 is a flow diagram of an embodiment of a Predictive Pricing Seller routine 500. The routine obtains predictive pricing information for one or more items, and uses the information to assist a seller (e.g., an intermediate seller) to perform selling decisions in one or more of a variety of ways.
The routine begins in step 505, where a request is received related to one or more items. In step 510, the routine determines current prices for the items, and in step 515 obtains predicted prices for the items, such as by interacting with the PP

Provider routine. In step 520, the routine then determines a price at which to currently offer the items based on the current prices and/or the predicted future prices, and in step 525 provides information about the determined offer and price to the requestor. In other embodiments, a variety of other additional types of functionality could be provided, such as to determine whether to ofiFer price protection insurance to a requestor based on the current prices and/or the predicted future prices. In addition, the determination of the price at which to offer an item can be made in various ways, such as to select prices lower than current offer prices based on predicted future prices dropping, or instead in some embodiments by negotiating with a supplier of the items to obtain a lower offered price from the supplier based on predicted lower future prices.
After step 525, the routine continues in step 530 to determine whether the requestor is interested in purchasing or otherwise acquiring one or more of the items at one of the offered prices. If so, .the routine continues to step 535 to determine whether to fulfill the requester's acquisition by actually acquiring the item from an item supplier now or instead by waiting until later (e.g., based on a predicted lower future price). If it is determined that it is preferable to buy now, the routine continues to step 540 to buy the item, and otherwise the routine continues to step 545 to store information about the item and to optionally schedule a later time to buy the item (e.g., to reflect a time at which it is predicted that the price will be lower, or instead to periodically check for lovi~er prices). In some embodiments, the decision to delay a purchase may further be made at least in part on the basis of a goal to aggregate multiple item purchase requests (e.g., for the same item, for related items such as items from a single supplier, etc.) in order to perform hedge activities or otherwise negotiate discounts. After steps 540 or 545, the routine continues to step 550 to provide confirmation to the requestor of the requestor's purchase. In situations in which the item has already been bought or is otherwise available, the item may in addition be supplied to the purchaser at this time, while in other situations (e.g., when the actual purchase is delayed), the supplying of the item may similarly be delayed. After step 550, or if it was instead determined in step 530 not to make a purchase, the routine continues in step 595 to determine whether to continue.
if so, the routine returns to step 505, and if not the routine continues to step 599 and ends.

Figure 6 is a flow diagram of a Predictive Pricing Advisor routine 600. The routine obtains predictive pricing information for items and uses the information to provide advice, such as to customers.
The routine begins in step 605, where a request is received related to one or more items. In the illustrated embodiment, the routine is illustrated as providing advice in an interactive manner, although in other embodiments such requests could be for future alerts and could be stored for periodic or scheduled processing to satisfy the requests. After step 605, the routine continues in step 610 to determine current prices for the items corresponding to the request, and in step 615 to obtain predicted prices for the items, such as by interacting with the PP Provider routine. In step 620, the routine determines what advice to give, such as based on a comparison of the current price to the predicted future price and on any other available information. The routine then continues to step 625 to determine how to provide the advice, such as via a notification displayed to the user along with other information or instead by alerting the user proactively in one or more of a variety of ways. After step 625, the routine continues to step 630 to provide the determined advice to the customer in the determined manner. In step 695, the routine then determines whether to continue. If so, the routine returns to step 605, and if not the routine continues to step 699 and ends.
Figure 7 is a flow diagram of an embodiment of a Predictive Pricing Buyer routine 700. The routine obtains and uses predictive pricing information for items in order to assist buyers in making buying decisions, such as for bulk buyers.
The routine begins In step 705, where one or more items of interest to purchase are determined, such as based on a request received from a user. In step 710, the routine determines current prices for the items, and in step 715 obtains predicted prices for the items, such as by interacting with the PP Provider routine. In step 720, the routine determines a price at which to buy some or all of the items and a time at which such purchases should be made. In some embodiments, such a determination could be made by interactively negotiating with an item supplier or intermediate seller in order to obtain discounted prices based on predicted lower future prices, while in other embodiments the determined may be made based on other factors. Similarly, some or all of such items could be determined to have their purchases held until later in order to aggregate for various purposes, such as for a 2s consolidator. In step 725, the routine then provides information about the determined price and optionally additional information about the predictive pricing in an appropriate manner, such as by providing the information to a requester from step 705.
After step 725, the routine continues in step 730 to determine whether an appropriate user is interested in purchasing or otherwise acquiring one or more of the items at one of the offered prices, such as based on a received request.
If so, the routine continues to step 735 to determine whether to fulfill that acquisition by actually acquiring the item from an item supplier now or instead by waiting until later (e.g., based on a predicted lower future price). If it is determined that it is preferable to buy now, the routine continues to step 740 to buy the item, and otherwise the routine continues to step 745 to store information about the item and to optionally schedule a later time to buy the item (e.g., to reflect a time at which it is predicted that the price will be lower, or instead to periodically check for lower prices). In some embodiments, the decision to delay a purchase may further be made at least in part on the basis of a goal to aggregate multiple item purchase requests in order to perForm hedge activities or otherwise negotiate discounts. After steps 740 or 745, the routine continues to step 750 to provide confirmation of the requested acquisition. After step 750, or if it was instead determined in step 730 not to make a purchase, the routine continues in step 795 to determine whether to continue.
If so, the routine returns to step 705, and if not the routine continues to step 799 and ends.
Figure 8 is a flow diagram of an embodiment of a Predictive Pricing Analyzer routine 800. The routine obtains predictive pricing information for items that corresponds to prior purchases of those items, and uses the predictive pricing information to analyze whether the buying decisions could have been performed more efficiently based on the predictive pricing. In addition, in the illustrated embodiment the routine further compares the previously purchased item prices to later actually available prices in order to determine how the actual and/or predicted prices compare to optimally available prices, although in other embodiments such use of actual later price information may not be used.
In step 805, a request is received to analyze historical purchases of items, and in step 807 the routine obtains information about the historical purchases, although in other embodiments such infiormation may instead be supplied as part of the request in step 805. In step 810, the routine determines the purchase prices for the items, and in step 815 determines the predicted prices that would have been made for those items at that time (e.g., based on data that was then available and/or a version of predictive pricing techniques that were then used), such as based on interactions with the PP Provider routine. In step 820, the routine then generates an analysis of the actual prior purchase prices versus the prices that would have been obtained based on following the predictive pricing advice that would have been provided at that time, and in the illustrated embodiment further generates an analysis based on a comparison to the optimal price that could have been obtained based on other actual offered prices (e.g., before and/or after the time of actual purchase). In step 825, the routine then continues to provide the generated analysis to the requestor. After step 825, the routine continues to step 895 to determine whether to continue. If so, the routine returns to step 805, and if not the routine continues to step 899 and ends.
In a similar manner, this or a related routine could use predictive pricing information to assist a user in analyzing historical and/or recent/current pricing information for a specified group of one or more item suppliers, such as on behalf of an item supplier to analyze pricing information for one or more competitors and/or affiliated business entities (e.g., customers, suppliers, partners, etc.).
When performed with respect to recent/current pricing information for one or more competitors, for example, such predictive pricing information may allow a user to anticipate likely price changes for those competitors and use that information to guide their own actions, whether in advance of any such actions by the competitors or instead as a response (e.g., an immediate response) if such actions by the competitors occur. If performed in advance, the user may be able to gain a first-mover advantage by use of the predictive pricing information.
Those skilled in the art will also appreciate that in some embodiments the functionality provided by the routines discussed above may be provided in alternative ways, such as being split among more routines or consolidated into less routines. Similarly, in some embodiments illustrated routines may provide more or less functionality than is described, such as when other illustrated routines instead lack or include such functionality respectively, or when the amount of functionality that is provided is altered. In addition, while various operations may be illustrated as being performed in a particular manner (e.g., in serial or in parallel) and/or in a particular order, those skilled in the art will appreciate that~in other embodiments the operations may be performed in other orders and in other manners. Those skilled in the art will also appreciate that the data structures discussed above may be structured in different manners, such as by having a single data structure split into multiple data structures or by having multiple data structures consolidated into a single data structure. Similarly, in some embodiments illustrated data structures may store more or less information than is described, such as when other illustrated data structures instead lack or include such information respectively, or when the amount or types of information that is stored is altered.
Appendix A provides additional details related to one example of techniques for performing predictive pricing, which in that illustrative example are in the context of airline ticket prices. In addition, those skilled in the art wiN appreciate that a variety of similar techniques could instead be used in alternative embodiments.
Some such additional techniques are discussed generally in "Machine Learning"
by Tom M. Mitchell, McGraw-Hill Companies Inc., 1997, which is hereby incorporated by reference in its entirety.
From the foregoing it will be appreciated that, although specific embodiments have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention.
Accordingly, the invention is not limited except as by the appended claims and the elements recited therein. In addition, while certain aspects of the invention are presented below in certain claim forms, the inventors contemplate the various aspects of the invention in any available claim form. For example, while only some aspects of the invention may currently be recited as being embodied in a computer-readable medium, other aspects may likewise be so embodied.

APPF,LMDTX A
To Buy or Not to Buy: Mining Airfare Data to Minimize Ticket Purchase Price Oren Etzioni Craig A. Knoblock Dept. Computer Science Information Sciences Institute University of Washington University of Southern California Seattle, Washington 98195 Marina del Rey, CA 90292 etzioni@cs.washington.edu knoblock@isi.edu Rattapoom Tuchinda Alexander Yates Dept. of Computer ScienceDept. Computer Science University of Southern University of Washington California Los Angeles, CA 90089 Seattle, Washington 98195 pipet@isi.edu ayates@cs.washington.edu ABSTRACT Keywords As product prices become increasingly available on the price mining, Internet, web mining, airline price prediction World Wide Web, consumers attempt to understand how corporations vary these prices over time. However, corpora-tions change prices based on proprietary algorithms and hid- 1, )~TTRODUC'TION
AND MOTIVATION
den variables (e.g., the number of unsold seats on a flight). Corporations often use complex policies to vary product Is it possible to develop data mining techniques that will prices over time.
The airline industry is one of the most enable consumers to predict price changes under these con- sophisticated in its use of dynamic pricing strategies in an ditions? attempt to maximize its revenue. Airlines have many fare This paper reports on a pilot study in the domain of air- ol~ses fox seats on the same flight, use different sales chan-line ticket prices where we recorded over 12,000 price obser- eels e. . travel a ents vations over a 41 day period. When trained on this data, ( g ' g , priceline.com, consolidators), and Hamlet - our mufti-strategy data mining algorithm - gen- frequently vary the price per seat over time based on a slew crated a predictive model that saved 341 simulated passen- of factors including seasonality, availability of seats, compet-gers $298,074 by advising them when to buy and when to itive moves by other airlines, and more. The airlines are said postpone ticket purchases. I~,emarkably, a clairvoyant algo- to use proprietary software to compute ticket prices on any rithm with complete knowledge of future prices could save given day, but the algorithms used are jealously guarded at most $320,572 in our simulation, thus HAntr.ET's savings trade secrets (191. Hotels, rental car agencies, and other were 61.8% of optimal. The algorithm's savings of $198,074 vendors with a "standing" inventory are increasingly using represents an average savings of 23.8% for the 341 passen- similar techniques.
gers for whom savings are possible. Overall, HAIYILET saved As product prices become increasingly available on the 4.4% of the ticket price averaged over the entire set of 4,488 World Wide Web, consumers have the opportunity to be-simulated passengers. Our pilot study suggests that mining come more sophisticated shoppers. They are able to com-of price data available over the web has the potential to save Parison shop efficiently and to track prices over time; they can attempt to identify pricing patterns and rush or delay consumers substantial sums of money per annum. purchases based on anticipated price changes (e.g., "I'll wait to buy because they always have a big sale in the spring...").
In this paper we describe the use of data mining methods to Categories and Subject Descriptors help consumers with this task. We report on a pilot study L2.6 iArtificial Tntelligence~: Learning in the domain of airfares where an automatically learned model, based on price information available on the Web, was able to save consumers a substantial sum of money in simulation.
The paper addresses the following central questions:
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are ~
What is the behavior of airline ticket prices not made or distributed for profit or commercial advantage and that copies over time? Do airfares change frequently? Do they boar this notice and the full citation on the first page. To copy otherwise, to move in small increments or in large jumps? Do they republish, to post on servers or to redistribute to lists, roquires prior specific tend to go up or down over time? Our pilot study en-permission andlor a fee. ables us to begin to characterize the complex behavior SIGKDD '03 August 24-27 2003 lvashingtan, DC, USA.
Copyright 2003 ACM 1-58113-737-0/03/0008 ...$5.00. of airfares.

~ What data mining methods are able to detect for each departure date was collected 8 times a day.2 Over-patterns in price data? In this paper we consider all, we collected over 12,000 fare observations over a 41 day reinforcement learning, rule learning, tune series meth- period for six different airlines including American, United, ods, and combinations of the above. etc. We used three-hour intervals to limit the number o~
http requests to the web site. For each flight, we recorded ~ Can Web price tracking coupled with data min- tile lonurat fare available for an emno~ny ticket. We also ing save consumers money in practice? Vendors recorded when economy tickets were no longer available; we vary prices based on numerous variables whose values refer to such flights as sold out,.
are not available on the Web. For example, an air- 2,1 Pricing Behavior in Our Data line Inay discount seats on a flight if the number of unsold seats, on a particular date; is high relative to We found that the price of tickets on a particular flight the airline's model. However, consumers do not have can change as often as seven tunes in a single day. We cate..
access to the airline's Illodel or to the number of avail- gorize price change into two types: dependent price changes able seats on the flights. Thus, ra yrior~i, price changes and independent price changes. Dependent changes occur could appear to be unpredictable to a consumer track- when prices of similar flights (i.e. having the same origin ing prices over the Web. In fact, we have found price and destination) frOlrl the S&Ille a1T11I1e ChaIlge at the 58,Irle C11aI1geS t0 be SllrpTlSlIlgly pTedlCtable lIl SOIne CaSeS. tlrTle. ThIS type of change can happen as often as once or twice a day when airlines adjust their prices to maximize The remainder of this paper is organized as follows. Seo- their overall revenue or "yield''. Independent changes occur tiara 2 describes our data collection mechanism and analyzes when the price of a particular flight changes independently the basic characteristics of airline pricing in our data. Seo- of similar flights from the same airline. We speculate that tion 3 considers related work in the areas of computational this type of change results from the change in the seat avail finance and tune series analysis. Section 4 introduces our ability of the particular flight. Table 1 shows the average data mining methods and describes how each method was number of changes per flight aggregated over all airlines for tailored to our domain. We investigated rule learning (8), G~- each route.
Overall, 762 price changes occurred across all learning (25), moving average models (13), and the cornbina~ the flights ill our data. 63% of the changes cars be classified tion of these methods via stacked generalization (28). Next, as dependent changes based on the behavior of other flights Section 5 deSCrlbeS OllT SlITlnlatlOn arid the peTfOTTIlarlCe Of by the same alrlllle.
each of the methods on our test data. The section also re-ports on a sensitivity allalySlS t0 flSS2SS the robustness of Route Avg.
nunt6er of price changes our results to changes in the simulation. We conclude with LAX-BO 6.8 a discussion of future work and a S111r17r1aTy of the paper's SEA-IAD 5.4 contributions.
Table 1: Average number of price changes per route.
2. DATA COLLECTION We found that the round-trip ticket price for flights can We collected airfare data directlyvary from a major travel web significantly over tune.
Table shows the Slte. IIl OTdeT to extract the price, large amount of data required lIlaX11I111II1 price, and the maxirrnlln difference in pTlCOS

fOr OllT IIlaCIllIle leaTIllIlg that algOTlthIT1&, We bllllt a flight Ca,II
data OCC11T
fOT
flights on each route.

COIIeCtlOII agent that r11r1S
at a scheduled lllteTVal, extracts the pricing data, and stores the Route Min Max Max Price result in a database. Price Price Change We built our flight data collection LAX-BOS275 2524 2249 agent using Agent-Builders for wrapping web sites SEA-IAD281 1668 1387 and Theseus for executing the agent (3). AgentBuilder exploits machine learning tech- Table nology (15) that enables the system2:
to automatically learn Minimum price, maximum price, and max-extraction rules that reliably imum convert information presented change in ticket price per route.
All prices in on web pages into XML. Once the this system has learned the paper refer to the lowest economy airfare avail-extraction rules, AgentBuilder able compiles this into a Theseus for purchase.

plan. Theseus is a streaming datafloW
execution system that supports highly optimized executionFor of a plan in a network Inarly flights there are easily discernible price tiers environment. The system maximizeswhere the parallelism across ticket prices fall into a relatively small price range.

different operators and streams The data between operations to unrulier of tiers typically varies from two to four, de-support the efficient eXeCllt10I1pending Of p1aI18 Wlth COIIIpIeX IlaVl- on the airline and the particular flight.
Even flights gation paths and extraction from from nnlltiple pages. the carne airline with the carne schedule (but with dif For the purpose of our pilot study,ferent we restricted our- departure dates) can have different numbers of tiers.

selves t0 COlleCtlIig data OIl For IIOII-Stop, round-trip flights example, for there are two price tiers for the flight in Figure two routes: Los Angeles (LAX) 1, to Boston (BOS) and Seat- four price tiers in Figure and tllTee price tiers in Figure tie (SEA) to Washington, DC (IAD).2 Our departure dates and Figure 3.

spanned Jarmary 2003 with the return flight 7 days after de- We expected to record (21 *
8) price observations for parture. For each departure date,each we began collecting pric- flight.
In fact, we found that on average each flight was ing data 21 days in advance at missing three-hour intervals; data 25 observations due to problems during data collec-tion including remote server failures, site changes, wrapper lwww, fetch. com bugs, etc.

2y,r0 .. . ..... .... .... . ..... . .... ... ..., ..... .. .. ..., . ...
..,.. .. . . . ..... .... .
1050 .............................................................
................ ....
1750 . . .. .... .... .. . .... .... .... . . ... . . .. . .... . . ... .. .
.. . . .. .. . .... . .. . .. .. . .... .
$ a 850 s 1250 ..................................... ..........~........................

750 . ..... .... . . .... ..... ..... .... . .. ..... .,.. . .. .... . . .
..... . . .. . .... ..,. .... .... ..,. .... .. 4r,,p . . .. . .. . .. . ,. .
.. . .. . .. . .. . .. . . . .. . .. . .. ... . .. . .. . .. . ., . .. . .. .
.. . .. . .. .
250 2'~
12118(2002121232002 12/28i2002 172!2003 1!712003 111212003 111712003 1218!2002 1211312002 12/18/1002 12!23/2002 1212812002 1!2/2003 1112003 oats pate Figure 1: Price change over time for United Air- Figure 4: Price change over time for Alaska Airlines lines roundtrip flight#168:169 LAX-BOS departing roundtrip flight#6:3, SEA-IAD
departing on Jan 4.
on Jan 12. This figure is an example of two price This figure shows an example of four price tiers.
tiers and how consumers might benefit from the price drop.
need to submit the change to the Airline Tariff Publish-ing Company(ATPCO),vs the organization formed by leading airlines around the world that collects and distributes airline . ..,.................. .. .. ........ pricing data. The whole process of detecting competitors' fare changes, deciding whether or not to match competitors' 1750 .......... ............. ... . ..... .. ...'..'....'.'.'.....""'. prices, and submitting the price update at ATPCO can take g up to orte day ~19).
a 1250 ........ ....."......._'...' ........".... . .-'..'.....-....'.'....w"
Price changes appear to be fairly orderly on some flights (e.g., Figure 3), and we see evidence of the well-known 7 7~ ..' "'_.._....-'_'.~.._.._'............'_.._.' and 14 day "advance purchaser fares. However, we also see plenty of surprising price changes. For example, flights that depart around holidays appear to fluctuate more (e.g., 121812002 12/13/2002 1211812002 1212312002 122812002 112/2003 1/7/2003 Figure 2. Figure 2 and Figure 3 show how pricing strategies differ between two flights from American Airlines that have the same schedule but fly on different dates. Figure 2 shows Figure 2: Price change over time for American Air- a flight that departs around the new year, while Figure 3 lines roundtrip flight#192:223, LAX-BOS departing shows the flight that departs one week after the first flight.
on Jan 2. This figure shows an example of rapid Both flights have the tier structure that we described earlier price fluctuation in the days priori to the New Year. in this section, but ticket prices in the first flight fluctuate more often.
In terms of pricing strategy, we can divide the airlines Price matching plays an important role in airline pricing into two categories.
The first category covers airlines that structure' Airlines use sophisticated software to track their are big players in the industry, such as United Airlines, and competitors' pricing history and propose adjustments that American Airlines.
The second category covers smaller air optinrize their overall revenue. To change the price, airlines lines that concentrate on selling low-price tickets, such as Air Trans and Southwest. We have found that pricing poli-cies tend to be similar for airlines that belong to the carne category. Fares for airlines in the first category are expen-2750 """""""""""""""""""""""""""""""""""""""""""""""""""""""""' live and fluctuate often, while fares for airlines in the second ,.. -, .., . _.. ., , ,.,. -.. ".., , .".... ".. ",.., . - -- category are moderate and appear relatively stable. How ever, there are corns policies that every airline seems to use.
~ l7so . .. .. For example, airlines usually increase ticket prices two weeks before departure dates and ticket prices are at a rnaxinnrrn a 1250 . ... . ..... ......... .............. .... .. .. .. ........ . on departure dates.
7~ ....................... ........ ..... ............. ...................3.
RELATED WORK
2so Previous work in the AI corrununity on the problem of 17!82002 12!132002 12/182002 12123f~002 121281d002 1/211003 1l7lZOD3 predicting product prices over tune has been limited to the oam Trading Agent Competition (TAC) ~27~. In 2002, TAC fo cased on the travel domain. TAC relies on a simulator of Figure 3: Price change over time for American Air- airline, hotel, and ticket prices and the competitors build lines roundtrip flight#192:223, LAX-BOS departing agents to hid on these. The problem is different from ours on Jan ?. This figure shows an example of three since the competition works as an auction (similar to Price-price tiers and low price fluctuation 'SSee http:/Jwww.atpco.net.

line.com). Whereas we gathered COIIIparISOII ShOpplrlg "hots"
actual flight price data from gather price data availal)le the web, TAC simulates flight on the web for a wide range of prices using a stochastic pro- products.4 These are de-cess that follows a random walk scendants of the Shopbot (11~
with an increasingly upward which automatically learned bias. Also, the TAC auction of to extract product and price information airline tickets assumes that from online mer-the supply of airline tickets ChaIltS' web SlteS. Norse of these is unlimited. Several TAC COIII- services attempts to analyze petitors have explored a range and predict the behavior of product of methods for, price predi~ prices over tune. Thus, tlOll lIlChldlng 111StOTICaI averaging,the data IIL1I11Itg IIlethOClS
neural nets, and 1)OOSt- 1rL t111S paper complement the body ing. It is difficult to know how of work on shopbots.
these methods would perform if reconfigured for our price raining task.

There has been some recent interest4, DATA MINING METHODS
in temporal data min-ing (see ~23J for a survey). However, the problems studied In this section we explain how we generated training data, under this heading are often quitearid then describe the various different from our own data mining methods we in-(e.g., (1J). There has also been and tune series (13, algorithmic work on tune se- (a-learning (25~
vestigated: Ripper (8J

ries methods within the data raining, cornrnunity (e.g., (4J). , 9~. We then explain how our data mining algorithm, HAM-We discuss tulle series methods LET, combines the results of these below. methods using a variant Problenls that are closely related to price prediction over tirlle have been studied in statisticsof stacked generalization (26, under the heading of 28).

"tulle series analysis" (7, 13, Our data consists of price observations 9) and in computational fi- recorded every 3 pence (20, 22, 21~ under the headinghours over a 41 day period. Our of "optimal stopping goal is to learn whether to these techniques have not been 1)uy a ticket or wait at a particular used time point, for a particu-problems''. However , lar flight, given the price history to predict price changes for consumerthat we have recorded. All goods based on data we combine these tech- of our experiments enforce the available over the web. Moreover following essential temporal , constraint: all the lrlfOTIIIatlOrl piques with rule learning techniquesUsed t0 lllake a decision at to improve their per-formance particular time, point was recorded t)efore that tune point.

. In this way, we ensure that we Computational finance is concernedrely on the past to predict with predicting prices and making buying decisions in tile future, but not vice versa.
markets for stock, options, and commodities. Prices in such 4:1 Role Learning markets are not determined by a hidden algoritlun, as in the product pricing case, but Our first step was to run the popular Ripper rule learning rather by supply and demand as system (8) on our training data.
determined by the actions Ripper is an efficient sep-of a large number of buyers and arate arid conquer rule learner, sellers. Thus, for example, We represented each price stock prices tend t0 rrlOVe 111 observation to Ripper as a vector Slllall incremental steps rather of the following features:

than lrl the large, tiered jumps observed in the airline data.

Nevertheless, there are well known~ Flight number.
problems in options trading that are related to ours.
First, there is the early ex-ercise of American Calls on stocks~ Number of hours until departure that pay dividends. The (denoted as hours-second problem is the exercise before-takeoff).
of American Puts on stocks that don't pay dividends. These problems are described in sections 11.12 and 7.6 respectively~ Current price.
of (14~. In both cases, there rnay be a tune before the , Airline.
expiration of an option at which its exercise is optimal.
Reinforcement learning meth-ods have been applied to both ~ Route (LAX-BOS or SEA-IAD).
problems, and that is one reason we consider reinforcement learning far our problem.

Tilne series analysis is a large The class lat)els on each training body of statistical tech- instance were 'buy' or piques that apply to a sequence 'wait'.
of values of a variable that varies over tune due to Borne We considered a host of additional underlying process or structure features derived from (7, 13, 9~. The observations of the data, but they did not improve product prices over tulle are Ripper's performance.

naturally viewed as tulle series We did not represent key variables data. Standard data mining like the number of ILIISOld teChIllqueS are "trallled~~ OTl seats on a flight, whether an a Set Of data t0 produce a pre- airline is running a pTOInOtlOrl, dlCtlVe IllOdel based OIl that OT SeaS011a1 variables 1)eCallSe data, Wh1C11 1S theIl tested HAnnLET did not have access OIl a separate set of test data. In to thlS lIIfOTIIlatlOrl. However, contrast, time series techniques see Section 6 for a dlSCl1$SlOrl wOllld attempt t0 predict the Of hOW HAMLET might be able t0 value of a variable 1>aSed OIl ObtalTl thlS lrlfOTIl1at10Il lIl 1tS ()WIl history. For example, the future.
Ollr IIIOVIIIg average model at-tempts to predict the future changesSome sample rules generated by in the price of a ticket Ripper are shown in Fig-on a flight from that flight's ure 5.
ovu~a price history, There is also significant interestIn our domain, classification in bidding and pricing accuracy is not the best xnet-strategies for online auctions. ric to optimize because tile cost For example, in (24~ Harshit of misclassified examples et al. use cluster analysis techniquesis highly variable. For example, to categorize the bidding rnisclassifying a single ex-strategies being used by the bidders.alllple Call COSt frOITI IlDthlI)g Alld lI1 ~17~, Lucking- t0 llpwardS Of $2,OOO. Meta-Reiley et al. explore the variousCost (10) is a well-known general factors that determine the method for training cost-final price paid lIl all OI111I1esensitive classifiers. In our allCtlOII, such as the length d()IIlaiIl, MetaCost will make of a the auction, whether there is learned classifier either more a reserve price, and the repute- conservative or more aggres-tion of the seller. However, thesesive about waiting for a better techniques are not readily price, depending on the cost applicable to our price mining 4See, for example, froogle,google,com problem. and mysimon.com.

IF hours-before-takeoff >= 252 AND price >= 2223 class and then use the learned model to generate predictions AND route = LAX-BOS THEN wait, for other states in the class.
To define our equivalence class we need to introduce some IF airline = United AND price >= 360 notation. Airlines typically use the same flight number (e.g., AND hours-before-takeoff >= 438 THEN mait UA 168) to refer to multiple flights with the carne route that depart at the carne tune on different dates. Thus, Figure 5: Sample Ripper rules. United flight 168 departs once daily from LAX
to Boston at 10:15pmr. We refer to a particular flight by a combination of its flight number and date. For example, UA168-Jam7 refers of misclassifying a 'buy' as a 'wait' compared with the cost to flight 168 which departs on January 7th, 2003. Since we of misclassifying a 'wait' as a 'buy'. We implemented Meta- observe the price of each flight eight times in every 24 hour Cost with mixed results. period, there are many price observations for each flight. We We found that MetaCost improves R,ipper's performance distinguish among there by recording the tune (number of by 14 percent, but that MetaCost hurts HAanLET's overall hours) until the flight departs. Thus, UA168-Jan7-120 is the performance by 29 percent. As a result, we did not use price observation for flight UA168, departing on January 7, MetaCost in HAMLET, which was recorded on January 2nd (120 hours before the 4.2 Q-learning flight departs on the 7th). Our equivalence class is the set of states with the carne flight numlber and the carne hours be As our next step we considered Q-learning, a species of fore takeoff, but different departure dates. Thus, the states reinforcement learning ~25j. Reinforcement learning seems demoted UAi68-Jan7-120 and UA168-JamlO-120 are in the like a natural fit because after making each new price ob- same equivalence class, but the state UA168-Jan7-117 is not.
nervation HAMLET has to decide whether to buy or to wait. We denote that s arid s* are in the carne equivalence class Yet the reward (or penalty) associated with the decision is by S ," s*, only determined later, when HAbILET determines whether it Thus, our revised Q-learning formula is:
saved or lost xnoney through its buying policy. , lheinforce-mlent leanlimg is also a popular technique in computational ~ Q(a, s) =
Avys*NB (1~(s*, a.) -~- rune. ((~(ri , s'))) finance ~20, 22, 21j.
The standard Q-learning formula is: . The reason for choosing -300,000 is now mlore apparent:
the large penalty can tilt the average toward a low value, Q(rr,, s) = Ii'(s, cr.) -1-7mcixdj (l~(at, s°)) even when xrlany la values are being averaged together. Sup pose, for example, that there are ten training examples in Here, 1;'(s, o,) is the immediate reward, ~y is the discount the carne a uivalence class, and each has a current rice of factor for future rewards, and s' is the state resulting from $2,500, Suppose now that in nine of the teal examples the taking action a in state s. We use the notion of state to price drops to $2,000 at some point in the future, but the model the state of the world after each price observation Wight in the tenth example sells out in the next state. The Q
(represented by the price, flight number, departure date, value for waiting in any state iIl this equivalence class will be and I171IrrbeZ Of hOtlTB prior to takeoff). Thus, there are two (-300 000-2 000*9)/10 = -31 800, or still much less then possible actions in each state: L for 'buy' and m for 'wait'. the Q value for any equivalence class where no flight sells Of course, the particular reward function used is critical out in the next state. Thus the choice of reward for a flight to the success (or failure) of (a-learning. In our study, the that sells out will determine how willing the (91,-Learning al-reward associated with b is the negative of the ticket price gorithnl will be to risk waiting when there's a chance a flight at that state, and the state resulting from L is a terminal may sell out.
Using a hill climrbimg search in the space of state so there is no future reward. The immediate reward penalties, we found -300,000 to be locally optimal.
associated with zv is zero as long as ecomomly tickets on the (a-learning can be very slow, but we were able to ex-flight do not sell out in the next time step. We set y = 1, ploit the structure of the problem and the close relationship so we do not discount future rewards. between dynamic progranrrming and reinforcement learning To discourage the algorithm from learning a model that (see )25j) to complete the learning in one pass over the traim-waits until flights sell out, we introduce a "penalty" for such ing set.
Specifically, the reinforcement learning problem we flights in the reward function. Specifically, in the case where face has a particularly nice structure, irl wrlicrl the value the flight does sell out at the next time point, we make of Q(6, s) depends only on the price in state s, and the the lrTlrrledlate T2WaTd fOr Waltlng a rlegatlVe COIIStaIit whose value of Q(TIJ, S) deperldS Orlly OIS the ~ Va111eS Of exactly absolute value is substantially greater than the price for any one other state: the state containing the carne flight rmmr-flight. We set the reward for reaching a sold-out state to be ber and departure date but with three hours less time left 300, 000. This setting can best be explained below, after until departure.
Applying dynamic progranuming is tlms we introduce a notion of equivalence classes among states. straightforward, and the initial training step requires only In short, we define the Q function by a single pass over the data. In order to compute averages Q(6, s) - -prices) over states in the carne equivalence class, we keep a running total and a count of the values in each a uivalence class.
-300000 if flight sells out after s. ~ q ~(r°, s) _ ~ max(G,7(b, s'), Q(zv, s')) otherwise. Thus, the reimforcernent learning algorithm just makes a sin gle pass over the training data, which bodes well for scaling To generalize from the training data we used a variant the algorithm to much larger data sets.
of the averaging step described in (18j. More specifically, The output of Ca-learning is the learned golicy, which de-we defined an equivalence class over states, which enabled termines whether to July or wait in unseen states by mapping the algorithm to train on a limited set of observations of the them to the appropriate equivalence class and choosing the action with the lowest learned cost. Let TS be the output of the Tirne Series algorithm, and let QL be the output of (a-Learning.
4.3 Time Series Time series analysis is a large and diverse subfield of IF hours-before-takeoff >= 480 AND ai~rli~te = U~aited statistics whose goal is to detect and predict trends. In this AND grrir:e >=
360 AND TS = tni~ AND QL = zurzit paper, we investigated a first order moving average model. THEN urait.
At tune step t, the model predicts the price one step into the future, Pt+l, based on a weighted average of prices already Figure 6: A sample rule generated by Hamlet.
seen. Thlls, whereas (a-learning and Ripper attempt to gen-eralize from the behavior of a set of flights in the training puted for each training example by our level-0 generalizers.
data to the behavior of future flights, the moving average model attempts to predict the price behavior of a flight in To add our three level-1 features to tile data, we applied the test data based on its own history, the model produced by each base-level generalizer (Ripper, At tune t, we pzedict the next price using a fixed window Q-learning, and tune series) to each instance in the training of price observations, pt-r~+l, . . ~ , p1. (In HAIvILET, we found data and labeled it with 'buy' or 'wait'. Thus, we added that setting k to one week's worth of price observations was features of the form TS = buy (tune series says to buy) and locally optimal.) We take a weighted average of these prices, QL = Wait ((~1-learning says to wait).
weighting the more recent prices more and more heavily. We then Used Ripper as our level-1 generalizer, running Formally, we predict that Pt+1 will be it over this augmented training data.
We omitted leave one-out cross validation because of the temporal nature of 1 a(x)pt-r+s our data. Although a form of cross validation is possible ~i 1 a(2) on temporal data, it was not necessary because each of our - base learners did not appear to overfit the training data.
where a(i) is some increasing function of i. We experi- Our stacked generalizer was our most successful data rnin-mented with different cr functions and chose a simple linearly ing method as shown in Table 3 and we refer to it as HAIvl-increasing function. LET.
Given the time series prediction, HAI41LET relies on the following simple decision rule: if the model predicts that 4.5 Hand-Crafted Rule Izt+1 > Ft, then tmy, otherwise wait. Thus, our time series After we studied the data in depth and consulted with model makes its decisions based on a one-step prediction travel agents, we were able to come up with a fairly simple of the ticket price change. The decision rule ignores the policy "1)y hand'' .
We describe it 1)elow, and include it in our magnitude of the difference between pt+1 and pt, which is results as a baseline for comparison with the more complex overly simplistic, and indeed the time series prediction does models produced by our data mining algorithms.
not do very well on its own (see Table 3). However, HAMLET The intuition underlying the hand-crafted r711es is as fol-uses the tiule series predictions extensively in its rules. In lows. First, to avoid sell outs we do not want to wait too effect, the tune series prediction provides information about long. By inspection of tile data, we decided to huy if the how the current price compares to a local average, and that price has not dropped within 7 days of the departure date.
turns out to be valuable information for HAh9LET. We can compute an expectation for the lowest,price of the 4.4 Stacked Generalization tight in the future based on similar flights in the training data." If the current price is higher than the expected rnin-EnSeIrlble-lJaSed learIllllg teChIllqlles SIICh AS bagglrlg (JJ, IIIIIlrTI
theIl It 1S best t0 Walt. OtherWlSe, we buy.
boosting (12J, and stacking (26, 28J, which combine the re- More formally, let MiraPrice(s, t) of a flight in the train-sllltS Of Irltlltlple generalizers, have teen shown to improve ing set denote tile Ixllrllrxllllxl price of that flight over the generalizer accuracy 011 many data sets. In our study, we interval starting from s days before departure up until investigated rnultiple data Il'11I11T1g methods with very differ- tulle t (or until the flight sells out). Let E~~Prire(s, t) ent characteristics (Ripper, Q-learning, and tune series) so for a particular flight number denote the average over all it makes sense to combine their outputs. ll-TirtPrice(s, t) for flights in the training set with that flight We preferred stacking to voting algorithms such as number. Suppose a passenger asks at tune t" to buy a ticket weighted majority (16J or bagging (5J because we believed that leaves in s"
days, and whose current price is C~zrPrice.
that there were identifiable a)rcrlitiorca under which one The hand-crafted rule is shown in Figure 7.
method's model would be more successful than another. See, for example, the sample rule in Figure 6. IF E~IZPrire(sr,, tr,) G CurPrice Standard stacking methods separate the original vec- AND s" > 7 days THEN
vuait, for representation of training examples (level-0 data in ELSE Lza Wolpert's terminology), and use the class labels from each level-0 generalizer, along with the example's true classifi- Figure 7: Hand-crafted rule for deciding whether to cation as input to a metarlevel (or level-1) generalizer. To wait or buy.
avoid over-fitting, "care is taken to ensure that the mod-els are formed from a batch of training data that does not We also considered simpler decision rules of the form "if include the instance in question" (26J. tile current time is less than IC days before the flight's de III OIIr lIIlpleIIleIltatI0I1 Of stacking, we collapsed level-0 parture then buy.'' Irl orlr SlrlllllatlOrl (described below) we and level-1 features. Specifically, we used the feature repre sentation described in Section 4.1 but added three additional For "similar"
flights we used flights with tile same airline features corresponding to the class lat)els (buy or wait) cool- arid flight number.

tested such rules for K ranging from 1 to 22, but none of of the ticket at the point when the predictive model reconl-these rules resulted in savings and some resulted in sulr mends buying. Net S8,V1I1gS 1S SaVlrlgS llet of both losses arld stantial losses. upgrade costs.

5. EXPERIMENTAL RESULTS S.2 Savings Table 3 shows the savings, losses, upgrade costs, and net In this section we describe the simulation we used to as- savings achieved in our simulation by each predictive model seas the SAVIIIgS due to each of the data raining 1r18thOds we generated. We also report on the frequency of upgrades described earlier. We then compare the methods in Table 3, ~ a percentage of the total passenger population, the net perform a sensitivity analysis of the comparison along sev- savings as a percent of tile total ticket price, and 'the perfor eral dimensions, and consider the implications of oar pilot rnance of each model as a percent of the maximal possible study. Sfl,VIIIgS.
5.1 Ticket Purchasing Simulation Tile xnodels we used are the following:
The most natural way to assess the quality of the predic- ~ Optimal: This model represents the rnaxirnal possi-tive models generated by the data mining methods described ble savings, which are computed by a "clairvoyant" al-ias Section 4 is to quantify the savings that each model would gorithrn with perfect information about future prices, generate for a population of passengers. For us, a passenger arid which obtained the best possible purchase price is a person wanting to buy a ticket on a particular flight for each passenger.
at a particular date arid tune. It is easy to imagine that ~ By hand: This model was hand-crafted by one of an online travel agent such as Expedia or Travelocity could the authors after consulting with travel agents and offer discounted fares to passengers on its web site, and use throughly analyzing our training data (see Figure 7).
HAMLET to appropriately tine ticket purchases behind the scenes. For example, if HAMLET anticipates that a fare will ~ Time series:
This model was generated by the mov-drop by $500, the agent could offer a $300 discount and keep ing average nlethod described earlier.
$200 as compensation and to offset losses due prediction er- ~ Ripper: This model was generated 1)y Ripper.
rOrS by HAMLET.
,5'lnCe HAMLET 1S rlOt yet ready for use by real passengers, ~ Pa-learning:
This model Was generated by Ollr C~-We SllIlulated paSSeIlgerS by generating a urllfOTIx1 dlStrlbll- lefl.rlliIlg IxlethOd.
tion of passengers wanting to purchase tickets on various , gamlet: This model was generated by our stacking flights as a function of tune. Specifically, tire Slrx1r11at10I1 generalizer which combined the results of Ii,ipper, f~-generated one passenger for each fare observation in oar set learning, and Tixne series.
of test data. The total I111TI11)eT Of paSS2IlgerS WAS 4,488.
Thus, each simulated passenger has a particular flight for Table 3 shows a comparison of the different methods. Note which they need to buy a ticket and arl earliest tune point that the savings measure we fOCrlS Or! 1S SB,VSrIgS rlet of IOSSeS
at which they could purchase that ticket (called the "earliest and upgrade costs. We see that HAMLET outperformed each purchase point''). The earliest purchase points, for different of the learning methods as well as the hand-crafted model simulated passengers, varied from 21 days before the flight to achieve a net savings of $198,074. Furthermore, despite to the day of the flight. the fact that HAMLET had access to a very limited price At each subsequent tune point, HAbILET decides whether history and Ilo lIIfOTIxlat1011 ab011t the Inlrrlber Of llIlSOld seats to buy a ticket immediately or to wait. This process con- on the flight, its net savings were a remarkable 61.8°l0 of tinues until either the passenger buys a ticket or economy optimal. Finally, while an average net savings of 4.4°/D may Seats Orl the flight Sell Orlt, in which case HAMLET Will buy IlOt SeeIII like rrnlch, passengers spend billions of dollars on a higher priced 1»lsiness-class ticket for the fiight.r' We de- air travel each year so 4.4°lo amounts to a substantial number fined upgrade costs as the difference between the cost of a , of dollars.
b11S1I1eSS CIaSS ticket and the cost of arl economy ticket at We believe that our simulation understates the savings the earliest purchase point. In our sin»llation, HAmiLET was that HAI'~1LET
would achieve in practice. For close to 7~% of forced to "upgrade" passengers to business class only 0.42vo the passengers in our test set, savings were not possible be-of the time, but the total cost of these upgrades was quite cause prices never dropped from the earliest purchase point high ($38,743 in Table 3).~ until the flight departed. We report the percent savings in We recorded for each simulated passenger, and for each ticket prices over the set of flights where savings was possible predictive model considered, the price of the ticket pur- ("feasible flights") in Table 4. These savings figures are of chased and the optimal price for that passenger given their interest because of the unrealistic distribution of passengers earliest time point and the subsequent price behavior for in our simulation.
Because we only gathered data for 21 that flight. The savings (or loss) that a predictive model days before each flight in our test set, passengers "arrived"
yields for a sinnllated passenger is the difference between the at most 21 days before a flight. Fylrthermore, due to the price of a ticket at the earliest purchase point and the price uniform distril)ution of passengers, 33°l0 of the passengers arrived at most 7 days before the flight's departure, when CIt'S pOSSIble, Of COIITSe, fOr br18111eSS ClaSS t0 Sell Ollt aS Well, S8,V1I1gS are hard t0 COIxIe by. Ill fa,Ct, Orl Orlr test data, HAn~I-in which case HAMLET would have to buy a first-class ticket LET lost money for passengers who "arrived'' in the last 7 or re-book the passenger on a different flight. However, l»1si- days prior to the flight. We believe that in practice we would ness class did not sell out 1x1 Orlr S1rr1111at10T1.
7Since we did not collect upgrade costs ~or all flights, our find-additional opportunities to save money for the bulk of upgrade costs are approximate but always positive azld often passengers who buy their tickets more than 7 days before as high as $1,000 or more. the flight date.

Method SavingsLossesUpgrade o UpgradesNet SavingsJo Savings!o of Cost Optimal ~

Optimal $320,572$0 $0 0% $320,5727.0% 100%

By hared$228,318$35,329$22,472 0.36~0 $170,5173.8% 53.2 -I~.ipper$211,031$4,689$33,340 0.45% $173,0023.8% 54.0l0 Time $269,879$6,138$693,10533.0% -$429,364-9.5% -134%
Series Q-learning$228,663$46,873$29,444 0.49% $152,3643.4l0 47.5%

Hamlet $244,868$8,051$38,743 0.42% $198,0744.4%a 61.8%

Table 3: Savings by Method.
Method Net Savings it saved more than any other method on all distributions Optimal 30.6% except the Quadratic Decrease distribution, where it per-By hand 21.8% formed slightly worse than the hand-crafted decision rule.

Ripper 20.1% HAMLET'S
savings were above 38%
of OptlIIlal in all cases.

Tinge 25.8% Table Series 5 reports on the performance of the different meth-Q-learning21.8Jo ods under the modified model where a passenger requests a Hamlet 23.8% ticket on a non-stop flight that departs at any tune during a particular three hour interval (e.g., rIIOIIIIrIg).
This different Table model 4: Comparison does of Net not Savings change (as our a percent results qualitatively.
HAMLET

of total still ticket achieves price) a on Feasible substantial Flights. percentage of the optimal sav-ings (59.2%) and its percentage of upgrades drops to only 0.1%.
Finally, HAMLET
still substantially outperforms the 5.3 Sensitivity other Analysis data mining methods.

esults t0 ChaIl eS 1r1 OllT

st ss of ur t th b t T

g o e ro u ne o r es ulation, we varied two key parameters.
First, we chaxlged the distribution of passengers requesting flight tickets.
Sec-OIId, We Changed the InOdel Of a passenger from one where a passenger wants to purchase a ticket on a particular flight to one where a passenger wants to fly at any tune during a three flour interval.
The interval model is similar to the interface offered at many travel web sites where a potential buyer specifies if they want to fly in the morning, afternoon, or evening. Table 5:
Performance of algorithms on multiple We used flights the over following three distributions hour to model interval.
the earliest purchase point (i.e., the first tune point at which passengers "arrive" Overall, and our need analysis to decide confirms whether that to buy HAD~ILET'S
a ticket perfor-or to wait): rnance on the test data is robust to the parameters we varied.

~ Uniform: a uniform distribution of simulated pas- (, FUTURE WORK
sengers over the 21 days before the flight's departure date; There are several prorrlising directions for future work on price mining. We plan to perform a more comprehensive ~ Linear Decrease: a distrik»ltion in which the number study on airline pricing with data collected over a longer of passengers arriving at the system decreased linearly period Of teens aTld over more routes. We plan to include as the amount of time left before departure decreased; nlulti-leg flights in this new data set. The pricing behavior of rnulti-leg flights is different than that of non-stop flights ~ Quadratic Decrease: a distrik»ltion like Linear De- because each leg in the flight can cause a change in the price, crease, but with a quadratic relationship; and because pricing through airline hubs appears to behave ~ Square Root Decrease: a distribution like Linear differently as well.
Decrease, but with a square root relationship; We also plan to exploit other sources of information to further improve HAMLET s predictions. We do not currently ~ Linear Increase: a distrit»ltion like Linear Decrease, have access to a key variable - the number of unsold seats except that the number of passengers increase as the on a flight. However, on-line travel agents and centralized arrlOtlIlt of time left before departure decreased; reservation systems such as Sabre or Calileo do leave this information. If we had access to the r»lmber of unsold seats ~ Quadratic Increase: a distribution like Linear In- ore a flight, HAMLET
Collld all trot eliminate the ' need to crease, 1>ut with a quadratic relationship; upgrade passengers, which is a major cost.
~ Square Root Increase: a distrillution like Linear To use the methods in this paper on the full set of domes Increase, but with a square root relationship. tic and lrlteTrlatl0llal filghtS OIl aIty glVerl day would require . collecting vast amounts of data. One possible way to address Table 6 reports the net savirlgs, as a percentage of the to- this problem is to build agents on demand that collect the tae ticket price, under the different distributions. HAbILET required data to make price predictions for ore a particular saved more than 2.5% of the ticket price in all cases, and future flight ore a particular day. The agents would still need Method Net lo of % upgrades SavingsOptimal Optimal $323,802100l0 0%

By haled$163,52355.5J 0%

Ii,ipper$173,23453.5% 0%

Time -$262,749-81.1% 6.3%
Series Q-Learning$149,58746.2% 0.2~

Hamlet ~ $191,64759.2 0 0.1%

DistributionBy C,,1-LearnTirne IB,ipperHamlet hand Series (auadratic 4.07% 3.77% -24.96l02.46%3.96%
Decrease Linear Decrease4.70% 4.30% -26.76%4.13%5.17 Sqrt Decrease4.47l04.04l0-29.05%4.23%5.03%

Uniform 3.77% 3.37% -32.55%3.83%4.38%

Sqrt Increase3.66/03.24% -34.63%4.05%4.39%

Linear Increase3.130102.72ojo-36.55%3.62%3.85%

(~luadratic 2.10 1.7400-39.90%2.48/02.60 Increase 0 0 Table 6: Sensitivity of Methods to Distribution of Passengers' Earliest Purchase Points. The numbers reported are the savings, as a percentage of total ticket price, achieved by each algorithm under each distri-bution. We see that Hamlet outperforms Q-learning, time series, and Ripper on all distributions.
to collect data for multiple flights, but tire amount of data tiOxr oxr key variables such as the nu rnber of seats available would be much smaller. This type of agent would fit well on a flight, our data mining algorithms performed surpris-within the Electric Elves system ~6, 2J, which deploys a set ingly well. Most notably, our HAbILET data ruining rrrethOd of personalized agents to monitor various aspects of a trip. achieved 61.8% of the possible savings by appropriately tinr-For example, Elves can notify you if your flight is delayed ing ticket purchases.
or canceled or let you know if there is an earlier connecting Our algorithms were drawn from statistics (tune series flight to your destination. methods), computational finance (reinforcement learning) Beyond airline pricing, we believe that the techniques de- and classical machine learning (R.ipper rule learning). Eaclr scribed in this paper will apply to other product categories. algorithm was tailored to the protrlem at hand (e.g., we In the travel industry, hotels and car rental agencies employ devised an appropriate reward firnction for reinforcement ITraTry of the same pricing strategies as tire a1T11IleS alld it learning), aTld tire algOr1t11Ir1S Wore COTIrblrl8d using a Va,rl-would be interesting to see how much HAMLET can save in ant of stacking to improve their predictive accuracy.
these product categories. Similarly, online shopping sites Additional experiments on larger airfare data sets acrd in such as Amazon and Wal-mart are beginning to explore other domains (e.g., hotels, reverse auctions) are essential, more sophisticated pricing strategies and HAIV1LET will al- but this initial pilot study provides the first demonstration of low consumers to make more informed decisions. Finally, the potential of price mining algorithms to save consumers reverse auction sites, such as hal~conr, also provide an op- substantial amounts of money using data available on the portunity for HAIVrLET to learn about pricing over tune and Internet, We believe drat price mining of this sort is a fertile make reconrnrendations about purchasing an item right away area for future research.
or waiting to buy it. In general, price mining over tixxre pro-vides a new dimension far comparison shopping engines to $. ACKNOWLEDGMENTS
exploit. We thank Haym Hirsh, John Moody, and Pedro Domingos We recognize that if a progeny Of HAjYILET wOUld achieve for helpful suggestions. This paper is based upon work sup wide spread use it could start to irnpact the airlines' (already ported in part by the Air Force Office of Scientific Research slim) profit margins. Could the airlines introduce noise into under grant nunrtrer F49620-Ol-1-0053 to USC. The views their pricing patterns in an attempt to fool a price miner? acrd conclusions contained herein are those of the authors While we have not studied this question in depth, the otr- and should not be interpreted as necessarily representing the vious problem is that changing fares on a flight in order to o~cial policies or endorsements, either expressed or implied, fool a price miner would impact all COTISllrrrer5 C011S1deTlIlg of any of tire abOVe OrgaI112at10r1S Or ally perSOI1 COrrrreCted buying tickets on that flight. If the price of a ticket moves with them.
up substantially, then consumers are likely to buy tickets on different flights resulting in a revenue loss for the airline. 9, REFERENCES
Sinrilarly, if the price moves down substantially, consumers ~1~ E,. Agrawal and Ii,. Srikant. Mining sequential will be buying tickets at a discount resulting in a revenue patterns. In P. S.
Yu and A. S. P. Chen, editors, loss again. Thus, to avoid these distortions, the airlines are Eleverctlc hcte~nritioreal Corcfer~etcre ore Drata forced to show the prices that they actually want to charge Erzyi9reeTZrry, pages 3-14, Taipei, Taiwan, 1995. IEEE
for tickets. Of course, there are more prosaic methods of Computer Society Press.
trying to block a price miner such as placing prices inside (2~ J. L. Anrbite, G. Barish, C. A. Knolrlock, M. Muslea, GIF files or trlocking the IP address of the price miner. How- J, Oh, and S.
Minton. Getting from here to there:
ever, an "industrial strength" price miner would not rely on Interactive plannrng and agent execution for "scraping'' information from web sites, but would access a optimizing travel.
In Pz~clreedi7cys of ttce Fourfieerctla fare database directly. Conference: orc Ircreovntive Applicratio~,s of Artificial ITLtFZI7,yBTLfe (IAAI 2002), pages 862-869, AAAI Press, ~. CONCLUSION Menlo Park, CA, 2002.
This paper reported on a pilot study in "price mining'' (3J G. Garish and C.
A. Knoblock. An efficient and over the web. We gathered airfare data from the web and expressive language for information gathering on the showed that it is feasible to predict price changes for flights wela. Irr Proceerlirrys of tfce AIPS-2002 LVorks)cop orc Is based on historical fare data. Despite the complex algo- t,)acr~ life after operator seyuertc27Ly? - Exployzrey yeral rithrns used by the airlines, and the absence of rIlfOTIIIa~ 7UUT'lrl TJl(r9L9L8Try, pages 5-12, Tolouse, France, 2002.

(4J D. Berndt and J. Clifford. (16J N. Littlestone and M. K.
Finding patterns in tune Warxnuth. The weighted series: a dynamic prograrnrning majority algorithm. Inforrrtr~tivn approach. In arcrl Cvrrcputativn, U. Fayyad, G. Shapiro, P. Smyth, 108(2):212-261, February 1994.
and R. Uthurusanly, editors, Advances in Ifnviulerlye(17) D. Lucking-Reiley, D. Bryan, Discovery and Data N. Prasad, and Mining. AAAI Press, 1996. D. I~,eeves. Pennies from shay:
The determinants of (5) L. Breirnan. Bagging predictors.price in online auctions. Technical Maclcine Lcarniny, report, University 24:123-140, 1996. of Arizona, 2000.

(6J H. Chalupsky, Y. Gil, C. A. (18) S. Mahadevan. Average reward Knoblock, K. Lerman, reinforcement J. Oh, D. V. Pynadath, T. A. Russ,learIllIlg: FO11I1dat10I1S, algorithms, axld M. Tarnbe. aIld empirical Electric elves: Applying agent results. Maclcine Learning, 22(1-3):159-195, technology to support 1996.

huxnau organizations. In Prvceerlinys(19) S. McCartney, Airlines Rely of ttte Conferenrx on Technology To vn htnvvrative Applications of Manipuate Fare Structure. Wall Artificial Irttelliyenre, Strr~.et. Jvur~ar~l, 2001. November 3 1997.

(7) C. Chatfield. Tlce Arcr~lysis(20) J. Moody alld M. Saffell.
of Tints Series: Art Reinforcement learning for Introduction. ChapIIlaII axld trading systems and portfolios.
Hall, LOrIdOII, UK, 1989. In KDD, pages (8J W. W. Cohen. Fast effective 279-283, 1998.
rule induction. In A. Prieditis and S. Russell, editors,(21) J. Moody and M. Saffell.
Pwc. of the l2tlc Minimizing downside risk via In,teT~rtatanno,l Cortfr:ucnce stochastic dynamic programming.
on Machine LfrlT7ti71.y, pages In Y. S.

115-123, Talloe City, CA, July Abu-Mostafa, B. LeBaron, A. W.
9-12, 1995. Morgall Lo, and A. S.

Kaufrnarlll. Weigend, editors, Cvrrcputativnral Firtance 1999, (9J F. Diebold. Elrncents of Fv~~ewasti~ay.Cambridge, MA, 2000. MIT Press.
South-Western College Publishing, 2nd edition, (22( J: Moody and M. Saffell.
2000. Learning to trade via direct (10) P. Dorningos. MetaCost: A reinforcement. In IEEE Trvnsrartiorts general method for making on Neural classifiers cost-sensitive. In Networks, lwl. 12, Nv. l, 2001.
Prvceedi~tys of the Fiftlt ACM SIGKDD InteTVtvtivnal Cv~tference(23) J. F. Roddick and M. Spiliopoulou.
on A bibliography of Knvwlerlye Discvver~ry anrl Data temporal, spatial and spatio-temporal Mining, pages data mining 155-164, San Diego, CA, 1999. research. SIGKDD E:cplorrativns, ACM Press. 1(1):34-38, 1999.

(11) R. Doorenbos, O. Etzioni, (24) H. S. Shah, N. R. Joshi, and D. Weld. A scalable A. Sureka, and P. R. Wurrnan.

comparison-shopping agent for Mining for bidding strategies the World-Wide Web. on shay. In Lecture In Proc. FiT,st, Irctl. Cv~tf. Nvtrs i~t Artifecial Intelliyertce.
A9ttOn071trJ9E8 AyeTtts, pages Springer-Verlag, 2003.

39-48, 1997. (25) R. S. Sutton and A. Barto.
Ileircfvrnerrtr:nt, Learrtirty:

(12) Y. F~eund and R. E. Schapire.An IntTYJrl7tCtirJ9t. MIT Press, Experiments with a Cambridge, MA, 1998.

new boosting algorithm. In Prvccerlinys(26) K. M. Ting and I. H. Witten.
of flee Issues in stacked Tlcirteentlt Ircterrcativrtal generalization. Journal of Artificial Cvnferenrx on Mractcin a Intelliyertce Learning, pages 148-156, Bari, IteseaTl:lt, 10:2?1-289, 1999.
Italy, 1996. Morgan Kaufrnanrl. (27) M. P. Wellrnan, D. M. Reeves, K. M. Lochner, and (13J C. W. J. Granger. Fvr~eroastirryy, Vorobeychik. Price prediction in Business arcrl in a trading agent Ecvrtvntics. Harcourt Brace, secondcompetition. Technical report, edition, 1989. University of Michigan, (14J J. C. Hull. Options, Futures,2002.
ranrl Otlter De~zvatives.

Prentice Hall College Div, 5th (28J D. Wolpert. Stacked generalization.
edition, 2002. Ne7irrrl Netvvurks , (15) C. A. Knoblock, K. Lermarl, 5:241-259, 1992.
S. Minton, and I. Muslea.

Accurately and reliably extracting data from the web:

A machine learning approach. In P. S. Szczepaniak, J. Segovia, J. Kacprzyk, and L.
A. Zadeh, editors, Intelligent E:cplvrafivn of the Web, pages 275-287.

Springer-Verlag, Berkeley, CA, 2003.

Claims (67)

1. A computer-implemented method for using predictive pricing for items, the method comprising:
analyzing prior prices for each of one or more items to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
automatically predicting that a current price for one of the items will change in the future based at least in part on at least one of the automatically determined price change patterns for the one item; and automatically determining whether to accept the current price for the one item based at least in part on the automatically predicted future price change for the one item.

2. The method of claim 1 wherein the method is performed by a computing system of an organization to provide information about airline tickets to customers by using predictive pricing that is based on historical airline ticket prices, wherein the items are airline tickets, wherein the analyzed prior prices are prices for airlines tickets that were previously offered to customers for multiple airline flights and that were each specified by an airline ticket provider unrelated to the organization, and wherein the analyzing of the prior prices to automatically determine patterns further includes, for each of the multiple airline flights, automatically determining pricing factors for the airline flight that are used to determine prices for the airline flight by the unrelated airline ticket provider for the airline flight, by identifying from the retrieved airline ticket price information multiple previously offered prices for airline tickets for the airline flight; and analyzing the identified previously offered airline ticket prices to detect the pricing factors for the airline flight, the pricing factors corresponding to changes in the identified previously offered airline ticket prices;
and wherein the method further includes, after the automatic determining of the pricing factors for the airline flights and in response to a request from a customer for information about airline flights, automatically advising the customer by:

identifying one or more of the multiple airline flights that each satisfy criteria in the request from the customer;
retrieving information about current prices offered for the identified airline flights that are specified by the unrelated airline ticket providers for the airline flights;
predicting future prices that will be offered for the identified airline flights by the unrelated airline ticket providers for those flights, the predicting based at least in part on the determined pricing factors for those airline flights;
predicting an optimal time to purchase airline tickets for each of the identified airline flights based at least in part on the predicted future offered prices and the current offered prices; and using the predicted optimal airline ticket purchase times to advise the customer related to a current purchase of one or more airline tickets;
and wherein the automatic determining of whether to accept the current price for the one item is performed for one of the airline tickets based on a predicted optimal airline ticket purchase time for that one airline ticket.

3. The method of claim 2 wherein a predicted future offered price for one of the identified airline flights is lower than the current offered price for the one airline flight, wherein the predicted optimal time to purchase an airline ticket for the one identified airline flight is at a later time corresponding to the predicted future offered price, and wherein the using of the predicted optimal airline ticket purchase times to advise a customer related to a current purchase of an airline ticket includes currently selling an airline ticket for the one identified airline flight to the customer at a price that is lower than the current offered price for that airline flight but at least as high as the predicted future offered price for that airline flight, and wherein the using of the predicted optimal airline ticket purchase times to advise the customer related to the current purchase of the airline ticket further includes delaying a purchase of that sold airline ticket from the unrelated airline ticket provider for that flight until the later time.

4. The method of claim 2 wherein a predicted future offered price for one of the identified airline flights is higher than the current offered price for the one airline flight, wherein the predicted optimal time to purchase an airline ticket for the one identified airline flight is at a current time, and wherein the using of the predicted optimal airline ticket purchase times to advise a customer related to a current purchase of an airline ticket includes notifying the customer that the current offered price for the one airline flight is a good buy such that the customer should purchase an airline ticket for the one identified airline flight at the current time.

5. The method of claim 2 wherein one of the requests from a customer is to be alerted if prices for an indicated airline flight are predicted to increase, wherein at a later time a predicted future offered price for the indicated airline flight is determined to be higher than a price that is currently offered at the later time for the indicated airline flight, and wherein the using of the predicted optimal airline ticket purchase times to advise the customer related to a current purchase of an airline ticket includes alerting the customer at the later time to purchase an airline ticket for the indicated airline flight at that time.

6. The method of claim 2 wherein a predicted future offered price for one of the identified airline flights is lower than the price currently offered for the one airline flight at a current time, wherein the predicted optimal time to purchase an airline ticket for the one identified airline flight is at a later time corresponding to the predicted future offered price, and wherein the using of the predicted optimal airline ticket purchase times to advise a customer related to a current purchase of an airline ticket includes notifying the customer at the current time to purchase the airline ticket at the later time.

7. The method of claim 2 wherein a predicted future offered price for one of the identified airline flights is higher than the current offered price for the one airline flight, and wherein the using of the predicted optimal airline ticket purchase times to advise a customer related to a current purchase of an airline ticket includes currently offering to sell an airline ticket for the one identified airline flight to the customer at a current sales price and offering to provide at least a partial refund if an actual future offered price for that airline flight is lower than the current sales price.

8. The method of claim 2 wherein one of the requests from a customer is to pay a specified price for an indicated airline flight at an indicated time, the specified price being lower than a current offered price for the indicated airline flight at the indicated time, wherein a predicted future offered price for the indicated airline flight at the indicated time is at least as low as the specified price, and wherein the using of the predicted optimal airline ticket purchase times to advise a customer related to a current purchase of an airline ticket includes currently selling an airline ticket for the indicated airline -flight at the indicated time to the customer at the specified price but delaying a purchase of that sold airline ticket from the unrelated airline ticket provider for that flight until a later time.

9. The method of claim 2 wherein the multiple customer requests correspond to prior requests for which the customers have already purchased airline tickets at prior times, and wherein the using of the predicted optimal airline ticket purchase times to advise a customer is performed at a current time but in a manner as if the predicting of the future prices and the predicting of the optimal time were performed at those prior times, so as to determine at the current time if the customers could likely have completed the purchases of the airline tickets near the prior times but at lower prices.

10. The method of claim 2 wherein multiple of the customer requests include requests to purchase airline tickets for an indicated airline flight at an indicated time, and including fulfilling those requests in an aggregate manner so as to hedge against price changes, the fulfilling including currently purchasing a subset of the requested airline tickets from the unrelated airline ticket provider for that flight and delaying purchasing of the other requested airline tickets from the unrelated airline ticket provider for that flight until a later time.

11. The method of claim 2 wherein the organization is an airline that supplies airline tickets for airline flights of the airline, wherein the retrieved information about airline ticket prices that were previously offered to customers is for airline tickets from one or more unrelated airline ticket providers that are each a competitor airline, wherein the predicted future offered prices for one or more of the identified airline flights of a competitor airline are lower than the currently offered prices for those airline flights such that the predicted optimal time to purchase an airline ticket for those airline flights is at a later time, and wherein the using of the predicted optimal airline ticket purchase times to advise a customer related to a current purchase of an airline ticket includes immediately lowering current prices on one or more of the airline flights of the organization and notifying the customer that the current prices on the airline flights of the organization are lower than the currently offered prices for airline flights of one or more of the competitor airlines.

12. The method of claim 2 wherein the using of the predicted optimal airline ticket purchase times to advise a customer related to a current purchase of one or more airline tickets is performed for a fee from the customer and/or from unrelated airline ticket providers that offer the tickets and/or from an intermediate seller from whom the one or more airline tickets can be acquired.

13. The method of claim 2 including responding to each request from a customer for information about airline flight prices by providing at least one Web page to the customer that includes information about a current offered price for each of one or more of the identified airline flights that satisfy criteria in the request from the customer, and wherein the using of the predicted optimal airline ticket purchase times to advise a customer related to a current purchase of one or more airline tickets includes providing information as part of the Web page for the customer that provides advice regarding purchasing one or more of the identified airline flights at the current offered prices for those flights.

14. The method of claim 2 wherein the method is further performed for other purchasable items distinct from airline tickets.

15. The method of claim 2 wherein the automatically determined pricing factors for each of at least one of the multiple airline flights includes multiple of an amount of time before the airline flight, a time of year of the airline flight, a degree of availability of airline tickets for the airline flight, a day of week for departure and/or arrival of the airline flight, a class code for the airline flight, a fair basis code for the airline flight, whether a current day is an advance purchase day for the airline flight, and behavior of competitors.

16. The method of claim 2 wherein the automatic determining of the pricing factors for the airline flights and/or the automatic predicting of the future prices that will be offered for identified airline flights and/or the automatic predicting of optimal times to purchase airline tickets for the identified airline flights includes using multiple of statistical-based learning, reinforcement-based learning, rule learning, machine learning, and ensemble-based learning.

17. The method of claim 2 including automatically determining one or more sell-out factors for each of the multiple airline flights based on information about prior instances of the airline flights selling out, and wherein the predicting of the optimal time to purchase airline tickets for an airline flight is further based on a predicted sell-out time for the airline flight, the predicted sell-out time being based at least in part on the determined sell-out factors for that airline flight.

18. The method of claim 1 wherein the one item has an associated expiration and/or use time, and wherein prices for the one item change based at least in part on a relationship between a current time and the associated time.

19. The method of claim 1 wherein prices for the one item change under control of a supplier of the one item.

20. The method of claim 1 wherein prices for the one item change in a controlled manner so as to maximize profit related to the one item.

21. The method of claim 1 wherein prices for the one item change in a controlled manner based at least in part on one or more factors and/or algorithms, and wherein those factors and/or algorithms are not identified.

22. The method of claim 21 wherein the analyzing of the prior prices for the one item further automatically determines at least one of the factors and/or algorithms, and wherein the automatic predicting that the current price for the one item will change is based at least in part on those determined factors and algorithms.

23. The method of claim 1 wherein the automatic predicting that the current price for the one item will change includes identifying a predicted future price for the one item.

24. The method of claim 1 wherein the automatic predicting that the current price for the one item will change includes identifying a predicted direction of the predicted future price change for the one item.

25. The method of claim 1 wherein the automatic predicting that the current price for the one item will change includes identifying a predicted magnitude of the predicted future price change for the one item.

26. The method of claim 1 wherein the automatic predicting that the current price for the one item will change includes identifying a predicted time of the predicted future price change for the one item.

27. The method of claim 1 wherein the automatic predicting that the current price for the one item will change includes identifying a likelihood associated with the predicted future price change.

28. The method of claim 1 wherein the automatic predicting that the current price for the one item will change includes predicting whether the one item will be available at a future time.

29. The method of claim 1 including receiving a fee based on the automatic predicting that the current price for the one item will change.

30. The method of claim 1 including receiving a fee based on the analyzing of the prior prices for the items.

31. The method of claim 1 including receiving a fee based on information and/or functionality provided after the automatic determining of whether to accept the current price for the one item.

32. The method of claim 1 including receiving a fee in response to an action taken based at least in part on the automatic determining of whether to accept the current price for the one item.

33. The method of claim 1 wherein the predicted future price change for the one item would result in a price for the one item that is lower than a current price for the one item, and wherein the automatic determining of whether to accept the current price for the one item includes determining that it is preferable to not accept the current price.

34. The method of claim 33 wherein the automatic determining of whether to accept the current price for the one item further includes causing the one item to currently be offered at a price that is lower than the current price.

35. The method of claim 34 wherein the current price for the one item is offered by an external supplier, wherein the causing of the current offering of the one item at the lower price is performed independent of the external supplier, and including delaying any acquisition of the one item from the external supplier until a later time.

36. The method of claim 33 wherein the automatic determining of whether to accept the current price for the one item further includes providing advice to a user to delay acquisition of the one item so as to wait for a lower price.

37. The method of claim 33 wherein the automatic determining of whether to accept the current price for the one item further includes providing advice to a user to delay acquisition of the one item so as to wait until a later time at which it is predicted that the price for the one item will be lower.

38. The method of claim 1 wherein the predicted future price change for the one item would result in a price for the one item that is higher than a current price for the one item, and wherein the automatic determining of whether to accept the current price for the one item includes determining that it is preferable to accept the current price.

39. The method of claim 38 wherein the automatic determining of whether to accept the current price for the one item further includes providing advice to a user to accept the current price for the one item.

40. The method of claim 39 wherein the automatic determining of whether to accept the current price for the one item is performed after a request from the user, and wherein the provided advice to the user is part of an interactive response to the user.

41. The method of claim 39 wherein the automatic determining of whether to accept the current price for the one item is performed independent of a current request from the user, and wherein the providing of the advice includes alerting the user regarding the advice based at least in part on previously obtained information about the user.

42. The method of claim 38 wherein the automatic determining of whether to accept the current price for the one item further includes automatically acquiring the one item at the current price.

43. The method of claim 38 wherein the automatic determining of whether to accept the current price for the one item further includes providing a price protection guarantee to a user based on future prices for the one item during a specified period of time not being above a specified price that is based on the current price.

44. The method of claim 1 including receiving an indication from a user of a specified price for the one time, and wherein the automatic determining of whether to accept the current price for the one item further includes automatically accepting the specified price when it is above a predicted future price for the one item.

45. The method of claim 1 further including comparing prior prices at which prior acquisitions of the one item occurred to alternative prices at which those prior acquisitions would have occurred if predictions for those prior acquisitions as to whether to accept those prior prices had been used, those predictions for those prior acquisitions based at least in part on the automatically determined price change patterns for the one item.

46. The method of claim 1 wherein the automatic determining of whether to accept the current price for the one item is performed in response to an indication related to acquiring the one item, and wherein the automatic determining of whether to accept the current price for the one item further includes determining to aggregate the indicated acquiring with other acquisitions of the one item in such a manner as to use the aggregated acquisitions to hedge against price changes and/or to obtain at least some of the acquisitions under preferable conditions.

47. The method of claim 1 wherein the one item is offered by an external third-party, and wherein the automatic determining of whether to accept the current price for the one item is performed as part of determining one or more conditions under which to offer another item that is a potential substitute for the one item.

48. The method of claim 1 wherein the automatic determining of the patterns and/or the automatic predicting of the future price changes includes using one or more of statistical-based learning, reinforcement-based learning, rule learning, machine learning, and ensemble-based learning.

49. The method of claim 1 wherein the items are each a ticket for an airline flight.

50. The method of claim 1 wherein the items are each one or more of a car rental, hotel rental, vacation package, cruise, gasoline, food product, jewelry, consumer electronic, book, CD, DVD, video tape, software, apparel, toy, game, automobile, ticket for a performance or event or occurrence, and furniture.

51. The method of claim 1 wherein the items are each a service provided by one or more unrelated providers.

52. A computer-readable medium whose contents cause a computing device to use predictive pricing for items, by performing a method comprising:
analyzing prior prices for each of one or more items to automatically determine factors that affect the prior prices in a predictable manner;
automatically predicting future price information for one of the items based at least in part on at least one of the automatically determined price factors for the one item; and automatically determining an action to take based at least in part on a comparison of the current price for the one item to the automatically predicted future price information for the one item.

53. The computer-readable medium of claim 52 wherein the computer-readable medium is a memory of a computing device.

54. The computer-readable medium of claim 52 wherein the computer-readable medium is a data transmission medium transmitting a generated data signal containing the contents.

55. The computer-readable medium of claim 52 wherein the contents are instructions that when executed cause the computing device to perform the method.

56. The computer-readable medium of claim 52 wherein the contents include one or more data structures for use in the automatic predicting of future price information, the data structure comprising a multiplicity of entries, each entry corresponding to an item and containing information comprising automatically determined factors and/or patterns related to prior prices for that item for use in the automatic predicting of future price information for that item.

57. A computing device for using predictive pricing for items, comprising:
a predictive price determiner system that is configured to analyze prior prices for each of one or more items to automatically determine information about changes in the prior prices that occur in a predictable manner and to automatically predict that a current price for one of the items will change in the future based at least in part on the automatically determined change information for the one item; and a predictive price use system that is configured to automatically determine whether to accept the current price for the one item based at least in part on the automatically predicted future price change for the one item.

58. The computing device of claim 57 wherein the predictive price determiner system consists of a means for analyzing prior prices for each of one or more items to automatically determine information about changes in the prior prices that occur in a predictable manner and for automatically predicting that a current price for one of the items will change in the future based at least in part on the automatically determined change information for the one item, and wherein the predictive price use system consists of a means for automatically determining whether to accept the current price for the one item based at least in part on the automatically predicted future price change for the one item.

59. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
automatically predicting that a current price for one of the airline flights will change in the future based at least in part on at least one of the automatically determined price change patterns for the one airline flight; and when the predicted future price change for the one airline flight would result in a price for the one airline flight that is lower than a current price for the one airline flight, providing advice to a user to delay acquisition of a ticket for the one airline flight so as to wait for a lower price.

60. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
automatically predicting that a current price for one of the airline flights will change in the future based at least in part on at least one of the automatically determined price change patterns for the one airline flight; and when the predicted future price change for the one airline flight would result in a price for the one airline flight that is higher than a current price for the one airline flight, notifying a user that current acquisition of a ticket for the one airline flight is preferable.

61. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
automatically predicting that a current price for one of the airline flights will change in the future based at least in part on at least one of the automatically determined price change patterns for the one airline flight, the prices at which the one airline flight is offered being specified by an external supplier; and when the predicted future price change for the one airline flight would result in a price for the one airline flight that is lower than a current price for the one airline flight, currently offering a ticket for the one airline flight at a price that is lower than the current price specified by the external supplier but that is not lower than a likely future price for the one airline flight after the predicted future price change.

62. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
automatically predicting that a current price for one of the airline flights will change in the future based at least in part on at least one of the automatically determined price change patterns for the one airline flight; and when the predicted future price change for the one airline flight would result in a price for the one airline flight that is higher than a current price for the one airline flight, offering to a user a price protection guarantee related to acquisition of a ticket for the one airline flight at the current price.

63. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
automatically predicting that a current price for one of the airline flights will change in the future based at least in part on at least one of the automatically determined price change patterns for the one airline flight; and when the predicted future price change for the one airline flight would result in a price for the one airline flight that is higher than a current price for the one airline flight, automatically acquiring at least one ticket for the one airline flight on behalf of a user.

64. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
automatically predicting that a current price for one of the airline flights will change in the future based at least in part on at feast one of the automatically determined price change patterns for the one airline flight; and when the predicted future price change for the one airline flight would result in a price for the one airline flight that is lower than a current price for the one airline flight, and after receiving an offer from a user of a specified price for a ticket for the one airline flight, automatically accepting the specified price when it is above a predicted lower future price for the one airline flight.

65. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
comparing prior prices at which prior acquisitions of one of the airline flights occurred to alternative prices at which those prior acquisitions would have occurred if predictions for those prior acquisitions as to whether to accept those prior prices had been used, those predictions for those prior acquisitions based at least in part on the automatically determined price change patterns for the one airline flight; and providing an analysis of whether the alternative prices based on the use of those predictions are preferable to the actual prior prices.

66. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
and automatically predicting that a current price for one of the airline flights will change in the future based at least in part on at least one of the automatically determined price change patterns for the one airline flight; and after receiving multiple indications that are each to acquire at least one ticket for the one airline flight, determining to aggregate at least some of the indicated ticket acquisitions in such a manner as to use the aggregated acquisitions to hedge against price changes and/or to obtain at least some of the acquisitions under preferable conditions.

67. A computer-implemented method for using predictive pricing for airline tickets, the method comprising:
analyzing prior prices for each of one or more airline flights to automatically determine patterns in changes in the prior prices that occur in a predictable manner;
automatically predicting that a current price for one of the airline flights wilt change in the future based at least in part on at least one of the automatically determined price change patterns for the one airline flight, the one airline flight being offered by an external third-party; and determining one or more conditions under which to offer tickets for another airline flight that is a potential substitute for the one airline flight based at least in part on the predicted future price change for the one airline flight.