US 7882127 B2 Abstract A system, method, and computer program product provides a multi-category apply operation in a data mining system that produces output with multiple class values, their associated measures including probabilities in case of supervised models, quality of fit and distance in case of clustering models, and the relative ranks of the predictions. A method for multi-category apply in a data mining system comprises the steps of receiving input data for scoring including a plurality of rows of data applied to a data mining model and generating multi-category apply output including a plurality of class values and their associated probabilities based on the received input data, the selected class values having probabilities meeting a selection criterion and their ranks.
Claims(28) 1. A computer-implemented method for multi-category apply in a data mining system, the method comprising the steps of:
receiving scoring data including a plurality of rows of data representing real-world quantities applied to a data mining model, including ranks, wherein the scoring data is received in a transactional format and is prepared to generate input data tables including active attributes and source attributes; and
generating a plurality of categories, each category representing a discrete value or a set of values of a class by generating multi-category apply data, the multi-category apply data generated by:
generating input data tables including active attributes and source attributes;
evaluating measures including probabilities and ranks of categories of a target attribute to determine those meeting a selection criterion for supervised models;
evaluating measures including probabilities, ranks, quality of fit, and distance of predicted values, to determine those meeting the selection criterion, for unsupervised models; and
generating an output data table including a plurality of class values of the target attribute meeting the selection criterion and their associated measures including probabilities, ranks, quality of fit and distance based on the received scoring data.
2. The method of
validating the received scoring data to ensure active attributes and a target attribute specified for the data mining model are present in the received scoring data and the source attributes specified for the multi-category apply output are present in the input data.
3. The method of
a topmost category including a class value or predicted value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or all categories including all class values with associated probabilities.
4. The method of
receiving the scoring data in a non-transactional format; and
preparing the scoring data in the non-transactional format to generate input data tables including active attributes and source attributes.
5. The method of
validating the received scoring data to ensure active attributes and a target attribute specified for the data mining model are present in the received scoring data and the source attributes specified for the multi-category apply output are present in the input data.
6. The method of
a topmost category including a class value or predicted value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or all categories including all class values with associated probabilities.
7. The method of
receiving a portion of the scoring data in a transactional format and a portion of the scoring data in a non-transactional format;
preparing the portion of the scoring data in the transactional format to generate input data tables including active attributes and source attributes; and
preparing the portion of the scoring data in the non-transactional format to generate input data tables including active attributes and source attributes.
8. The method of
validating the received scoring data to ensure active attributes and a target attribute specified for the data mining model are present in the received scoring data and the source attributes specified for the multi-category apply output are present in the input data.
9. The method of
a topmost category including a class value or predicted value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or all categories including all class values with associated probabilities.
10. A system for multi-category apply in a data mining system comprising:
a processor operable to execute computer program instructions;
a memory operable to store computer program instructions executable by the processor; and
computer program instructions stored in the memory and executable to perform the steps of:
receiving scoring data including a plurality of rows of data representing real-world quantities applied to a data mining model including quality of fit, wherein the scoring data is received in a transactional format and is prepared to generate input data tables including active attributes and source attributes; and
selecting a plurality of categories, each category representing a discrete value or a set of values of a class by generating multi-category apply data, the multi-category apply data generated by:
generating input data tables including active attributes and source attributes;
evaluating measures including probabilities and ranks of categories of a target attribute to determine those meeting a selection criterion for supervised models;
evaluating measures including probabilities, ranks, quality of fit, and distance of predicted values to determine those meeting the selection criterion for unsupervised models; and
generating an output data table including a plurality of class values of the target attribute meeting the selection criterion and their associated measures including probabilities, ranks, quality of fit, and distance based on the received scoring data.
11. The system of
12. The system of
13. The system of
receiving the scoring data in a non-transactional format; and
preparing the scoring data in the non-transactional format to generate input data tables including active attributes and source attributes.
14. The system of
15. The system of
a topmost category including a class value predicted value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or all categories including all class values with associated probabilities.
16. The system of
receiving a portion of the scoring data in a transactional format and a portion of the scoring data in a non-transactional format;
preparing the portion of the scoring data in the transactional format to generate input data tables including active attributes and source attributes; and
preparing the portion of the scoring data in the non-transactional format to generate input data tables including active attributes and source attributes.
17. The system of
18. The system of
a topmost category including a class value predicted value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or all categories including all class values with associated probabilities.
19. A computer program product for multi-category apply in a data mining system comprising:
a computer readable medium encoded with computer program instructions, recorded on the computer readable medium, executable by a processor, for performing the steps of
receiving scoring data including scores of rows of data representing real-world quantities applied to a data mining model, including distance, wherein the scoring data is received in a transactional format and is prepared to generate input data tables including active attributes and source attributes; and
selecting a plurality of categories, each category representing a discrete value or a set of values of a class by generating multi-category apply data, the multi-category apply data generated by:
generating input data tables including active attributes and source attributes;
evaluating measures including probabilities and ranks of categories of a target attribute to determine those meeting a selection criterion for supervised models; and
evaluating measures including probabilities, ranks, quality of fit, and distance of predicted values to determine those meeting the selection criterion for unsupervised models; and
generating an output data table including a plurality of class values of the target attribute meeting the selection criterion and their associated measures including probabilities, ranks, quality of fit and distance based on the received scoring data.
20. The computer program product of
21. The computer program product of
a topmost category including a class value predicted value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or all categories including all class values with associated probabilities.
22. The computer program product of
receiving the scoring data in a non-transactional format; and
preparing the scoring data in the non-transactional format to generate input data tables including active attributes and source attributes.
23. The computer program product of
24. The computer program product of
25. The computer program product of
receiving a portion of the scoring data in a transactional format and a portion of the scoring data in a non-transactional format;
preparing the portion of the scoring data in the transactional format to generate input data tables including active attributes and source attributes; and
preparing the portion of the scoring data in the non-transactional format to generate input data tables including active attributes and source attributes.
26. The computer program product of
27. The computer program product of
28. A computer-implemented method for multi-category apply in a data mining system, comprising:
receiving scoring data including a plurality of rows of data representing real-world quantities applied to a data mining model and associated measures, including quality of fit, wherein the scoring data is received in a transactional format and is prepared to generate input data tables including active attributes and source attributes; and
generating a plurality of categories, each category representing a discrete value or a set of values of a class by generating multi-category apply data including a plurality of class values by generating multi-category apply data, the multi-category apply data generated by:
generating input data tables including active attributes and source attributes;
evaluating measures including probabilities and ranks of categories of a target attribute to determine those meeting a selection criterion for supervised models; and
evaluating measures including probabilities, ranks, quality of fit, and distance of predicted values to determine those meeting the selection criterion for unsupervised models; and
generating an output data table including a plurality of class values of the target attribute meeting the selection criterion and their associated measures including probabilities, ranks, quality of fit and distance based on the received scoring data.
Description The benefit of provisional application 60/379,060, filed May 10, 2002, under 35 U.S.C. §119(e), is hereby claimed. The present invention relates to a system, method, and computer program product that provides a multi-category apply operation in a data mining system that produces output with multiple class values and their associated probabilities. In data mining, supervised learning is a collection of techniques that are used to build a model from a given set of records, known as a training set, whose class values are known a priori. Once the model is built, it is tested against another set of records with known class values, known as a test set, in order to quantify the quality of the model. It is then used to predict (or score) unknown class values of real-world records. This last stage where the model is used for prediction is termed apply. The traditional applications of such supervised learning techniques include retail target marketing, medical diagnosis, weather prediction, credit approval, customer segmentation, and fraud detection. Based on the application, it is required that the result of the apply operation contain various class values and their probabilities, as well as some attributes that can be used to characterize the input records. In conventional data mining systems, a user can describe the result of apply operation in the form of a database table. The user is allowed to specify the columns of the table. The output columns include the predicted class value and its probability, and source attributes of the input data. Typically, such conventional data mining systems allow the user to select only the class value with the highest probability. In real-world applications, however, the data miner may want to get several class values and their associated probabilities. For example, a need may arise for a recommendation engine to choose 10 items with the highest probabilities in order to provide the customers with appropriate recommendations. Thus, a need arises for a data mining system that provides a multi-category apply operation that produces output with multiple class values and their associated probabilities. This technique can also be used for unsupervised models such as clustering models. Clustering analysis identifies clusters embedded in the data where a cluster is a collection of records in the data that are similar to one another. Once clusters are identified from a given set of records, one can get predictions for new records on which cluster each record is likely to belong. Such predictions may be associated with probability, the quality of fit, which describes how well a given record fits in the predicted cluster, and the distance from the center of the predicted cluster. The present invention is a system, method, and computer program product that provides a multi-category apply operation in a data mining system that produces output with multiple class values and their associated probabilities. In one embodiment of the present invention, a method for multi-category apply in a data mining system comprises the steps of receiving input data for scoring including a plurality of rows of data applied to a data mining model and generating multi-category apply output with a plurality of predicted class values and their associated probabilities based on the received input data and a selection criterion. The step of generating multi-category apply output may comprise the steps of generating input data tables including active attributes and source attributes, evaluating probabilities of categories of a target attribute to determine those meeting the selection criterion, and generating an output data table including a plurality of class values of the target attribute and their associated probabilities, the selected class values having probabilities meeting the selection criterion. In one aspect of the present invention, the step of generating multi-category apply output further comprises the steps of receiving the input data for scoring in a transactional format and preparing the input data in the transactional format to generate input data tables including active attributes and source attributes. The method may further comprise the step of validating the received input data to ensure active attributes and a target attribute specified for the data mining model are present in the received input data and the source attributes specified for the multi-category apply output are present in the input data. The selection criterion may comprise one of a topmost category including a class value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or a set of select class values specified by the user and their associated probabilities and ranks. In one aspect of the present invention, the step of generating multi-category apply output may further comprise the steps of receiving the input data in a non-transactional format and preparing the input data for scoring in the non-transactional format to generate input data tables including active attributes and source attributes. The method may further comprise the step of validating the received input data to ensure active attributes and a target attribute specified for the data mining model are present in the received input data and the source attributes specified for the multi-category apply output are present in the input data. The selection criterion may comprise one of a topmost category including a class value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or a set of select class values specified by the user and their associated probabilities and ranks. In one aspect of the present invention, the step of generating multi-category apply data may further comprise the steps of receiving a portion of the input data for scoring in a transactional format and a portion of the input data in a non-transactional format, preparing the portion of the input data in the transactional format to generate input data tables including active attributes and source attributes and preparing the portion of the input data in the non-transactional format to generate input data tables including active attributes and source attributes. The method may further comprise the step of validating the received scoring data to ensure active attributes and a target attribute specified for the data mining model are present in the received input data and the source attributes specified for the multi-category apply output are present in the input data. The selection criterion may comprise one of a topmost category including a class value having a highest associated probability, top N categories including N class values having highest associated probabilities, bottom N categories including N class values having lowest associated probabilities, or a set of select class values specified by the user and their associated probabilities and ranks. The details of the present invention, both as to its structure and operation, can best be understood by referring to the accompanying drawings, in which like reference numbers and designations refer to like elements. An exemplary data flow diagram of a data mining process, including model building (supervised learning or unsupervised) and scoring of models (model apply), is shown in The inputs to training/model building step Training/model building step Apply (scoring) step For example, a user can score a classification model that is produced by the Naive Bayes algorithm. The data to be scored must have attributes compatible with the training data that was used for model building. Specifically, it must have a superset of attributes with the same respective data types and/or a suitable mapping. The result of apply operation is placed in the location specified by the user. Trained model Scoring data Scored data A target attribute is a data mining attribute whose value is to be predicted by applying the model. Each value that the target attribute can have is called class, and the cardinality of the class values is usually low. The lowest cardinality is two, a case of which is specifically called binary target and typically constitutes a major part of the applications. Each class is represented as a category. In other words, a category is a discrete value or a set of values of the target attribute. There can be many different ways of describing a target. For example, if age is the target attribute then there can be following types of category definitions: - 1. Binary target: If the goal is to predict whether the given person can vote, the target values will be divided into two based on this question: Is age less than 21? Again, the categories can be represented in many ways: yes/no, true/false, 0/1, child/adult, non-voter/voter, etc.
- 2. Individual value: Each value of the domain becomes a class value. The goal of this application is to predict the age of a person. One can decide upon the categories such that each number between 0 and 150 is a category. In this case, the number 150 is arbitrarily chosen, based on the knowledge that the maximum age of human being cannot exceed 150. A special value may be necessary in order to handle rare cases where someone's age is over 150 (outlier handling). For example, a value 151 may imply any age greater than 150.
- 3. Value set: Each category represents a group of ages. For example, the age groups (1-18), (18-25), (25-35) are represented as 1, 2, 3, respectively. This would make more sense in target marketing than having as many categories as ages.
A score/prediction is a category associated with probability as the result of applying to a supervised model a record whose target value is unknown. A single-target apply operation produces the target value (or category) whose probability is the highest among the all target values. A process In step If, in step If, in step In step An example of the usage of multi-category apply, according to the present invention, is described below. An online retailer recently developed a recommendation engine that provides its customers with a recommendation for a given number of products, based on the customer profile, purchase history and click stream data. Since the number of products is very high, they would have to generate as many single-category apply outputs as the number of products they want to recommend, if they did not use multi-category apply. This would cause not only a serious maintenance problem and delays, but also require significant resources. Another problem is that it is impossible to produce an output that contains a specified number of products with specified probabilities, such as those products whose probabilities are the lowest. This problem can be avoided with the use of multi-category apply. Once a model is built, a predetermined number of products can be specified to appear in the output table together with their associated probabilities. Alternatively, an analyst will be able to get a number of products with specified probabilities. An exemplary table of data mining attributes of training data used in building a supervised data mining model is shown in Examples of typical output table formats that may be generated include: -
- The topmost Category—the user wants to find the most likely category for customers to buy.
- Top N Categories—the user wants to find the top N most likely categories for customers to buy.
- Bottom N Categories—the user wants to find the top N least likely categories for customers to buy
- All Categories probabilities—the user can generate an output table with all the categories with their probabilities.
An example of an apply output table is shown in An exemplary block diagram of a data mining system Input/output circuitry Memory In the example shown in Prediction parameters Multi-category apply routines As shown in It is important to note that while the present invention has been described in the context of a fully functioning data processing system, those of ordinary skill in the art will appreciate that the processes of the present invention are capable of being distributed in the form of a computer readable medium of instructions and a variety of forms and that the present invention applies equally regardless of the particular type of signal bearing media actually used to carry out the distribution. Examples of computer readable media include recordable-type media such as floppy disc, a hard disk drive, Ram, and CD-ROM's. Although specific embodiments of the present invention have been described, it will be understood by those of skill in the art that there are other embodiments that are equivalent to the described embodiments. Accordingly, it is to be understood that the invention is not to be limited by the specific illustrated embodiments, but only by the scope of the appended claims. Patent Citations
Referenced by
Classifications
Legal Events
Rotate |