US 6353767 B1 Abstract A method of developing a confidence level in reliability or producibility prediction scores that includes generating data for a set of factors, inserting the data into scorecards and dashboards, and calculating an overall Z score and Z confidence range. The scorecards and dashboards are used to calculate the confidence level in reliability scores. Each reliability or producibility factor may be weighted as to its importance to the overall confidence level. Factors included in the method include accuracy of the critical to quality flowdown; completeness of the critical to quality flowdown; percentage of the design analyzed; comprehensiveness of the design analysis; risk assessment analysis; percentage of component. subassembly or product reuse from other projects; percentage of the design complete; verification of the process capability; plant integration; order to remittance process integration; extent of pilot run; and effectiveness of a test plan.
Claims(25) 1. A method of developing, a confidence level using six sigma prediction scoring comprising: determining a customer expectation value; determining a Z factor comprising a set of factors; selecting said set of factors; generating respective data sets for said set of factors; collecting said respective data sets in at least one scorecard; calculating at least one Z score for said at least one scorecard; generating a total Z score as a function of the respective scorecard Z scores scorecards; comparing said total Z score for said scorecards with a selected Zst value; calculating a Z confidence range; scoring a confidence level based upon said Z confidence range and said total Z value; and reporting said confidence level.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
9. The method of
10. A method of
11. The method of
12. The method of
13. The method of
14. The method of
15. The method of
16. The method of
17. The method of
18. A storage medium encoded with machine-readable computer program for developing a confidence level in six sigma prediction scoring, the storage medium including instructions for causing a computer to implement a method comprising: determining a customer expectation value; determining an availability value; determining a life calculation prediction value; determining a Z factor comprising a set of factors; selecting said set of factors for at least one category; generating a plurality of respective data sets for said set of factors; collecting said data sets in at least one scorecard; generating a transfer function to quantify said data sets in said at least one scorecard; calculating at least one respective Z score for said at least one scorecard; generating a total Z score for all of said scorecards; comparing said total Z score for all of said scorecards with a selected Zst value; calculating a Z confidence range; scoring a confidence level based upon said Z confidence range and said total Z value; and reporting said confidence level.
19. The storage medium of
20. The storage medium of
21. The storage medium of
22. The storage medium of
23. The storage medium of
24. The storage medium of
25. A method for using a computer to develop a confidence level in six sigma prediction scoring, comprising: determining a customer expectation value; determining an availability value; determining a life calculation prediction value; determining a Z factor comprising a set of factors; selecting said set of factors; generating a plurality of respective data sets for said set of factors; collecting said data sets in at least one scorecard; generating a respective trransfer function to quantify said respective data sets at said scorecards; calculating at least one Z score for said at least one scorecard; generating a total Z score for said scorecards; comparing said total Z score for said scorecards with a selected Zst value; calculating a Z confidence range; scoring a confidence level based upon said Z confidence range and said total Z value; and reporting said confidence level.
Description The invention relates generally to a method and system of predicting and tracking a confidence level or score in reliability prediction within engineering and business processes that use design for six sigma techniques (DFSS). The invention is a subset of a system for implementing a DFSS process. For any process (business, manufacturing, service, etc.), the “Z” statistic is a metric that indicates how well that process is performing. The higher the “Z” value, the better the output. The “Z” statistic measures the capability of the process to perform defect-free-work, where a defect is synonymous with customer dissatisfaction. With six sigma the common measurement index is defects-per-unit where a unit can be virtually anything—a component, a component of a jet engine, an administrative procedure, etc. The “Z” value indicates how often defects are likely to occur. As the “Z” value increases, customer satisfaction goes up along with improvement of other metrics (e.g., cost and cycle time). “Six sigma” generally refers to a quality improvement system in which statistical processes are used to assess and measure process or product capabilities (with six sigma referring to an extremely small defect rate corresponding to six standard deviations of the desired process capability). Most uses of six sigma have been for improving a specific application, such as semiconductor manufacturing, through a quality improvement project. The basic steps in a quality improvement project are first to define the real problem by identifying the customer's critical-to-quality requirements and related measurable performance that is not meeting customer expectations. This real problem is then translated into a statistical problem through the collection of data related to the real problem. By the application of the scientific method (observation, hypothesis and experimentation), a statistical solution to this statistical problem is arrived at. This solution is deduced from the data through the testing of various hypotheses regarding a specific interpretation of the data. Confidence (prediction) intervals provide a key statistical tool used to accept or reject hypotheses that are to be tested. The arrived at statistical solution is then translated back to the customer in the form of a real solution. In common use, data is interpreted on its face value. However, from a statistical point of view, the results of a measurement cannot be interpreted or compared without a consideration of the confidence that measurement accurately represents the underlying characteristic that is being measured. Uncertainties in measurements will arise from variability in sampling, the measurement method, operators and so forth. The statistical tool for expressing this uncertainty is called a confidence interval depending upon the exact situation in which the data is being generated. Confidence interval refers to the region containing the limits or band of a parameter with an associated confidence level that the bounds are large enough to contain the true parameter value. The bands can be single-sided to describe an upper or lower limit or double sided to describe both upper and lower limits. The region gives a range of values, bounded below by a lower confidence limit and/or from above by an upper confidence limit, such that one can be confident (at a pre-specified level such as 95% or 99%) that the true population parameter value is included within the confidence interval. Confidence intervals can be formed for any of the parameters used to describe the characteristic of interest. In the end, confidence intervals are used to estimate the population parameters from the sample statistics and allow a probabilistic quantification of the strength of the best estimate. A prediction interval for an individual observation is an interval that will, with a specified degree of confidence, contain a randomly selected observation from a population. The inclusion of the confidence interval at a given probability allows the data to be interpreted in light of the situation. The interpreter has a range of values bounded by an upper and/or lower limit that is formed for any of the parameters used to describe the characteristic of interest. Meanwhile and at the same time, the risk associated with and reliability of the data is fully exposed allowing the interpreter access to all the information in the original measurement. This full disclosure of the data can then be used in subsequent decisions and interpretations for which the measurement data has bearing. A drawback to specific applications of the six sigma process is that there is a lack of flexibility to allow for the existing implementation to be applied to other business processes. There is a need to develop a confidence level in the DFSS results of a project while in the early stages of its development. With such a confidence level prediction, decisions can be made on improving the confidence of achieving a six sigma level in the final stages of the project development. It is well known to those skilled in the art of project management, that changes in the early stage of a product are easier and less expensive than if such changes are made at the customer's location. An exemplary embodiment of the invention is a method of developing a confidence level in six sigma prediction scores that includes determining a customer expectation value. A Z factor comprising a set of factors is determined. The set of factors are selected and a plurality of data is generated for the set of factors. The data is collected in at least one scorecard. At least one Z score is calculated for at least one scorecard. A total Z score is generated for the scorecards. The total Z score for said scorecards is compared with the Zst value. A Z confidence range is calculated. A confidence level based upon said Z confidence range and said total Z value is scored. The confidence level is reported. Another embodiment uses a storage medium encoded with machine-readable computer program for developing a confidence level in six sigma prediction scores. The storage medium includes instructions for causing a computer to implement a method comprising determining a customer expectation value, an availability value and a life calculation prediction value. A Z factor comprising a set of factors is determined. The set of factors are selected and a plurality of data is generated for the set of factors. The data is collected at least one scorecard. A transfer function is generated to quantify the data at the scorecards. At least one Z score is calculated for at least one scorecard. A total Z score is generated for the scorecards. The total Z score for the scorecards is compared with the Zst value. A Z confidence range is calculated. A confidence level based upon said Z confidence range and said total Z value is scored. The confidence level is reported. These and other features and advantages of the present invention will be apparent from the following brief description of the drawings, detailed description, and appended claims and drawings. The invention will be further described in connection with the accompanying drawings in which: FIG. 1 is a chart that illustrates nominal confidence in reliability predictions and costs to fix product defects at various stages of the product life; FIG. 2 illustrates the various metrics used in predicting life with confidence; FIG. 3 illustrates the product structure used in confidence prediction; FIG. 4 illustrates the relationship between confidence and Zst over the life of a project; and FIGS. 5A and 5B are two segments of a scorecard containing the three categories comprising the overall Z factor. The Design for Six Sigma (“DFSS”) product development process envisions both matching designs with manufacturing capabilities to achieve six sigma producibility and matching designs with the operating environment to achieve six sigma reliability. The predictions of component reliability (life) and producibility are routinely made in the product development process. When this is done in the context of six sigma the probability of achieving these predictions is often given as a statistical “Z” value (the overall sigma rating for a system is commonly referred to as Zst (Z short term). Another statistical term is that of “confidence” usually given as a percent value that indicates the degree of certainty that the predicted “Z” value is realistic and likely to be realized once the process or component is put into its designed use. When enough data is measured, both “Z” and confidence can be calculated using well know statistical methods. When no product exists to measure, these parameters can be estimated using historical data from legacy products. In the past, only the predicted “Z” value was considered prior to drawing release, that is, approval of the design documents for an article or a process, or prior to measurement of the actual performance of a product. However, there exists a need to evaluate the confidence of a product design prior to its drawing release. FIG. 1 illustrates an exemplary embodiment of a product life cycle. As is shown, changes implemented at an early stage of the design process are easy and inexpensive, whereas, changes in a product s design made after its release, production, or use are prohibitive. If the manufacturer waits until there are product failures in the field to measure performance, it can become difficult and expensive to make the necessary changes in the design. At that point a re-design program can be implemented to addres' the poor performance of the product, as well as the immediate and negative impact experienced by the customers. To address this concern a confidence score can be estimated in the early stages of the design process using manufacturing management tools, i.e., a scorecard(s), prior to the drawing release. An exemplary embodiment of the invention is a process including methods and tools for scoring the predicted reliability and/or producibility of components or systems throughout the product life cycle, including the conceptual and development period prior to approval of drawings and specifications to go into the production process. A first embodiment of the method disclosed herein scores confidence for producibility predictions, or, alternatively, design(s) for assembly, and a second embodiment scores confidence for reliability predictions. However, both of these embodiments can have the following common features: scoring confidence without new product use or production data; scoring at any point in the product cycle; tracking scoring over time; depicting growth in confidence to an acceptable risk level as might be assessed in the design process based on performance, cost, and other salient factors; confidence scoring can be product specific in a general framework; percent confidence scores can be shown on scorecards; and validation of scoring methods can be provided. FIG. 2 illustrates how the confidence scoring of a Z factor, or a life prediction, can be generalized as a combination of three exemplary categories Each factor for each category (E, V, A) can include a confidence score. The confidence scores for each factor can be combined to generate a confidence score for the respective category, i.e., E, V, and A. The confidence scores for each category can be arithmetically averaged to generate an overall confidence score that provides a qualitative assessment of the confidence of the Z factor for that particular design:
In the alternative, each category can be weighted differently when, for example, generating confidence scores for each category using historical data from similar legacy products. Products previously designed for a customer are typically referred to as legacy products. Legacy products can be scored, using this method, for reliability of the legacy product's design. The legacy product scoring can be examined to determine if a specific factor or a set of factors influenced that design process. T he relative significance of one factor with respect to another can be reflected as coefficients “C” in the equation for generating the overall confidence score.
The C values can be based on regression using the legacy product data. Likewise, the C values can be weighted to place more emphasis on factors that have more impact on Reliability. For instance, a particular set of factors for Use and Environment FIG. 3 illustrates an exemplary embodiment of a flow diagram for a product structure In the instant application, for example, the product being designed can be an aircraft engine. The aircraft engine design can be examined as a whole at Plants & Major Systems The breakdown of the system into features and materials of individual components coincides with the customer CTQs generated at Dashboards A CTQ flowdown can then take place from Dashboards The Dashboards Tables 1 and 2 herein illustrate examples of determining confidence in Z factors for a portion of a typical design project. The factors listed in Table 1 are defined in Table 2. Referring now to Table 2, Z factors such as the Accuracy of the CTQ Flowdown, Completeness of CTQ Flowdown, Comprehensiveness of Analysis, Risk Assessment Effectiveness, Verified Process Capability of Entitlement Database. Plant Integration, OTR Process Integration, Extent of Pilot Run, and Effectiveness of Test Plan can be assigned a qualitative numerical score such as, for example, a 9, a 5, or a 1. The numerical value corresponds to a definition of the status of the design process, which can be determined using the feedback from the Product Structure The descriptions provided for each Z factor further improve the Scorecards Referring again to Table 1, in this exemplary embodiment, the “Z” value from the Scorecard FIG. 4 illustrates graphically the impact of time on the percentage of confidence calculated using the method disclosed herein for an exemplary completed product design project. The x-axis frequently represents the duration of the product design project in terms of months, or in another measure of time such as days, annually, semi-annual, decades, and the like. The y-axis frequently represents the Z values calculated using the method disclosed herein. The project duration can be further measured using tollgates referred to as DP As is illustrated in FIG. 4, the value of a typical confidence range of a particular Zst generally improves over the life of the project. For example, the value Zst at the fourth month, or DP By way of example, the individual Z values for the factors of each category of Table 1 can be compiled from the Scorecards
wherein factors A-L represent the factors listed in Tables 1 and 2 (Accuracy of CTQ Flowdown, etc.). The Scorecards
where E represents the confidence score for Use and Environment
The weighting approach is used when it is desirable to place the value of more emphasis upon a category having a significant impact upon the overall life prediction of the product. As experience develops in the use of the method of establishing confidence factors, the level of confidence is expected to improve. As time progresses calibrated field data on parts used in multiple products (legacy parts) will determine what level of confidence is an acceptable level. This quantification of confidence at an early stage is an important factor in the development of a reliable product that meets the customer's requirements. The ability to develop over time a confidence factor at an early stage of the product cycle is an important factor towards the efficient production and introduction of a product. The testing stage is too far down the product development cycle to be an efficient location for reliability or producibility predictions. Likewise, reliability or producibility predictions made during production and/or use are based upon a prediction using ill defined or undefined factors. The method of developing confidence scores enables the user to avoid engaging in extensive testing, while improving the overall efficiency and quality of the development and production cycles. For example, safety issues may be weighted higher for an aircraft engine than for another product. Although cost factors are typically excluded in the evaluation, the cost to achieve a certain confidence level might be taken into account. If the costs were not considered, it might be possible to produce a product having a very high six sigma value, such as 100% confidence, at a price that would be unprofitable. The method disclosed herein typically is used to determine what confidence level changes may transpire when altering the number of factors, and their corresponding values, in order to evaluate the economic impact of choosing one factor over another to achieve a particular confidence level for reliability and producibility predictions. Prediction scoring for reliability and producibility comprises a disciplined approach to ensure products will meet their quality goals. The approach integrates the efforts of all engineering functions into a standard framework for product modeling and specification. The method disclosed herein provides enhanced producibility by serving as a key driver of hardware cycle time, reducing risk of test failures, reducing the number of redesigns, lowering the part introduction cost, reducing risk of manufacturing defects, and reducing, engineering costs associated with manufacturing. The method disclosed herein also serves to enhance reliability by reducing warranty and concession costs, and reducing engineering costs associated with field support. The present invention can be embodied in the form of computer-implemented processes and apparatuses for practicing those processes. The present invention can also be embodied in the form of computer program code containing instructions embodied in tangible media, such as floppy diskettes, optical media such as compact discs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. The present invention can also be embodied in the form of computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits. While the invention has been described with reference to preferred embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiments disclosed as the best mode contemplated for this invention, but that the invention will include all embodiments falling within the scope of the appended claims. [t2]
[t3]
[t4]
Patent Citations
Referenced by
Classifications
Legal Events
Rotate |