US 7533294 B2
A functional coverage based test generation technique for pipelined architectures is presented. A general graph-theoretic model is developed that can capture the structure and behavior (instruction-set) of a wide variety of pipelined processors. A functional fault model is developed and used to define the functional coverage for pipelined architectures. Test generation procedures are developed that accept the graph model of the architecture as input and generate test programs to detect all the faults in the functional fault model. A graph model of the pipelined processor is automatically generated from the specification using functional abstraction. Functional test programs are generated based on the coverage of the pipeline behavior. Module level property checking is used to reduce test generation time.
1. A method of validating a pipelined processor, comprising:
receiving a functional description of a processor architecture;
creating a graph-theoretic model of the processor architecture using the functional description, wherein the graph-theoretic model comprises a structure graph and a behavior graph;
constructing a set of functional fault models for the processor architecture, which model a set of functional faults; and
constructing a plurality of test programs, wherein each test program covers a functional fault for the processor architecture.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
executing the plurality of test programs on the pipelined processor.
7. The method of
8. The method of
9. A method of validating a pipelined processor, comprising:
receiving a functional description of a processor architecture;
creating a graph-theoretic model of the processor architecture using the functional description;
constructing a set of functional fault models for the processor architecture, which model a set of functional faults, wherein the set of functional fault models comprises a register read/write fault, an instruction execution fault, an execution path fault and a pipeline execution fault; and
constructing a plurality of test programs, wherein each test program covers a functional fault for the processor architecture.
10. A method of validating a pipelined processor, comprising:
receiving a functional description of a processor architecture;
creating a graph-theoretic model of the processor architecture using the functional description;
constructing a set of functional fault models for the processor architecture, which model a set of functional faults, wherein the set of functional fault models comprises a pipeline execution fault, and wherein the pipeline execution fault is selected from the group consisting of: a hazard detection fault, a stalling fault, a flushing fault, and an exception handling fault; and
constructing a plurality of test programs, wherein each test program covers a functional fault for the processor architecture.
11. A method of generating a test program for validating a pipelined processor comprising:
receiving an architecture specification of the pipelined processor;
receiving a set of generic templates which model a set of components of the pipelined processor;
automatically combining the architecture specification and the set of templates to generate a graph-theoretic model of the pipelined processor;
selecting a component to analyze;
receiving a property of the selected component;
generating a counterexample for the property; and
creating a test program based on the counterexample.
12. The method of
13. The method of
determining an input requirement of the component.
14. The method of
determining an output requirement of a parent component of the component;
modifying the property based on the output requirement.
15. The method of
This application claims the benefit of provisional application Ser. No. 60/609,045 filed on Sep. 9, 2004. The priority of this prior application is expressly claimed, and the disclosure of this prior application is hereby incorporated by reference in its entirety.
This invention was made with Government support under Grant Nos. 0203813 and 0205712 awarded by the National Science Foundation. The Government has certain rights in this invention.
Embodiments of the invention relate to microprocessor design and validation tools, and more particularly to systems and methods for validating the correct functioning of a microprocessor design.
As embedded systems continue to face increasingly higher performance requirements, deeply pipelined processor architectures are being employed to meet desired system performance. Functional validation of such programmable processors is one of the most complex and expensive tasks in the current Systems-on-Chip (SOC) design methodology. Simulation is the most widely used form of microprocessor verification: millions of cycles are spent during simulation using a combination of random and directed test cases in traditional validation flow. Several coverage measures are commonly used, such as code coverage, toggle coverage and fault coverage, to attempt to ensure that all aspects of a microprocessor design have been validated by the simulation. Unfortunately, these measures do not have any direct relationship to the functionality of the device. For example, none of these measures determine if all possible interactions of hazards, stalls and exceptions are tested in a processor pipeline. Additionally, certain heuristics and design abstractions are used to generate directed random test cases. However, due to the bottom-up nature and localized view of these heuristics the generated test cases may not yield a good coverage. The problem is further aggravated due to the lack of a comprehensive functional coverage metric.
Specification driven test generation has been introduced as a top-down validation technique for pipelined processors. The processor is specified using an Architecture Description Language (ADL). A SMV (Symbolic Model Verifier) description of the processor is generated from the ADL specification of the architecture. The SMV system is a tool for checking finite state systems against logic specifications for those systems. Further details about SMV are available in K. L. McMillan, The SMV System for SMV version 2.5.4. Specific properties are applied to the processor model using the SMV model checker. For example, to generate a test case to stall the decode unit, the property states that the decode unit is not stalled. The model checker produces a counter-example that stalls the decode unit. The generated counterexample is converted into a test program consisting of processor instructions. Since, the complete processor is modeled using SMV, this approach is limited by the capacity restrictions of the tool. As a result, it is not possible to model a detailed description of the processor and generate test programs which cover all of the functional aspects of the processor. Furthermore, the test generation time is long. Thus there is a need for a coverage metric based on the functionality of the processor design, and for which test program generation is automated.
To define a useful functional coverage metric, it is preferable to define a fault model of the design that is described at the functional level and independent of the implementation details. In an embodiment of the invention, a functional fault model for pipelined processors is presented. The fault model is applicable to the wide variety of today's microprocessors from various architectural domains (such as Reduced Instruction Set Computer (RISC), Digital Signal Processor (DSP), Very Long Instruction Word (VLIW) and Superscalar) that differ widely in terms of their structure (organization) and behavior (instruction-set). A graph-theoretic model has been developed, that can capture a wide spectrum of pipelined processors, coprocessors, and memory subsystems. Functional coverage has been defined based on the effects of faults in the fault model applied at the level of the graph-theoretic model. This allows for computation of functional coverage of a pipelined processor for a given set of random or constrained-random test sequences.
Embodiments of the invention present test generation procedures that accept the graph model of the pipelined processor as input and generate test programs to detect all the faults in the functional fault model. In two inventive aspects of the techniques of an embodiment of the invention, functional coverage can be used in an existing validation flow that uses random or directed-random test programs and the number of test sequences generated by the method of an embodiment of the invention to obtain a given fault (functional) coverage is several orders of magnitude less than the random or constrained-random test programs.
To make the ADL driven test generation applicable to realistic embedded processors, an embodiment of the invention automates a top-down validation method for pipelined processors. The processor model generation from the ADL specification is automated using functional abstraction. A comprehensive functional coverage metric of the pipeline behavior in terms of pipeline graph coverage is crafted, that can be used to automatically generate test programs. The pipeline graph is generated from the ADL specification of the processor. Each node in the graph corresponds to a functional unit (module) or storage component in the processor. The behavior of each node is described using SMV language. An edge in the graph represents instruction (or data) transfer between the nodes. Finally, an efficient test generation technique is provided that traverses the pipeline graph to generate test programs based on the coverage metric. The technique of an embodiment of the invention breaks a processor level property into multiple module level properties and applies them. The technique can model complex designs and can enable fast generation of functional test programs. Since, the SMV is applied at the module level, this technique can handle larger designs and also drastically reduces the test generation time.
The accompanying drawings are included to provide a further understanding of embodiments of the invention and together with the Detailed Description, serve to explain the principles of the embodiments disclosed.
In an embodiment of the invention, generation of test programs for validation of pipelined processors begins with design of an architectural model of the processor to be validated. Among the aspects that should be considered in designing an efficient architectural model are the following considerations. First, the architecture model should be based on a functional description of the processor, such as the description available in a typical user's manual for the processor to be modeled. Second, the model should be able to treat the processor organization and instruction-set as parameters of the test generation procedures. Finally, the architecture model should be able to support a functional fault model describing faults in various computations performed by the processor. Consideration of these aspects will allow fault model developers to describe faulty behavior without requiring the developer to know the implementation details of the processor.
As shown in
Vunit is a set of functional units, which includes the instruction fetch unit 20, the instruction decode unit 22, the ALU 24, the address calculation unit 26, the memory control unit 28, the write back unit 30, the first external memory interface unit 32 (EMIF_1), the co-processor 34, the second external memory interface unit 36 (EMIF_2), and the direct memory access unit 38. Vstorage is a set of storage units, including the program counter 40, a register file 42, an L1 instruction cache 44, an L1 data cache 46, an L2 unified cache 48, local memory 50 and main memory 52. Edata-transfer is a set of data transfer edges, and Epipeline is a set of pipeline edges. An edge (pipeline or data-transfer) indicates connectivity between two components. A data-transfer edge transfers data between functional units and storage units. A pipeline edge transfers program operation from one functional unit to another functional unit.
The structure sets are described mathematically as follows:
When the processor is in operation, the instruction fetch unit 20 fetches instructions from the memory sub-system 16, according to the value of the program counter 40. The instructions are fetched from the L1 instruction cache 44 or the L2 unified cache 48 if they have been previously cached therein, otherwise the instructions are fetched from main memory 52. The instruction decode unit 22 receives the fetched instructions from the instruction fetch unit 20 and decodes them for execution by the processor 12 or the co-processor, 14 as indicated by each instruction. The instruction decode unit 22 receives data from the register file 42, if appropriate. For example if the instruction is to perform an arithmetic operation on data in the register file 42, the data is fetched by the instruction decode unit 22. Similarly, if the instruction is to write data from the register file 42 to the main memory 52, the data is fetched from the register file 42. The instruction decode unit 22 then directs the decoded instructions to the ALU 24 if the instruction is an arithmetic instruction, to the address calculation unit 26 if the instruction is a memory access instruction, or to the first external memory interface 32 in the co-processor 14 if the instruction is a co-processor instruction.
For arithmetic instructions, the ALU 24 receives arithmetic instructions, performs the specified arithmetic operation on the specified data, updates the program counter 40 if necessary, and hands control off to the write back unit 30 to write the results of the ALU processing back to the register file 42.
For memory access instructions, the address calculation unit 26 calculates the memory address to be accessed by the instruction, and passes this address to the memory control unit 28. The memory control unit 28 uses the address to access the specified memory location, by accessing either the L1 data cache 46 or the L2 unified cache 48 if the specified memory location has been cached, or otherwise accessing the specified memory location in main memory 52. The memory control unit 28 also includes a feedback path to the ALU 24, should the ALU 24 need to receive a data value fetched from memory. If the instruction was a memory read instruction, the memory control unit 28 then passes control to the write back unit 30, where the read-in data value is written to the register file 42.
For co-processor instructions, the first external memory interface unit 32 receives co-processor instructions, for example multiplication instructions for a mathematic co-processor, or graphics processing instructions for a graphics co-processor. The first external memory interface unit 32 requests the data specified by the instruction from the direct memory access unit 38. The direct memory access unit 38 retrieves the requested data from the main memory 52 and populates the local memory 50 with the requested data. The co-processor 34 then accesses the requested data in the local memory 50 and performs the co-processing function (e.g. multiplication, interpolation, etc.) specified by the instruction on the requested data, and writes the modified data back to the local memory 50. The second external memory interface unit 36 then instructs the direct memory access unit 38 to write the modified data back from the local memory 50 to the main memory 52.
In addition to constructing a graph of the structure of the processor architecture, the behavior of the architecture is also captured in graph form. The behavior of a processor architecture is typically expressed by the instruction-set (ISA) description in the processor's user manual. This description includes a set of instructions that can be executed on the architecture, such as ADD, STORE, MOVE, etc. These instruction sets are well-known to those skilled in the art and will not be discussed in further detail herein. Each instruction includes a set of fields, such as an opcode field which specifies the instruction to be performed, and one or more argument fields which specify the input and output data locations for the data to be operated on and generated by the instruction. These fields specify, at an abstract level, the execution semantics of the instruction.
The behavior of the processor architecture is modeled as a graph GB=(VB, EB), where VB is a set of nodes and EB is a set of edges. The nodes represent the fields of each operation, while the edges represent orderings between the fields. The behavior graph GB is a set of disjointed sub-graphs, one for each instruction. Each sub-graph is called an instruction graph.
The behavior sets are described mathematically as follows:
Nodes are of two types. Vopcode is a set of opcode nodes that represent the opcode (i.e. the mnemonic for the instruction), and Vargument is a set of argument nodes that represent argument fields (i.e. source and destination arguments). Edges are also of two types, Eoperation is a set of operation edges that link the fields of the instruction and also specify the syntactical ordering between them. Eexecution is a set of execution edges that specify the execution ordering between the fields.
An illustrative portion of the behavior graph 53 for the processor of
Once the structure and the behavior of the processor architecture have been captured as a graph, the instructions defined in the behavior are mapped to the components defined in the structure graph of the architecture. The architecture manual provides information regarding this mapping between the structure and the behavior, to indicate which components perform which instructions. A set of mapping functions are defined which map nodes in the structure, representing components, to nodes in the behavior, representing instructions, and vice-versa.
The unit-to-opcode (opcode-to-unit) mapping is a bi-directional function that maps unit nodes in the structure graph 10 to opcode nodes in the behavior graph 53. For example, the unit-to-opcode mappings for the architecture in
The argument-to-storage (storage-to-argument) mapping is a bi-directional function that maps argument nodes in the behavior graph 53 to storage unit nodes in the structure graph 10. For example, the argument-storage mappings for the ADD operation are mappings from the DEST node 60, the SRC1 node 56 and the SRC2 node 58 to the register file 42. The other storage units of
Once the architectural model has been designed, and the structure graph 10 and the behavior graph 53 have been constructed, a set of functional fault models is constructed, for the various functions of the pipelined processor of
For the register read/write function, to ensure fault-free operation all registers should be written and read correctly. That is, when a value is written to a register and then read back from the register, the value read should be the same as the value previously written. If there is a register read/write fault, then the value read will be different from the value previously written. This fault could be due to an error in reading from the register, or in decoding the register read/write instructions, or in writing to the register, or in the register storage itself. Mathematically, if the value VRi is written into register Ri and read back, the output should be the value VRi in the fault-free case. In the presence of a fault, output≠VRi.
For the instruction execution function, all instructions must execute correctly for there to be a fault-free state. In the presence of a fault, the output of the execution of an instruction is different from the expected output for that instruction. This fault could be due to an error in instruction decoding, in control generation, or in the computation of the output of the instruction. Erroneous operation decoding might return an incorrect opcode, thus causing the wrong instruction to be executed. This can happen if incorrect bits are decoded for the opcode. Selection of incorrect bits can also lead to erroneous decoding of the source and destination operands of the instruction, which would cause either incorrect input data to be provided, or the result to be written to an incorrect destination. Even if the decoding is correct, an incorrect computation unit can be enabled, due to an error in control generation. For example a faulty instruction decode unit 22 could incorrectly enable the address calculation unit 26 for an ADD instruction, instead of enabling the ALU 24. Finally, the functional unit which executes the instruction and performs the computation can be faulty. For example, the ALU 24 could perform incorrect addition or other mathematic functions. The outcome of an instruction execution fault is an unexpected result. Expressed mathematically, let vali=fopcodei(src1, src2, . . . ), denote the result of computing the instruction “opcodei, dest, src1, src2, . . . ” In the fault-free case, the destination will contain the value vali. Under a fault, the destination is not equal to vali.
For the execution path, an execution path epopi is faulty if it produces an incorrect result during execution of instruction opi in the pipeline. During execution of an instruction in the pipeline, one pipeline path and one or more data-transfer paths get activated. The combination of these activated paths is the execution path for that instruction. The fault could be due to an error in one of the paths (pipeline or data-transfer) in the execution path. A path is faulty if any one of its nodes or edges are faulty. A node is faulty if it accepts valid inputs and produces incorrect outputs. An edge is faulty if it does not transfer the data/instruction correctly to the next node.
Without loss of generality, assume that the processor has p pipeline paths (PP=∪i=1 to p ppi) and q data transfer paths (DP=∪j=1 to q dpj). Furthermore, each pipeline path ppi is connected to a set of data-transfer paths DPgrpi (DPgrpi ⊂ DP). During execution of an instruction opi in the pipeline path ppi, a set of data-transfer paths DPopi (DPopi ⊂ DPgrpi) are used (activated). Therefore, the execution path epopi for instruction opi is, epopi=ppi ∪ DPopi. Assuming that operation opi has one opcode (opcodei), m sources (∪j=1 to m srcj) and n destinations (∪k=1 to n destk). Each data-transfer path dpi (dpi ε DPopi) is activated to read one of the sources or write one of the destinations of opi in execution path epopi. Let vali, where vali=fopcodei(∪j=1 to m srcj), denote the result of computing the instruction opi in execution path epi. The vali has n components (∪k=1 to n valj k). In the fault-free case, all the destinations will contain correct values, i.e., ∀k destk=vali k. Under a fault, at least one of the destinations will have incorrect value, i.e., ∃k destk≠vali k.
For pipeline execution, an implementation of a pipeline is faulty if it produces incorrect results due to the execution of multiple instructions in the pipeline. Note that the previous fault models consider only one instruction at a time, whereas the pipeline execution fault model examines the execution of multiple instructions in the pipeline. The pipeline execution fault could be due to incorrect implementation of the pipeline controller. The faulty controller might have errors in the implementation of hazard detection, stalling, flushing, or exception handling schemes.
Let us define stall set for a unit u (SSu) as all possible ways to stall that unit. Therefore, the stall set StallSet=∪∀uSSu. Let us also define an exception set for a unit u (ESu) as all possible ways to create an exception in that unit. We define the set of all possible multiple exception scenarios as MESS. Hence, the exception set ExceptionSet=∪∀uESu∪MESS. We consider two types of pipeline interactions: stalls and exceptions. Therefore, all possible pipeline interactions (PIs) can be defined as: PIs=StallSet∪ExceptionSet. Let us assume a sequence of operations opspi causes a pipeline interaction pi (i.e., pi ε PIs), and updates n storage locations. Let valpi denote the result of computing the operation sequence opspi. The valpi has n components (∪k=1 to n valpi k). ∀k destk=vali k. Under a fault, at least one of the destinations will have incorrect value, i.e., ∃k destk≠vali k.
In an embodiment, functional coverage of a processor architecture model is based on the fault models described above. Consider the following cases for the fault coverage:
a fault in register read/write is covered if the register is written first and read later.
a fault in instruction execution is covered if the instruction is performed, and the result of the computation performed by the instruction is read.
a fault in an execution path is covered if the execution path is activated, and the result of a computation performed by the instruction that uses the execution path is read.
a fault in pipeline execution is covered if the fault is activated due to execution of multiple instructions in the pipeline, and the result of the computation performed by one or more of the instructions is read.
The faults discussed above are detected by executing a test program that is designed to cover a particular fault for a particular component of the architecture. For example, test programs are generated that cover the register read/write fault for each register present in the architecture (in the architecture of
Once the graph model has been constructed and the fault models identified for the processor, test programs are generated to test each component of the processor to detect the faults identified by the fault models. For clarity, the following discussion refers to load and store instructions for making the register and memory contents observable at the data bus. Different architectures may use other specific instructions to access and observe the contents of registers and memories.
Each of the test generation procedures discussed below use the procedure createTestProgram shown in Table 1 to generate the actual test programs.
The procedure createTestProgram accepts a list of instructions as input, and returns a modified list of instructions, which is the test program. The instructions generated by the test generation procedures and received as input may be incompletely specified by the test generation procedures. For example, if a particular fault model does not require that a particular field of an instruction be assigned a value, then the test generation procedures discussed below do not assign a value to that field. However, when the test program is executed, all of the instructions must be fully specified, or else program execution will fail. Thus the procedure createTestProgram fills out these instructions with appropriate values for any unspecified locations, either opcodes or operands of the instructions. The procedure createTestProgram then creates initialization instructions for any uninitialized source operands, and then creates instructions to read the destination operands for each instruction in the instruction list. Finally, the procedure returns the modified list of instructions, which contains the initialization instructions, the modified input instructions, and the read instructions, in that order. This instruction list is in assembly format suitable for execution by a simulator or hardware implementation, to test the functioning of the microprocessor being designed.
The procedure of Table 2 below presents the procedure for generating test programs for detecting faults in the register read/write function.
The fault model for the register read/write function is discussed above. For each register in the architecture, the procedure generates an instruction sequence that includes a write of a pre-determined value to the register, followed by a read from the register. The function GenerateUniqueValue determines and returns a unique value for each register, based on the name of the register (e.g. “1” for register 1, “2” for register 2, etc.). For example, a test program for register Ri will include two assembly instructions: “MOVI Ri, #vali” and “STORE Ri, Rj, #0”. The move-immediate (MOVI) instruction writes vali in register R1. The STORE instruction reads the contents of Ri and writes it in memory addressed by Rj (offset 0).
The test sequence generated using the procedure of Table 2 is capable of detecting any detectable fault in the register read/write fault model. The test sequence generates one test program for each register in the architecture. A test program includes two instructions—a write followed by a read. Each register is written with a specific value, as provided in the procedure. If there is a fault in the register read/write function, the value read would be different from the value written.
The procedure of Table 3 presents the procedure for generating test programs for detecting faults in instruction execution.
The fault model for the instruction execution is discussed above. The procedure traverses the behavior graph of the architecture, and generates one test program for each instruction sub-graph in the behavior graph, using the createTestProgram discussed above. For example, a test program for the instruction sub-graph with the opcode ADD (ADD node 54 in
The test sequence generated using the procedure of Table 2 is capable of detecting any detectable fault in the instruction execution fault model. The procedure generates one test program for each instruction in the architecture, as those instructions are specified in the behavior graph. If there is a fault in instruction execution, the computed result of the instruction as executed would be different than the expected result of the instruction.
The procedure of Table 4 presents the procedure for generating test programs for detecting faults in the execution paths.
The fault model for the execution paths is described above. The procedure traverses the structure graph of the processor architecture, such as the structure graph 10 of
The test sequence generated using the procedure of Table 4 is capable of detecting any detectable fault in the execution path fault model. The only way a detectable fault will be missed is if a pipeline or data-transfer edge is not activated (used) by the generated test programs. Assume that an edge Epp is not activated by any instructions. If the Epp is not part of (connected to) any pipeline path, the fault is not a detectable fault. If Epp is part of pipeline path pp, then if the pipeline path pp does not support any instructions, the fault is also not detectable. If the pipeline path pp does support at least one instruction, then the procedure of Table 4 will generate one or more instruction sequences that exercises the pipeline path and all of the data-transfer paths connected to it. Since the edge Epp is connected to the pipeline path pp, the edge is activated by the test program generated according to the procedure of Table 4.
The procedure of Table 5 presents a procedure for generating test programs for detecting faults in pipeline execution.
The fault model for pipeline execution is described above. The procedure of Table 5 contains five loops, each of which is discussed below. The first loop (L1) traverses the structure graph of the architecture in a bottom-up manner, starting at the leaf nodes, and iteratively calls each of the loops L2, L3 and L4 for each node of the structure graph. The second loop (L2) computes test programs for generating all possible exceptions in each unit node in the architecture, based on a program template for each possible exception. The program template includes the instructions required to cause the exception. This template is passed to the procedure createTestProgram where it is filled out as discussed above to create a complete test program, and joined with the rest of the test programs for execution.
The third loop (L3) computes test programs for generating all possible stall conditions due to data and control hazards in each unit node in the architecture, based on a program template for each possible stall condition. The program template includes the instructions required to cause the stall condition. Each unit node is checked to see if it is a node that can generate a stall condition, and if it can, the template is passed to the procedure createTestProgram where it is filled out as discussed above to create a complete test program, and joined with the rest of the test programs for execution.
The fourth loop (L4) computes test programs for generating all possible stall conditions due to structural hazards in each unit node in the architecture, based on a program template for each possible stall condition. The program template includes the instructions required to cause the stall condition. A test program is created for each parent node for the node being tested, and this test program is joined with a template-based test program that stalls the node being tested, if such a stall program exists. If it does, the template is passed to the procedure createTestProgram where it is filled out as discussed above to create a complete test program, and joined with the test program for the parent and the rest of the test programs for execution.
Finally, the last loop (L5) computes test sequences for multiple exceptions involving more than one unit. The loop L5 investigates every ordered n-tuple of units in the structure graph, and generates a test program to cause an exception in each node of the n-tuple being investigated (or retrieves the test program if it has already been generated). The test programs are then combined into a composite test program using the composeTestProgram function, which combines test programs in an ordered union of programs, adding the test program for the unit closer to completion first. The composeTestProgram function also removes dependencies across test programs, to ensure the generation of multiple exceptions during the execution of the combined test program.
The test sequence generated using the procedure of Table 5 is capable of detecting any detectable fault in the pipeline execution fault model. The procedure generates test programs for all possible interactions during pipeline execution. The first loop (L1) generates all possible hazard and exception conditions for each functional unit in the pipeline. The test programs for creating all possible exceptions in each node are generated by the second loop (L2). The third loop (L3) generates test programs for creating all possible data and control hazards in each node. Similarly, the fourth loop (L4) generates test programs for creating all possible structural hazards in a node. Finally, the last loop (L5) generates test programs for creating all possible multiple exception scenarios in the pipeline.
An example of the application of the graph model of the architecture, the fault models and the test program generation procedures to two pipelined architectures will now be discussed. The two architectures discussed in this example are a Very Long Instruction Word (VLIW) implementation of the DLX architecture, presented in J. Hennessy and D. Patterson, Computer Architecture: A Quantitative Approach, Morgan Kaufmann Publishers, Inc. San Mateo, Calif., 1990, and a Reduced Instruction Set Computing (RISC) implementation of the SPARC V8 architecture, presented in The SPARC Architecture Manual, Version 8, available from Sparc International of Campbell, Calif.
The test generation and coverage analysis framework of this example were developed using Specman Elite, available from Cadence Design Systems of San Jose, Calif. The architecture specifications were captured in executable form using Cadence's “e” language. These specifications include description of 91 instructions for the DLX, and 106 instructions for the SPARC v8 architectures.
This example also implements a VLIW version of the DLX architecture using Cadence's “e” language.
The framework of this example generates test programs in three different ways: random, constrained-random, and the functional coverage approach discussed herein. Specman Elite was used to generate the random and constrained-random test programs from the specification. For the constrained-random approach, several constraints are applied. For example, to generate test programs for the register read/write fault model, the Specman Elite software was instructed to use the highest possibility for choosing register-type operations in DLX. Since register-type instructions have three register operands, the chances of reading/writing registers are higher than immediate type (two register operands) or branch type (one register operand) instructions. The test programs generated by the function coverage approach uses the procedures discussed in Tables 1-5 above.
To ensure that the generated test programs are executed correctly, the framework applies the test programs both on the implementations as well as the specifications, and compares the contents of the program counter, registers and memory locations after execution of each test program, as shown in the method of
At step 110 of
The Specman Elite framework allows definition of various coverage measures, which are used to compute the functional coverage discussed above. Each entry in the instruction definition (e.g. opcode, destination and sources) is defined in the framework as a coverage item. The coverage for the destination operand gives the measure of which registers are written. Similarly, the coverage of the source operands gives the measure of which registers are read. A different variable is used corresponding to each register, to identify a read after a write. Computation of coverage for instruction execution is done by observing the coverage of the opcode field. That is, the coverage for instruction execution is the ratio of opcodes generated by the test programs to all defined opcodes in the architecture. The computation of coverage for the execution paths is performed by observing if all the registers are used for computation of all or selected opcodes. This is performed by using cross coverage of instruction fields in Specman Elite that computes every combination of values of the fields. Finally, the coverage for pipeline execution is computed by maintaining variables for all stalls and exceptions in each functional unit. The coverage for multiple exceptions is obtained by performing cross coverage of the exception variables (events) that occur simultaneously.
Table 6 shows a comparison of the results generated by the functional coverage approach discussed above against the random and constrained-random test programs generated by Specman Elite.
The rows indicate the fault models, and the columns indicate the test generation techniques. Each entry in Table 6 has two, numbers. The first number represents the minimum number of test programs generated by that test generation technique for that fault model. The second number (in parentheses) represents the functional coverage obtained by the generated test programs for that fault model, expressed as a percentage. A 100% coverage means that the generated test programs covered all the detectable faults in that fault model. For example, the Random technique covered all the faults in “Register Read/Write” function, using 3,900 test programs. The number of test programs used for operation execution are similar for both random and constrained-random approaches. This is because the constraint used in this case (same probability for generation of all opcodes) appears to be the default option used in the random test generation approach.
The functional coverage approach was also compared to the conventional code coverage measure for determining the coverage of a set of test programs. Table 7 compares the functional coverage against conventional HDL code coverage. The first column indicates the functional fault models. The second column presents the minimum number of test programs necessary to cover all the functional faults in the corresponding fault model. The last column presents the code coverage obtained for the DLX implementation using the test programs mentioned in the second column. The test programs generated by the functional approach discussed above resulted in a small number of test programs which generate a high coverage ratio as measured by the code coverage metric.
Table 8 shows the comparative results for the different test generation approaches as applied to the LEON2 processor. The trend is similar to the results of Table 6, in terms of number of operations and fault coverage for both the DLX and the LEON2 architectures. The random and constrained-random approaches obtained 100% functional coverage for the first three fault models using an order of magnitude more test programs than the functional approach discussed above.
The random and constrained-random approaches both generated a low fault coverage for the Pipeline Execution fault model, for both processors. This occurred because these two approaches were unable to activate any of the multiple exception scenarios, and were also unable to activate some of the single exception faults. The DLX architecture has a lower coverage percentage than the LEON2 architecture because DLX has a larger set of pipeline interactions. This functional coverage problem experienced by the conventional techniques will be even greater as processors are implemented with ever larger pipelines.
Generation of test programs according to the functional approach discussed above is further automated with improved test generation time using model checking according to the following discussion. Turning to
The operation of the method of
The ADL contains information regarding the structure, behavior and mapping (between structure and behavior) of the processor. As discussed above with reference to
In creating the graph model of the architecture from the ADL description, the structure of each functional unit is captured using parameterized functions. For example, an instruction fetch unit such as the instruction fetch unit 20 of
Each sub-function is defined using appropriate parameters. For example, ReadInstMemory reads n instructions from the instruction cache using the current PC address (returned by ReadPC) and writes them to the reservation station. Using generic sub-functions in the manner shown in Table 10 allows the system to be specified in finer detail, and also allows for the re-use of the sub-functions in other units, without incurring the extra overhead of regenerating the functionality expressed in the sub-functions.
The behavior of the processor being modeled is captured through the definition of opcodes, as discussed with reference to
The graph model is organized as discussed above with reference to
The Fetch module has two inputs, PC and InstMemory (instruction memory), and one output, operation, which outputs the operation type to the next node in the graph. The fetch module has two instructions that it supports, the initialization instruction and the next instruction.
The functional coverage metric used in the functional approach is a coverage metric based on functional coverage of the processor pipeline, which covers all possible interactions between opcodes (instructions) and pipeline stages (paths) in the processor model, as represented by the structure graph of the processor. When the structure graph is completely covered, then the processor model is considered covered by the functional coverage metric. The structure graph is considered covered if all of the graph nodes are covered and all of the graph edges are covered. A node in the graph is covered if it has been in all of the four possible states for a processor node: active, stalled, exception and flushed. A node is active when it is executing an instruction. A node can be stalled due to structural, control or data hazards. A node can be in an exception state if it generates an exception while executing an instruction. It is possible to have multiple exception scenarios and stall conditions for a node. However, a node may be considered covered as long as it enters at least one exception scenario and one stall condition. A node is in the flushed state if an instruction in the node is flushed due to the occurrence of an exception in any of its children nodes.
Similarly, an edge in the graph is considered covered if it has been in all of the three states possible for an edge in the processor model: active, stalled, and flushed. An edge is active when it is used to transfer an instruction in a clock cycle. An edge is stalled if it does not transfer an instruction in a clock cycle from a parent node to a child node. An edge is flushed if the parent node is flushed due to an exception in the child node. The edge coverage conditions are redundant if a node has only one child, because the node coverage conditions will necessarily also cover the edges. However, if a node has multiple children (or parents), edge coverage conditions are necessary.
The test generation method discussed with reference to
An example of the results of an implementation of the method of
Table 12 shows the number of test programs generated for node and edge coverage of the DLX processor, according to the method discussed above.
Although 20 test cases would suffice for the active node coverage, this example covers all 91 instructions. Also, there are many ways of making a node stalled, flushed or in an exception condition. This example chose one such condition for each node. If all possible scenarios were considered, the number of test programs would increase. In this example, the method discussed above generated 223 test programs in 91 seconds on a 333 Mhz Sun UltraSPARC-II with 128M RAM.
As mentioned above, some of the test programs are redundant. For example, since the processor model of
A specific example of creating a test program according to the method of
Using the language of the SMV verifier, a property is created which is used to generate the instruction sequence to initialize Ain and Bin with values 2 and 3 respectively at clock cycle 9. This property reads as follows:
The property implies that if the current clock cycle is 8, in the next cycle DIV.Ain should not be 2 or DIV.Bin should not be 3. Recall that the property are expressed in negation form when using the SMV language. This property is then slightly modified to make it applicable at the module level, by removing the “DIV.” header from the input names.
The modified property is then applied to the division unit 80 using the SMV description of the division unit 80 (step 220 of
The counterexample generated by this property (step 225 of
This modified property is applied to the fetch unit 70 (step 220 of
In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. For example, the reader is to understand that the specific ordering and combination of process actions shown in the process flow diagrams described herein is merely illustrative, and the invention can be performed using different or additional process actions, or a different combination or ordering of process actions. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense, and the invention is not to be restricted or limited except in accordance with the following claims and their legal equivalents.