CA1252900A - Parser-based attribute analysis - Google Patents
Parser-based attribute analysisInfo
- Publication number
- CA1252900A CA1252900A CA000506446A CA506446A CA1252900A CA 1252900 A CA1252900 A CA 1252900A CA 000506446 A CA000506446 A CA 000506446A CA 506446 A CA506446 A CA 506446A CA 1252900 A CA1252900 A CA 1252900A
- Authority
- CA
- Canada
- Prior art keywords
- attribute
- grammar
- program
- attributes
- parser
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
- G06F8/42—Syntactic analysis
- G06F8/427—Parsing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
- G06F8/43—Checking; Contextual analysis
- G06F8/436—Semantic checking
Abstract
ABSTRACT
A table-driven attribute parser for checking the con-sistency and completeness of attribute assignments in a source program is generated by expressing the syntax rules semantic restrictions and default assignments as a single context-free grammar compatible with a grammar processor, or parser generator, and processing the context-free grammar to generate an attribute parser including a syntax table and a parse driver. The table-driven attribute parser is incor-porated into a compiler.
A table-driven attribute parser for checking the con-sistency and completeness of attribute assignments in a source program is generated by expressing the syntax rules semantic restrictions and default assignments as a single context-free grammar compatible with a grammar processor, or parser generator, and processing the context-free grammar to generate an attribute parser including a syntax table and a parse driver. The table-driven attribute parser is incor-porated into a compiler.
Description
;~ ?~ S~ ~ V
IBM DOCKET No~: SA8-84-038 P.~RSER-BASED ATTRIBUTE ANALYSIS
sAcKGRouND OF THE INVENTION
, The present invention is directed to an improved attri-bu~e analyeis technique for use in compilers~ and more part~cularly to a parser-based attribute analysis technique S for usa in compilers. The invention will be described in ~ha context of PL/I programming, although it will be appre-ci~ted ~hat this is by way of example only and ~hat the lnvention is applicable to other programming languages as well.
1~ ~hen creatinq a program to run on a general purpose computer, the progr m is first written in source code, and ~he source code program is then passed through a compiler which converts the source code into object code, o~ machine languaqe. There are several tasks performed by the compiler lS be~ore inally generating an object code output, one of ~ha~e tasks being performed by an "attribute proce~sor" in th~ compiler front end. More particularly, when writing a P~II source pragram, various "attributes" are assigned to data items used in ,the program, e.g., an item can be ~ ~ssigned the attribute of a variable or a constant, and a varlable can be further assigned the attrLbutes of PARA-M~TER, CONTROLLED, or STATIC. The most common way of assigning the-~e attributes to data items is by way of a DECL~RE statement, although attributes can also be assigned by a number of other techniques, e.g., a data description statement, a LI~E statement, or even inferentially by using ~2~ t~
IBM Docket No.: SA9-84-038 a data item in a context which would require certain attributes. The ~asks performed by the attri~ute processor of a PL/I compiler during compi-lation of ~ PL/I source progriam include: the checking of syntax correctness of attribu~es specified for each declared data item in the defining PL/~ DECLAR~ sta~ement, and repre-senting this initial set of valid a~:~ributes in a concise internal notation and comple~ing the initi~l attribu~2 se~
of each ~ata item by providin~ ~he appropria~e default values for any missing attributes, whil~ checking the con-sistency and completeness o~ the final attribute set.
The attrLbute processor tasks must be performed in accordance with the syntax gri~mmar defined in the American National Standard Institute 1ANSI~ PLJI standard, particu-larly chapter 4 thereof. The test for attribu~e consistencyis set forth in 4.3.6.1 at pages 83-85. The purpose of the attribute consistency check is to ensure that no data item has been assi~ned inconsistent attribu~es, e.g., REAL and COMPLEX. The completion of the attribute set with default ~alue~ is described in ~4.3.6.3 through 4.3.6.5 at pages;~
86-88 of the ANSI P~/I standard. Thie checking to determine~
that the final attribute se~ is complete and consistent i5 described in 4.3.7.1 at page 93 o~ the ANSI PL~I standard.
The tasks performed by:the attribut~ processar o~ a PL/I
compiler front end are not limited to only those steps :~et for~h in the above-cited sections of the AN5I PL/I standard,~
, "r~
~ S2~
IBM DOCKET NO.: SA8-84-03B
but these sections exemplify the gr~matical rules which must be followed in the attribute processor.
In conventional PL/I compilexs, the attribute analysis is caxried ollt by various ad hoc techniques, whereas defaul~
values are added, and completeness and consis~ency of the final at~ribute se~s checked, by using specialized proce-dures, e.gO, breakin~ the program down into small parts and sub-parts and then checking from inside to ou~side. This requires the writing of a great deal of code, typically generated manually by ~he programmer. This is not only tLme consuming and quite ~edious, but the manual generation of the code is error-prone and substantial de~ugging of the compiler program is usually r~yuired~ Investigating and correc~ing errors in a very long listing is a time consuming process.
Further, even after a working compiler is obtained, it is sometimes necessary to update the compiler to accommodate new or changed language features. ~s in the original gene-ration of the compiler pro~ram, updating and dehugging extensive object code listings implementing complex gramma-tical rules is a task which reguires a sub~tantial amount of programmer time.
Fox the above reasons, the generation of the a~ribu~e processor por~ion of the compiler program by conventional techniques is error prone, inef~icien~ a~d difficul~ to adapt to new or chan~ed language features/ and it would be highly advanta~eous to provide a simplified and much ~aster ~ 25~
IBM DOCKET NO.: SA8-84-038 technique for implementing th attribute processor portion of a compiler.
S~AIIV ~r r~ ~ rlc l It is, therefore, an object of this invention ~o pro-vid~ a simplified technique for implementing an att~ibu~eproces~or.
Briefly, this and other objects of the invention ar~
realized by implementing the attribute processor as a table-drive~ attribute parser, and the invention comprises both a table-driven parser ih th~ attribu~e processor as well as a simple and efficient technique for genexating the syntax tables and parse d river which make up the attri~ute parser.
In the parser-based a~tribute analysis of the present inven-tion, the consistency of attributes of PL/I declarations is checked, and default values for missing attribu~es are added, based on th~ efficient and reliable LALR~l) parsing technique. The attribu~e analysis grammar speclfied in the ANSI standard is ~xpressed in terms of a context-free grammar which is provided as an input to a LALR(l) grammar processor. In a well-known manner, ~he gra~mar proces~or generat2s a finite sta~e mac~ine ~SM) consisting of syntax tables and a parse driver. These are included i.n ~he compiler fron~ end and can be used ~o check ~he consistency and comple~eness of a~tribute set~ while at the same time 25 adding any needed default a~ributes~ .
The parser-based at~ribute analysis of the presen~
invention is a sisnificant improvement ov~r conven~ional implemen~ations in tha~ it is systematlc~ efficien~ and I~M DOCKET NO.: SA8-84-038 eliminates dealin~ with special cases. Further, since a parser genarator is used to ~enerate the syntax tables and parse driver, the technique Qf the present invention is far less error prone than conventional techniques requiring programmers ~o manually generate large amoun~s of object code. In addi~ion, the attribu~e processor will be easily modiiable for additional syntax rules or seman~ic checks.
BRIEF DESCRIPTIO~ OF THE DRAWINGS
The invention wlll be more clearly understood with reference to ~he ~ollowing description in conjunction with the accompanying drawings, wherein:
Figure 1 is a brief conceptual diagram o~ a compiler operation;
Fi~ures 2-11 are diagrams collec~ively r pres~nting a Group 1 grammar during derivation of a context-free PL/I
gra~mar for use in the present inven~ion;
Figures 12-21 are diagrams collectively represen~ing a Group 2 grammar during a subsequen~ stage in the derivation o~ a context-fr~e grammar for use in the present inYen~ion;
Figures 22-31 are diagrams collectively illustrating a Group 3 grammar duri~ a s~bsequent stage in the derivation of a context-free grammar for u~e in the present invention;
Figures 32-41 are diagrams collectivaly illustrating a Group 4 grammar during a subsP~uent stage in the deriv~tion of a context-free grammar for use in the.pre~en~ invention;
Figure 42 illustrates a construct used for avoiding null paths when ~ddin~ ~Iskip~ in the Group 3 gram~ar;
~' ~ ~;~5~0 I BM DOCKET NO .: SA8 - 8 4 - ~ 3 8 Figure 43 is a brief diagram illustrating the flow-chart-t~pe form characteristic of a typical computer program.
DETAILED DESCRIPTION OF THE INVENTION
The present invention is based on a recognition by th~
inventor that the attributes of PL/I declarations, as defined in ~he ANSI standard, have a syntac~ic s~ructure which can be expressed as a context-fre~ gramm~r. ~urther7 the syntax of this grammar can be modified to include ~he 1~ semantic checks requi'red for verifying the consistency of attributes 9 and the syntax can be structured in such a way that any missin~ attributes are automatically provided from a set of user-defined and/or system defaults. With such an attribute-checkin~ grammar, a parser generator can be used to cr~ate a table-driven attribute parser, i.e., a finite state machine (FSM) consisting of syntax tables and the parse driver, ~hich is capable of checking the consistency and completeness of attribute sets, while at the same time adding any needed default attributes~
After the ~able-driven attribute parser is genera~ed, all that is necessary is to provide a simple attribute scanner which can supply the at~ributa parser wi~h attri-butes from each initial attribute set in the proper order, i.e., the order in which attributes re defined in the attribute grammar, and the attribute parser can then be used to per~oxm the complete attribute analysis. If there is an inconsistency, the paxser will reject the stream. Each
IBM DOCKET No~: SA8-84-038 P.~RSER-BASED ATTRIBUTE ANALYSIS
sAcKGRouND OF THE INVENTION
, The present invention is directed to an improved attri-bu~e analyeis technique for use in compilers~ and more part~cularly to a parser-based attribute analysis technique S for usa in compilers. The invention will be described in ~ha context of PL/I programming, although it will be appre-ci~ted ~hat this is by way of example only and ~hat the lnvention is applicable to other programming languages as well.
1~ ~hen creatinq a program to run on a general purpose computer, the progr m is first written in source code, and ~he source code program is then passed through a compiler which converts the source code into object code, o~ machine languaqe. There are several tasks performed by the compiler lS be~ore inally generating an object code output, one of ~ha~e tasks being performed by an "attribute proce~sor" in th~ compiler front end. More particularly, when writing a P~II source pragram, various "attributes" are assigned to data items used in ,the program, e.g., an item can be ~ ~ssigned the attribute of a variable or a constant, and a varlable can be further assigned the attrLbutes of PARA-M~TER, CONTROLLED, or STATIC. The most common way of assigning the-~e attributes to data items is by way of a DECL~RE statement, although attributes can also be assigned by a number of other techniques, e.g., a data description statement, a LI~E statement, or even inferentially by using ~2~ t~
IBM Docket No.: SA9-84-038 a data item in a context which would require certain attributes. The ~asks performed by the attri~ute processor of a PL/I compiler during compi-lation of ~ PL/I source progriam include: the checking of syntax correctness of attribu~es specified for each declared data item in the defining PL/~ DECLAR~ sta~ement, and repre-senting this initial set of valid a~:~ributes in a concise internal notation and comple~ing the initi~l attribu~2 se~
of each ~ata item by providin~ ~he appropria~e default values for any missing attributes, whil~ checking the con-sistency and completeness o~ the final attribute set.
The attrLbute processor tasks must be performed in accordance with the syntax gri~mmar defined in the American National Standard Institute 1ANSI~ PLJI standard, particu-larly chapter 4 thereof. The test for attribu~e consistencyis set forth in 4.3.6.1 at pages 83-85. The purpose of the attribute consistency check is to ensure that no data item has been assi~ned inconsistent attribu~es, e.g., REAL and COMPLEX. The completion of the attribute set with default ~alue~ is described in ~4.3.6.3 through 4.3.6.5 at pages;~
86-88 of the ANSI P~/I standard. Thie checking to determine~
that the final attribute se~ is complete and consistent i5 described in 4.3.7.1 at page 93 o~ the ANSI PL~I standard.
The tasks performed by:the attribut~ processar o~ a PL/I
compiler front end are not limited to only those steps :~et for~h in the above-cited sections of the AN5I PL/I standard,~
, "r~
~ S2~
IBM DOCKET NO.: SA8-84-03B
but these sections exemplify the gr~matical rules which must be followed in the attribute processor.
In conventional PL/I compilexs, the attribute analysis is caxried ollt by various ad hoc techniques, whereas defaul~
values are added, and completeness and consis~ency of the final at~ribute se~s checked, by using specialized proce-dures, e.gO, breakin~ the program down into small parts and sub-parts and then checking from inside to ou~side. This requires the writing of a great deal of code, typically generated manually by ~he programmer. This is not only tLme consuming and quite ~edious, but the manual generation of the code is error-prone and substantial de~ugging of the compiler program is usually r~yuired~ Investigating and correc~ing errors in a very long listing is a time consuming process.
Further, even after a working compiler is obtained, it is sometimes necessary to update the compiler to accommodate new or changed language features. ~s in the original gene-ration of the compiler pro~ram, updating and dehugging extensive object code listings implementing complex gramma-tical rules is a task which reguires a sub~tantial amount of programmer time.
Fox the above reasons, the generation of the a~ribu~e processor por~ion of the compiler program by conventional techniques is error prone, inef~icien~ a~d difficul~ to adapt to new or chan~ed language features/ and it would be highly advanta~eous to provide a simplified and much ~aster ~ 25~
IBM DOCKET NO.: SA8-84-038 technique for implementing th attribute processor portion of a compiler.
S~AIIV ~r r~ ~ rlc l It is, therefore, an object of this invention ~o pro-vid~ a simplified technique for implementing an att~ibu~eproces~or.
Briefly, this and other objects of the invention ar~
realized by implementing the attribute processor as a table-drive~ attribute parser, and the invention comprises both a table-driven parser ih th~ attribu~e processor as well as a simple and efficient technique for genexating the syntax tables and parse d river which make up the attri~ute parser.
In the parser-based a~tribute analysis of the present inven-tion, the consistency of attributes of PL/I declarations is checked, and default values for missing attribu~es are added, based on th~ efficient and reliable LALR~l) parsing technique. The attribu~e analysis grammar speclfied in the ANSI standard is ~xpressed in terms of a context-free grammar which is provided as an input to a LALR(l) grammar processor. In a well-known manner, ~he gra~mar proces~or generat2s a finite sta~e mac~ine ~SM) consisting of syntax tables and a parse driver. These are included i.n ~he compiler fron~ end and can be used ~o check ~he consistency and comple~eness of a~tribute set~ while at the same time 25 adding any needed default a~ributes~ .
The parser-based at~ribute analysis of the presen~
invention is a sisnificant improvement ov~r conven~ional implemen~ations in tha~ it is systematlc~ efficien~ and I~M DOCKET NO.: SA8-84-038 eliminates dealin~ with special cases. Further, since a parser genarator is used to ~enerate the syntax tables and parse driver, the technique Qf the present invention is far less error prone than conventional techniques requiring programmers ~o manually generate large amoun~s of object code. In addi~ion, the attribu~e processor will be easily modiiable for additional syntax rules or seman~ic checks.
BRIEF DESCRIPTIO~ OF THE DRAWINGS
The invention wlll be more clearly understood with reference to ~he ~ollowing description in conjunction with the accompanying drawings, wherein:
Figure 1 is a brief conceptual diagram o~ a compiler operation;
Fi~ures 2-11 are diagrams collec~ively r pres~nting a Group 1 grammar during derivation of a context-free PL/I
gra~mar for use in the present inven~ion;
Figures 12-21 are diagrams collectively represen~ing a Group 2 grammar during a subsequen~ stage in the derivation o~ a context-fr~e grammar for use in the present inYen~ion;
Figures 22-31 are diagrams collectively illustrating a Group 3 grammar duri~ a s~bsequent stage in the derivation of a context-free grammar for u~e in the present invention;
Figures 32-41 are diagrams collectivaly illustrating a Group 4 grammar during a subsP~uent stage in the deriv~tion of a context-free grammar for use in the.pre~en~ invention;
Figure 42 illustrates a construct used for avoiding null paths when ~ddin~ ~Iskip~ in the Group 3 gram~ar;
~' ~ ~;~5~0 I BM DOCKET NO .: SA8 - 8 4 - ~ 3 8 Figure 43 is a brief diagram illustrating the flow-chart-t~pe form characteristic of a typical computer program.
DETAILED DESCRIPTION OF THE INVENTION
The present invention is based on a recognition by th~
inventor that the attributes of PL/I declarations, as defined in ~he ANSI standard, have a syntac~ic s~ructure which can be expressed as a context-fre~ gramm~r. ~urther7 the syntax of this grammar can be modified to include ~he 1~ semantic checks requi'red for verifying the consistency of attributes 9 and the syntax can be structured in such a way that any missin~ attributes are automatically provided from a set of user-defined and/or system defaults. With such an attribute-checkin~ grammar, a parser generator can be used to cr~ate a table-driven attribute parser, i.e., a finite state machine (FSM) consisting of syntax tables and the parse driver, ~hich is capable of checking the consistency and completeness of attribute sets, while at the same time adding any needed default attributes~
After the ~able-driven attribute parser is genera~ed, all that is necessary is to provide a simple attribute scanner which can supply the at~ributa parser wi~h attri-butes from each initial attribute set in the proper order, i.e., the order in which attributes re defined in the attribute grammar, and the attribute parser can then be used to per~oxm the complete attribute analysis. If there is an inconsistency, the paxser will reject the stream. Each
2~2~
I B~ DOCKET NO.: SA8~84-038 system default a~ribute addition ls associated wlth a specific reduction detected by the parser.
The first step in the creation of the attribute parser of the present invention is to derive an attribute checking grammar which can be used by the parser generator. The parser generator is a grammar processor of a type known in the art, e.g., the LALR ~1~ grammar processor available under the name Translator Writing 5ystem from Metaware o~
Santa Cruz, California. As is well-known, this grammar processor, when provided with a suitable cQntext-frse grammar definition, will generate a table-driven parser which can parse statements made in that grammar.
For a PL/I compiLer, it is nece~sary to derive a context-free grammar based on the grammar defined in th~
~NSI PL/~ standard at 4.3.6.1, with modifications re~uired for the consis~ency checking and application of a~tribute defaults, and aLso for different language levels, e.g., including desired extensions to the "standard" PL/I. In deriving the context-free grammar, a four-step approach was followed. As ~entioned, the ANSI s~andard PL/I provides a syn ax or determining when a set of da~a attributes is consistentO The consistency check involves s~eps 4, 5 and 6 o~ 4.3.6.1 of the Standard entitled l'test-attribu~e-consistency", with steps 4 and 5 being semantic checks and step 6 referring to ~h~ syntax. Thus, for the purposes of m~rely ~esting the consistency of a given se~ of a~ributes, the ANSX syn~ax is not completeO However, by modifying this 5~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 syntax, the ANSI semantic consistency checks can be included in the syntax, and the syntax can be made LALR(1).
The ANSI attribute syntax is first represented in a Group 1 grammar, and is then reordere~ in a Group 2 grammar.
5 The Group 2 syntax incorporates syntactically th~ semantic restriction in step 5 of 4.3.6.1 of the standard.
The complete syntax ~or consistency checking must allow all subsets of legal sentences from ~he Group 2 syntax.
This is accomplished in a Group 3 syntax which provides op~ional skipping over each keyword while still preventing multiple null paths. Then, if a "lexical scanner" is used that produces encoded data attributes in a canonical order, a standard LALR(l) parser may be used to check consistency.
When there are user supplied defaults, this consistency checker may be used in ~he standard ANSI algorithm for pro-cessing the user defaults.
~ y properly structuring the productions in this syntax, the reduction of null rules, or of rules with missin~ compo-nents, can be m~de to correspond exactly to the reguir~d addition of system default attributes. The syntax diagrams for ~his are shown in a Group 4 grammar, and include references to the system de~ault statements. Thus, a set of attributes can have i~s consistency checked and syst~m default attributes added in essentially one step. This is 25 done without multiple checks for attxibute combinations. A
final "Group 5" version is ob~ained by cambinin~ ~roups 3 and 4 using a Backus-Naur Form (BNF). Th~s is the fi~al syn~ax used in the parser genPrator.
` ~ ~25~S~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 The syntax has 155 rules which, when processed by the YACC parser generator on the UNIX system, produce~ 168 s~ates and 306 table entries~ 240 entries being non-zero.
When th~re are user-supplied defaults followed by system defaults, i.e., when "NONE'I is not used, the user default attributes can be added using the above consistency checker~ and afterwards, the process of this invention ~or adding system deaults can be used to complete and do final checks on the resulting at~ributes. This avoids the inefficiency of processing s~stem defaults as if they were user defaults.
In addition to the standard consistency checXs, there are some "final checks" in "check-attribute-completenes~-and-delete-attributes", in 4.3.7.1 of the ANSI PL/I
standard, that are done after all defaul~ attributes have been added. Some of ~he~e final checks are included in the final syntax, while the remainder of the ch~cks can be more easily added not syntactially, bu~ as semantic actions.
The complete derivation o~ ~he contex~-~ree grammar for 2 0 u~e in the LALR(1) parser generator will now be described in de~ail.
- In ~he first step, a context-free "Group 1" grammar, coLlectively illustra~ed in Figures 2~ is created from the original ~NSI grammar specified in 4.3.6.1 of the PL/I
2~ standard, with the addition o~ value attributes: ~Ipll with POSITION; SIZE with <string>~ PRECISIO~ and AREA; and SC.~LE
wi~h PRECISION~ In addition, a G~AP~IC attribut~ has been ~ ~ ~25~2~
I BM DOCKET No.: SA8-84-038 added as an extension of the original ANSI grammar. This grammar could be used only for checking that a given attri-bute set was consistent. Also, the Group 1 grammar is not suitable for input to a LALR(l) type parser generator.
It should be noted that the at~ribute consistency check, speoified in 4.3.6.1 of the ANSI PL/I standard, checks fcr consistency by verifying that each attribute is contained in a subse~ of the attributes frorn some path of ~he syntax tree having a root-node at either the <consistent-description>, <consistent-literal-constant~, or <consistent-declara~ian>. Figures 2 and 5-11 illus~rate ~he syntax tree for the <consistent-declaration> roo~-node, Figures 3 and 7 illustrate the syntax tree for the <consistent-literal-constant> root-node, and Figures 4 and 6-9 illustrate the syn~ax tree for the ~consisten~-description~ root-node.
In the second stage of grammar derivation, a "Group 2"
grammar is crea~ed. The Group 2 grammar is collectivel~
illustrated in Figures 12-21, which correspond to Fiqures 2-~0 11, respectively. The Group 2 grammar is obtained byreordering the &roup 1 grammar in such a way that it incorporates some o~ the ANSI-specified semantic restric-tions on the use of attributes. As can be seen from a review o~ 4.3.6.1 of the ANSI PL/I standard, i~ is possible ~5 for a <data-descrip~ion> ~o be asslgned the S~RUCTU~E attri-bu~e while simultaneously being defined as <alignment> da~a.
On the o~her hand, <alignment~ data can ~e assign~d the ALI&NE~ or UNALI~NED a~tributes. When diagrammin~ the ~roup ~ 2S2~
I BM DOCKET NO .: SA8 - B 4 - 0 3 8 1 grammar, therefore, the STRUCTURE and calignmen~> attri-butes would appear in the same path. However, Step S of 4.306.1 of the PL/I standard imposes a semantic restriction that the STRUCTURE attribute and <alignment> attributes s cannot be specified together. This semantic restriction is easily incorporated into the Group 2 grammar by moving the <alignment> attributes onto a d~fferent path from the path which included the S~RUCTURE a~tribute, i~e., by moving the <alignment> attribute of F.igure 6 to the left-hand branch in ~igure 16.
Next, note that DIMENSION is inconsistently ordered, in that it occurs before ENTRY, FILE, FORMAT and L~EL in <variable> (see Figures 5, ~ and 9~, while it occurs after those same attributes in <constant> (see Figure 10). Thus, DIMENSION can be factor~d I5 out o~ <data~description> ~Figure 6) and <constant~ ~Figure 10).
and ins~ead added to Figure 12. ~lso, MEMBER is inconsistently ordered in that it occurs hefore <data-description> in the <variable> attributes, whereas i~ occurs after cdata-description>
in <consisten~-description> ~Figure 4). Thus, M~$BER can be moved in Fi~ure 4 as shown in Figure 14.
There are further lookahead problems created when optional "skips" will be added in subsequent derivation steps to be described below. First, INITIAL only goes wi~h <variable>, so that INITI~L mus~ ~e moved ahead in Figure 6 to resolve <variable> ~rom <constant> in case CONST~NT is left outO This is shown in Figure 16~ ~lso, for ~NTRY as a variable, a similar si~uation exlsts with ~ETU~NS and ~ ~ ~2S~
IBM DOCKET NO .: SA8 - 8 4 - 0 3 8 OPTIONS, so that these should be moved before ENTRY in the <access-type> diagram of Figure 9. This is shown in Figure 19. Still urther, the key words in <file set> (Figure 11) distinguish a constant FILE, so that they should come before , FILE i~ the <constant> diagram of Figure 10~ This is shown in Figure 20.
Finally, th~ INTERNAL or EXTFRNAL specification is moved to assist in dealing with semantic restrictions, described be~ow, involving EXTERNAL, GENERIC or MEMBER.
The Group 2 grammar would be effective for checking that a given attribute set is consistent and complies with all incorporated semantic restrictions.
The next stage in the grammar derivation is a Group 3 ~rammar obtained by extending the Group 2 grammar in such a way that it would allow all subsets of attributes ~ha~ could ~e used with each attribute, More particularly, while the Group 2 grammar would resul~ in a list of all attributes which could result by following any particular "path"
through the Group 2 grammar, not all at~ributes in any given attribu~e set need be specified. The Group 3 grammar incor-porates 'Iskips" needed for bypassing certain optional attri-butes in order ~o accommodata the allowable attribu~e subsets. Some additional semantic restrictions omitted from the Group 2 grammar are also incorporated into the Group 3 grammar.
The Group 3 grammar is collectiv~ly illustrated in Figures 22-31, which correspond to Figures 12-~1, respec-t~vely. The syntax illustrated in Figures 22~31 is ~ 52~
IBM DOCKET NO .: SA8 - 8 4 - 0 3 8 hasically the same as the Group 2 grammar of Figures 12-21 except that it allows any subset of key words allowed by the Group 2 grammar. To prevent ambigui~y, some factoring and other modifications are required. Some choices for unique-ness are done to facilitate the DEFAULT processing to beadded in a subsequent grammar derivation step. For instance, ENTRY or FILE occurring alone (i.e., without IN~TIAL, ~LIGNED, OR UNALIGNED) belong only in the syntax for <constant> ~Figures 20 and 30), while FORMAT or LABEL
occurring alone (i.e., without CONSTANT) belong only in <access-type> (Fi~ure 19~ within ~variable~ (Figures 15-19).
Thus, EN~RY and FILE occurring alone are removed from <access-type> within ~da~a-description> (Figures 16~19) leaving <description> (Figure~ 26-29). Further comparison 15 o~ the cdata~description> and <descrip~ion> diasxams shows that the alignment attrib~te for stxings and pictures is separated out for special treatmen~, to be described later, and the ENTRY and FILE attri~ute~ are treated specially so that ~hey cannot occur alone ~otherwise they would belong in 20 the <constant> diagram).
In addition, paths are modi~ied in accordance with Step ~ 4 af ~4, 3, 6.1 of the ANSI PL~I standard, which dictates tha~
EXTERNAL never occurs with GENERIC or MEM~ER.
The rest of the changes involve the provision of 25 unambiguous s~ips. More particularly, one way to allow subsets of attributes would ~e to provide an optional skip over each keyword. ~owever, this would result in an s~
IBM DOCKET NO.: SA8-84-038 am~iguity which would be incompatibl~ with the LALR(l) grammar processor, due tu the occurrence of multiple null paths~ In order to avoid this problem, the construct as shown in Flgure 42 is used. The arrangement shown in Figure 42 will allow either x or y, or both, but does not allow the null path. Note~ for instance, that the ~file set> diagram of Figure 31 includes no null pa~hs, yet the allowable paths include any ordered subse~ of keywords in <file set>.
The Group 3 grammar of Figures 22-31 could be used for checking tha~ a gi~en attribu~e set is consistent, eYen though some of it~ attributes might be missing, an~ for checking th~ the set complies with all ANSI semantic restrictions.
The next step in the grammar derivation is to deriv~ a Group 4 grammar by au~menting the grammar syntax tree of Group 3 with references to sets of defauLt attributes, and by including some of the final complet~ness checks ~ound in 4.3.7.1 of the ANSI PL/I ~tandard. In Figures 32-41, which correspond respectively to Figure~ 22-31, the numbers in the 20 skip paths refer to the system default statements set forth in the SYSTEM DEFAULT ~ABLE below, in whi~h the "¦" sym~ol represen~s a logical O~ and the " " symbol indicates NOT.
SYSTEiM DEFAULT TABLE
1~ DEFAULT ( RETUR~S
ENTRY;
2 . DEFAULT ( DIRECT ¦ INPUT ¦ KEXED ¦ OUTPUT ¦ PRINT
FILE; RECORD t SEQIIENTIAL ¦ STREAM ~ UPDATE ) 5~
IBM DOCKET NO.: SA8-84-038
I B~ DOCKET NO.: SA8~84-038 system default a~ribute addition ls associated wlth a specific reduction detected by the parser.
The first step in the creation of the attribute parser of the present invention is to derive an attribute checking grammar which can be used by the parser generator. The parser generator is a grammar processor of a type known in the art, e.g., the LALR ~1~ grammar processor available under the name Translator Writing 5ystem from Metaware o~
Santa Cruz, California. As is well-known, this grammar processor, when provided with a suitable cQntext-frse grammar definition, will generate a table-driven parser which can parse statements made in that grammar.
For a PL/I compiLer, it is nece~sary to derive a context-free grammar based on the grammar defined in th~
~NSI PL/~ standard at 4.3.6.1, with modifications re~uired for the consis~ency checking and application of a~tribute defaults, and aLso for different language levels, e.g., including desired extensions to the "standard" PL/I. In deriving the context-free grammar, a four-step approach was followed. As ~entioned, the ANSI s~andard PL/I provides a syn ax or determining when a set of da~a attributes is consistentO The consistency check involves s~eps 4, 5 and 6 o~ 4.3.6.1 of the Standard entitled l'test-attribu~e-consistency", with steps 4 and 5 being semantic checks and step 6 referring to ~h~ syntax. Thus, for the purposes of m~rely ~esting the consistency of a given se~ of a~ributes, the ANSX syn~ax is not completeO However, by modifying this 5~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 syntax, the ANSI semantic consistency checks can be included in the syntax, and the syntax can be made LALR(1).
The ANSI attribute syntax is first represented in a Group 1 grammar, and is then reordere~ in a Group 2 grammar.
5 The Group 2 syntax incorporates syntactically th~ semantic restriction in step 5 of 4.3.6.1 of the standard.
The complete syntax ~or consistency checking must allow all subsets of legal sentences from ~he Group 2 syntax.
This is accomplished in a Group 3 syntax which provides op~ional skipping over each keyword while still preventing multiple null paths. Then, if a "lexical scanner" is used that produces encoded data attributes in a canonical order, a standard LALR(l) parser may be used to check consistency.
When there are user supplied defaults, this consistency checker may be used in ~he standard ANSI algorithm for pro-cessing the user defaults.
~ y properly structuring the productions in this syntax, the reduction of null rules, or of rules with missin~ compo-nents, can be m~de to correspond exactly to the reguir~d addition of system default attributes. The syntax diagrams for ~his are shown in a Group 4 grammar, and include references to the system de~ault statements. Thus, a set of attributes can have i~s consistency checked and syst~m default attributes added in essentially one step. This is 25 done without multiple checks for attxibute combinations. A
final "Group 5" version is ob~ained by cambinin~ ~roups 3 and 4 using a Backus-Naur Form (BNF). Th~s is the fi~al syn~ax used in the parser genPrator.
` ~ ~25~S~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 The syntax has 155 rules which, when processed by the YACC parser generator on the UNIX system, produce~ 168 s~ates and 306 table entries~ 240 entries being non-zero.
When th~re are user-supplied defaults followed by system defaults, i.e., when "NONE'I is not used, the user default attributes can be added using the above consistency checker~ and afterwards, the process of this invention ~or adding system deaults can be used to complete and do final checks on the resulting at~ributes. This avoids the inefficiency of processing s~stem defaults as if they were user defaults.
In addition to the standard consistency checXs, there are some "final checks" in "check-attribute-completenes~-and-delete-attributes", in 4.3.7.1 of the ANSI PL/I
standard, that are done after all defaul~ attributes have been added. Some of ~he~e final checks are included in the final syntax, while the remainder of the ch~cks can be more easily added not syntactially, bu~ as semantic actions.
The complete derivation o~ ~he contex~-~ree grammar for 2 0 u~e in the LALR(1) parser generator will now be described in de~ail.
- In ~he first step, a context-free "Group 1" grammar, coLlectively illustra~ed in Figures 2~ is created from the original ~NSI grammar specified in 4.3.6.1 of the PL/I
2~ standard, with the addition o~ value attributes: ~Ipll with POSITION; SIZE with <string>~ PRECISIO~ and AREA; and SC.~LE
wi~h PRECISION~ In addition, a G~AP~IC attribut~ has been ~ ~ ~25~2~
I BM DOCKET No.: SA8-84-038 added as an extension of the original ANSI grammar. This grammar could be used only for checking that a given attri-bute set was consistent. Also, the Group 1 grammar is not suitable for input to a LALR(l) type parser generator.
It should be noted that the at~ribute consistency check, speoified in 4.3.6.1 of the ANSI PL/I standard, checks fcr consistency by verifying that each attribute is contained in a subse~ of the attributes frorn some path of ~he syntax tree having a root-node at either the <consistent-description>, <consistent-literal-constant~, or <consistent-declara~ian>. Figures 2 and 5-11 illus~rate ~he syntax tree for the <consistent-declaration> roo~-node, Figures 3 and 7 illustrate the syntax tree for the <consistent-literal-constant> root-node, and Figures 4 and 6-9 illustrate the syn~ax tree for the ~consisten~-description~ root-node.
In the second stage of grammar derivation, a "Group 2"
grammar is crea~ed. The Group 2 grammar is collectivel~
illustrated in Figures 12-21, which correspond to Fiqures 2-~0 11, respectively. The Group 2 grammar is obtained byreordering the &roup 1 grammar in such a way that it incorporates some o~ the ANSI-specified semantic restric-tions on the use of attributes. As can be seen from a review o~ 4.3.6.1 of the ANSI PL/I standard, i~ is possible ~5 for a <data-descrip~ion> ~o be asslgned the S~RUCTU~E attri-bu~e while simultaneously being defined as <alignment> da~a.
On the o~her hand, <alignment~ data can ~e assign~d the ALI&NE~ or UNALI~NED a~tributes. When diagrammin~ the ~roup ~ 2S2~
I BM DOCKET NO .: SA8 - B 4 - 0 3 8 1 grammar, therefore, the STRUCTURE and calignmen~> attri-butes would appear in the same path. However, Step S of 4.306.1 of the PL/I standard imposes a semantic restriction that the STRUCTURE attribute and <alignment> attributes s cannot be specified together. This semantic restriction is easily incorporated into the Group 2 grammar by moving the <alignment> attributes onto a d~fferent path from the path which included the S~RUCTURE a~tribute, i~e., by moving the <alignment> attribute of F.igure 6 to the left-hand branch in ~igure 16.
Next, note that DIMENSION is inconsistently ordered, in that it occurs before ENTRY, FILE, FORMAT and L~EL in <variable> (see Figures 5, ~ and 9~, while it occurs after those same attributes in <constant> (see Figure 10). Thus, DIMENSION can be factor~d I5 out o~ <data~description> ~Figure 6) and <constant~ ~Figure 10).
and ins~ead added to Figure 12. ~lso, MEMBER is inconsistently ordered in that it occurs hefore <data-description> in the <variable> attributes, whereas i~ occurs after cdata-description>
in <consisten~-description> ~Figure 4). Thus, M~$BER can be moved in Fi~ure 4 as shown in Figure 14.
There are further lookahead problems created when optional "skips" will be added in subsequent derivation steps to be described below. First, INITIAL only goes wi~h <variable>, so that INITI~L mus~ ~e moved ahead in Figure 6 to resolve <variable> ~rom <constant> in case CONST~NT is left outO This is shown in Figure 16~ ~lso, for ~NTRY as a variable, a similar si~uation exlsts with ~ETU~NS and ~ ~ ~2S~
IBM DOCKET NO .: SA8 - 8 4 - 0 3 8 OPTIONS, so that these should be moved before ENTRY in the <access-type> diagram of Figure 9. This is shown in Figure 19. Still urther, the key words in <file set> (Figure 11) distinguish a constant FILE, so that they should come before , FILE i~ the <constant> diagram of Figure 10~ This is shown in Figure 20.
Finally, th~ INTERNAL or EXTFRNAL specification is moved to assist in dealing with semantic restrictions, described be~ow, involving EXTERNAL, GENERIC or MEMBER.
The Group 2 grammar would be effective for checking that a given attribute set is consistent and complies with all incorporated semantic restrictions.
The next stage in the grammar derivation is a Group 3 ~rammar obtained by extending the Group 2 grammar in such a way that it would allow all subsets of attributes ~ha~ could ~e used with each attribute, More particularly, while the Group 2 grammar would resul~ in a list of all attributes which could result by following any particular "path"
through the Group 2 grammar, not all at~ributes in any given attribu~e set need be specified. The Group 3 grammar incor-porates 'Iskips" needed for bypassing certain optional attri-butes in order ~o accommodata the allowable attribu~e subsets. Some additional semantic restrictions omitted from the Group 2 grammar are also incorporated into the Group 3 grammar.
The Group 3 grammar is collectiv~ly illustrated in Figures 22-31, which correspond to Figures 12-~1, respec-t~vely. The syntax illustrated in Figures 22~31 is ~ 52~
IBM DOCKET NO .: SA8 - 8 4 - 0 3 8 hasically the same as the Group 2 grammar of Figures 12-21 except that it allows any subset of key words allowed by the Group 2 grammar. To prevent ambigui~y, some factoring and other modifications are required. Some choices for unique-ness are done to facilitate the DEFAULT processing to beadded in a subsequent grammar derivation step. For instance, ENTRY or FILE occurring alone (i.e., without IN~TIAL, ~LIGNED, OR UNALIGNED) belong only in the syntax for <constant> ~Figures 20 and 30), while FORMAT or LABEL
occurring alone (i.e., without CONSTANT) belong only in <access-type> (Fi~ure 19~ within ~variable~ (Figures 15-19).
Thus, EN~RY and FILE occurring alone are removed from <access-type> within ~da~a-description> (Figures 16~19) leaving <description> (Figure~ 26-29). Further comparison 15 o~ the cdata~description> and <descrip~ion> diasxams shows that the alignment attrib~te for stxings and pictures is separated out for special treatmen~, to be described later, and the ENTRY and FILE attri~ute~ are treated specially so that ~hey cannot occur alone ~otherwise they would belong in 20 the <constant> diagram).
In addition, paths are modi~ied in accordance with Step ~ 4 af ~4, 3, 6.1 of the ANSI PL~I standard, which dictates tha~
EXTERNAL never occurs with GENERIC or MEM~ER.
The rest of the changes involve the provision of 25 unambiguous s~ips. More particularly, one way to allow subsets of attributes would ~e to provide an optional skip over each keyword. ~owever, this would result in an s~
IBM DOCKET NO.: SA8-84-038 am~iguity which would be incompatibl~ with the LALR(l) grammar processor, due tu the occurrence of multiple null paths~ In order to avoid this problem, the construct as shown in Flgure 42 is used. The arrangement shown in Figure 42 will allow either x or y, or both, but does not allow the null path. Note~ for instance, that the ~file set> diagram of Figure 31 includes no null pa~hs, yet the allowable paths include any ordered subse~ of keywords in <file set>.
The Group 3 grammar of Figures 22-31 could be used for checking tha~ a gi~en attribu~e set is consistent, eYen though some of it~ attributes might be missing, an~ for checking th~ the set complies with all ANSI semantic restrictions.
The next step in the grammar derivation is to deriv~ a Group 4 grammar by au~menting the grammar syntax tree of Group 3 with references to sets of defauLt attributes, and by including some of the final complet~ness checks ~ound in 4.3.7.1 of the ANSI PL/I ~tandard. In Figures 32-41, which correspond respectively to Figure~ 22-31, the numbers in the 20 skip paths refer to the system default statements set forth in the SYSTEM DEFAULT ~ABLE below, in whi~h the "¦" sym~ol represen~s a logical O~ and the " " symbol indicates NOT.
SYSTEiM DEFAULT TABLE
1~ DEFAULT ( RETUR~S
ENTRY;
2 . DEFAULT ( DIRECT ¦ INPUT ¦ KEXED ¦ OUTPUT ¦ PRINT
FILE; RECORD t SEQIIENTIAL ¦ STREAM ~ UPDATE ) 5~
IBM DOCKET NO.: SA8-84-038
3. DEFAULT ( CONSTANT ~ PICTURE) FIXED, BINARY, REAL;
4. DEFAULT (FIXED & BINARY & CONSTANT~ PREcIsIoNtdl~o);
5. DEFAULT (FIXED & DECIMAL & CONSTANT) PRECISION(d2,0);
6. DEFAULT (FLOAT & ~INARY & CONSTANT) PRECISIONtd3);
7. DEFAULT (F~OAT & DECIMAL & CONSTANT3 PRECISXON(d4);
8. DEFAULT (C~ARACTER L BIT ¦ GRAPHIC) CHARACTER(1), BIT(1), GRAPHIC~l), NONVARYING;
9o DEFAULT (~REA~
AREA(d5);
10. DEFAULT (POSITION) POSITION~l);
11. DEFAULT ( (ENTRY ¦ FILE) &
(AUTOMATIC J BASED ¦ DEFINED J PAR~METER ¦
STATIC ¦ CONTROLLED.J MEMBER ¦ ALIGN D ¦
UNALIGNED ¦ INITIAL) ) VARIABLE;
12. DEFAULT ~ tENTR~ ¦ FILE) & RANGE(*) ) CONSTANT;
13. DEFAULT (RAN~E(*) & CONSTANT) VARIABLE;
14. DEEAULT ( CONDITION ¦ ( (FILE ¦ ENTRY~ & CONSTANT ) EXTERNAL;
15. DEFAULT (R~NGE~*)) INTERNAL;
5 16. DEFAULT (VARIABLE & EXTERNAL) STATIC;
17. DEFAULT ~VARIABLE) AUTOMATIC;
~3 'f~ 2S~
IBM DOCKET NO .: SA8 ~ 8 4 - 0 3 8 18. DEFAULT ( (CHARACTER ¦ BIT ¦ PICTURE) & CONSTANT) UNALIGNED;
19 . DEFAULT ~ CONSTANT ) ALI GNED
Note that in the <arit~netic> diagram of Figure 37, each of the double skips around SIZE uses the given default, since if only P~ECISION is present, the "test-invalid-duplicates" procedure described in 4.3O6.2 o~ the ANSI PL/I
standard allows the addition o~ either SIZE or SIZE and SCALE.
Aside from adding the de~ault table numbers and separating the skip path on the left side of <variable> in Figure 35, the only changes from the Group 3 grammar of Figures 22-31 are the modification of some paths to exclude some cases made illegal hy the "final" checks found in ~4.3.7~1 of the Standard. For example, step 3 of ~4.3.7.1 states that EXTERNAL canno~ occur with AUTOMArIC, B~SED, DEFINED, PARAMETER or BUILTIN, and step 4 of 4.3.7.1 states the combination o~ EXTE~NAL and CON~TANT cannot occur ~o~ether with either FORMAT or LABEL. ~lso, some illegal sLngle attributes are removed, i.e., POSITION, CONSTANT, ENVIRONMENT, VARYING, ~NONV M YING, LOCAL, and OPTIONS.
Not~ tha~ in the syntax fox 1'cconstan~~1iteral-constant>" in Figure 33, <arithmetic> has now been changed . 25 to <crea~ed-const>~ This <created-const> identifies a literal, which has different defaulting rules ~han ~hose for variables. The adding of default attribute~ for literals could be performed in a conv~ntional manner, or th~ attri-I BM DOCKET NO.: SA8-84-038 bute defaulting syntax for literals could be derived from "cxeate-constant" in 4.4.7 of the ~NSI PL/I standard. The dexivation would be performed in a manner similar t~ that described above for variables, and need not be described in detail here for a complete understanding of the invention.
The next step is to combine the Group 4 grammar with the Group 3 grammar to produce a "Group 5" grammar to be used in the attribute pxocessor for consistency checking, applying of defaults, and verifying that the final attribute lQ set is complete. ~his Group 5 grammar is illustxated in the BNF grammax to be described below.
The Group 5 grammar consists of the basic rules ~or checking consistency and completeness, plus the subset rules and the default rules. By using some sta~e variables, the 15 subset rules or the default rules, or both, may be effec-tively deactiva~ed. Consequently, the one grammar can be used for the multiple purposes of:
~asic: consistency and completeness check;
Basic and subxet: consistency check only;
Basic and default: consistency and completeness wi~h de~ault attribute addition all at once;
Basic, subset and default: consistency with default attribute addition.
Having de~ine~ the PL/I a~tribu~e syntax structure in 25 ~erms of a context~fr2e grammar~ the grammar speciica~ions can now be conver~ed to a Backus-Naur Form (BNF) grammar in a well-known manner and provided ~o a grammar processor, 252~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 B
-preferably a LALR~1) processox, which will generate the syntax tables and parse driver which make up the table-driven attribute parser according ~o this invention. In the operation o~ the LALR(1) processor, a "lexical scanner" will be used to supply encoded attributes to the grammar pro-cessor as tokens in a canonical order, i.e., an order con-sistent with the syntax. For example, the token ordering may be as given below in the TOKEN ORDER TABLE.
TOKEN OR~
1o ~token VARIABL~ CONSTANT
%token INTERNAL EXTERNAL
%token AUTOMATIC BASED CONTROLLED STATIC MEMBER
PARAMETER DEFINED
%token POSITION
~token P
%token INITIAL
%tok~n ALIGNED UNALIGNED
~token ENVIRONMENT
~token STREAM RECORD
0 %token INPUT OUTPUT UPDATE
%token PRINT
~stoken SEQUENTIAI; DIRECT
%token KEYED
~token RETURNS
%token OPTIONS
%token BUILTIN CONDITION GENE:RIC
~token STRUCTURE
%t.oken ENTRY FILE FORMAT LA~EL
%token AREA POINTER OFFSET
%token CHARACTER BIT GRAPHIC PIC~URE
%toXen REAL COMPLEX
%token DECIMAL BIN~RY, %token FIXED FLOAT
%token PREC~SION
~S %tok~n SIZE
%token SCALE
%token VARYING NONVARYING
%token LOCAL
%token DIMENSION
%token END
With this ordering the B~F grammar may be defined as ~llows.
:~Z5~
IBM DOCKET NO .: SA8 - 8 4 - 0 3 8 Note that the non-terminals are all lower case,- and associated actions are in { }. For example, the notation { add: cattribute>
indicates the system default addition of the particuLar S attribute mentioned to the exi~ting set of attributes. The "su~set" rule~ are within comment delimiters, i.e., /*...*/
and the "default rules" are those with an associated "add"
action. The subset production rules in comments are legal for checking consis~ency, but not for checking a final set of attrihutes when there are no more user dafault attributes to be added.
BNF GRAMMAR
pgm : decl.list decl.list : decl.
¦ decl.llst decl decl : decl.type END
{ declaration valid 20 op~.in~.e~rn : INTERNAL
¦ EXTEF~TAL
, r add:internal }
2 5 op~ . ext . intrn : INTERNAI
IEXTERNAL
{ add: external 52~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 opt . intrn : XNTERNAL
{ add: internal ;
decl. type : variahle DIMENSION
variable ¦ const DIMENS~ON
I const I opt.ext.intrn CONDITION
L o l opt.intrn BUILTIN
opt.i~trn GENERIC
;
variable : explicit.var I implicit.var 15 '{ add: variable }
explicit.var : VARIABLE storage data,desc ¦ VARIA~LE data.des~.
{ add: internal }add: auto J VARIA~LE INTERNAL data.desc.
~5 {}add: auto ¦ VARIABLE EXTERNAL data.desc { add: static I V~RI~BLE storage ENTRY
¦ VARIABLE storage FILE
implicit.var : storage data.desc I data.desc.
{ add: internal .}add: autO
¦ INT~RN~L da~a.desc { add: auto I EXTERNAL data.d~sc { add: static I storage ENTRY
I storage FILE
s~
I ~M DOCKET NO.: SA~-84-038 init . or . al ign : INITIAL align ¦ INITIAL
~ add: aligned 1align , ;
storage opt.int.extrn CONTROLLED
¦ opt.int.extrn STATIC
~3 opt~intrn stor.type stor.type : AUTOMATIC
¦ BASED
¦ DEFINED POSITIO~ opt.pos.value ¦ DEFINED
1~ /* I ~ POSITION */
I PARAMETER
¦ MEMBER
;
opt.pos.value : P
{ add: position-value= 1 }
data.desc :STRUCTURE
¦ INITIAL data.type data.type init.or.align ENTRY
I init . or . align FILE
opt.align :ALIGNED
~ UNALIGNED
{ add: aligned , }
align : ALIGNED
¦ UNALIGMED
;
~r~ 0 IBM DOCKET No.: SA8-84-038 data.t~e : opt.align arith op~.align noncomp string { add: unaligned }
align string pictured ~ add: unaligned 1 align pictured noncomp : MEA
t add: size = 1 ¦ AREA SIZE
J RETURNS OPTIONS ENTRY
¦ RETURNS OPTIONS
{ add: entry }
~ RETURNS ENTRY
IOPTIONS ENTRY
¦ RETURNS
{ add: entry ¦ FORMAT LOCAL
I FORMAT
I LABEL LOCAL
I LABEL
/* ¦ LOCAL */
¦ POI~TER
I OFFSET
arith : mode arith.rest ;
35 mode : REAL
I COMPLEX
{ add: real }
40 arith.rest : opt.bin op~.~ixed precision.4 I DECI~LUL op~.fixed precision.5 I opt.bin FLOAT precision.6 I DECIMAL FLOAT precision,7 ~ 2~9U~
IBM DOCKET NO.: SA8-84-038 ~3 opt.bin : I BINARY
{ add: ~inary ;
5 opt.fixed :~ FIXED
I
{ add~ fixed ;
precision.4 : PRECI5ION SIZE SCALE
¦ PRECISION SIZE
¦ PRECISION
. { add: size- dl, scale= O
{ add: precision, size= dl, . scale= 0 }
precision.5 PRECISION SIZE SCALE
¦ PRECISION SIZE
¦ PRECISION
{ add: size= d2, scale= 0 }
{ add: precision, size= d2, scale= O
}
precision~6 : PRECISION.SIZE
¦ PRECISION
{ add: siæe- d3 1 }
{ add: precision, size= d3 precision.7 : PRECISION SIZE
¦ PRECISIOM
{ add: size~ d4 .}
{ add: precision, size- d4 ~0 I~ DOCKET NO.: SA8-84-038 string :char.bit variance ¦ char.bit { add: nonvarying /* I variance *~
;
char.bit : - CHARACTER SIZE
CHARACTER
{ add: length 1 }
BIT SIZE
BIT
{ add: len~th 1 ¦ GRAPHIC SIZE
;
variance : ~A~YING
¦ NONVARYING
20 pictured : PICTURE REAL
~ PICTURE COMPLEX
¦ PICTURE
;
const : explicit.const 1 implicit.const { add: cons~ant ;
explicit.const : CONSTANT opt.Lntrn FORMA~
/* ¦ CONSTANT EXTERNAL FORMAT */
I CONSTANT opt.intrn LABEh /* I CONSTANT EXTERNAL 1A8EL */
I CONSTANT opt.ext. intrn ENTR~ -I CONST~NT opt.ex~.in~rn fileset FILE
I CONSTANT opt.ext.intrn fileset { add:file CONSTANT opt.ext.intrn FILE
;
0 implicit.const : op~.ext~intrn ENTRY
opt.ext.intrn fileset FILE
op~ext.intrn filese~
{ add:file 4s i opt.ext.intrn FILE
IBM DOC~ET NO.: SA8-84-038 fileset : ENVIRONMENT filet~pe { add: file S Ifiletype { add: file /* ¦ ENVIRONMENT */
,.
input.output : INPUT
;
opt.in.out.update : INPUT
I OUTPUT
I UPDATE
filetype : stream I record I input.output 2U I input.output record.attr stream : STR~AM input.output I STREAM OUTPUT PRINT
¦ STREAM
I PRINT
;
record : RECORD opt.in.out.update record.attr ¦ RECORD optOin.out.updat~
1 UPDATE record.attr UPDATE
I record. attr record.attr . SEQUENTIAL KEYED
¦ S~QUENTIAL
KEYED
DIRECT KEYED
DIRECT
;
Given ~he BNF grammar defined above, ~he operation of the LAL~(l) gram~lar processor is well knGwn and nPed not be described in detail herein for a complete understanding o the invention. In general, there will be a leæical scanner S29~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 function associated with the dictionary structure which supplies tokens in a canonical order. The dictionary or symbol table abstraction, "dict", may have a control abstrac~ion, or iterator:
all_attrs - ITER (r: name_reference) returns at~r.
This returns all symbol attributes, for the name reference r, in the required order. The use of the construct would appear as follows: ~
complete attrs(r):
initialize internal state;
FOR a: attr IN dict.all a~trs(r) DO
skandard parsing algorithm run from generated ~ables END;
END complete attrs;
The operation of the attribute processor of this inven-ti.on wilL now be described, after firs~ defining some terminology to be used in the description.
Any PL/I source code program can be represented in the form of a "program tree", which is an acyclic lloop-less) graph of various nodes which represent the program~ For example, ~here are nodes representing procedures and begin blocks, executabLe statements, expression~, etc., as may be conceptually illustrated in Figure 48.
A "Block_m" will hereinafter refer to a node of the program tree, with one ~lock m fox every procedure and begin ~lock dafined in ~he PL~I source pr~gram (including the main ex~exnal procedure comprising the en~ire program)~ Each BlocX m node also iden~ifie-~ ~ data struc~ure referred to as ~S2~
I ~M DOCKET NO.: SA8-84-038 a Declaration Table which describes data items declared in the corresponding procedure or begin block.
A Declaration Table represents and describes data items declared in a procedure or begin ~lock. There is one Declaration Table for every procedure and begin block defined in a given PL/I source program. Each table is a set of Decl_m entries, one entry for every declared data item in the procedure or begin block.
Each one of the Decl_m en~ries in a Declaration Table corresponds to a particular one of the data item~ declared in that procedure or begin block. Each Decl_m entry identi-fies the attributes assigned to the correspondin~ data item.
The assigned attributes include those which have been explicitly assisned, e.g., by a DECLARE sta~ement in the PL/I source program, as well a~ attributes which have been implicitly assigned, e.g., default values. The representa-tion of a set of attributes corresponding to a data item is in the form of a structure called an Attribute Record, the components of which identify and describe the attributes in the set. For example:
The Attribute Sçt component of a given Attribute Record indica~es which attributes are explicitly defined for this data item. This includes simple attributes, e.g., AUTOMA-TIC, BINAR~, etc., as well ~s composi~e attributes, e.g., 2~ BASED, DEFINED, etc., which have another independent declaration associated with them.
~ ~ ~2~Z~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 ;~8 The Subnode Set component of an Attribute Record indi-cates which of the composite attributes in an Attribute Set ; have additional descriptors called Attribute Subnodes; and The Su~nod~s components of the Attribute Record identi-fies data base objects compri~in~ the Attribute Subnode~.
Additional terms will be described as necessary in thefollowing description of ~he operation of the ta~le driven parser.
The attributa analysis algorithm is performed by the 10 compiler during a traversal of the source program tre~. The attribu~e analysis can be performed during the same traversal of the program tree in which other analysis functions are performed, e.g. ~ the analysi~ of name references. The operations performed during the travers~l, and the corresponding data produced in each opera~ion, can be described as follow.s.
During a top-down p~ss, whe~ processing a particular Block m node r~presenting a procedure, the formal parameters of this procedure are identiied and represented by Decl m entriesO These en~ries are placed in th~ Block m's Declara-~ion TableO In add~tion~ an initial se~ Q~ attributes for each parameter is represented by an At~ribute Record, with the composite attribu~es being represen~ed by Attribute Subnodes~ Each Attribute Record is placed in the corre-sponding Decl_m entry.
In the bot~om-up pass of the compiler through the source program, the remainder of the attribute analysis ~ 529(~3 I~M DOCKET NO.: sA8-84-o38 algorithm is performed for all data items represented in the Declaration Table of each Block_m node. This involves the following operations:
For data items which have LIK~ attribu~es, these attri-butes are expanded and the resulting additional data items are rPpresented by Decl_m en~ries. These entries are placed in the Block m's Declaration Table. In addition, an initial attribute set of each such item is represented by an Attri-bute Record~ and the composite attributes are also repre sented by Attribu~e Subnodes. Each Attribute ~ecord is placed in the corresponding Decl_m entry. Thus, at this point, the Declaration Table represents all data items tha~
are to be subjected to the final attribu~e analysis by the attribute parsex.
Next, for every data i~em represented in the final Declaration Table, the at~ribute parser is invoked ~o-perform the following steps:
Parse the initial attribute set, verifyi~g its syntax and pPrforming the initial semantic checks;
~0 Complete the a~tribute set hy applying default values;
and Check the con~istency and completeness of the final attribute set.
- ~11 of these func~ions are peEformed hy the table-driven attribute parser, using the syntax table and parse driver produ~ed rom the PL/I context-free attribute checking grammax by the LAL~ (1) parser generator, The invoking of the parser from the compiler is done in a straight-~orward ~ S~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 manner which need not be described to those skilled in the field of computer programming.
As can be seen from the above description, the tedious task of writing lar~e amounts of object code to perform attribute analysis in the compiler front end can be avoided by converting the attribute ~xammar ~o a context-free grammar compatible with a known grammar processor. It is known how to use such grammar processors to generate synta~
tables and a parse driver for syntax checking of standard grammar, e.g., English parsing, and the same technigue can be used for checking the syntax and comple~eness of attri-butes in a source program. The operation of the table-driven parser need not be described in de~ail for a proper understanding of the invention, since i~ is a substantially conventio~al operation merely adapted for a heretofore unrecognized application. It will be appreciated that the cumbersome tasX of manually generating large amounts of object code for the attribute analysis portion of the compiler fron~ end has been substan~ially replaced by the ~as~ of mere~y expressing the a~tribute grammar as a context-free grammar compatible with a grammar processor.
The grammar processor then generates the table-driven parser in an efficien~ and relia~le manner. Adapting the compiler to subsequent changes in the programming language becomes a relatively straight-forward matter, substantially decreasing the time required to generate a new compiler.
I ~M DOCKET NO .: SA8-84-038 The invention has been described above as utilizing a LALR(1) grammar processor to generate the table-driven attribute parser, although it should be readily apparent that other ~rammar processor may be sui~able.
It should also be noted tha~, although the invention has been described above for use in a PL/l compiler, the techni~ue is equally applicable to other programming languages w~ose attribute grammar can be expressed as a context-free grammar compatible with the LALR (1) or some other suitable grammar processor.
9o DEFAULT (~REA~
AREA(d5);
10. DEFAULT (POSITION) POSITION~l);
11. DEFAULT ( (ENTRY ¦ FILE) &
(AUTOMATIC J BASED ¦ DEFINED J PAR~METER ¦
STATIC ¦ CONTROLLED.J MEMBER ¦ ALIGN D ¦
UNALIGNED ¦ INITIAL) ) VARIABLE;
12. DEFAULT ~ tENTR~ ¦ FILE) & RANGE(*) ) CONSTANT;
13. DEFAULT (RAN~E(*) & CONSTANT) VARIABLE;
14. DEEAULT ( CONDITION ¦ ( (FILE ¦ ENTRY~ & CONSTANT ) EXTERNAL;
15. DEFAULT (R~NGE~*)) INTERNAL;
5 16. DEFAULT (VARIABLE & EXTERNAL) STATIC;
17. DEFAULT ~VARIABLE) AUTOMATIC;
~3 'f~ 2S~
IBM DOCKET NO .: SA8 ~ 8 4 - 0 3 8 18. DEFAULT ( (CHARACTER ¦ BIT ¦ PICTURE) & CONSTANT) UNALIGNED;
19 . DEFAULT ~ CONSTANT ) ALI GNED
Note that in the <arit~netic> diagram of Figure 37, each of the double skips around SIZE uses the given default, since if only P~ECISION is present, the "test-invalid-duplicates" procedure described in 4.3O6.2 o~ the ANSI PL/I
standard allows the addition o~ either SIZE or SIZE and SCALE.
Aside from adding the de~ault table numbers and separating the skip path on the left side of <variable> in Figure 35, the only changes from the Group 3 grammar of Figures 22-31 are the modification of some paths to exclude some cases made illegal hy the "final" checks found in ~4.3.7~1 of the Standard. For example, step 3 of ~4.3.7.1 states that EXTERNAL canno~ occur with AUTOMArIC, B~SED, DEFINED, PARAMETER or BUILTIN, and step 4 of 4.3.7.1 states the combination o~ EXTE~NAL and CON~TANT cannot occur ~o~ether with either FORMAT or LABEL. ~lso, some illegal sLngle attributes are removed, i.e., POSITION, CONSTANT, ENVIRONMENT, VARYING, ~NONV M YING, LOCAL, and OPTIONS.
Not~ tha~ in the syntax fox 1'cconstan~~1iteral-constant>" in Figure 33, <arithmetic> has now been changed . 25 to <crea~ed-const>~ This <created-const> identifies a literal, which has different defaulting rules ~han ~hose for variables. The adding of default attribute~ for literals could be performed in a conv~ntional manner, or th~ attri-I BM DOCKET NO.: SA8-84-038 bute defaulting syntax for literals could be derived from "cxeate-constant" in 4.4.7 of the ~NSI PL/I standard. The dexivation would be performed in a manner similar t~ that described above for variables, and need not be described in detail here for a complete understanding of the invention.
The next step is to combine the Group 4 grammar with the Group 3 grammar to produce a "Group 5" grammar to be used in the attribute pxocessor for consistency checking, applying of defaults, and verifying that the final attribute lQ set is complete. ~his Group 5 grammar is illustxated in the BNF grammax to be described below.
The Group 5 grammar consists of the basic rules ~or checking consistency and completeness, plus the subset rules and the default rules. By using some sta~e variables, the 15 subset rules or the default rules, or both, may be effec-tively deactiva~ed. Consequently, the one grammar can be used for the multiple purposes of:
~asic: consistency and completeness check;
Basic and subxet: consistency check only;
Basic and default: consistency and completeness wi~h de~ault attribute addition all at once;
Basic, subset and default: consistency with default attribute addition.
Having de~ine~ the PL/I a~tribu~e syntax structure in 25 ~erms of a context~fr2e grammar~ the grammar speciica~ions can now be conver~ed to a Backus-Naur Form (BNF) grammar in a well-known manner and provided ~o a grammar processor, 252~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 B
-preferably a LALR~1) processox, which will generate the syntax tables and parse driver which make up the table-driven attribute parser according ~o this invention. In the operation o~ the LALR(1) processor, a "lexical scanner" will be used to supply encoded attributes to the grammar pro-cessor as tokens in a canonical order, i.e., an order con-sistent with the syntax. For example, the token ordering may be as given below in the TOKEN ORDER TABLE.
TOKEN OR~
1o ~token VARIABL~ CONSTANT
%token INTERNAL EXTERNAL
%token AUTOMATIC BASED CONTROLLED STATIC MEMBER
PARAMETER DEFINED
%token POSITION
~token P
%token INITIAL
%tok~n ALIGNED UNALIGNED
~token ENVIRONMENT
~token STREAM RECORD
0 %token INPUT OUTPUT UPDATE
%token PRINT
~stoken SEQUENTIAI; DIRECT
%token KEYED
~token RETURNS
%token OPTIONS
%token BUILTIN CONDITION GENE:RIC
~token STRUCTURE
%t.oken ENTRY FILE FORMAT LA~EL
%token AREA POINTER OFFSET
%token CHARACTER BIT GRAPHIC PIC~URE
%toXen REAL COMPLEX
%token DECIMAL BIN~RY, %token FIXED FLOAT
%token PREC~SION
~S %tok~n SIZE
%token SCALE
%token VARYING NONVARYING
%token LOCAL
%token DIMENSION
%token END
With this ordering the B~F grammar may be defined as ~llows.
:~Z5~
IBM DOCKET NO .: SA8 - 8 4 - 0 3 8 Note that the non-terminals are all lower case,- and associated actions are in { }. For example, the notation { add: cattribute>
indicates the system default addition of the particuLar S attribute mentioned to the exi~ting set of attributes. The "su~set" rule~ are within comment delimiters, i.e., /*...*/
and the "default rules" are those with an associated "add"
action. The subset production rules in comments are legal for checking consis~ency, but not for checking a final set of attrihutes when there are no more user dafault attributes to be added.
BNF GRAMMAR
pgm : decl.list decl.list : decl.
¦ decl.llst decl decl : decl.type END
{ declaration valid 20 op~.in~.e~rn : INTERNAL
¦ EXTEF~TAL
, r add:internal }
2 5 op~ . ext . intrn : INTERNAI
IEXTERNAL
{ add: external 52~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 opt . intrn : XNTERNAL
{ add: internal ;
decl. type : variahle DIMENSION
variable ¦ const DIMENS~ON
I const I opt.ext.intrn CONDITION
L o l opt.intrn BUILTIN
opt.i~trn GENERIC
;
variable : explicit.var I implicit.var 15 '{ add: variable }
explicit.var : VARIABLE storage data,desc ¦ VARIA~LE data.des~.
{ add: internal }add: auto J VARIA~LE INTERNAL data.desc.
~5 {}add: auto ¦ VARIABLE EXTERNAL data.desc { add: static I V~RI~BLE storage ENTRY
¦ VARIABLE storage FILE
implicit.var : storage data.desc I data.desc.
{ add: internal .}add: autO
¦ INT~RN~L da~a.desc { add: auto I EXTERNAL data.d~sc { add: static I storage ENTRY
I storage FILE
s~
I ~M DOCKET NO.: SA~-84-038 init . or . al ign : INITIAL align ¦ INITIAL
~ add: aligned 1align , ;
storage opt.int.extrn CONTROLLED
¦ opt.int.extrn STATIC
~3 opt~intrn stor.type stor.type : AUTOMATIC
¦ BASED
¦ DEFINED POSITIO~ opt.pos.value ¦ DEFINED
1~ /* I ~ POSITION */
I PARAMETER
¦ MEMBER
;
opt.pos.value : P
{ add: position-value= 1 }
data.desc :STRUCTURE
¦ INITIAL data.type data.type init.or.align ENTRY
I init . or . align FILE
opt.align :ALIGNED
~ UNALIGNED
{ add: aligned , }
align : ALIGNED
¦ UNALIGMED
;
~r~ 0 IBM DOCKET No.: SA8-84-038 data.t~e : opt.align arith op~.align noncomp string { add: unaligned }
align string pictured ~ add: unaligned 1 align pictured noncomp : MEA
t add: size = 1 ¦ AREA SIZE
J RETURNS OPTIONS ENTRY
¦ RETURNS OPTIONS
{ add: entry }
~ RETURNS ENTRY
IOPTIONS ENTRY
¦ RETURNS
{ add: entry ¦ FORMAT LOCAL
I FORMAT
I LABEL LOCAL
I LABEL
/* ¦ LOCAL */
¦ POI~TER
I OFFSET
arith : mode arith.rest ;
35 mode : REAL
I COMPLEX
{ add: real }
40 arith.rest : opt.bin op~.~ixed precision.4 I DECI~LUL op~.fixed precision.5 I opt.bin FLOAT precision.6 I DECIMAL FLOAT precision,7 ~ 2~9U~
IBM DOCKET NO.: SA8-84-038 ~3 opt.bin : I BINARY
{ add: ~inary ;
5 opt.fixed :~ FIXED
I
{ add~ fixed ;
precision.4 : PRECI5ION SIZE SCALE
¦ PRECISION SIZE
¦ PRECISION
. { add: size- dl, scale= O
{ add: precision, size= dl, . scale= 0 }
precision.5 PRECISION SIZE SCALE
¦ PRECISION SIZE
¦ PRECISION
{ add: size= d2, scale= 0 }
{ add: precision, size= d2, scale= O
}
precision~6 : PRECISION.SIZE
¦ PRECISION
{ add: siæe- d3 1 }
{ add: precision, size= d3 precision.7 : PRECISION SIZE
¦ PRECISIOM
{ add: size~ d4 .}
{ add: precision, size- d4 ~0 I~ DOCKET NO.: SA8-84-038 string :char.bit variance ¦ char.bit { add: nonvarying /* I variance *~
;
char.bit : - CHARACTER SIZE
CHARACTER
{ add: length 1 }
BIT SIZE
BIT
{ add: len~th 1 ¦ GRAPHIC SIZE
;
variance : ~A~YING
¦ NONVARYING
20 pictured : PICTURE REAL
~ PICTURE COMPLEX
¦ PICTURE
;
const : explicit.const 1 implicit.const { add: cons~ant ;
explicit.const : CONSTANT opt.Lntrn FORMA~
/* ¦ CONSTANT EXTERNAL FORMAT */
I CONSTANT opt.intrn LABEh /* I CONSTANT EXTERNAL 1A8EL */
I CONSTANT opt.ext. intrn ENTR~ -I CONST~NT opt.ex~.in~rn fileset FILE
I CONSTANT opt.ext.intrn fileset { add:file CONSTANT opt.ext.intrn FILE
;
0 implicit.const : op~.ext~intrn ENTRY
opt.ext.intrn fileset FILE
op~ext.intrn filese~
{ add:file 4s i opt.ext.intrn FILE
IBM DOC~ET NO.: SA8-84-038 fileset : ENVIRONMENT filet~pe { add: file S Ifiletype { add: file /* ¦ ENVIRONMENT */
,.
input.output : INPUT
;
opt.in.out.update : INPUT
I OUTPUT
I UPDATE
filetype : stream I record I input.output 2U I input.output record.attr stream : STR~AM input.output I STREAM OUTPUT PRINT
¦ STREAM
I PRINT
;
record : RECORD opt.in.out.update record.attr ¦ RECORD optOin.out.updat~
1 UPDATE record.attr UPDATE
I record. attr record.attr . SEQUENTIAL KEYED
¦ S~QUENTIAL
KEYED
DIRECT KEYED
DIRECT
;
Given ~he BNF grammar defined above, ~he operation of the LAL~(l) gram~lar processor is well knGwn and nPed not be described in detail herein for a complete understanding o the invention. In general, there will be a leæical scanner S29~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 function associated with the dictionary structure which supplies tokens in a canonical order. The dictionary or symbol table abstraction, "dict", may have a control abstrac~ion, or iterator:
all_attrs - ITER (r: name_reference) returns at~r.
This returns all symbol attributes, for the name reference r, in the required order. The use of the construct would appear as follows: ~
complete attrs(r):
initialize internal state;
FOR a: attr IN dict.all a~trs(r) DO
skandard parsing algorithm run from generated ~ables END;
END complete attrs;
The operation of the attribute processor of this inven-ti.on wilL now be described, after firs~ defining some terminology to be used in the description.
Any PL/I source code program can be represented in the form of a "program tree", which is an acyclic lloop-less) graph of various nodes which represent the program~ For example, ~here are nodes representing procedures and begin blocks, executabLe statements, expression~, etc., as may be conceptually illustrated in Figure 48.
A "Block_m" will hereinafter refer to a node of the program tree, with one ~lock m fox every procedure and begin ~lock dafined in ~he PL~I source pr~gram (including the main ex~exnal procedure comprising the en~ire program)~ Each BlocX m node also iden~ifie-~ ~ data struc~ure referred to as ~S2~
I ~M DOCKET NO.: SA8-84-038 a Declaration Table which describes data items declared in the corresponding procedure or begin block.
A Declaration Table represents and describes data items declared in a procedure or begin ~lock. There is one Declaration Table for every procedure and begin block defined in a given PL/I source program. Each table is a set of Decl_m entries, one entry for every declared data item in the procedure or begin block.
Each one of the Decl_m en~ries in a Declaration Table corresponds to a particular one of the data item~ declared in that procedure or begin block. Each Decl_m entry identi-fies the attributes assigned to the correspondin~ data item.
The assigned attributes include those which have been explicitly assisned, e.g., by a DECLARE sta~ement in the PL/I source program, as well a~ attributes which have been implicitly assigned, e.g., default values. The representa-tion of a set of attributes corresponding to a data item is in the form of a structure called an Attribute Record, the components of which identify and describe the attributes in the set. For example:
The Attribute Sçt component of a given Attribute Record indica~es which attributes are explicitly defined for this data item. This includes simple attributes, e.g., AUTOMA-TIC, BINAR~, etc., as well ~s composi~e attributes, e.g., 2~ BASED, DEFINED, etc., which have another independent declaration associated with them.
~ ~ ~2~Z~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 ;~8 The Subnode Set component of an Attribute Record indi-cates which of the composite attributes in an Attribute Set ; have additional descriptors called Attribute Subnodes; and The Su~nod~s components of the Attribute Record identi-fies data base objects compri~in~ the Attribute Subnode~.
Additional terms will be described as necessary in thefollowing description of ~he operation of the ta~le driven parser.
The attributa analysis algorithm is performed by the 10 compiler during a traversal of the source program tre~. The attribu~e analysis can be performed during the same traversal of the program tree in which other analysis functions are performed, e.g. ~ the analysi~ of name references. The operations performed during the travers~l, and the corresponding data produced in each opera~ion, can be described as follow.s.
During a top-down p~ss, whe~ processing a particular Block m node r~presenting a procedure, the formal parameters of this procedure are identiied and represented by Decl m entriesO These en~ries are placed in th~ Block m's Declara-~ion TableO In add~tion~ an initial se~ Q~ attributes for each parameter is represented by an At~ribute Record, with the composite attribu~es being represen~ed by Attribute Subnodes~ Each Attribute Record is placed in the corre-sponding Decl_m entry.
In the bot~om-up pass of the compiler through the source program, the remainder of the attribute analysis ~ 529(~3 I~M DOCKET NO.: sA8-84-o38 algorithm is performed for all data items represented in the Declaration Table of each Block_m node. This involves the following operations:
For data items which have LIK~ attribu~es, these attri-butes are expanded and the resulting additional data items are rPpresented by Decl_m en~ries. These entries are placed in the Block m's Declaration Table. In addition, an initial attribute set of each such item is represented by an Attri-bute Record~ and the composite attributes are also repre sented by Attribu~e Subnodes. Each Attribute ~ecord is placed in the corresponding Decl_m entry. Thus, at this point, the Declaration Table represents all data items tha~
are to be subjected to the final attribu~e analysis by the attribute parsex.
Next, for every data i~em represented in the final Declaration Table, the at~ribute parser is invoked ~o-perform the following steps:
Parse the initial attribute set, verifyi~g its syntax and pPrforming the initial semantic checks;
~0 Complete the a~tribute set hy applying default values;
and Check the con~istency and completeness of the final attribute set.
- ~11 of these func~ions are peEformed hy the table-driven attribute parser, using the syntax table and parse driver produ~ed rom the PL/I context-free attribute checking grammax by the LAL~ (1) parser generator, The invoking of the parser from the compiler is done in a straight-~orward ~ S~
I BM DOCKET NO .: SA8 - 8 4 - 0 3 8 manner which need not be described to those skilled in the field of computer programming.
As can be seen from the above description, the tedious task of writing lar~e amounts of object code to perform attribute analysis in the compiler front end can be avoided by converting the attribute ~xammar ~o a context-free grammar compatible with a known grammar processor. It is known how to use such grammar processors to generate synta~
tables and a parse driver for syntax checking of standard grammar, e.g., English parsing, and the same technigue can be used for checking the syntax and comple~eness of attri-butes in a source program. The operation of the table-driven parser need not be described in de~ail for a proper understanding of the invention, since i~ is a substantially conventio~al operation merely adapted for a heretofore unrecognized application. It will be appreciated that the cumbersome tasX of manually generating large amounts of object code for the attribute analysis portion of the compiler fron~ end has been substan~ially replaced by the ~as~ of mere~y expressing the a~tribute grammar as a context-free grammar compatible with a grammar processor.
The grammar processor then generates the table-driven parser in an efficien~ and relia~le manner. Adapting the compiler to subsequent changes in the programming language becomes a relatively straight-forward matter, substantially decreasing the time required to generate a new compiler.
I ~M DOCKET NO .: SA8-84-038 The invention has been described above as utilizing a LALR(1) grammar processor to generate the table-driven attribute parser, although it should be readily apparent that other ~rammar processor may be sui~able.
It should also be noted tha~, although the invention has been described above for use in a PL/l compiler, the techni~ue is equally applicable to other programming languages w~ose attribute grammar can be expressed as a context-free grammar compatible with the LALR (1) or some other suitable grammar processor.
Claims (8)
1. A compiler of the type for receiving and compiling a source program in a predetermined programming language, and wherein said source program assigns attributes to data items employed in said program and said programming language including attribute grammar rules with which said attribute assignments must comply, said compiler including attribute analysis means for checking the consistency of attributes assigned to a particular data item by said program and for selectively assigning default attributes to said data items, characterized in that said attribute analysis means com-prising a table-driven attribute parser including a syntax table and a parse driver for verifying that said attribute assignments comply with said attribute grammar rules.
2. A compiler as defined in Claim 1, wherein said table-driven attribute parser includes means for adding said default values to obtain a final attribute set for each data item.
3. A compiler as defined in Claim 2, wherein said table driven attribute parser includes means for checking for consistency of the various attributes in said final attribute set in accordance with said attribute grammar rules and for checking the completeness of said final attri-bute set in accordance with said attribute grammar rules.
4. A method of checking the consistency of attribute assignments in a program in accordance with attribute grammar rules associated with the programming language of said program, said program being defined by a program tree having a plurality of nodes interconnected by program segments, said method including the steps of:
generating a declaration table for each program segment, said declaration table listing all data items declared in its corresponding program segment;
generating a plurality of attribute records corre-sponding to respective data items listed in each said declaration table, each said attribute record comprising an initial set of attributes assigned to a corresponding data item declared in said corresponding program segment;
parsing said initial set of attributes for each data item to verify the syntax of said initial attribute set and verify that said initial attribute set conforms to semantic restrictions included in said attribute grammar;
adding default values to selected attribute records to obtain final attribute sets for each said data item; and checking the consistency of said attributes in each said final attribute set in accordance with said attri-bute grammar rules and the completeness of each final attri-bute set in accordance with said attribute grammar rules.
generating a declaration table for each program segment, said declaration table listing all data items declared in its corresponding program segment;
generating a plurality of attribute records corre-sponding to respective data items listed in each said declaration table, each said attribute record comprising an initial set of attributes assigned to a corresponding data item declared in said corresponding program segment;
parsing said initial set of attributes for each data item to verify the syntax of said initial attribute set and verify that said initial attribute set conforms to semantic restrictions included in said attribute grammar;
adding default values to selected attribute records to obtain final attribute sets for each said data item; and checking the consistency of said attributes in each said final attribute set in accordance with said attri-bute grammar rules and the completeness of each final attri-bute set in accordance with said attribute grammar rules.
5. A method of checking the consistency and complete-ness of attribute assignments in a program expressed in a predetermined programming language, said programming language having an associated set of grammar specifications defining attribute requirements, said method comprising the steps of:
generating a context-free grammar consistent with said grammar specifications;
providing said context-free grammar to a grammar processing means to obtain a table-driven attribute parser comprising a syntax table and a parse driver; and processing said program with said table-driven attribute parser to check the consistency of said attributes for compliance with said attribute grammar.
generating a context-free grammar consistent with said grammar specifications;
providing said context-free grammar to a grammar processing means to obtain a table-driven attribute parser comprising a syntax table and a parse driver; and processing said program with said table-driven attribute parser to check the consistency of said attributes for compliance with said attribute grammar.
6. A method as defined in Claim 5, wherein said context-free grammar incorporates default value assignments, and wherein said processing step comprises adding default values as further attribute assignments for selected data items where necessary to comply with said attribute grammar.
7. A method as defined in Claim 6, wherein said processing step further comprises checking the completeness and consistency of attribute assignments with said attribute grammar after the addition of said further attribute assignments.
8. A method of generating a table-driven attribute parser, including a syntax table and a parse driver, for checking the consistency and completeness of attribute assignments in a source program in accordance with attribute grammar rules associated with the programming language of said source program, said method comprising the steps of:
generating a context-free grammar in accordance with said attribute grammar; and processing said context-free grammar in a parser generator to obtain said syntax table and parse driver.
generating a context-free grammar in accordance with said attribute grammar; and processing said context-free grammar in a parser generator to obtain said syntax table and parse driver.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US06/742,493 US4686623A (en) | 1985-06-07 | 1985-06-07 | Parser-based attribute analysis |
| US742,493 | 1985-06-07 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA1252900A true CA1252900A (en) | 1989-04-18 |
Family
ID=24985059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA000506446A Expired CA1252900A (en) | 1985-06-07 | 1986-04-11 | Parser-based attribute analysis |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US4686623A (en) |
| EP (1) | EP0204942A3 (en) |
| JP (1) | JPS61282935A (en) |
| CA (1) | CA1252900A (en) |
Families Citing this family (45)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4931928A (en) * | 1988-11-09 | 1990-06-05 | Greenfeld Norton R | Apparatus for analyzing source code |
| US5193191A (en) * | 1989-06-30 | 1993-03-09 | Digital Equipment Corporation | Incremental linking in source-code development system |
| US5325531A (en) * | 1989-06-30 | 1994-06-28 | Digital Equipment Corporation | Compiler using clean lines table with entries indicating unchanged text lines for incrementally compiling only changed source text lines |
| US5201050A (en) * | 1989-06-30 | 1993-04-06 | Digital Equipment Corporation | Line-skip compiler for source-code development system |
| JPH04211830A (en) * | 1990-02-05 | 1992-08-03 | Matsushita Electric Ind Co Ltd | Parallel compiling system |
| WO1992003782A1 (en) * | 1990-08-23 | 1992-03-05 | Super-Computer Systems Limited Partnership | Parsing program data streams |
| US5560009A (en) * | 1990-09-21 | 1996-09-24 | Hewlett-Packard Company | Generating symbolic debug information by merging translation and compiler debug information |
| JPH04280320A (en) * | 1991-03-08 | 1992-10-06 | Nec Corp | Automatic generation device for program |
| US5487147A (en) * | 1991-09-05 | 1996-01-23 | International Business Machines Corporation | Generation of error messages and error recovery for an LL(1) parser |
| US5488727A (en) * | 1991-09-30 | 1996-01-30 | International Business Machines Corporation | Methods to support multimethod function overloading with compile-time type checking |
| US5613115A (en) * | 1991-12-09 | 1997-03-18 | Total Control Products, Inc. | Method for using PLC programming information to generate secondary functions such as diagnostics and operator interface |
| US5511213A (en) * | 1992-05-08 | 1996-04-23 | Correa; Nelson | Associative memory processor architecture for the efficient execution of parsing algorithms for natural language processing and pattern recognition |
| US5367685A (en) * | 1992-12-22 | 1994-11-22 | Firstperson, Inc. | Method and apparatus for resolving data references in generated code |
| US5491628A (en) * | 1993-12-10 | 1996-02-13 | Xerox Corporation | Method and apparatus for document transformation based on attribute grammars and attribute couplings |
| US5696973A (en) * | 1995-02-17 | 1997-12-09 | International Business Machines Corporation | Index-based method for supporting multimethod function overloading with compile-time type checking and run-time dispatch |
| US5687378A (en) * | 1995-06-07 | 1997-11-11 | Motorola, Inc. | Method and apparatus for dynamically reconfiguring a parser |
| AU743193B2 (en) * | 1997-02-26 | 2002-01-17 | Nec Corporation | Decoder using a finite state machine in decoding an abstract syntex notation-message and an encoder for carrying out encoding operation at a high speed |
| US5991539A (en) * | 1997-09-08 | 1999-11-23 | Lucent Technologies, Inc. | Use of re-entrant subparsing to facilitate processing of complicated input data |
| US6321376B1 (en) * | 1997-10-27 | 2001-11-20 | Ftl Systems, Inc. | Apparatus and method for semi-automated generation and application of language conformity tests |
| US5949993A (en) * | 1997-10-31 | 1999-09-07 | Production Languages Corporation | Method for the generation of ISA simulators and assemblers from a machine description |
| US6405365B1 (en) * | 1999-07-02 | 2002-06-11 | Cisco Technology, Inc. | Computer program command generator and parser |
| US9916134B2 (en) * | 1999-10-05 | 2018-03-13 | Dietrich Charisius | Methods and systems for accessing distributed computing components through the internet |
| US7089541B2 (en) * | 2001-11-30 | 2006-08-08 | Sun Microsystems, Inc. | Modular parser architecture with mini parsers |
| US20030144978A1 (en) * | 2002-01-17 | 2003-07-31 | Zeine Hatem I. | Automated learning parsing system |
| US7080352B2 (en) * | 2002-01-30 | 2006-07-18 | Dloo, Incorporated | Method and system for creating programs using code having coupled syntactic and semantic relationships |
| US6925541B2 (en) | 2002-06-12 | 2005-08-02 | Hitachi, Ltd. | Method and apparatus for managing replication volumes |
| US7603661B2 (en) * | 2003-01-30 | 2009-10-13 | Hamilton Sunstrand | Parse table generation method and system |
| US20040181707A1 (en) * | 2003-03-11 | 2004-09-16 | Hitachi, Ltd. | Method and apparatus for seamless management for disaster recovery |
| US7058864B1 (en) * | 2003-05-21 | 2006-06-06 | Hewlett-Packard Development Company, L.P. | Test for processor memory cache |
| US8190723B2 (en) * | 2003-12-14 | 2012-05-29 | Cisco Technology, Inc. | Method and system for automatically determining commands for a network element |
| US7945903B1 (en) | 2004-04-16 | 2011-05-17 | At&T Intellectual Property Ii, L.P. | System and method for converting graphical call flows into finite state machines |
| US8683437B1 (en) * | 2004-04-16 | 2014-03-25 | At&T Intellectual Property Ii, L.P. | System and method for the automatic validation of dialog run time systems |
| US20060015591A1 (en) * | 2004-06-08 | 2006-01-19 | Datla Krishnam R | Apparatus and method for intelligent configuration editor |
| US8010952B2 (en) * | 2004-06-08 | 2011-08-30 | Cisco Technology, Inc. | Method and apparatus for configuration syntax and semantic validation |
| US7735140B2 (en) * | 2004-06-08 | 2010-06-08 | Cisco Technology, Inc. | Method and apparatus providing unified compliant network audit |
| US7721304B2 (en) | 2004-06-08 | 2010-05-18 | Cisco Technology, Inc. | Method and apparatus providing programmable network intelligence |
| US20070113221A1 (en) * | 2005-08-30 | 2007-05-17 | Erxiang Liu | XML compiler that generates an application specific XML parser at runtime and consumes multiple schemas |
| US8037092B2 (en) | 2005-10-05 | 2011-10-11 | International Business Machines Corporation | System and method for merging manual parameters with predefined parameters |
| US8843906B1 (en) * | 2006-10-16 | 2014-09-23 | The Mathworks, Inc. | Inferring data types from compiler call site |
| US20090024366A1 (en) * | 2007-07-18 | 2009-01-22 | Microsoft Corporation | Computerized progressive parsing of mathematical expressions |
| US9158519B2 (en) * | 2008-05-21 | 2015-10-13 | Optumsoft, Inc. | Dynamic collection attribute-based computer programming language methods |
| EP2454661A1 (en) * | 2009-07-15 | 2012-05-23 | Proviciel - Mlstate | System and method for creating a parser generator and associated computer program |
| CN102426550B (en) * | 2011-10-26 | 2014-05-14 | 中国信息安全测评中心 | Source code analysis method and system |
| WO2014152800A1 (en) * | 2013-03-14 | 2014-09-25 | Massively Parallel Technologies, Inc. | Project planning and debugging from functional decomposition |
| US9875087B2 (en) * | 2015-04-10 | 2018-01-23 | Oracle International Corporation | Declarative program engine for large-scale program analysis |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4398249A (en) * | 1970-08-12 | 1983-08-09 | Pardo Rene K | Process and apparatus for converting a source program into an object program |
| JPH0630066B2 (en) * | 1983-10-24 | 1994-04-20 | 株式会社日立製作所 | Table type language translation method |
-
1985
- 1985-06-07 US US06/742,493 patent/US4686623A/en not_active Expired - Fee Related
-
1986
- 1986-04-11 CA CA000506446A patent/CA1252900A/en not_active Expired
- 1986-04-30 EP EP86105949A patent/EP0204942A3/en not_active Withdrawn
- 1986-05-07 JP JP61103255A patent/JPS61282935A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| EP0204942A2 (en) | 1986-12-17 |
| EP0204942A3 (en) | 1988-08-03 |
| JPS61282935A (en) | 1986-12-13 |
| US4686623A (en) | 1987-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA1252900A (en) | Parser-based attribute analysis | |
| Holub | Compiler design in C | |
| Beazley | Python essential reference | |
| Clark et al. | Super-languages: Developing languages and applications with XMF | |
| Gil et al. | Formal language recognition with the Java type checker | |
| Parr et al. | Pccts reference manual: version 1.00 | |
| Gopinath et al. | Input algebras | |
| Thain | Introduction to compilers and language design | |
| Schmid et al. | Document spanners-a brief overview of concepts, results, and recent developments | |
| Matsumura et al. | A declarative extension of parsing expression grammars for recognizing most programming languages | |
| Leino et al. | Dafny reference manual | |
| Anderson et al. | SEMANOL (73) A metalanguage for programming the semantics of programming languages | |
| Summerfield | Programming in Python 3 | |
| Jeuring et al. | Grammars and parsing | |
| Lenz | Parsing with Perl 6 Regexes and Grammars: A Recursive Descent Into Parsing | |
| Papoulias | Parsing multi-ordered grammars with the Gray algorithm | |
| Kifer et al. | ERGOAI Reasoner User’s Manual | |
| Jackson | Some theoretical and practical results in context-sensitive and adaptive parsing | |
| Goncalves | A Reference Implementation of ECMAScript Built-in Objects | |
| Taylor | Lily: A parser generator for LL (1) languages | |
| Kubicek et al. | F# parsing expression grammar | |
| Campobadal Forés | The Design of an experimental programming language and its translator | |
| Johnstone et al. | A tutorial guide to rdp for new users | |
| Lieberherr et al. | Demeter User's Guide | |
| Horowitz et al. | Defining Syntax |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MKEX | Expiry |