J3/06-168 To: J3 From: Malcolm Cohen Subject: Intelligent Macros - Edits Date: 2006/04/24 Reference: J3-014, J3/05-280, J3/06-123r1, 06-142. 1. Introduction This paper contains edits for the intelligent macro facility whose specs and syntax are in 06-123r1, excluding the block-local variable edits which appear in 06-142. Rationale for this facility appears in 05-280. 2. Changes to the Specifications from 06-123r1 3. Changes to the Syntax from 06-123r1 3.1 PRESENT intrinsic The suggested syntax in 06-123r1 for handling optional macro dummy arguments is defective in that although it successfully handles absent ones, it forces present ones to expand to a single name. This is obviously bad! Therefore, in a macro expression only, PRESENT() should *expand* either to .TRUE. or .FALSE., i.e. it is "evaluated" by expansion rather than normal evaluation. 4. Edits to 04-007 [10:11+] 2.1, R207 , add new production in alphabetic order, "<> " {A macro definition shall be a specification construct.} [11:38+] Insert new paragraph "Additionally, an EXPAND statement may occur anywhere that any statement may occur other that as the first statement of a program unit. The syntax rules are apply to the program after macro expansion, i.e. with each EXPAND statement replaced by the statements it produces." {Instead of inserting macro expansion into the syntax, apply it first and then apply the syntax rules. An alternative approach (which certainly has some advantages) is described in section 5 of this paper.} [31:0+] Add new subclause at the end of clause 3. "3.5 Macro processing 3.5.1 Macro definition A macro definition defines a macro. A defined macro may only be referenced by a USE statement, IMPORT statement, or macro expansion statement. A defined macro shall not be redefined. R313 <> [ ]... R314 <> DEFINE MACRO [ , ] :: [ ( [ ] ) ] C3nn (R314) A shall not appear more than once in a . R315 <> The DEFINE MACRO statement begins the definition of the macro . Appearance of an in the DEFINE MACRO statement explicitly gives the macro the specified attribute. Each is a macro dummy argument. A macro dummy argument is a macro local variable. R316 <> <> R317 <> MACRO :: R318 <> MACRO OPTIONAL :: R319 <> INTEGER [ ( [ KIND= ] ) ] C3nn (R317) A shall not be the same as the name of a dummy argument of the macro being defined. C3nn (R318) A shall be the name of a dummy argument of the macro being defined. C3nn (R319) If appears, when the macro is expanded it shall be of type integer, and have a non-negative value that specifies a representation method that exists on the processor. A macro type declaration statement specifies that the named entities are macro local variables of the specified type. If the kind is not specified, they are of default kind. A macro local variable that is not a macro dummy argument shall appear in a macro type declaration statement. R319a <> [ ]... R320 <> <> <> <> <> C3nn A statement in a macro definition that is not a or shall not appear on a line with any other statement." {Note: This is in the specs. It was for ease of use for external macro tools. I am not sure it is worth it though.} "R321 <> R322 <> MACRO DO = , [ , ] C3nn (R322) A shall be a local variable of the macro being defined, and shall not be a dummy argument. R322a <> C3nn (R322a) A shall expand to an expression of type integer. R323 <> MACRO END DO A macro DO construct iterates the expansion of its enclosed macro body block at macro expansion time. The number of iterations is determined by the value of the expanded macro expressions in the MACRO DO statement. R324 <> [ ]... [ ] R325 <> MACRO IF ( ) THEN R326 <> MACRO ELSE IF ( ) THEN R327 <> MACRO ELSE R328 <> MACRO END IF R329 <> C3nn (R329) A macro condition shall expand to an expression of type logical. A macro IF construct provides conditional expansion of its enclosed macro body blocks at macro expansion time. Whether the enclosed macro body blocks contribute to the macro expansion is determined by the logical value of the expanded macro expressions in the MACRO IF and MACRO ELSE IF statements. R330 <> [ ]... [ && ] C3nn (R327) The first shall not be MACRO unless the second is not a keyword or name." {Alternatively, "is a left parenthsis, equals sign, or percent sign.} "R331 <> [ %% ]... Constraint: The concatenated textual s in a shall have the form of a lexical token. R332 is any lexical token including labels, keywords, and semi-colon. Constraint: && shall not appear in the last of a macro definition. Constraint: When a macro is expanded, the last processed shall not end with &&. R333 <> END MACRO [ ] Constraint: The in the END MACRO statement shall be the same as the in the DEFINE MACRO statement. R334 <> C3nn (R334) A shall expand to a scalar initialization expression. Macro expressions are used to control the behaviour of the MACRO DO and MACRO IF constructs when a macro is being expanded. The type, type parameters and value of a macro expression are determined when that macro expression is expanded." {NOTE: This is because we (obviously) need to plug macro arguments and macro local variable values into macro expressions. It does seriously limit the amount of checking that can be done at macro definition time.} "3.5.2 Macro expansion" {Note: I have subdivided this section to give it better structure. It is perhaps unfortunate that the division into definition and expansion keeps the BNF away from the syntax of its effects.} "3.5.2.1 General Macro expansion is the conceptual replacement of the EXPAND statement with the Fortran statements that it produces. The semantics of an EXPAND statement are those of the Fortran statements that it produces. It is recommended that a processor be capable of displaying the results of macro expansion. It is processor-dependent whether comments in a macro definition appear in the expansion. It is processor-dependent whether consecutive blanks and continuations are preserved or replaced by a single blank character. The process of macro expansion produces Fortran statements consisting of tokens. The combined length of the tokens for a single statement, plus inter-token spacing, shall not be greater than 256*130 characters. If a statement contains any character that is not of default kind, the maximum number of characters allowed is processor dependent. Note 3.xx This length is so that the result of macro expansion can be formed into valid free form Fortran source, consisting of an initial line and 255 continuation lines, times 130 which allows for beginning and ending continuation characters (&) on each line. Note also that breaking tokens across continuation lines in macro definitions and in EXPAND statements does not affect macro expansion: it is as if they were joined together before replacement. R334 <> EXPAND [ ( ) ] C3nn (R334) shall be the name of a macro that was previously defined or accessed via use or host association. C3nn (R334) The macro shall expand to a sequence or zero or more complete Fortran statements. C3nn (R334) The statements produced by a macro expansion shall conform to the syntax rules and constraints as if they appeared in the program source instead of the EXPAND statement." {I have put this in as a constraint, but it is stated also in clause 2. Perhaps overkill?} "C3nn (R334) The statements produced by a macro expansion shall not include a statement which ends the scoping unit containing the EXPAND statement. C3nn (R334) If a macro expansion produces a statement which begins a new scoping unit, it shall also produce a statement which ends that scoping unit. C3nn (R334) If the EXPAND statement appears as the of an , it shall expand to exactly one that is not an , , , or . \obs{ C3nn (R334) If the EXPAND statement appears as a , it shall expand to exactly one that is not a , a , a , a , an , a , an , an , an , or an . } C3nn (R334) If the EXPAND statement is labelled, the expansion of the macro shall produce at least one statement, and the first statement produced shall not be labelled. C3nn (R334) A shall appear corresponding to each nonoptional macro dummy argument. C3nn (R334) At most one shall appear corresponding to each optional macro dummy argument. Expansion of a macro is performed by the EXPAND statement. If the EXPAND statement is labelled, the label is interpreted after expansion as belonging to the first statement of the expansion." {NOTE: An alternative would be to simply prohibit labelling of the EXPAND statement. (The third alternative, of accepting the label but not associating it with the first statement of the expansion, is horrible to contemplate.)} "R335 <> [ = ] C3nn (R335) shall be the name of a macro dummy argument of the macro being expanded. C3nn (R334) The = shall not be omitted unless it has been omitted from each preceding in the . R336 <> R337 <> <> [ ] [ ] <> R338 is any lexical token except comma, parentheses, array constructor delimiters, and semi-colon. R339 <> ( [ ]... ) <> (/ [ ]... /) <> [ ]... R340 <> <> ," Macro expansion processes any macro declarations of the macro definition, and then expands its macro body block. Any KIND= macro expressions are evaluated and the kind of the macro variables thereby declared is determined for that particular expansion." {NOTE: The specs are silent on how this was meant to be. Another reasonable approach would be to evaluate macro declarations at macro DEFINITION time instead of macro EXPANSION time. For subgroup discussion. It doesn't make much odds for just KIND=, but if we do decide to bind at definition time we could then sensibly have MACRO PARAMETERs to encapsulate stuff from the scope containing the macro definition.} "Macro expansion of a macro body block processes each macro body construct of the macro body block in turn, starting with the first macro body construct and ending with the last macro body construct. Expansion of a statement within a macro body construct consists of three steps: (1) token replacement, (2) token concatenation, and (3) statement-dependent processing. Token replacement replaces each token of a macro body statement or macro expression that is a macro local variable with the value of that variable. In a macro expression, a reference to the PRESENT intrinsic function with a macro dummy argument name as its actual argument is replaced by the token \cf{.TRUE.} if the specified macro dummy argument is present, and the token \cf{.FALSE.} if the specified macro dummy argument is not present. Otherwise, the value of a macro dummy argument that is present is the sequence of tokens from the corresponding actual argument. The value of a macro dummy argument that is not present is a zero-length token sequence. The value of an integer macro variable is its minimal-length decimal representation; if negative this will produce two tokens, a minus sign and an unsigned integer literal constant." {NOTE: This makes "PRESENT" special, in that it operates on macro dummy arguments in the token replacement phase instead of in the expression evaluation phase. The reason for this is so that it will work even when a macro dummy argument is present and expands to a sequence of tokens - PRESENT doesn't accept a sequence of tokens so trying to do this in expression evaluation would be bad. However, this use of PRESENT counts as use of the intrinsic (because that's how the text describes it) and works with USE-renaming, should the user do something silly like USE module,sillyname=>present. An alternative would be to make the form "PRESENT ( )" special, and then it would not count as use of the intrinsic, but keeping to our usual rules seems best.} "Token concatenation is performed with the %% operator, which is only permitted inside a macro definition. After expansion, each sequence of single tokens separated by %% operators is replaced by a single token consisting of the concatenated text of the sequence of tokens. The result of a concatenation shall be a valid Fortran token, and may be a different kind of token from one or more of the original sequence of tokens. Note 3.xx For example, the sequence 3 %% .14159 %% E %% + %% 0 forms the single real literal constant 3.14159E0. 3.5.2.2 Macro body statements Processing a macro body statement produces a whole or partial Fortran statement. A macro body statement that is either the first macro body statement processed by this macro expansion or the next macro body statement processed after a macro body statement that did not end with the continuation generation operator &&, is an initial macro body statement. The next macro body statement processed after a macro body statement that ends with && is a continuation macro body statement. An initial macro body statement that does not end with && produces a whole Fortran statement consisting of its token sequence. All other macro body statements produce partial Fortran statements, and the sequence of tokens starting with those produced by the initial macro body statement and appending the tokens produced by each subsequent continuation macro body statement form a Fortran statement. The && operators are not included in the token sequence. 3.5.2.3 The macro DO construct The macro DO construct specifies the repeated expansion of a macro body block. Processing the macro DO statement performs the following steps in sequence:" {Note: The following insertion is the literal LaTeX source, taken from the c08 DO loop description with modifications.} "\begin{enum} \item The initial parameter $m_1$, the terminal parameter $m_2$, and the incrementation parameter $m_3$ are of type integer with the same kind type parameter as the \si{macro-do-variable-name}. Their values are given by the first \si{macro-expr}, the second \si{macro-expr}, and the third \si{macro-expr} of the \si{macro-do-stmt} respectively, including, if necessary, conversion to the kind type parameter of the \si{macro-do-variable-name} according to the rules for numeric conversion (Table \ref{T:Numeric conversion and the assignment statement}). If the third \si{macro-expr} does not appear, $m_3$ has the value 1. The value of $m_3$ shall not be zero. \item The macro DO variable becomes defined with the value of the initial parameter $m_1$. \item The \tdef{iteration count} is established and is the value of the expression $(m_2-m_1 + m_3) / m_3$, unless that value is negative, in which case the iteration count is 0. \end{enum} After this, the following steps are performed repeatedly until processing of the macro DO construct is finished: \begin{enum} \item The iteration count is tested. If it is zero, the loop terminates and processing of the macro DO construct is finished. \item If the iteration count is nonzero, the macro body block of the macro DO construct is expanded. \item The iteration count is decremented by one. The macro DO variable is incremented by the value of the incrementation parameter $m_3$. \end{enum} 3.5.3.4 The MACRO IF construct The MACRO IF construct provides conditional expansion of macro body blocks. At most one of the macro body blocks of the macro IF construct is expanded. The macro conditions of the construct are evaluated in order until a true value is found or a MACRO ELSE or MACRO END IF statement is encountered. If a true value or a MACRO ELSE statement is found, the macro body block immediately following is expanded and this completes the processing of the construct. If none of the evaluated conditions is true and there is no MACRO ELSE statement, the processing of the construct is completed without expanding any of the macro body blocks within the construct." "3.5.3.5 Macro definitions Processing a macro definition defines a new macro. If a macro definition is produced by a macro expansion, all of the statements of the produced macro definition have token replacement and concatenation applied to them before the new macro is defined. 3.5.2.6 Examples Note 3.nn This is a macro which loops over an array of any rank and processes each array element. DEFINE MACRO loop_over(array,rank,traceinfo) MACRO INTEGER :: i BLOCK MACRO DO i=1,rank INTEGER loop_over_temp_%%i MACRO END DO MACRO DO i=1,rank DO loop_over_temp_%%i=1,size(array,i) MACRO END DO CALL impure_scalar_procedure(array(loop_over_temp_%%1 && MACRO DO i=2,rank ,loop_over_temp%i && MACRO END DO ),traceinfo) MACRO DO i=1,rank END DO MACRO END DO END BLOCK END MACRO Note 3.nn One can effectively pass macro names as macro arguments, since expansion of arguments occurs before analysis of each macro body statement. For example: DEFINE MACRO :: iterator(count,operation) DO i=1,count EXPAND operation(i) END DO END MACRO DEFINE MACRO :: process_element(j) READ *,a(j) result(j) = process(a(j)) IF (j>1) PRINT *,'difference =',result(j)-result(j-1) END MACRO EXPAND iterator(17,zero_element) This expands into 17 sets of 3 statements." {Note: Not a particularly good example. Has anyone a nicer one lying around?} "Note 3.nn Using the ability to evaluate initialization expressions under macro control and test them, one can create interfaces and procedures for all kinds of a type, for example: DEFINE MACRO :: i_square_procs() MACRO INTEGER i MACRO DO i=1,1000 MACRO IF (SELECTED_INT_KIND(i)>=0 .AND. (i==1 .OR. SELECTED_INT_KIND(i)/=SELECTED_INT_KIND(i-1))) THEN FUNCTION i_square_range_%%i(a) RESULT(r) INTEGER(SELECTED_INT_KIND(i)) a,r r = a**2 END FUNCTION MACRO END IF MACRO END DO END MACRO" {Note: OK, not the best example in the world either, but maybe we should keep it?} [86:10] 5.2.1 Accessibility statements, second constraint, Change "or namelist group" to "namelist group, or macro". {Allow macro names in PUBLIC and PRIVATE statements.} [250:9] 2.1, R1104 , Replace "[ ]" with "[ [ ]... ]" on two lines. [251:10] 11.2.1 The USE statement and use association, first paragraph, first sentence, before "and namelist groups" insert "macros,". {Macros shall be accessible via (both) use (and host) association.} NOTE TO THE EDITOR: Immediately before this delete the pointless cross-ref for generic identifiers (see later rant in host assoc.). [406:7] 16.2 Scope of local identifiers, first numbered list, item (1), Before "and statement labels", insert "macros,". {Macro names shall be class 1 names.} [409:16] 16.3 Statement and construct entities, first paragraph, Append new sentence "A macro local variable is a construct entity". [410:14+] 16.3, after the last paragraph, immediately before 16.4 Association, Append new paragraph "The macro local variables of a macro definition have the scope of the macro definition. If a global or local identifier accessible in the scoping unit of a macro definition is the same as a macro dummy argument, the name is interpreted within the macro definition as that of the macro dummy argument. Elsewhere in the scoping unit, the name is interpreted as the global or local identifier." {The macro local variables have the scope of the construct, i.e. they shall be construct entities.} NOTE TO THE EDITOR: Factor out this boilerplate everywhere it occurs, replacing it with a single paragraph stating "If a global or local identifier is the same as a construct entity, the name is interpreted within the construct as that of the construct entity. Elsewhere in the scoping unit, the name is interpreted as the global or local identifier." {This is at [406:2-5] and [406:11-14]. Note that the FORALL statement part of [406:2-5] was already redundant with [405:37-40].} [411:8] 16.4.1.3 Host association, first paragraph, last sentence, before "and namelist groups" insert "macros,". {Macros shall be accessible via (both use and) host association.} NOTE TO THE EDITOR: immediately before this, delete the pointless cross-ref for "generic identifiers". We don't have cross-refs for anything else in the list, and "generic identifiers" appears in the index already. NOTE TO THE EDITOR: The first two sentences of this paragraph are wrong. Host association applies to identifiers, not just names (both these sentences say "named entities"). Fix, perhaps by incorporating the last sentence into the first and then we can just say "accessed entities" throughout the rest. This is basically how we do it for use assoc... [411:29+] 16.4.1.3, numbered list, insert new item near the end "(14a) The name of a macro". {Local macro definitions have to block host association.} [somepage:someline] Annex A, entry for "entity", add macros to the list. {This is helpful, but not necessary. I'll remind people that Annex A is not normative, and any "definition" therein is not a definition, just a helpful hint or reminder of what we mean. As it happens, "entity" is a formal and very general way of saying "thing"... and macros are of course "things" in the most general sense.} 5. Alternative approach to syntax insertion 5.1 Description "Macro expansion shall be a statement", plus a macro shall not end the scope it starts in, plus a macro shall end every scope it starts, means that we have the following cases to consider only: Simple cases: (0) macro generating nothing. It should be ok just to treat this as any one of the next three cases, depending on context. (1) macro generating specifications only, ...treat this is a specification. (2) macro generating executables only. ...treat this as an executable. (3) macro generating whole contained subprograms only. ...treat this as a contained subprogram. The combinations: (4) macro with 1+ specs and 1+ executables. There can be only one. ...just insert this in between the spec-part and the exe-part. (5) macro with a bare CONTAINS. There can be only one. This has the subcases: (5a) 1+ specs, 0+ executables, CONTAINS, 0+ contained subprograms (5b) 0 specs, 0+ executables, CONTAINS, 0+ contained subprograms. ... the only difference is that 5a cannot be preceded by an exe-part nor by a (case 4) macro expansion. ... we can do this either by doubling up the syntax, or by simply stating the restriction in a constraint. Since we need a constraint anyway to draw the distinction between 5a and 5b, I've chosen to do this case by constraint. I think it looks cleaner. Note that inserting macro expansion in this way automatically enforces not only scope nesting but also construct nesting. If we want to allow construct non-nesting, as per the specs, this can be done with this approach but would require constructs to be broken up into opening and closing statements, with nesting requirements expressed by constraint instead of following from the syntax. 5.3.1 Empty internal-subprogram-part and module-subprogram-part. It is simpler to describe things if we allow emptiness after CONTAINS. Furthermore, the existing requirement that the CONTAINS actually be followed by something can be problematic for automatic generation using macros (e.g. when it so happens that nothing is required). This is already J3-010 (completed by 05-196), so I assume this is not controversial. Since there is an interaction here, the edits below will replace those from 05-196, as noted. 5.4 Edits [9:15-17] 2.1 High level syntax, R1101 , Replace "[ ] ... [ ]" with "". {We are going to factor this out from several places.} [9:22-24] 2.1, R1223 , Replace "[ ] ... [ ]" with "". {Factoring.} [9:27-29] 2.1, R1231 , Replace "[ ] ... [ ]" with "". {Factoring.} [9:33] 2.1, R1104 , Replace "[ ]" with "[ [ ]... ]" on two lines. [9:37+] 2.1, Immediately before R204 , insert new productions: " <> [ ] [ ] [ ] [ [ ]... ]" {This will allow a macro to contain both specifications and executables; the usual "specs precede execs" rule means there can be only one.} [10:13+] 2.1, R207, add new production in alphabetic order, "<> ". {This handles macros that produce only specification constructs.} [10:23-25] 2.1, R210 , DELETE and insert new "R210 <> <> " {This handles macros with a non-nested CONTAINS. There is no longer any need for an term.} NOTE TO EDITOR: Subsumes 05-196:[10:24]. [10:27+] 2.1, R211 , add new production "<> ". {This handles macros that contain only whole subprograms.} [10:28-30] 2.1, R1107 , DELETE and insert new "R1107 <> <> " {This handles macros with a non-nested CONTAINS in a module.} NOTE TO EDITOR: Subsumes 05-196:[10-29]. [11:18+] 2.1, R214 , add new production in alphabetic order, "<> ". {This handles macros that contain only executable statements.} [near the end of the 3.5.3 insertion] "3.5.3.6 Macro expansion in context The following syntax rules provide the context within which specific kinds of macro expansions are permitted to appear in the high level syntax. R341 <> C3nn (R341) The macro expansion shall produce only declaration constructs. R342 <> C3nn (R342) The macro expansion shall produce only executable constructs. R343 <> C3nn (R343) The macro expansion shall produce only internal subprograms. R344 <> C3nn (R344) The macro expansion shall produce one or more declaration constructs followed by one or more executable constructs. R345 <> C3nn (R345) The macro expansion shall produce zero or more declaration constructs, followed by zero or more executable constructs, followed by a CONTAINS statement, followed by zero or more internal subprograms. C3nn (R345) A shall not produce any declaration constructs if it is preceded by or . R346 <> C3nn (R346) The macro expansion shall produce zero or more declaration constructs, followed by a CONTAINS statement, followed by zero or more module subprograms. R347 <> C3nn (R347) The macro expansion shall produce zero or more component definition statements." [45:6] 4.5.1 Derived-type definition, BNF R429 , Change "" to "". NOTE: Changed name because the macro with the contains might also have component definitions ahead of the contains. [50:2+] 4.5.3 Components, R439 , append new production "<> ". {Allow EXPAND within a type definition ... this one generates component definitions only.} [56:2-4] 4.5.4 Type-bound procedures, BNF R448 , DELETE R448 and insert instead: "R448 <> [ ] [ ]... <> [ ] [ ]... <> [ ]... [250:13-15] 2.1, R1107 , DELETE and insert new "R1107 <> <> " {This handles macros with a non-nested CONTAINS in a module.} NOTE TO EDITOR: Subsumes 05-196:[250:14]. 5.5. Things we get for free with the alternative "A macro expansion shall be properly nested with respect to scope; that is, it shall not generate a statement which terminates the scope the EXPAND statement appears in." This comes straight out of the grammar with this approach 6. Possibly better examples? (a) use macros to create modules (b) use macros to create types and procedures; and (c) use macros to create inline sections of code conveniently (like an inline procedure call). Perhaps one or more of these would be better than especially the "passing a macro name as an actual argument" one? ===END===