J3/06-174r2 Date: 10 May 2006 To: J3 From: Bill Long and John Reid Subject: Edits for co-arrays References: Feature UK-001, J3/05-208, J3/05-272r2, J3/06-132 ----------------------------------------------------------------------- Begin Preface ============= This paper represents a draft of the edits for feature uk-001, co-arrays. The major changes with respect to 06-122 are: 1) BACKSPACE, REWIND and ENDFILE are not allowed for files connected to more than one image. 2) Added SYNC_TEAM statement, which must be executed by all members of the team. 3) Added an opaque derived type IMAGE_TEAM in ISO_FORTRAN_ENV and a FORM_TEAM collective subroutine that calculates a value of this type for a set of images. The TEAM argument to the collective subroutines changed to be of type IMAGE_TEAM. 4) Require a co-array declared in a recursive procedure to have the SAVE attribute. 5) Co-arrays are not permitted in COMMON or EQUIVALENCE statements. 6) Addition of the intrinsics CO_LBOUND and CO_UBOUND to enquire about the co-bounds of an allocated allocatable co-array. End Preface =========== Note 1. < and > are used to delimit italic font. << and >> are used to delimit bold font. Note 2. All page and line numbers refer to j3/04-007. EDITS ....................................................................... [xiii] Introduction, the list of new Fortran 2008 features should include Parallel programming support: SPMD parallel programming, co-arrays for data exchange between images, image control statements, and collective procedures. ....................................................................... In section 1.6 Compatibility, retain paragraph 3:5-9 regarding new intrinsic procedure names. There are new intrinsics introduced in this proposal. ....................................................................... [11:1-] Add to R213 <> ....................................................................... [11: 6+] In R214 , add in alphabetic order: <> <> <> <> <> <> ....................................................................... [13:17++] Add new section before '2.3 Execution concepts' 2.2a Images A Fortran program executes as if it were replicated a number of times (which may be one), the number of replications remaining fixed during execution of the program. Each copy is called an <> and has its own set of data objects. The images execute asynchronously. The programmer controls the progress of execution in each image through explicit use of Fortran control constructs. The image control statements (8.5.1) affect the relative progress of execution between images. Co-arrays provide a mechanism for accessing data on one image from another image. Each image is identified by an <>, which is an integer value in the range one to the number of images. NOTE 2.2a Processors provide the means to execute, on a single image, programs that do not use co-arrays or image control statements (8.5.1). [end NOTE] NOTE 2.2b A processor might allow the number of images to be chosen at compile time, link time, or run time. It might be the same as the number of CPUs but this is not required. Compiling for a single image might permit the optimizer to eliminate overhead associated with parallel execution. Portable programs should not make assumptions about the exact number of images. The maximum number of images may be limited due to architectural constraints. [end NOTE] A <> is a set of images formed by invoking the intrinsic collective subroutine FORM_TEAM (13.7.25l) for the purposes of collaboration. A team is identified by a scalar variable of type IMAGE_TEAM (13.8.2). ....................................................................... [15:19+] After the second paragraph of 2.3.4 Execution sequence, add Coordination of the execution sequences of different images is discussed in 8.5. ....................................................................... [15:20] Replace the first sentence of the last paragraph of 2.3.4 by Normal termination of execution of the program occurs on all images if a STOP statement or is executed immediately after a SYNC_ALL (8.5.2) statement on all images. The stop code, if any, and warnings of any exceptions that are signaling (8.4) shall be made available only for the image 1. ....................................................................... [15:25+] Add paragraph and note at the end of 2.3.4 When a STOP or that does not immediately follow a SYNC_ALL statement is executed in an image, normal termination occurs on that image. The stop code, if any, and warnings of any exceptions that are signaling (8.4) shall be made available only for that image. The executions of all other images are terminated. NOTE 2.2c When a STOP or statement is executed on an image, how soon termination takes place on another image is processor-dependent. [end NOTE] ....................................................................... [18:20+] Add new subsection after 2.4.5 2.4.5a Co-array A <> is a data entity that is declared with nonzero co-rank and may be referenced or defined by any image. It may be a scalar or an array. For each co-array on an image, there is a corresponding co-array with the same type, type parameters, and bounds on every other image. The co-array on any image may be addressed by using s. On its own image, a co-array may be accessed without use of co-subscripts. The <> of a co-array is the number of co-dimensions. The <> are given within square brackets in the co-array's declaration or allocation. The number of co-subscripts shall be equal to the co-rank. If the co-rank is , the co-array has lower co-bounds , , upper co-bounds , , and co-extents , . The <> of a co-array is always equal to the number of images. An object whose designator includes an is a <>. For each co-array, co-subscripts are mapped to image indices in the same way as Fortran array subscripts are mapped to the position of the array element in array element order (6.2.2.2). THIS_IMAGE(A) returns the co-subscript values for a co-array A that corresponds to the invoking image. IMAGE_INDEX(A,SUB) returns the image index that corresponds to co-subscript values SUB for a co-array A. NOTE 2.4a For example, consider the co-array declared as REAL :: A(10,20)[10,0:9,0:*] On image 5, THIS_IMAGE() has the value 5 and THIS_IMAGE(A) has the value [5,0,0]. For the same co-array on image 213, THIS_IMAGE(A) has the value [3,1,2] and IMAGE_INDEX(A,[3,1,2]) has the value 213. [end NOTE] NOTE 2.4b The mechanism for an image to reference and define a co-array on another image might vary according to the hardware. On a shared-memory machine, a co-array could be implemented as a section of an array of higher rank. On a distributed-memory machine with separate physical memory for each image, an implementation might store a co-array at the same virtual address in each physical processor. [end NOTE] ....................................................................... [19:5] In the last line of the first paragraph of 2.5.1, after 'array element selectors,' add 'image selectors,'. ....................................................................... [28:4+] Insert 'END CRITICAL' in the table in 3.3.1 ....................................................................... [50:6] Replace the second line of R441 by <> ....................................................................... [50:9+] After the first line of R442, add [] ....................................................................... [50:10+] After R442, add R442a <> DIMENSION () <> DIMENSION [()] ....................................................................... [50:25+] After C440, add C440a (R440) If a appears, it shall be a and the component shall be specified with the ALLOCATABLE attribute. C440b (R440) If the type specified has a co-array subcomponent, none of the components in the shall be an array or co-array. C440c (R440) The ALLOCATABLE or POINTER attribute shall not appear if the type specified has a co-array subcomponent. ....................................................................... [51:16-20] Replace the first paragraph of 4.5.3.1 by: In a that does not contain a , each data component is an array if and only if its contains a and it is a co-array if and only if its contains a . The specifies the rank and if it is an , it specifies the array bounds. The specifies the co-rank. A that contains a is interpreted as if the were removed and its specifications were attached to each data component that has neither a nor a . ....................................................................... [67:8,11,12] In R465, R467, R468, change to and to . ....................................................................... [71:21] In R503, replace the DIMENSION line by <> ....................................................................... [72:9] Separate R504 into two lines after '[()]' and insert line: [] ....................................................................... [72:10+] After R504, add R504a <> DIMENSION () <> DIMENSION [()] ....................................................................... [72:25+] After C510, add C510a (R501) A co-array that has the ALLOCATABLE attribute shall be specified with a that is a . C510b (R501) If the type specified has a co-array subcomponent, none of the entities in the shall be an array or a co-array. C510c (R501) If the type specified has a co-array subcomponent, none of the entities in the shall have the POINTER or ALLOCATABLE attribute. ....................................................................... [72:33] In the second line of C514, after 'an allocatable entity,' add 'a co-array'. ....................................................................... [73:1+] After C517, add C517a (R501) The SAVE attribute shall be specified for a co-array unless it is a dummy argument, declared in the main program, or allocatable and declared in a non-recursive procedure. NOTE 5.2a An allocatable co-array is required to have the SAVE attribute in a recursive procedure because of the difficulties associated with defining the separate levels of recursion. Without the SAVE attribute, each would have a separate instance of the co-array and implicit synchronization would be required for each allocation and deallocation of each instance. Automatic co-arrays are not permitted; for example, the co-array WORK in the following code fragment is not valid: SUBROUTINE SOLVE3(N,A,B) INTEGER :: N REAL :: A(N)[*], B(N) REAL :: WORK(N)[*] ! Not permitted If this were permitted, it would require an implicit synchronization on entry to and exit from the procedure. Note that local allocatable co-arrays are available. Explicit shape co-arrays that are not dummy arguments are required to have the SAVE attribute because otherwise they might be implemented as if they were automatic co-arrays. [end NOTE] C517a (R501) A co-array shall not have the POINTER attribute. C517b (R501) If the type specified has a co-array subcomponent, each entity in the shall have the SAVE attribute unless it is a dummy argument, declared in the main program, or declared in a non-recursive procedure. NOTE 5.2b A co-array is permitted to be of a derived type with pointer or allocatable components. The targets of such components are always on the same image (6.3.1). If an array is needed on a subset of images and is specified as an allocatable component of a co-array, it need be allocated only on the images on which it is needed. Malcolm replacementr: An allocatable component of a co-array need be allocated only on those images where it is referenced or defined. [end NOTE] ....................................................................... [74:20-] After NOTE 5.3, add: NOTE 5.3a Unless it is a dummy argument, a co-array has the same bounds and co-bounds on every image. NOTE 12.21b contains an example of a dummy argument co-array that might have bounds and co-bounds that differ between images. [end NOTE] ....................................................................... [78:3-9] Replace the first paragraph of 5.1.2.5 by The <> specifies that an entity is an array, a co-array, or both. The appearance of an or a in an specifies the DIMENSION attribute for the entity. Such an entity is an array if an appears and is a co-array if a appears. The DIMENSION attribute alternatively may be specified for an array that is not a co-array in the specification statements DIMENSION, ALLOCATABLE, POINTER, TARGET, or COMMON. The DIMENSION attribute alternatively may be specified for a co-array in the specification statements DIMENSION, ALLOCATABLE, or TARGET. For an array, the rank or rank and shape are specified by the in its DIMENSION attribute. For a co-array, the co-rank or co-rank and co-bounds are specified by the in its DIMENSION attribute. ....................................................................... ***Note to Editor: This edit replaces the one for this line from 05-234r2 [78:14] Replace C541 by R510a <> <> C541 (R510 and R510a) The sum of the rank and co-rank shall not exceed fifteen. ....................................................................... [85:6] Add a new sentence at the end of the first paragraph of 5.1.2.16: A co-array variable with the VOLATILE attribute might be modified by another image without synchronization (8.5). ....................................................................... [86:21-22] Replace R520 by R520 <> ALLOCATABLE [::] R520a <> [()] [] ....................................................................... [87:34+] After R529, add C552a (R525). A shall not contain any image selectors (6.2a). ....................................................................... [90:2-5] Replace the contents of 5.2.6 by R535 <> DIMENSION [::] R535a <> () <> [()] This statement specifies the DIMENSION attribute (5.1.2.5) and the array and co-array properties for each object named. ....................................................................... [92:4-5] Replace R546 by R546 <> TARGET [::] R546a <> [()] [] ....................................................................... [95:28+] Before 5.5.1, add NOTE 5.36a For efficiency, it is expected that processors will hold co-arrays in an area of memory that is distinct from that used for other data. Co-arrays are therefore not allowed in COMMON and EQUIVALENCE statements. A revision of the standard could allow co-arrays in COMMON and EQUIVALENCE statements and add restrictions to keep them separate, but it would not be possible to check all such restrictions during compilation. [end NOTE] ....................................................................... [96:12+] After C577, add C577a (R556). An shall not be a co-array, an element of a co-array, a substring of a co-array, or a substring of an element of a co-array. ....................................................................... [98:22] In C588, change ",or a result name." to ", a result name, or a co-array." ....................................................................... [105:2] Append to the end of R613 '[]' ....................................................................... [105:9+] After C613, add C613a (R613). If appears and is an array, () shall appear. C613b (R613). A that contains an and is of a derived type that has a subcomponent that is a pointer or of derived type IMAGE_TEAM (13.8.2.x), C_PTR, or C_FUNPTR (15.2.2) shall not appear as a variable in a variable definition context (16.5.7). NOTE 6.3a This restriction is needed to avoid a disallowed pointer assignment for a component, such as Z[P] = Z ! Not allowed if Z has a pointer component Z = Z[P] ! Not allowed if Z has a pointer component Such pointer assignments would associate a pointer with a target that is on another image. [end NOTE] ....................................................................... [110:2-] Add new section before 6.3 Dynamic association. 6.2a Image selectors An <> specifies the image index for co-array data. R622a <> R622b <> The value of a co-subscript in an image selector shall be within the bounds for its co-dimension. An image selector shall specify an image index in the range 1, ..., NUM_IMAGES(). NOTE 6.15a For example, if there are 16 images and the co-array A is declared REAL :: A(10)[5,*] A(:)[1,4] is valid because it specifies image 16, but A(:)[2,4] is invalid because it specifies image 17. [end NOTE] ....................................................................... [110:20] In R628, after '[()]' add the new line '[]'. ....................................................................... [111:5+] After R632, add 'R632a <> [,] [:]* '. ....................................................................... [111:17] Add at the end of C628 'An shall appear if and only if the is a co-array.' ....................................................................... [111:19] Add at the end of C629 'The number of s in an shall be one less than the co-rank of the .' ....................................................................... [111:25+] After C633, add C633a (R629) An shall not be a co-indexed object. NOTE 6.15b If a co-array is of a derived type that has an allocatable component, the component shall be allocated by its own image: TYPE(SOMETHING), ALLOCATABLE :: T[:] ... ALLOCATE(T[*]) ! Allowed - implies synchronization ALLOCATE(T%PTR(N)) ! Allowed - allocated by its own image ALLOCATE(T[Q]%PTR(N)) ! Not allowed, because it is not ! necessarily executed on image Q. [end NOTE] ....................................................................... ***Note to the Editor: This new sentence is appended to the replacement paragraph [111:32-36] in 06-137. [111:36] At the end of the third paragraph after C633, add 'The dynamic type shall not have a co-array subcomponent.' ....................................................................... [113:21+] At the end of 6.3.1.1, Allocation of allocatable variables, add The value of each and each in an shall be the same on each image. If an specifies a co-array, its dynamic type and the values of each length type parameter shall be the same on each image. There is implicit synchronization of all images in association with each ALLOCATE statement that involves one or more co-arrays. On each image, execution of the segment (8.5.1) following the statement is delayed until all other images have executed the same statement the same number of times. The segments executed before the statement on any image precede the segments executed after the statement on any image. NOTE 6.19a When an image executes an ALLOCATE statement, communication is not necessarily involved apart from any required for synchronization. The image allocates its co-array and records how the corresponding co-arrays on other images are to be addressed. The processor is not required to detect violations of the rule that the bounds are the same on all images, nor is it responsible for detecting or resolving deadlock problems (such as two images waiting on different ALLOCATE statements). [end NOTE] ....................................................................... [116:19-] At the end of 6.3.3.1, Deallocation of allocatable variables, add There is implicit synchronization of all images in association with each DEALLOCATE statement that involves one or more co-arrays. On each image, execution of the segment (8.5.1) following the statement is delayed until all other images have executed the same statement the same number of times. The segments executed before the statement on any image precede the segments executed after the statement on any image. There is also an implicit synchronization of all images in association with the execution of a RETURN or END statement that causes the deallocation of a co-array or a subcomponent that is a co-array. ***Note to the Editor: In view of the edit [115:8+] in paper 06-142, the second paragraph might need to be replaced by: There is also an implicit synchronization of all images in association with the deallocation of a co-array or co-array subcomponent caused by the execution of a RETURN or END statement or the termination of a BLOCK construct. ....................................................................... [129:10+] Add a new paragraph after note 7.15 If a statement contains a function reference in a part of an expression that need not be evaluated, no invocation of that function in that part of the expression shall execute an image control statement other than CRITICAL or END CRITICAL. NOTE 7.15a Violation of this restriction might cause deadlock to occur. For example, consider the case of a function that allocates a temporary co-array, but is not invoked on all images. [end NOTE] ....................................................................... [130:1-] Add a new paragraph after note 7.16 During an execution of a statement that contains more than one reference to a procedure, image control statements other than CRITICAL or END CRITICAL shall be executed in at most one of the procedures invoked from the statement. ....................................................................... [138:20] In item (2) of the numbered list in 7.4.1.2, change "array of" to "array that is not a co-array or a co-indexed object and has". ....................................................................... ***Note to editor: Replace the edit for [139:1] in 05-014r1 with the following edit. The change from 05-278r2 has been incorporated. [139:1] 7.4.1.2 Intrinsic assignment statement, numbered list, beginning of item (3), replace "The" by "If the assigned variable is polymorphic it shall be type compatible with and have the same rank. If the assigned variable is polymorphic and is an allocatable co-array or a co-indexed object, its dynamic type shall be that of . Otherwise the". ....................................................................... [139:22&24] In lines 1 and 3 the third paragraph of 7.4.1.3, add "that is not a co-array" after "allocatable variable". ....................................................................... [141:19-22] In item (1) of the numbered list that follows note 7.39, add "and is not a co-array" after "is allocated". In item (2), add "and is not a co-array" after the first "is allocated". Make the final sentence of item (2) into a new item (3) and delete "then" from its first line. ....................................................................... [143:31+] After C723, add C723a (R739) A shall not be a co-indexed object. NOTE 7.42a A pointer and its target are on the same image. A co-array may be of a derived type with pointer or allocatable subcomponents. For example, if PTR is a pointer component, Z[P]%PTR is a reference to the target of component PTR of Z on image P. This target is on image P and its association with Z[P]%PTR must have been established by the execution of an allocate statement or a pointer assignment on image P. [end NOTE] ....................................................................... [155:2] In the first sentence of 8: Execution control, after the first 'The execution sequence' add 'of a single image'. ....................................................................... [155:3] After the first sentence of 8: Execution control, add a new sentence: 'This is the subject of 8.1, 8.2, 8.3, and 8.4; image synchronization for exchange of data between images is the subject of 8.5.' ....................................................................... [155:7+] To the numbered list of executable constructs at the start of 8.1 Executable constructs containing blocks, insert new list item in alphabetical order (2a) CRITICAL construct ....................................................................... ***Note to editor: This edit should be applied following the edit for the BLOCK construct at [169:1-] in 06-142 [169:1-] After the end of 8.1.6: DO construct and before the beginning of 8.2 Branching, and a new subsection: 8.1.8 CRITICAL construct A <> limits execution of a block to one image at a time. R844a <> R844b <> [:] CRITICAL R844c <> END CRITICAL [] C829a (R844a) If the of a specifies a , the corresponding shall specify the same . If the of a does not specify a , the corresponding shall not specify a . C829b (R844a) The of a shall not contain an image control statement. A critical construct completes execution if the END CRITICAL statement is executed or if control is transferred to a statement outside the block. The processor shall ensure that once an image has commenced executing , no other image shall commence executing it until the image has completed executing it. If image T is the next to execute the construct after image M, the segments (8.5.1) that executed before the construct on image M precede the segments that execute after the construct on image T. No image control statement shall be executed during the execution of a critical construct by the image executing the critical construct. NOTE 8.19a If more than one image executes the block of a critical construct, its execution by one image always either precedes or succeeds its execution by another image. Typically no other statement ordering is needed. Consider the following example: CRITICAL global_counter[1] = global_counter[1] + 1 END CRITICAL The definition of global_counter[1] by a particular image will always precede the reference to the same variable by the next image to execute the block. [end NOTE] NOTE 8.19b The following example permits a large number of jobs to be shared among the images: INTEGER :: NUM_JOBS[*], JOB IF (THIS_IMAGE() == 1) READ(*,*) NUM_JOBS SYNC_ALL DO CRITICAL JOB = NUM_JOBS[1] NUM_JOBS[1] = JOB - 1 END CRITICAL IF (JOB > 0) THEN ! Work on JOB ELSE EXIT END IF END DO SYNC_ALL [end NOTE] ....................................................................... [170:25-26] Replace the first sentence, beginning "Execution of a STOP statement" of 8.4 STOP statement with: The processor shall ensure that once an image has commenced executing a STOP or an statement, no other image shall commence executing such a statement. It causes normal termination (2.3.4) of execution of the program on all images. ....................................................................... ***Note to Editor: This edit should be applied following the edit for [170:29+] from 05-231r4 ***Note to Editor: In the sixth paragraph of this edit, the "default bits" text assumes the integration of the edits for the bits data type. [170:29+] After 8.4 STOP statement, add new section: 8.5 Image execution control 8.5.1 Image control statements The execution sequence on each image is as specified in 2.3.4. An <> statement affects the execution ordering between images. Each of the following is an image control statement: (1) SYNC_ALL statement, (2) SYNC_TEAM statement, (3) SYNC_IMAGES statement, (4) SYNC_MEMORY statement, (5) NOTIFY statement, (6) QUERY statement, (7) ALLOCATE or DEALLOCATE statement involving a co-array, (8) CRITICAL or END CRITICAL statement (8.1.8), (9) OPEN statement with a TEAM= specifier, (10) CLOSE statement for a file that is open with a TEAM= specifier, (11) END, END BLOCK, or RETURN statement that involves an implicit deallocation of a co-array, (12) CALL statement for a collective subroutine (13.1), and On each image, the sequence of statements executed before the first image control statement, between the execution of two image control statements, or after the last image control statement is known as a <>. The segment executed immediately before the execution of an image control statement includes the evaluation of all expressions within the statement. By execution of image control statements or user-defined ordering (8.5.6), the program can ensure that the execution of the i-th segment on image P, S(i,P), either precedes or succeeds the execution of the j-th segment on another image Q, S(j,Q). If the program does not ensure this, segments S(i,P) and S(j,Q) are <>; depending on the relative execution speeds of the images, some or all of the execution of the segment S(i,P) may take place at the same time as some or all of the execution of the segment S(j,Q). NOTE 8.22 The set of all segments on all images is partially ordered: the segment S(i,P) precedes segment S(j,Q) if and only if there is a sequence of segments starting with S(i,P) and ending with S(j,Q) such that each segment of the sequence precedes the next either because they are on the same image or because of the execution of image control statements or user-defined ordering. [end NOTE] A scalar co-variable that is of type default integer, default logical, default real, or default bits, and has the VOLATILE attribute (5.1.2.16) may be referenced during the execution of a segment that is unordered relative to the execution of a segment in which the co-variable is defined. Otherwise, (1) if a co-array variable is defined on an image in a segment, it shall not be referenced or defined in a segment on another image unless the segments are ordered, (2) if the allocation of an allocatable subobject of a co-array or the association of a pointer subobject of a co-array is changed on an image in a segment, that subobject shall not be referenced or defined in a segment on another image unless the segments are ordered, and (3) if a procedure invocation on image P is in execution in segments S(i,P), S(i+1,P), ..., S(k,P) and defines a non-co-array dummy argument, the corresponding actual argument shall not be referenced or defined on another image Q in a segment S(j,Q) unless S(j,Q) precedes S(i,P) or succeeds S(k,P). NOTE 8.22a Apart from the effects of volatile variables, the processor may optimize the execution of a segment as if it were the only image in execution. [end NOTE] NOTE 8.22b The model upon which the interpretation of a program is based is that there is a permanent memory location for each co-array variable and that all images can access it. In practice, an image may make a copy of a non-volatile co-array variable (in cache or a register, for example) and, as an optimization, defer copying a changed value back to the permanent location while it is still being used. Since the variable is not volatile, it is safe to defer this transfer until the end of the current segment and thereafter to reload from permanent memory any co-array variable that was not defined within the segment. It would not be safe to defer these actions beyond the end of the current segment since another image might reference the variable then. [end NOTE] If an image P writes a record during the execution of segment S(i,P) to a file that is opened for direct access with a TEAM= specifier, no other image Q shall read or write the record during execution of a segment that is unordered with S(i,P). Furthermore, it shall not read the record in a segment that succeeds S(i,P) unless (1) after image P writes the record, it executes a FLUSH statement (9.8) for the file during the execution of a segment S(k,P), and (2) before image Q reads the record, it executes a FLUSH statement for the file during the execution of a segment S(j,Q) that succeeds S(k,P). 8.5.2 SYNC_ALL statement R851 <> SYNC_ALL [([])] R852 <> STAT = <> ERRMSG = C835 (R851) No specifier shall appear more than once in a given . C836 (R852) A scalar variable in a STAT= specifier or ERRMSG= specifier shall not contain an (6.2a). The STAT= and ERRMSG= specifiers are described in 8.5.7. Execution of a performs a synchronization of all images. Execution on an image, M, of the segment following the is delayed until each other image has executed a as many times as has image M. The segments that executed before the on an image precede the segments that execute after the on another image. NOTE 8.23 The processor might have special hardware or employ an optimized algorithm to make the execute efficiently. Here is a simple example of its use. Image 1 reads data and broadcasts it to other images: REAL :: P[*] ... SYNC_ALL IF (THIS_IMAGE()==1) THEN READ (*,*) P DO I = 2, NUM_IMAGES() P[I] = P END DO END IF SYNC_ALL [end NOTE] NOTE 8.23a If synchronization is required when the images commence statement execution, a SYNC_ALL statement should be the first executable statement of the main program. This is necessary if the code relies on the initialization of a co-array variable on another image. [end NOTE] 8.5.3 SYNC_TEAM statement R853 <> SYNC_TEAM ( [,]) R854 <> C837 (R853) No specifier shall appear more than once in a given . C838 (R854) shall be a scalar variable of the type IMAGE_TEAM defined in the intrinsic module ISO_FORTRAN_ENV. Execution of a performs a <>, which is a synchronization of the images in a team. The team is specified by the value of and shall include the executing image. All images of the team shall execute a with a value of that was constructed by invoking FORM_TEAM for the team. They do not commence executing subsequent statements until all images in the team have executed a statement for the team an equal number of times since FORM_TEAM was invoked for the team. If images M and T are any two members of the team, the segments that execute before the statement on image M precede the segments that execute after the statement on image T. Team synchronization is also performed by an OPEN statement with a TEAM= specifier, a CLOSE statement for a file that is open with a TEAM= specifier, or the invocation of a collective subroutine. NOTE 8.24 Here is an example where the images are divided into two teams, one for an ocean calculation and one for an atmosphere calculation. USE, INTRINSIC :: ISO_FORTRAN_ENV TYPE(IMAGE_TEAM) :: TEAM INTEGER :: N2, STEP, NSTEPS LOGICAL :: OCEAN N2 = NUM_IMAGES()/2 OCEAN = (THIS_IMAGE()<=N2) IF(OCEAN) THEN CALL FORM_TEAM(TEAM, (/ (I, I=1,N2) /) ) ELSE CALL FORM_TEAM(TEAM, (/ (I, I=N2+1,NUM_IMAGES()) /) ) END IF : ! Initial calculation SYNC_ALL DO STEP = 1, NSTEPS IF (OCEAN) THEN DO : ! Ocean calculation SYNC_TEAM(TEAM) IF ( ... ) EXIT ! Ready to swap data END DO ELSE DO : ! Atmosphere calculation SYNC_TEAM(TEAM) IF ( ... ) EXIT ! Ready to swap data END DO END IF SYNC_ALL : ! Swap data END DO In the inner loops, each set of images first works entirely with its own data and each image synchronizes with the rest of its team. The number of synchronizations for the ocean team might differ from the number for the atmosphere team. The SYNC_ALL statement that follows is needed to ensure that both teams have done their calculations before data are swapped. [end NOTE] 8.5.4 SYNC_IMAGES statement R855 <> SYNC_IMAGES ( [,]) R856 <> <> * C839 (R855) No specifier shall appear more than once in a given . C840 (R856) An that is an shall be scalar or of rank one and shall not contain an (6.2a). If is an array expression, the value of each element shall be in the range 1, ..., NUM_IMAGES(), and there shall be no repeated values. If is a scalar expression, its value shall be in the range 1, ..., NUM_IMAGES(). An that is an asterisk specifies all images. Execution of a performs a synchronization of the image with each of the other images in the . Execution on an image, M, of the segment following the is delayed until all other images T in the have executed a with M in its as many times as image M has executed a with T in the . The segments that executed before the on image M precede the segments that execute after the on image T. NOTE 8.25 A SYNC_IMAGES statement that specifies the single image value THIS_IMAGE() in its image set is allowed. This simplifies writing programs for an arbitrary number of images by allowing correct execution in the limiting case of the number of images being equal to one. [end NOTE] NOTE 8.25a Execution of a SYNC_IMAGES(*) statement is not equivalent to the execution of a SYNC_ALL statement. SYNC_ALL causes all images to wait for each other. SYNC_IMAGES statements are not required to specify the same image set on all the images participating in the synchronization. In the following example, image 1 will wait for each of the other images to complete its use of the data. The other images wait for image 1 to set up the data, but do not wait on any of the other images. IF (THIS_IMAGE() == 1) then ! Set up co-array data needed by all other images SYNC_IMAGES(*) ELSE SYNC_IMAGES(1) ! Use the data set up by image 1 END IF [end NOTE] NOTE 8.25b Execution of a SYNC_TEAM statement causes all the images of the team to wait for each other. There might, however, be situations where this is not efficient. In the following example, each image synchronizes with its neighbor. INTEGER :: ME, NE, STEP, NSTEPS NE = NUM_IMAGES() ME = THIS_IMAGE() ! Initial calculation SYNC_ALL DO STEP = 1, NSTEPS IF (ME > 1) SYNC_IMAGES(ME-1) ! Perform calculation IF (ME < NE) SYNC_IMAGES(ME+1) END DO SYNC_ALL The calculation starts on image 1 since all the others will be waiting on SYNC_IMAGES(ME-1). When this is done, image 2 can start and image 1 can perform its second calculation. This continues until they are all executing different steps at the same time. Eventually, image 1 will finish and then the others will finish one by one. The SYNC_IMAGES syntax involves rather than to allow the set of images to vary from image to image. [end NOTE] 8.5.5 NOTIFY and QUERY statements R857 <> NOTIFY ( [,]) R858 <> QUERY ( [,]) R859 <> READY = <> C841 (R857) No specifier shall appear more than once in a given . C842 (R858) No specifier shall appear more than once in a given . Execution on image M of a NOTIFY statement with a different image T in its increments by 1 a record of the number of times, n(M,T), image M executed such a NOTIFY statement. The execution of a QUERY statement on an image is called <> if the QUERY statement has no READY= specifier or if it has a READY= specifier whose value after the statement has been executed is true. Execution on image M of a blocking QUERY statement with a different image T included in its image set increases by 1 a record of the number of times, b(M,T), image M executed such a QUERY statement. This increase is made after its value has been compared with n(T,M). If a READY= specifier appears, execution on image M of a QUERY statement causes the to become defined. The value is false if, for a different image T in the image set, n(T,M) <= b(M,T). Otherwise, the value is true. If a READY= specifier does not appear, increasing b(M,T) and completing execution of the statement is delayed until, for each different T in the image set, n(T,M) > b(M,T). For a blocking QUERY statement on image M with a different image T in its image set, the NOTIFY statement execution on image T for which n(T,M) = b(M,T) + 1 is said to correspond to it. Segments on image T executed before the execution of this NOTIFY statement precede the executions of the segments on image M that follow this QUERY statement. NOTE 8.26 The NOTIFY and QUERY statements can be used to order statement executions between a producer and consumer image. INTEGER,PARAMETER :: PRODUCER = 1, CONSUMER = 2 INTEGER :: VALUE[*] LOGICAL :: READY SELECT CASE (THIS_IMAGE()) CASE(PRODUCER) VALUE[CONSUMER] = 3 NOTIFY(CONSUMER) CASE(CONSUMER) WaitLoop: DO QUERY(PRODUCER,READY=READY) IF(READY) EXIT WaitLoop ! Statements not referencing VALUE[CONSUMER], or causing it to ! become defined or undefined END DO WaitLoop ! references to VALUE CASE DEFAULT ! Statements not referencing VALUE[CONSUMER], or causing it to ! become defined or undefined END SELECT Unlike SYNC_IMAGES statements, the number of notifications and corresponding blocking queries may be unequal. A program can complete with an excess number of notifies. [end NOTE] NOTE 8.27 NOTIFY/QUERY pairs can be used in place of SYNC_ALL and SYNC_IMAGES to achieve better load balancing and allow one image to proceed with calculations while another image is catching up. Here is an example: IF (THIS_IMAGE()==1) THEN DO I=1,100 ! Primary processing of column I NOTIFY(2) ! Done with column I END DO SYNC_IMAGES(2) ELSE IF (THIS_IMAGE()==2) THEN DO I=1,100 QUERY(1) ! Wait until image 1 is done with column I ! Secondary processing of column I END DO SYNC_IMAGES(1) END IF [end NOTE] NOTE 8.28 The incorrect sequencing of image control statements can halt execution indefinitely. For example, one image might be executing a while another is executing an allocate statement for a co-array; or one image might be executing a blocking QUERY statement for which an image in its image set never executes the corresponding NOTIFY statement. [end NOTE] 8.5.6 SYNC_MEMORY statement The SYNC_MEMORY statement provides a means of dividing a segment on an image into two segments, each of which can be ordered in some user-defined way with respect to segments on other images. R860 <> SYNC_MEMORY [([])] C843 (R860) No specifier shall appear more than once in a given . NOTE 8.29 SYNC_MEMORY usually suppresses compiler optimizations that might reorder memory operations across the segment boundary defined by the SYNC_MEMORY statement and ensures that all memory operations initiated in the preceding segments in its image complete before any memory operations in the succeeding segment in its image are initiated. It needs to do this unless it can establish that failure to do so could not alter processing on another image. All of the other image control statements include the effect of executing a SYNC_MEMORY statement. In addition, the other image control statements cause some form of cooperation with other images for the purpose of ordering execution between images. SYNC_MEMORY in combination with user-written code that forces synchronization between images can provide an effective and flexible alternative to the image control statements discussed in 8.5.2-8.5.5. A common example of user-written code that can be used in conjunction with SYNC_MEMORY to implement specialized schemes for segment ordering is the spin-wait loop. For example: LOGICAL,VOLATILE :: LOCKED[*] = .TRUE. INTEGER :: IAM, P, Q IAM = THIS_IMAGE() IF (IAM == P) THEN ! Preceding segment SYNC_MEMORY ! A LOCKED[Q] = .FALSE. ! segment S(i,P) SYNC_MEMORY ! B ELSE IF (IAM == Q) THEN DO WHILE (LOCKED); END DO ! segment S(j,Q) SYNC_MEMORY ! C ! Following segment END IF Here, image Q does not complete the segment S(j,Q) until image P executes segment S(i,P). This ensures that executions of segments before S(i,P) on image P precede executions of segments on image Q after S(j,Q). The first SYNC_MEMORY statement (A) ensures that the compiler does not reorder the following statement (locking) with the previous statements, since the lock should be freed only after the work has been completed. The definition of LOCKED[Q] might be deferred to the end of segment S(i,P). The second SYNC_MEMORY statement (B) ends that segment immediately after the definition, minimizing any delay in releasing the lock in segment S(j,Q). The third SYNC_MEMORY statement (C) marks the beginning of a new segment, informing the compiler that the values of co-array variables referenced in that segment might have been changed by other images in preceding segments, so need to be loaded from memory. As a second example, the user might have access to an external procedure that performs synchronization between images. That library procedure will not necessarily be aware of the mechanisms used by the Fortran implementation to manage remote data references and definitions, and therefore not, by itself, be able to ensure the correct memory state before and after its reference. The SYNC_MEMORY statement provides the needed memory ordering that enables the safe use of the external synchronization routine. For example: INTEGER :: IAM REAL :: X[*] IAM = THIS_IMAGE() IF (IAM == 1) X = 1.0 SYNC_MEMORY CALL EXTERNAL_SYNC() SYNC_MEMORY IF (IAM == 2) WRITE(*,*) X[1] where executing the subroutine EXTERNAL_SYNC has an image synchronization effect similar to executing a SYNC_ALL statement. [end NOTE] 8.5.7 STAT= and ERRMSG= specifiers in image execution control statements If the STAT= specifier appears, successful execution of the SYNC_ALL, SYNC_TEAM, SYNC_IMAGES, SYNC_MEMORY, NOTIFY, or QUERY statement causes the to become defined with the value zero. If an error occurs during execution of one of these statement, the becomes defined with a processor-dependent positive integer value. If an error condition occurs during execution of a SYNC_ALL, SYNC_TEAM, SYNC_IMAGES, SYNC_MEMORY, NOTIFY, or QUERY statement that does not contain the STAT= specifier, the effect is processor dependent. If the ERRMSG= specifier appears and an error condition occurs during execution of the SYNC_ALL, SYNC_TEAM, SYNC_IMAGES, SYNC_MEMORY, NOTIFY, or QUERY statement, the processor shall assign an explanatory message to . If no such condition occurs, the processor shall not change the value of . NOTE 8.30 Which errors, if any, are diagnosed is processor dependent. The processor might check that a valid set of images has been provided, with no out-of-range or repeated values. It might test for network time-outs. While the overall program would probably not be able to recover from the failure of an image, it could perhaps provide information on what failed and be able to save some of the program state to a file. [end NOTE] ....................................................................... [172:26+] Before the final paragraph of 9.2 External files, add NOTE 9.3a Named files that are opened with the TEAM= specifier (9.4.5.16) have the same name on each image of the team. Apart from this, whether a named file on one image is the same as a file with the same name on another image is processor dependent. For code portability, if different files are needed on each image, different file names should be used. A common technique is to incorporate the image number as part of the name. [end NOTE] ....................................................................... [173:29] At the end of 9.2.2.1 Sequential access, in the third list item, after 'statements.', add : Each record shall be read or written by a single image. The processor shall ensure that once an image commences transferring the data of a record to the file, no other image transfers data to the file until the whole record has been transferred. ....................................................................... [178:36] At the end of the second paragraph of 9.4 File connection, after the second 'program', add 'and on all images'. ....................................................................... [179:27] In the first sentence of 9.4.2 Unit existence, after 'exist for', add 'all the images of'. ....................................................................... [179:33] At the end of the second sentence of 9.4.3 Connection of a file to a unit, after 'external file', add ', and the connection is for a team of images'. ....................................................................... [181:39+] In 9.4.5 OPEN statement, insert into R905 , alphabetically: <> TEAM = ....................................................................... [181:41+] In 9.4.5 OPEN statement, add a new rule After R907 R907a <> ....................................................................... [185:4+] Add new subsection before 9.4.6 The CLOSE statement: 9.4.5.16 TEAM= specifier in the OPEN statement The specifies the <> for the unit, which is the set of images that are connected to the unit. The team shall include the executing image. If there is no TEAM= specifier, the connect team consists of only the executing image. All images in the connect team, and no others, shall invoke the same OPEN statement with identical values for the s. There is an implicit team synchronization. The OPEN statement connects the file on the invoking images only. If the OPEN statement does not have a FILE= specifier and the unit is not connected to a file, the processor shall connect the same file to the unit on all images in the connect team. If the connect team contains more than one image, the OPEN statement shall (1) have an ACCESS= specifier that evaluates to DIRECT or (2) have an ACTION= specifier that evaluates to WRITE, and shall not specify an ACCESS= specifier or shall have an ACCESS= specifier that evaluates to SEQUENTIAL. NOTE 9.21a Writing to a sequential file from more than one image is permitted for situations in which the ordering of records is unimportant. An examples is for diagnostic output that is labeled with the image index. [end NOTE] An OPEN statement on a unit connected to a file by a previously executed OPEN statement shall have the same connect team as currently in effect. NOTE 9.21b The rule in 9.4.3 that a unit shall not be connected to more than one file at the same time and a file shall not be connected to more than one unit at the same time applies even if the connect teams for the units have no images in common. [end NOTE] Pre-connected units and the units identified by the values INPUT_UNIT, OUTPUT_UNIT, and ERROR_UNIT in the intrinsic module ISO_FORTRAN_ENV have a connect team consisting of all the images. If an image with index greater than one executes an input/output statement on one of these units, it shall be a WRITE or PRINT statement. NOTE 9.21c The input unit identified by * is therefore only available on the image with index one. [end NOTE] ....................................................................... [185:19+] Add new paragraph at the end of 9.4.6 The CLOSE statement: Execution of a CLOSE statement on a unit open with a connect team that contains more than one image consists of a wait operation, followed by a team synchronization, followed by an implied FLUSH operation for the unit. ....................................................................... [208:4+] Add new paragraph at the end of 9.7.1 BACKSPACE statement: A BACKSPACE statement shall not specify a that is connected with a connect team with more than one image. ....................................................................... [208:19+] Add new paragraph at the end of 9.7.2 ENDFILE statement: An ENDFILE statement shall not specify a that is connected with a connect team with more than one image. ....................................................................... [208:23+] Add new paragraph at the end of 9.7.3 REWIND statement: A REWIND statement shall not specify a that is connected with a connect team with more than one image. ....................................................................... [209:18-20] Replace the second paragraph of 9.8 FLUSH statement that begins "Execution of a FLUSH statement causes" by: Execution of a FLUSH statement (1) causes data written to an external file to be made available to other images of the same connect team and to other processes, (2) causes data written to an external file by another image of the same connect team and made available there to be made available to a READ statement, and (3) causes data placed in an external file by other processes to be made available to a READ statement. ....................................................................... [257:25+] In the first itemized list, (1), of 12.3.1.1 Explicit interface, add (a1) With an argument that is a co-indexed object (2.4.5a), ....................................................................... [257:33+] In the second itemized list, (2), of 12.3.1.1 Explicit interface, add (b1) is a co-array, ....................................................................... [266:27+] In '12.4 Procedure reference', add a new paragraph before R1220: If the in a is a co-indexed object, the procedure referenced is on the executing image. ....................................................................... ***Note to editor: This edit should precede edit [269:14-] in 06-108r1. [269:13+] In 12.4.1.2, Actual arguments associated with dummy data objects, after the fifth paragraph, add a new paragraph and note: If the actual argument is a co-indexed object, the dummy argument shall not be a co-array and shall have the INTENT(IN) or the VALUE attribute. A co-indexed object that has a pointer subcomponent shall not be an actual argument. NOTE 12.21a If the actual argument is a co-indexed object and the corresponding dummy argument is not a co-array, it is expected that implementations will make a copy on the executing image of the actual argument, including copies of any allocated allocatable subcomponents, before argument association occurs. [end NOTE] ....................................................................... [269:17+] After the sixth paragraph of 12.4.1.2 Actual arguments associated with dummy data objects, add new paragraphs and notes: If a dummy argument is an allocatable co-array, the actual argument shall be an allocatable co-array of the same rank and co-rank. If a dummy argument is allocatable and is not a co-array, the actual argument shall not be an allocatable co-array. If a dummy argument is a co-array that is not allocatable, the co-rank and co-bounds are specified by the local declaration and are independent of those of the actual argument. The actual argument shall be a co-array or a subobject of a co-array without any co-subscripts, vector-valued subscripts, allocatable component selection, or pointer component selection. NOTE 12.21b The co-array on the executing image is specified as the actual argument and this is associated with the dummy co-array argument on the same image. There are corresponding co-arrays on the other images. Here is an example: INTERFACE SUBROUTINE SUB(X,Y) REAL :: X(:)[*], Y(:)[*] END SUBROUTINE SUB END INTERFACE : REAL, ALLOCATABLE :: A(:)[:], B(:,:)[:] : CALL SUB(A(:),B(1,:)) [end NOTE] The executing image associates each non-allocatable co-array dummy argument with a co-array that is not a dummy argument (perhaps through several layers of procedure calls). Any reference to a co-indexed object of the dummy argument uses the rank, co-rank, bounds, and co-bounds declared on the image. NOTE 12.21c It is expected that a procedure with a co-array dummy argument will often be called simultaneously on all the images of a team with the same actual co-array and the same bounds and co-bounds. However, there are other possibilities; for example, a procedure might be called on a single image to perform a calculation that needs data from other images. [end NOTE] If an assumed-shape co-array that does not have the CONTIGUOUS attribute or a subobject of an assumed-shape co-array that does not have the CONTIGUOUS attribute is an actual argument associated with a dummy co-array, the dummy co-array shall be of assumed shape. If an array section is an actual argument associated with a dummy co-array that is not of assumed shape or has the CONTIGUOUS attribute, the section shall be contiguous (5.1.2.4a). ***Note to editor: Section number 5.1.2.4a is specified in paper 05-273r3, for feature J3-043. NOTE 12.21d It is intended to ensure that neither copy-in nor copy-out takes place for a dummy co-array. [end NOTE] ....................................................................... [270:6] In the first sentence of the 12th paragraph of 12.4.1.2 Actual arguments associated with dummy data objects (first full paragraph on page 270), replace "corresponding actual argument has the TARGET attribute" with "corresponding actual argument is not a co-indexed object and has the TARGET attribute". ....................................................................... [271:6] In the penultimate constraint (C1232) of 12.4.1.3 Actual arguments associated with dummy data objects, replace the beginning of the first sentence: "If an actual argument is an array section or an assumed-shape array," with If an actual argument is an array section or an assumed-shape array that is not a co-indexed object ....................................................................... [279:21+] In 12.5.2.1 Function subprogram, after R1224 add a new constraint: C1233a (R1223) A function result shall not be a co-array. ....................................................................... ***Note to Editor: This text should be added before the Note edited by 05-237r3 [287:7+] After the final constraint (C1276) in 12.6 Pure procedures, add two new constraints: C1276a A co-indexed object shall not appear in a variable definition context in a pure subprogram. C1276b A pure subprogram shall not contain an image control statement (8.5.1). ....................................................................... [291:14] In the last sentence of the first paragraph of 13.1 Classes of intrinsic procedures, change 'standard intrinsic' to 'inquiry, elemental, and transformational intrinsic'. ....................................................................... [291:16+] After the second paragraph, and before the Note, in 13.1 Classes of intrinsic procedures, add a new paragraph: A <> is one that has an argument of type IMAGE_TEAM (13.8.2). If it is invoked by one image of a team, it shall be invoked by the same statement on all images of the team. There is an implicit team synchronization (8.5.3) at the beginning and end of the execution of a collective subroutine. ....................................................................... [292:18+] At the end of 13.2 Arguments to intrinsic procedures, add new subsection: 13.2.3 Arguments of collective subroutines Each argument of a collective subroutine shall have the same shape on all images of the team. A co-array argument shall have the same bounds on all images of the team. An INTENT(IN) argument of type IMAGE_TEAM shall have a value that was constructed by invoking FORM_TEAM for the team. ....................................................................... [296:6+] In 13.5.7 Array inquiry functions, add alphabetically: CO_LBOUND (CO_ARRAY [, DIM, KIND]) Lower co-dimension bounds of a co-array CO_UBOUND (CO_ARRAY [, DIM, KIND]) Upper co-dimension bounds of a co-array ....................................................................... [297:25+] After 13.5.14 Array location functions, add a new subsection: 13.5.14a Collective subroutines CO_ALL(MASK,RESULT[,TEAM]) True if all values are true CO_ANY(MASK,RESULT[,TEAM]) True if any value is true CO_COUNT(MASK,RESULT[,TEAM]) Numbers of true elements CO_MAXLOC(CO_ARRAY,RESULT[,TEAM]) Image indices of maximum values CO_MAXVAL(CO_ARRAY,RESULT[,TEAM]) Maximum values CO_MINLOC(CO_ARRAY,RESULT[,TEAM]) Image indices of minimum values CO_MINVAL(CO_ARRAY,RESULT[,TEAM]) Minimum values CO_PRODUCT(CO_ARRAY,RESULT[,TEAM]) Products of elements CO_SUM(CO_ARRAY,RESULT[,TEAM]) Sums of elements FORM_TEAM(TEAM,IMAGES) Forms a team of images NOTE 13.6a The subroutine FORM_TEAM forms a team of images. The other collective subroutines return an array of the same shape as the given co-array after having applied an operation across the images in a team. The simple case of wanting the operation to be applied to all the elements of the co-array can easily be programmed, for example REAL FUNCTION GLOBAL_SUM(ARRAY) REAL :: ARRAY(:,:)[*] REAL,SAVE :: TEMP[*] TEMP = SUM(ARRAY) ! Sum on the executing image CALL CO_SUM(TEMP,GLOBAL_SUM) END FUNCTION GLOBAL_SUM Versions with other arguments can be programmed similarly, see C.9a.2. [end NOTE] ....................................................................... [298:9+,11+,12+] In 13.5.18 System environment procedures, add three new items alphabetically: IMAGE_INDEX(CO_ARRAY, SUB) Index of an image NUM_IMAGES() Number of images THIS_IMAGE( ) or Index of the invoking image THIS_IMAGE(CO_ARRAY [,DIM]) ....................................................................... [309:32+] After 13.7.25, add new subsections 13.7.25a CO_ALL(MASK, RESULT [,TEAM]) <> Determine whether all values are true across a team of images. <> Collective subroutine. <> MASK shall be a co-array of type logical. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have type logical and the shape of MASK. It is an INTENT(OUT) argument. If RESULT is a scalar, it has the value true if the value of MASK is true on all the images of TEAM, and the value false otherwise. If RESULT is an array, an element has the value true if the corresponding elements of MASK are true on all the images of TEAM, and the value false otherwise. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and MASK has the values (/.TRUE.,.FALSE.,.TRUE./) and (/.TRUE.,.TRUE.,.TRUE./) on the two images, executing CALL CO_ALL(MASK,RESULT) gives RESULT the value [true, false, true]. 13.7.25b CO_ANY(MASK, RESULT [,TEAM]) <> Determine whether any value is true across a team of images. <> Collective subroutine. <> MASK shall be a co-array of type logical. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have type logical and the shape of MASK. It is an INTENT(OUT) argument. If RESULT is scalar its value is true if any value of MASK is true on the images of TEAM, and false otherwise. If RESULT is an array, an element has the value true if any of the corresponding elements of MASK are true on the images of TEAM, and the value false otherwise. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and MASK has the values (/.TRUE.,.FALSE.,. FALSE./) and (/.TRUE.,.TRUE.,.FALSE./) on the two images, executing CALL CO_ANY(MASK,RESULT) gives RESULT the value [true, true, false]. 13.7.25c CO_COUNT(MASK, RESULT [,TEAM]) <> Count the numbers of true elements across a team of images. <> Collective subroutine. <> MASK shall be a co-array of type logical. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have type integer and the shape of MASK. It is an INTENT(OUT) argument. If RESULT is a scalar, it has the a value equal to the number of images of TEAM for which MASK has the value true. If RESULT is an array, an element has a value equal to the number of corresponding elements of MASK on the images of TEAM that have the value true. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and MASK has the values (/.TRUE.,.FALSE.,. FALSE./) and (/.TRUE.,.TRUE.,.FALSE./) on the two images, executing CALL CO_COUNT(MASK,RESULT) gives RESULT the value [2, 1, 0]. 13.7.25d CO_LBOUND (CO_ARRAY [, DIM, KIND]) <> Returns all the lower co-bounds or a specified lower co-bound of a co-array. <> Inquiry function. <> CO_ARRAY shall be a co-array of any type. It may be a scalar or an array. If it is allocatable it shall be allocated. DIM (optional) shall be scalar and of type integer with a value in the range 1 <= DIM <= n, where n is the co-rank of CO_ARRAY. The corresponding actual argument shall not be an optional dummy argument. KIND (optional) shall be a scalar integer initialization expression. <> Integer. If KIND is present, the kind type parameter is that specified by the value of KIND; otherwise, the kind type parameter is that of default integer type. The result is scalar if DIM is present; otherwise, the result is an array of rank one and size n, where n is the co-rank of CO_ARRAY. <> : CO_LBOUND (CO_ARRAY, DIM) has a value equal to the lower co-bound for co-subscript DIM of CO_ARRAY. : CO_LBOUND (CO_ARRAY) has a value whose ith component is equal to CO_LBOUND (CO_ARRAY, i), for i = 1, 2,..., n, where n is the co-rank of CO_ARRAY. < If A is allocated by the statement ALLOCATE ( A [2:3, 7:*] ) then CO_LBOUND (A) is [2, 7] and CO_LBOUND (A, DIM=2) is 7. 13.7.25e CO_MAXLOC(CO_ARRAY, RESULT [,TEAM]) <> Image indices of maximum values of the elements across a team of images. <> Collective subroutine. <> CO_ARRAY shall be a co-array of type integer, real, or character. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have type integer and the shape of CO_ARRAY. It is an INTENT(OUT) argument. If RESULT is a scalar, it has a value equal to the smallest image index of a maximum value of CO_ARRAY on the images of TEAM. If RESULT is an array, an element of RESULT has a value equal to the smallest image index of a maximum value of all the corresponding elements of CO_ARRAY on the images of TEAM. If CO_ARRAY has type character, the result is the value that would be selected by application of intrinsic relational operators; that is, the collating sequence for characters with the kind type parameter of the argument is applied. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and CO_ARRAY has the values (/1,5,6/) and (/4,1,6/) on the two images, executing CALL CO_MAXLOC(CO_ARRAY,RESULT) gives RESULT the value (/2,1,1/). 13.7.25f CO_MAXVAL(CO_ARRAY, RESULT [,TEAM]) <> Maximum values of the elements across a team of images. <> Collective subroutine. <> CO_ARRAY shall be a co-array of type integer, real, or character. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have the same type, type parameters, and shape as CO_ARRAY. It is an INTENT(OUT) argument. If RESULT is a scalar, it has a value equal to the maximum value of CO_ARRAY on the images of TEAM. If RESULT is an array, an element of RESULT has a value equal to the maximum value of all the corresponding elements of CO_ARRAY on the images of TEAM. If CO_ARRAY has type character, the result is the value that would be selected by application of intrinsic relational operators; that is, the collating sequence for characters with the kind type parameter of the argument is applied. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and CO_ARRAY has the values (/1,5,3/) and (/4,1,6/) on the two images, executing CALL CO_MAXVAL(CO_ARRAY,RESULT) gives RESULT the value (/4,5,6/). 13.7.25g CO_MINLOC(CO_ARRAY, RESULT [,TEAM]) <> Image indices of minimum values of the elements across a team of images. <> Collective subroutine. <> CO_ARRAY shall a co-array of type integer, real, or character. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have type integer and the shape of CO_ARRAY. It is an INTENT(OUT) argument. If RESULT is a scalar, its value is equal to the smallest image index of a minimum value of CO_ARRAY on the images of TEAM. If RESULT is an array, an element has a value equal to the smallest image index of a minimum value of all the corresponding elements of CO_ARRAY on the images of TEAM. If CO_ARRAY has type character, the result is the value that would be selected by application of intrinsic relational operators; that is, the collating sequence for characters with the kind type parameter of the argument is applied. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and CO_ARRAY has the values (/1,5,6/) and (/4,1,6/) on the two images, executing CALL CO_MINLOC(ARRAY,RESULT) gives RESULT the value (/1,2,1/). 13.7.25h CO_MINVAL(CO_ARRAY, RESULT [,TEAM]) <> Minimum values of the elements across a team of images. <> Collective subroutine. <> CO_ARRAY shall be a co-array of type integer, real, or character. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have the same type, type parameters, and shape as CO_ARRAY. It is an INTENT(OUT) argument. If RESULT is scalar it has a value equal to the minimum value of CO_ARRAY on the images TEAM. If RESULT is an array, an element of RESULT has a value equal to the minimum value of all the corresponding elements of CO_ARRAY on the images of the team. If CO_ARRAY has type character, the result is the value that would be selected by application of intrinsic relational operators; that is, the collating sequence for characters with the kind type parameter of the argument is applied. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and CO_ARRAY has the values (/1,5,3/) and (/4,1,6/) on the two images, executing CALL CO_MINVAL(CO_ARRAY,RESULT) gives RESULT the value (/1,1,3/). 13.7.25i CO_PRODUCT(CO_ARRAY, RESULT [,TEAM]) <> Products of elements across a team of images. <> Collective subroutine. <> CO_ARRAY shall be a co-array of type integer, real, or complex. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have the same type, kind type parameter, and shape as CO_ARRAY. It is an INTENT(OUT) argument. If RESULT is a scalar, it has a value equal to a processor-dependent and image-independent approximation to the product of the values of CO_ARRAY on the images of TEAM. If RESULT is an array, an element of RESULT has a value equal to a processor-dependent and image-dependent approximation to the product of all the corresponding elements of CO_ARRAY on the images of TEAM. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and CO_ARRAY has the values (/1,5,3/) and (/4,1,6/) on the two images, executing CALL CO_PRODUCT(CO_ARRAY,RESULT) gives RESULT the value (/4,5,18/). 13.7.25j CO_SUM(CO_ARRAY, RESULT [,TEAM]) <> Sums of elements across a team of images. <> Collective subroutine. <> CO_ARRAY shall be a co-array of type integer, real, or complex. It may be a scalar or an array. It is an INTENT(IN) argument. RESULT shall have the same type, kind type parameter, and shape as CO_ARRAY. It is an INTENT(OUT) argument. If RESULT is a scalar, it has a value equal to a processor-dependent and image-dependent approximation to the sum of the values of CO_ARRAY on the images of TEAM. If RESULT is an array, an element of RESULT has a value equal to a processor-dependent and image-dependent approximation to the sum of all the corresponding elements of CO_ARRAY on the images of the team. TEAM (optional) shall be a scalar of type IMAGE_TEAM (13.8.2). It is an INTENT(IN) argument. The case with TEAM absent is treated as if it were present with a team consisting of all images. <> If NUM_IMAGES() has the value 2 and CO_ARRAY has the values (/1,5,3/) and (/4,1,6/) on the two images, executing CALL CO_SUM(CO_ARRAY,RESULT) gives RESULT the value (/5,6,9/). 13.7.25k CO_UBOUND (CO_ARRAY [, DIM, KIND]) <> Returns all the upper co-bounds or a specified upper co-bound of a co-array of co-rank greater than one. <> Inquiry function. <> CO_ARRAY shall be a co-array of any type and of co-rank greater than one. It may be a scalar or an array. If it is allocatable it shall be allocated. DIM (optional) shall be scalar and of type integer with a value in the range 1 <= DIM <= n-1, where n is the co-rank of CO_ARRAY. The corresponding actual argument shall not be an optional dummy argument. KIND (optional) shall be a scalar integer initialization expression. <> Integer. If KIND is present, the kind type parameter is that specified by the value of KIND; otherwise, the kind type parameter is that of default integer type. The result is scalar if DIM is present; otherwise, the result is an array of rank one and size n-1, where n is the co-rank of CO_ARRAY. <> : CO_UBOUND (CO_ARRAY, DIM) has a value equal to the upper co-bound for co-subscript DIM of CO_ARRAY. : CO_UBOUND (CO_ARRAY) has a value whose ith component is equal to CO_UBOUND (CO_ARRAY, i), for i = 1, 2,..., n-1, where n is the co-rank of CO_ARRAY. < If A is allocated by the statement ALLOCATE ( A [2:3, 0:7, *] ) then CO_UBOUND (A) is [3, 7] and CO_UBOUND (A, DIM=2) is 7. ...................................................................... [317:1+] After 13.7.39 FLOOR, add a new subsection: 13.7.39a FORM_TEAM(TEAM, IMAGES) <> Form a team of images. <> Collective subroutine. <> TEAM shall be a scalar of the derived type IMAGE_TEAM that is defined in the intrinsic module ISO_FORTRAN_ENV (13.8.2). It is an INTENT(OUT) argument. IMAGES shall be of rank one and type integer. It is an INTENT(IN) argument that specifies the image indices of the team members and has the same value on all images of the team. All the elements shall have values in the range 1, ..., NUM_IMAGES() and there shall be no repeated values. It shall not have size zero. <> The following splits images into two equal groups and implicitly synchronizes each of the teams if there are two or more images. If there is only one image, that image becomes the only team member. USE, INTRINSIC :: ISO_FORTRAN_ENV INTEGER :: I TYPE(IMAGE_TEAM) :: TEAM IF(THIS_IMAGE()<=NUM_IMAGES()/2) THEN CALL FORM_TEAM(TEAM,[(I,I=1,NUM_IMAGES()/2)]) ELSE CALL FORM_TEAM(TEAM,[(I,I=NUM_IMAGES()/2+1,NUM_IMAGES())]) END IF ...................................................................... [322:23+] After 13.7.51, insert new subsection 13.7.51a IMAGE_INDEX(CO_ARRAY, SUB) <> Returns the index of the image corresponding to the co-subscripts SUB for CO_ARRAY. <> Inquiry function. <> CO_ARRAY shall be a co-array of any type. SUB shall be a rank-one array of size equal to the co-rank of CO_ARRAY and type integer. <> Scalar of type default integer. <> If the value of SUB is a valid sequence of co-subscripts for CO_ARRAY, the result is the index of the corresponding image. Otherwise, the result lies outside the range 1, ..., NUM_IMAGES(). <> If A is declared A[0:*], IMAGE_INDEX(A,(/0/)) has the value 1. NOTE 13.16a A module containing an implementation of this function with a slightly changed argument list is available in C9a.1 [end NOTE] ...................................................................... [341:30+] After 13.7.88, insert new subsection 13.7.88a NUM_IMAGES() <> Returns the number of images. <> Inquiry function. <> None. <> Scalar of type default integer. <> The number of images. <> To employ image 1 to read data and broadcast it to other images: REAL :: P[*] ... IF (THIS_IMAGE()==1) THEN READ(6,*)P DO I = 2, NUM_IMAGES() P[I] = P END DO END IF SYNC_ALL NOTE 13.16b These two functions are useful in connection with NUM_IMAGES. INTEGER FUNCTION LOG2_IMAGES() ! Returns the base-2 logarithm of the number of images, ! truncated to an integer. LOG2_IMAGES = LOG(NUM_IMAGES()+0.5)/LOG(2.0) END FUNCTION LOG2_IMAGES INTEGER FUNCTION REM_IMAGES() ! Returns MOD(NUM_IMAGES(),2**LOG2_IMAGES()). REM_IMAGES = MOD(NUM_IMAGES(),2**LOG2_IMAGES()) END FUNCTION REM_IMAGES [end NOTE] ...................................................................... [356:1+] After 13.7.119, insert new subsection 13.7.119a THIS_IMAGE() or THIS_IMAGE(CO_ARRAY [,DIM]) <> Returns the index of the invoking image. <> Inquiry function. <> CO_ARRAY (optional) shall be a co-array of any type. DIM (optional) shall be scalar and of type default integer. Its value shall be in the range 1<=DIM<=n where n is the co-rank of CO_ARRAY. It shall not be an absent optional dummy argument. <> Type default integer, scalar or of rank one. <> <> The result of THIS_IMAGE() is a scalar with value equal to the index of the invoking image. It is in the range 1, ..., NUM_IMAGES(). <> The result of THIS_IMAGE(CO_ARRAY) is an array of size equal to the co-rank of CO_ARRAY. Element K of the result has value equal to co-subscript K of CO_ARRAY on the invoking image. <> The result of THIS_IMAGE(CO_ARRAY,DIM) is a scalar with value equal to co-subscript DIM of CO_ARRAY on the invoking image. <> The following employs the first image to read data. The other images then copy the data. IF (THIS_IMAGE()==1) READ(*,*)P SYNC_ALL P = P[1] NOTE 13.17a A module containing an implementation of this function with a slightly changed argument list is available in C.9a.1. [end NOTE] ................................................... ***Note to Editor: This edit interacts with the edits proposed in paper 05-252. [360:2+] Add new sentence: This module shall define a derived type IMAGE_TEAM with private components. Scalars of this type define teams of images that collaborate in computations. NOTE 13.19 To allow an implemention to define IMAGE_TEAM to have pointer properties, co-arrays are not permitted to be of this type. [end NOTE] ................................................... [421:25+] In section 16.5.5, events that cause variables to become defined, add after list item (17): (17a) Execution of a SYNC_ALL, SYNC_TEAM, SYNC_IMAGES, SYNC_MEMORY, NOTIFY, or QUERY statement with a STAT= specifier causes the variable specified by the STAT= specifier to become defined. (17b) If an error condition occurs during execution of a SYNC_ALL, SYNC_TEAM, SYNC_IMAGES, SYNC_MEMORY, NOTIFY, or QUERY statement that has an ERRMSG= specifier, the variable specified by the ERRMSG= specifier becomes defined. ................................................... [423:43+] In section 16.5.7, Variable definition context, add after list item (10): (10a) A variable in a , , , , or , (10b) A variable in a , ...................................................................... Add to glossary: [427:1-] <> (2.4.5a) : A data entity that is declared with nonzero co-rank. It has the same shape on every image and its values may be referenced or defined by any image. <> (2.4.5a) : The bounds given by the data in square brackets in a co-array's declaration or allocation. <> (2.4.5a) : An object whose designator includes square brackets. <> (2.4.5a) : The number of co-subscripts of a co-array. <> (2.4.5a) : A co-array on any image may be addressed by using co-subscripts in square brackets. Each co-subscript shall be a scalar integer expression. [427:2+] <> (13.1) : An intrinsic subroutine that has an argument of type IMAGE_TEAM. [427:18+] <> (9.4.5) : The set of images that are associated with an input-output unit. [430:26+] <> (1.2a) : A Fortran program executes as if it were replicated a number of times, the number of replications remaining fixed during execution of the program. Each copy is called an <> and each image executes asynchronously. <> (1.2a) : An integer in the range 1, ..., NUM_IMAGES() that indexes an image. [436:1-] <> (1.2a) : A set of images formed by invoking the intrinsic collective subroutine FORM_TEAM (13.7.25l) for the purposes of collaboration. <> (8.5.3) : A synchronization of the images in a team. ...................................................................... [484:9+] Before C.10, add new section C.9a Section 13 notes C.9a.1 Module for THIS_IMAGE and IMAGE_INDEX The intrinsics THIS_IMAGE(CO_ARRAY) and IMAGE_INDEX(CO_ARRAY,SUB) cannot be coded in Fortran since CO_ARRAY may be of any type and THIS_IMAGE(CO_ARRAY) needs to know the index of the image on which the code is running. As an example, here are simple versions that require the co-bounds to be specified as integer arrays and require the image index for THIS_IMAGE(CO_ARRAY). MODULE INDEX CONTAINS INTEGER FUNCTION IMAGE_INDEX(LBOUND,UBOUND,SUB) INTEGER,INTENT(IN) :: LBOUND(:),UBOUND(:),SUB(:) INTEGER :: I,N N = SIZE(SUB) IMAGE_INDEX = SUB(N) - LBOUND(N) DO I = N-1,1,-1 IMAGE_INDEX = IMAGE_INDEX*(UBOUND(I)-LBOUND(I)+1) + & SUB(I) - LBOUND(I) END DO IMAGE_INDEX = IMAGE_INDEX + 1 END FUNCTION IMAGE_INDEX INTEGER FUNCTION THIS_IMAGE(LBOUND,UBOUND,ME) RESULT(SUB) INTEGER,INTENT(IN) :: LBOUND(:),UBOUND(:),ME INTEGER :: SUB(SIZE(LBOUND)) INTEGER :: EXTENT,I,M,ML,N N = SIZE(SUB) M = ME - 1 DO I = 1,N-1 EXTENT = UBOUND(I)-LBOUND(I)+1 ML = M M = M/EXTENT SUB(I) = ML - M*EXTENT + LBOUND(I) END DO SUB(N) = M + LBOUND(N) END FUNCTION THIS_IMAGE END MODULE INDEX C.9a.2 Collective co-array subroutine variations The subroutines of 13.5.14a return an array of the same shape as the given co-array after having applied an operation across the images involved. A note there shows how to program the operation being applied to all the elements of the co-array to produce a scalar result. Versions with other arguments can similarly be programmed, for example REAL FUNCTION GLOBAL_SUM_MASK(ARRAY,MASK) REAL :: ARRAY(:,:)[*],MASK(:,:) REAL,SAVE :: TEMP[*] TEMP = SUM(ARRAY,MASK=MASK) ! Sum on the executing image CALL CO_SUM(TEMP,GLOBAL_SUM_MASK) END FUNCTION GLOBAL_SUM_MASK FUNCTION GLOBAL_SUM(ARRAY,DIM) REAL :: ARRAY(:,:)[*] INTEGER :: DIM REAL,ALLOCATABLE :: GLOBAL_SUM(:) REAL,ALLOCATABLE :: TEMP(:)[:] ALLOCATE(GLOBAL_SUM(SIZE(ARRAY,3-DIM)) ) ALLOCATE( TEMP(SIZE(ARRAY,3-DIM))[*] ) TEMP = SUM(ARRAY,DIM) ! Sum on the executing image CALL CO_SUM(TEMP,GLOBAL_SUM) END FUNCTION GLOBAL_SUM