J3/98-211R1 Date: 12 Nov 1998 To: J3 From: R. Maine Subject: Edits for M.25, stream I/O Specs and syntax for M.25, stream I/O were approved in paper 98-209r2. The following are edits to implement those specs and syntax. A previous draft of the edits was separated into two papers, 98-210r0 and 98-211. The current draft subsumes both of those previously separate papers, plus revisions to accomodate the approved specs and syntax. These edits are all relative to J3/98-007R3. In 9.0 [155:11] add ", WAIT" after "REWIND". This edit not really related to stream I/O. It was already a bit anomalous to start talking about records before we mentioned that files have (or now may have) records. This anomaly gets worse when files might not have records. So we will do a minor reorganization. The (very little) material in 9.2.0 can be readily moved up into 9.0 and expended to cover the new concepts. And we then promote all the 9.2 subsections up one level (they went a little deep anyway). Thus: [155:15+] Add as new paragraphs "A file is composed of either a sequence of file storage units or a sequence of records, which provide an extra level of organization to the file. A file composed of records is called a <>. A file composed of a sequence of file storage units without a record structure is called a <>. A processor may allow a file to be viewed both as a record file and as a stream file; in this case the relationship between the file storage units when viewed as a stream file and the records when viewed as a record file is processor dependent. A file is either an external file or an internal file." [155:38] "processor-dependent" -> "file storage (9.2.1.4)" In 9.2, Files [156:19-23] Delete all of 9.2 (including the heading), except for the note 9.3. Then promote all former subsections of 9.2 up one level. [156:24-25] Add "external" before "files" in note 9.3 and then move the note to [156:36+] In C.6.1 [387:32] "Files" -> "External files" and change the xref to be to the external files section in 9. In 9.2.1.2.*, File access [157:8] Replace the first sentence by "There are three methods of accessing the data of an external file: sequential, direct, and stream." [157:16+] Add para "<> is a method of accessing the records of an external record file in the order of their record numbers." [157:17] Unbolden "sequential access" [157:30-31] replace these lines by "(3) The records of the file shall be read or written only by sequential access input/output statements." [157:32+] Add para "<> is a method of accessing the records of an external record file in arbitrary order." [157:17] Unbolden "direct access" [157:45-46] replace these lines by "(3) The records of the file shall be read or written only by direct access input/output statements." [158:8+] Add new section "9.2.2.3 Stream access <> is a method of accessing the file storage units of an external stream file. When connected for stream access, an external file has the following properties: (1) The file storage units of the file shall be read or written only by stream access input/output statements." (2) Each file storage unit in the file is uniquely identified by a positive integer called the position. The order of the file storage units in the file is the order of their positions. (3) File storage units need not be read or written in order of their position. For example, it is permissable to write the file storage unit at position 3, even though the file storage units at positions 1 and 2 have not been written. Any file storage unit may be read from the file while it is connected to a unit, provided that the file storage unit has been written since the file was created." In 9.2.1.3.*, File position [158:12-14] Replace "record(s)" by "record(s) or file storage unit(s)" 3 times. [158:15] "file" -> "record file" [158:25+] Add a para "In a file connected for stream access, the file position is either between two file storage units, at the initial point of the file, at the terminal point of the file, or undefined." [158:27,29] "the file" -> "a record file" twice. [158:35-36] "sequential or direct" -> "sequential, direct, or stream" [159:4+] Add para "For stream access, the file is positioned immediately before the file storage unit specified by the POS= specifier; if there is no POS= specifier, the file position is not changed." [159:8+] Add para "For stream access, if no error condition occurred, the file position is not changed." [159:14+] Add new section "9.2.1.4 File storage units [Add file storage unit to the index pointing here] A <> is the basic unit of storage in a stream file or an unformatted record file. It is the unit of file position in stream access, the unit of record length in unformatted direct access, and the unit of file size for all external files. Every value in a stream file or an unformatted record file shall occupy an integer number of file storage units, which shall be the same for all scalar values of the same type and type parameters. The number of file storage units required for an item of a given type and type parameters may be determined using the IOLENGTH= specifier of the INQUIRE statement (9.7.3). In a file connected for stream access, the processor shall not have alignment restrictions that prevent a value of any type and type parameters from being stored at any positive integer file position. It is recommended that the file storage unit be an 8-bit octet where this choice is practical. [BEGIN NOTE] The requirement that every data value occupy an integer number of file storage units implies that data items inherently smaller than a file storage unit will require padding. This suggests that the file storage unit be small to avoid wasted space. Ideally, the file storage unit would be choosen such that padding is never required. A file storage unit of one bit would always meet this goal, but would likely be impractical because of the alignment requirements. The prohibition on alignment restrictions prohibits the processor from requiring file data alignments larger than the file storage unit. The 8-bit octet is recommended as a good compromise that is small enough to accomodate the requirements of many applications, yet not so small that the data alignment requirements are likely to cause significant performance problems. [END NOTE]" In 9.2.2, Internal files [159:19] "has" -> "is a record file with" The two subsections of this section aren't clearly distinguishable. Although the first is labelled as "properties" and the second as "restrictions", there are things in the first list that are clearly restrictions, and there are things in the second list that directly correspond to "properties" listed for external files. Thus: [159:18] Delete the section heading [160:1-2] Delete the section heading and the lead-in to the list [160:3-6] Renumber these as items 10 and 11. [160:4] Delete "that do not specify namelist formatting" In 9.3.4, OPEN statement [163:26] "or DIRECT" -> ", DIRECT, or STREAM" [163:27] "or direct" -> ", direct, or stream" [163:38+] Add a para "If the file is being connected for stream access, then it is connected for both formatted and unformatted input/output; in this case the OPEN statement shall not have a FORM= specifier." [163:42] After "." insert "This specifier shall not be present when a file is being connected for stream access." [164:5] "processor-dependent" -> "file storage" [164:22] "sequential" -> "sequential or stream" [164:47] "input of a formatted record" -> "formatted input" [165:7-8] "output of a formatted record" -> "formatted output" In 9.4.1 Control information list [167:29+] Add new line "<> POS = " [168:11] Change "or a " to "or a POS= specifier". [168:15] After "sequential" insert "or stream" [168:25+] Add new constraint "Constraint: If a POS= specifier is present, the shall not contain a REC= specifier." [169:24] before the "." insert "or a <>, depending on whether the file connection is for sequential access or stream access." [169:24+] Add new section "9.4.1.4 File position The POS= specifier specifies a file position in file storage units. This specifier may be present only in an input/output statement that specifies a unit connected for stream access. A processor may prohibit the use of POS= with particular files that do not have the properties necessary to support random positioing. A processor may also prohibit positioning a particular file to anywhere prior to its current file position if the file does not have the properties necessary to support such positioning. [BEGIN NOTE] A file that represents connection to a device or data stream might not be positionable. [END NOTE]" [169:37] after "sequential" insert "or stream" [170:25] after "external" insert "record" [171:1] delete "formatted sequential". [171:2] after "omitted" insert "from a formatted sequential input/output statement". (Otherwise we manage to imply that, for example, unformatted direct access input/output statements all perform advancing formatted sequential input/output, which would seem a bit bizzare). [171:28] After "Records" insert "and file storage units" In 9.4.2, Data transfer I/O list This isn't directly related to stream I/O, but its something I got reminded of while doing the stream I/O stuff. It does cause some confusion in stream I/O, but only because it causes exactly the same confusion elsewhere. I commented about this once sometime before, but my comment apparently got lost. I don't really feel like diverting my stream I/O work to deal with this, so I'll just add it as an unresolved issue. [173:43+] Add J3 note after Note 5.29 "[begin J3 note] Unresolved issue xx. The definition of the term "effective item" as used in I/O lists is dispersed and confused. Note 9.29 says that the rules in 9.4.2 determine the list off effective items. This is true, except that the term "effective item is never once mentioned there outside of note 9.29 itself. The rules are there, but not the term. The first para of 9.4.4.4, data transfer, appears to define the term "next effective item" and even puts in in bold face as befits the defining occurance. It would seem more sensible to define the term "effective item". I think we can rely on the English definition of "next". In addition to that, if this is supposed to be the definition, it is wrong, because it does NOT mention the rules of 9.4.2. For example, it does not say anything about the treatment of derived types and distinguish the cases where a scalar of derived type is treated as a single effective item from the cases where it is decomposed into its components. Speaking of which, note 9.29 is wrong because its statement that effective items in formatted i/o are ALWAYS of intrinsic type is now incorrect with derived type I/O. I suggest that section 9.4.2 say somewhere that it is defining the term "effective item". Then the words about this in 9.4.4 can be removed and replaced by a reference to 9.4.2. Section 10 uses the term extensively, but should not need modifying, except possibly to refer to wherever the term is defined. [end J3 note]" In 9.4.3, Error, etc. [174:25] delete "either of" [174:28+] Add list item "(3) When an attempt is made to read beyond the end of a stream file." [174:31] After the "." add "An end-of-file condition also may occur during execution of a stream input statement." [176:14] "gg" -> "g" (typo) In 9.4.4, Execution of data transfer... [177:2] "records" -> "the file" [177:36] "current record" -> "file" [177:37] "Exactly" -> "If the file is connected for sequential or direct access, exactly" The following 2 edits doen't really have anything to do with stream i/o. The sentences appear to be no-ops that are more confusing than helpful in that they implies that there are other possibilities. The only other possibility is that it might be a formatted record when we expect an unformatted one or vice versa (as we've defined record files to consist of those 3 kinds of records). We have also already said that all of the records of a file are either all formatted or all unformatted, except for the endfile record. And we've said that formatted records can only be read/written by formatted i/o statements, and simillarly for unformatted. So there is nothing left for these sentences to actually mean. They confusingly implies that a file might have a mixture of formatted and unformatted records - perhaps they are holdovers from some draft where that was allowed. [177:40-41] Delete the sentence "On input, the file shall be positioned so that the record read is an unformatted record or an endfile record." and change the following "The" to "For sequential or direct access input, the". [178:27-28] Delete the para. [177:41-42] "values" -> "file storage units" (twice) [177:42-178:7] Replace "Each value...[end note]" by "A value in the file is stored in a contiguous sequence of file storage units, begining with the file storage unit immediately following the current file position. After each value is transfered, the current file position is moved to the point immediately after the last file storage unit of the value. On input, if the file storage units transfered do not contain a value with the same type and type parameters as the input list entity, then the resulting value of the entity is processor-dependent except in the following cases: (1) A complex list entity may correspond to two real values of the same kind stored in the file, or visa versa. (2) A default character list entity of length n may correspond to n default characters stored in the file, regardless of the length parameters of the entities that were written to these storage units of the file. If the file is connected for stream input, the characters may have been written by formatted stream output." [178:15] "connected for formatted" -> "not connected for unformatted" [178:19] delete ", if any". (It doesn't add anything) [178:20-21] Delete "The current..written." and add new para "If the file is connected for sequential or direct acess, the current record and possibly additional records are read or written. If the file is connected for stream access, there is no record structure; anything that would otherwise have the effect of terminating a record does not have that effect. [BEGIN NOTE] In a file connected for stream access, a slash edit descriptor has no effect and format reversion (10.3) does not start a new record. It is allowed to specify an ADVANCE= specifier, but it has no effect. [END NOTE] For formatted input from a file connected for stream access, all of the file storage units read shall contain default character data. This data may have been written by formatted stream output or by unformatted stream output of character data." [178:29] "connected for unformatted" -> "not connected for formatted" [178:30, 32, 35, 38] "a file" -> "a record file" (4 times). [179:25] Before the "." insert "or at the current file position" [180:6,34] Align the ampersands. (Not related to stream I/O) [182:19] Delete the first "other". (Not related to stream I/O) [182:29] "A" -> "For formatted record output, a" [182:35] "A" -> "For formatted record input, a" [182:38+] Add new J3 note "Unresolved issue xx The derived type I/O section has some wording in the above 2 paras about "where the last edit descriptor finished". This only makes sense for formatted I/O (and not all cases of that). This wording either needs to be generalized, or we need to say that it is for formatted I/O, and then add comparable statements about unformatted." In 9.6, File positioning statements [185:22-23] replace the sentence spanning these line with "A file that is connected for direct access shall not be referred to by a BACKSPACE, ENDFILE, or REWIND statement. A file that is connected for stream access shall not be referred to by a BACKSPACE statement." [186:9,14] (twice) after "ENDFILE statement" insert "for a file connected for sequential access" [186:16+] Add new para "Execution of an ENDFILE statement for a file connected for stream access causes the terminal point of the file to become equal to the current file position. Only file storage unis before the current position are considered to have been written; thus only those file storage units may be subsequently read. Subsequent stream output statements may be used to write further data to the file. [186:17-18] after "creates the file" insert "; if the file is connected for sequential access it is created" In 9.7.1, Inquiry specifiers [187:24+] Add new line "<> STREAM = " [187:29+] Add new lines "<> POS = <> SIZE = " [188:40] Delete "and". Before the "." insert ", or STREAM is the file is connected for stream access" [189:11+] Duplicate lines [189:7-11], changing all 5 occurances of "DIRECT" to "STREAM" and incrementing the subsection number. [189:32] "processor-dependent" -> "file storage" [189:33] after "connection" insert ", or if the connection is for stream access" [189:42+] Add new sections "9.7.1.15 POS= specifier in the INQUIRE statement The in the POS= specifier is assigned the number of the file storage unit immediately following the current position of a file connected for stream access. If the file is positioned at its terminal position the variable is assigned a value one greater than the number of the last storage unit in the file. If the file is not connected for stream access or if the position of the file is indeterminate because of previous error conditions, the variable becomes undefined. 9.7.1.16 SIZE= specifier in the INQUIRE statement The in the SIZE= specifier is assigned the size of the file in file storage units. If the file size cannot be determined the variable is assigned the value -1. For a file that may be connected for stream access, the file size is the number of the highest numbered file storage unit in the file. For a file that may be connected for sequential or direct access, the file size may be greater than the number of storage units implied by the data in the records; the exact relatiosnhip is processor-dependent." [192:35] Replace "value...statement" with "number of file storage units that would be required to store the data of the output list in an unformatted file." The above statement didn't make sense before anyway. It was an awkward attempt to work around the lack of a term for "file storage unit". The only sensible interpretation I could make of "value that would result..." was the value of the actual data; I certainly didn't see anything about length or size in the old definition. Luckily the specifier name gives a hint, and the requirement that it be useable as a RECL is pretty explicit. But these didn't adequately make up for a nonsensical definition. In 10.6 [207:13] After "record" insert "or the current position of the stream file" [208:3] For a file connected for stream access, a slash edit descriptor has no effect. In 10.9.1.6 [217:38+] Add new para "Namelist comments are not allowed in stream input because they depend on record structure." In 14.6.3.1 [329:23] "or" -> "a file storage unit (9.2.1.4), or" In Annex A [363:14+] Add glossary entry "<> (9.2.1.4) : The basic unit of storage in an unformatted file." [367:20] "or" -> "a , or" In Annex C [392:16] "or direct" -> ", direct, or stream"