J3/06-179r1 To: J3 From: Dan Nagle Subject: Reply to DARPA (Project of Drs. Leuke & Gordon) Date: 9 May 2006 J3 is pleased that Drs. Leuke and Gordon were able to present the concerns of DARPA to us, and to give J3 the opportunity to understand those concerns and to discuss with them the issues involved. It is difficult from the point of view of a standards technical committee to address all these concerns within the confines of a language specification for a number of reasons. We discuss these reasons in the following. In short, a language standard specifies the means for writing a correct program, and the effects of executing a correct program. It cannot easily describe the effects of arbitrary occurrences within an incorrect program. The Fortran standard explicitly disavows any specification of the means of file storage. Specifically, there is no statement regarding source files. The immediate implication is that requirements expressed for line numbers within a source file as part of error messages are difficult to describe in the standard. Some programs are created from source text stored in databases, or are heavily preprocessed, with the result that no file name and line number combination meaningful to the application programmer exists anywhere. Further complications such as inlining procedures or other aggressive optimizations make the reporting of file-based line numbers difficult. Specific examples such as detection of use of undefined variables are difficult to implement and can require a large penalty in execution resources. J3 believes this problem is best treated by platform-specific engineering solutions. Others, such as subscript bounds checking for assumed-size arrays for most vendors requires a change to the calling sequences, that in turn negatively affects library usage and interlanguage references. The Fortran standard has features that promote the detection of common programming errors. These features include explicit interfaces to subprograms (for detection during compilation of argument list mismatches), the implicit none statement (to detect misspelled or incorrectly declared names), provision for return status values for input/output statements and memory allocation statements (to provide an indication of the success or failure of the request), and the strict rules for arrays of different ranks (which may be enforced during compilation) and array extent bounds (which are usually enforced optionally during execution) which promote detecting array addressing errors. Many cases, notably input/output statements and memory allocation statements require the program execution to be terminated if there is an error and the application programmer does not include an explicit status variable. Also, a compiler is required to have the means of detecting syntax errors, or the use of any syntax extensions. How well a particular compiler documents its usage and makes available indications of errors is beyond the standard's means to specify. These factors are what are described as "Quality of Implementation" issues. Along with the size and complexity of a program that will cause any particular processor to fail, these issues are not now specified by the standard, and they could be specified by the standard only with great difficulty in order to avoid inhibiting opportunities for optimization. Language design involves a number of tradeoffs, including a tradeoff by the standard between ease of application debugging versus efficiency of application execution. The marketplace for compilers is the best arbitrator of the balance between these competing goals. Different vendors have chosen to emphasize different aspects of compilers in their respective products. Some have placed the emphasis on execution efficiency, others have selected error detection; some vendors have selected support for legacy programs, others have placed correct compilation according to the latest revision of the standard as the more important goal. With varying emphasis in different compilers, use of a variety of compilers often catches more problems than use of any one compiler. J3 strongly feels that the use of application coding standards to encourage the best use of available features of any programming language is a necessary ingredient when attempting to reduce the overall effort needed to make and use application software. Failing to use features already present in a language weakens the case that new features are needed. Use of straightforward coding constructs and avoidance of constructs where the intention is provided by subtle effects (and therefore errors are made difficult to detect) is a part of the programming discipline. A further issue arises in deciding whether to detect an error during execution, or to detect the error during compilation. Fortran has long required the detection of many errors during compilation. J3 believes this early detection of errors provides more efficient use of the application programmer's time than detection of errors during execution. Furthermore, it allows the detection of consequent errors during compilation as well, and it eases the diagnosis of other errors caught during execution. Examination of status values returned from allocation statements and input/output statements, for example, may lead to the discovery of an incorrect value having been passed to the request thereby causing the failure and program termination. The standard's interface to the IEEE floating point arithmetic capabilities of the underlying hardware is another example of where the application programmer can control the application program's behavior to report the occurrence of errors. Many vendors provide compilers to support more than one programming language, yet use a common code generator and run-time libraries to provide the actual hardware instruction sequences. Many times, this imposes conflicting demands on code generators and library writers, as different languages require different behavior in the face of invalid operations. Thus, a vendor might be unable diagnose a programming error in a particular language because of the requirements of another language. These engineering decisions are beyond the ability of the standard to control. Other tools exist beyond compilers and are almost always available with the compiler. These tools include symbolic debuggers, along with profilers and other source code checking tools. Symbolic debuggers supply many of the requirements presented to J3. J3 believes that these tools are not so difficult to use as to prevent their use to achieve the goals described to J3. J3 is pleased to have heard the concerns expressed to it. However, the issues raised are more difficult to resolve than might have been imagined. J3 will keep these concerns in mind when developing future revisions of the Fortran standard. However, too much effort is required to fully comply with the specific requests made for the revision of the Fortran standard scheduled to be finalized during 2008 that J3 feels it is unable to do so and achieve the results desired.