To: J3                                                     J3/20-139
From: Richard Bleikamp
Subject: UTI's 2, 3, and 4
Date: 2020-October-05

Reference: 20-134

Unresolved technical issues 2, 3, and 4 concern the newly added SPLIT
intinsic, and "unnecessary" complexity therein to provide certain
funcionality that is readily available without this additional
complexity.  In particular:

  - the optional SEPARATOR argument returns the positions of the
    token delimiters.  When SPLIT is invoked with FIRST and LAST,
    these positions are easily derivable using the FIRST and LAST
    arguments, and indexing into the original string, i.e.
        SEPARATOR(i) = STRING(LAST(i)+1 : LAST(i)+1)
    for 1 <= i < SIZE(LAST), similarly when BACK is true.

    The case when POS is present is representable as:
        if (POS .ge. 1 .and POS .le. len(STRING)) then
          SEPARATOR = STRING(POS:POS)
        end if
    (slightly different if BACK is present and true)

    This paper proposes allowing SEPARATOR only for the first
    form of a call to SPLIT.

  - the optional BACK argument is unneeded, except when POS
    is present.  You can reference the returned TOKENS array
    backwards to get the tokens in reverse order. This paper
    proposes allowing BACK only when POS is present (3rd form).

  - Invoking SPLIT with POS present allows for returning partial
    tokens, if POS is not set to the index of the start of a token
    in STRING on entry to SPLIT. This would possbly be an unintended
    error on the users part, but we should clarify the description.

Edits to 20-007 to address these 3 technical issues follow.

{16.9.194}
[432:13]
  Replace
    "SPLIT (STRING, SET, TOKENS [, SEPARATOR, BACK]) or"
  with
    "SPLIT (STRING, SET, TOKENS [, SEPARATOR]) or"

  Replace
    "SPLIT (STRING, SET, FIRST, LAST [, SEPARATOR, BACK]) or"
  with
    "SPLIT (STRING, SET, FIRST, LAST)"

  and replace
    "SPLIT (STRING, SET, POS [, SEPARATOR, BACK])"
  with
    "SPLIT (STRING, SET, POS [, BACK])"

[432:31]
  Replace
    "If POS does not appear, SEPARATOR "
  with
    "It "
  and combine this paragraph with the preceeding paragraph.

[432:34-36] same paragraph as preceeding edit
  Delete
    "; if BACK is present with the value true, the direction of
     counting is from the back, so that SEPARATOR(1) will contain
     the last delimiter in STRING"

[432:37-40] Delete these lines.

[433:23+]
  Add this new paragraph
    "When SPLIT is invoked with a value for POS of
     1 <= POS <= LEN(STRING) and STRING(POS:POS) is not a
     token delimiter present in SET, the token identified by
     SPLIT does not comprise a complete token as described in the
     description of the SET argument, but rather a partial token."

end of edits