------------------------------------------------------------------------
Comments on DCG draft 2012-11-20, Joachim Schimpf, 2013-01-11
------------------------------------------------------------------------


General
-------
The proposal tries to provide the means for using DCGs without knowledge
of the detail of the expansion (i.e. essentially the fact that a grammar
rule is expanded into a like-named predicate with two extra arguments).
This is achieved by way of providing (1) "nonterminal indicators" and
(2) phrase/2,3.  The use of these facilities is the recommended style
for using DCGs.

On the other hand, the proposal specifies the expansion in detail.
As a result, code that explicitly calls expanded predicates or explicitly
references their names must necessarily be correct (even though undesirable).
In addition, even the naive DCG user must be aware of a few things,
in particular the possibility of name clashes between user predicates
and predicates resulting from expansion.

It might therefore be helpful to add a section that states that:
- the use of nonterminal indicators and phrase/2,3 is the recommended style
- explicit references to expanded predicates and predicate names are legal,
but considered bad style
- name clashes with user predicates (and built-ins) can occur
- debuggers etc may show the expanded form


Omissions
---------
I may have missed something, but I can not deduce from the text whether
assert/1 is supposed to do the grammar rule expansion.  From an implementation
point of view it would be nice not to, in order to avoid the overheads
(expansion can explicitly be programmed if needed).

There are a few problems here anyway, since "nonterminal indicators" can
be declared dynamic, and thus dynamic grammar rules are possible:

How does clause/2 work on a grammar rule?  If it is supposed to retrieve
the expanded clause, then this breaks the intended hiding of the expansion
detail.  If it is supposed to retrieve the DCG source, then there is no
mechanism to specify that in a call to clause/2.

Is retract((foo-->X)) allowed and what should it do?


7.16 Execution
---------------
I agree with other reviewer's reservations about the requirement for
the error term existence_error(procedure, N//A).  The implementation
effort required is out of proportion with its usefulness.
Suggestion: make this optional.


6.3 Operator op(1105,xfy,'|')
-----------------------------
I have yet to see a rationale for the precedence of 1105.
Traditionally, the bar was an alias for semicolon and thus had the
same precedence of 1100.

While the change is unlikely to break old code, it is a change.
I think the motivation for the change stems from CHRs and other
language variants where the bar serves as a commit-operator.  However,
as these languages would better be served by an xfx operator, it is
not clear why their requirements should reflect back on DCGs.

Also, if a new precedence is being introduced here, why such a small
precedence gap to 1100, given that the minimum spacing between all
other ISO-defined operator precedences is 50?


8.1.2 and 10.1.1 expand_term and term_expansion
-----------------------------------------------
Can we simply remove these sections?  There is no need to mix up the
definition of DCGs and their expansion with details of the actual
expansion mechanism (as indeed alluded to in 8.1.2 note 3).

term_expansion/2 has nothing whatsoever to do with grammar rules,
so let's simply not define it here.

expand_term/2 seems to be defined here solely for providing a means
of accessing the result of the dcg translation.  Most modern systems
do no longer use expand_term/2, but more complex mechanisms to
control source preprocessing.  It was wise of the ISO standard
to stay away from the details and refer only to "preparation for
execution".  Let's keep it this way, and not say how the translator
is hooked in.

Suggestion: require a differently named built-in, say expand_dcg/2
that gives access specifically to the DCG expansion, and nothing else.


Terminology
-----------
I do not understand the logic behind the "grammar-body-XXX" names.
Some control constructs have such a name, some don't.
Many of these names occur only in one place in the text, and are
therefore redundant (e.g. grammar-body-cut).
Some of these names appear sometimes with, sometimes without hyphens
(e.g. grammar body goal).
Strangely, "control construct" is not among the definitions,
neither is "empty terminal sequence".
What is the point of the grammar-body-elem group (it looks redundant)?
"grammar-body-alternative" designates only the semicolon, not the bar,
while the bar construct has no name at all.

Maybe we can do without all (or most of) these definitions?

I would expect a clear partitioning into: terminals, nonterminals
and control constructs.  Currently, call/1 and phrase/1 are referred
to as both nonterminal and control construct, and [] as both terminal
sequence and control construct.


Miscellaneous
-------------
7.4.2
The second paragraph is pointless and can be omitted.

7.15
Drop the sentence: Because the expansion ... somewhat difficult.

Why the provision that GR expansion may take place when phrase/3 is called?

7.15.5
Why the fuss about forbidding ->/2 instead of simply defining it
analogous to Prolog's ->/2 control construct, for consistency?
We may all agree that ->/2 is somewhat useless, but as it is historically
allowed in Prolog code, it is curious to suddenly forbid it in DCGs.
Am I missing something?

7.15.10
Surely, call//1 is only needed and useful for variable arguments,
so the examples seem somewhat pointless.

7.16
Second line: this is meta-syntax, so simply use = instead of ==.


11 Expansion
phrase/1 is missing from dcg_constr/1 (or redundant in dcg_body/4).




------------------------------------------------------------------------
Further comment on DCG draft 2012-11-20, Joachim Schimpf, 2013-01-17
------------------------------------------------------------------------


On 15/01/2013 19:42, Ulrich Neumerkel wrote:
> ...
> The precise way how the grammar is
> implemented is not defined.  The expansion serves as reference
> implementation only for executing a non-terminal.

If this were indeed the intention, or ever was the intention, then
it is not corroborated by the version of the document that I have
commented on (dcgsdin211120).  This document says (6.2.1.3):

"Clause 11 of this TR defines how a grammar rule in Prolog text is
expanded into an equivalent clause when Prolog text is prepared for execution."

and (7.15):

"Section 11 provides a reference implementation that further defines
the semantics of expansion."

These statements clearly establish the Prolog code in section 11 as the
normative text for the expansion (dcg_rule/2), while e.g. section 7.15
is explicitly said to be "informal".



The only statement in the text that might be taken as a qualification of
the strong statements above is (8.1.2 Note 1):

"..., users should not rely on a specific translation of a grammar rule,
which is implementation-dependent."

Now, even an implementation-dependent translation must implement the
semantics required for DCGs.  As I can see no formal specification
apart from section 11, I conclude that every implementation must
conform to section 11.

The question is now what flexibility section 11 allows.  There is no
information about this in the current document (except a comment that
section 11 omits error checking).  In the absence of further information,
my reading (as a member of the audience for this document) is that
aspects such as the mapping from F//N to F/(N+2) in dcg_non_terminal/4
are indeed required, because they affect the observable behaviour of
the program (e.g. by causing or no causing certain predicate name clashes).
If any other reading is intended, this must be explicitly stated.



I strongly believe it would be wise to stick to, and standardise,
the F//N to F/(N+2) mapping.  Doing otherwise opens a can of worms
and is at the same time not terribly useful.  Some detail:

(1) By opening it up, we would again yield to the temptation to
innovate rather than standardise common, tried-and-tested practice.
This means half-baked solutions in the standard, and endless
discussions.

(2) If an implementation (in a laudable attempt not to pollute the
predicate name space) were to map foo/N to 'dcg$foo'/(N+2), then any
_portable_ DCG application still would have to consider the name
foo/(N+2) as taboo, because another Prolog _could_ use that name as
the mapping target.  So what is gained by this provision, apart
from uncertainty?

(3) What about the flexibility of using a different arity from N+2?
Obviously, it must be at least N+2, and the first two must be the
difference list (implied by the call//1 definition).  If allowed,
then DCG users must be warned to be aware of name clashes with
_any_ F/M where M>=N+2.  Is see no such warning in the document.

(4) One would need a more abstract description than section 11 provides.
Or one would need to specify, somehow, which aspects of the Prolog code
in section 11 are normative, and which aren't.  As an example,
dcg_non_terminal/4 would have to invoke an implementation defined subgoal
that provides the implementation defined translation from the grammar
rule functor to the resulting predicate functor.

(5) How is a module system supposed to know what to export when asked
to export F//N?  By making a dummy call to phrase/2?  There is no
convenient interface.

(6) What about the flexibility that existed in traditional DCG expansions,
of being able to pass a different data structure in the argument pair
(which was possible thanks to the now eliminated 'C'/3 hook, and was
of practical value).  Is this still allowed?

Etc.

-- Joachim Schimpf