Extra commas ignored?

[Tim McDaniel]

So I figured that allowing one trailing comma was merely special-cased
to allow for neat diffing, like in a revision-control system diffing
old
%a = (
FIELD1 => 23,
FIELD2 => 45,
);
versus new
%a = (
FIELD1 => 23,
FIELD2 => 45,
FIELD3 => 67,
);

But then I accidentally did
FIELD67 => , # FIXME
and learned that the extra comma is ignored. In fact, it seems to
ignore extra commas wherever:

$ perl -e 'use strict; use warnings; use Data:umper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
$VAR1 = [
15,
16
];

The exception is that it doesn't allow a leading comma just after the
"(".

Why in the dickens are extra commas permitted in most places?

 

[$Bill]

$ perl -e 'use strict; use warnings; use Data:umper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
$VAR1 = [
15,
16
];

The exception is that it doesn't allow a leading comma just after the
"(".

Why in the dickens are extra commas permitted in most places?

perlop
....
Comma Operator
....
In list context, it's just the list argument separator, and inserts both its
arguments into the list. These arguments are also evaluated from left to
right.

Basically you just have null arguments to the comma operator after the
15 and 16.

 

[Tim McDaniel]

$ perl -e 'use strict; use warnings; use Data:umper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
$VAR1 = [
15,
16
];

The exception is that it doesn't allow a leading comma just after the
"(".

Why in the dickens are extra commas permitted in most places?

perlop
...
Comma Operator
...
In list context, it's just the list argument separator, and inserts both its
arguments into the list.

I saw that, but most of these commas are missing an argument on one
side or both.

Basically you just have null arguments to the comma operator after
the 15 and 16.

I don't know of anywhere in Perl where there's a concept of "null
argument", or even where "null" might occur at all (except for null
string and NUL character). They certainly aren't undef arguments
(which I might have expected) or the output would have shown them.

 

[Tim McDaniel]

I don't know of anywhere in Perl where there's a concept of "null
argument", or even where "null" might occur at all (except for null
string and NUL character).

And null statement.

I shouldn't have gone down that path. Anyway, I don't know of any
"null arguments" anywhere.

$a = $b + ;

is a syntax error (hell, I'd better check that ... yup) as is

$c = >> $d;

And things that might look like something with an implied operand are
actually a different operator, like

$e = * $f; # Can't use an undefined value as a symbol reference at -e line 1.

$g = - h; # Results in "-h"

 

[$Bill]

$ perl -e 'use strict; use warnings; use Data:umper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
$VAR1 = [
15,
16
];

The exception is that it doesn't allow a leading comma just after the
"(".

Why in the dickens are extra commas permitted in most places?

perlop
...
Comma Operator
...
In list context, it's just the list argument separator, and inserts both its
arguments into the list.

I saw that, but most of these commas are missing an argument on one
side or both.

Basically you just have null arguments to the comma operator after
the 15 and 16.

I don't know of anywhere in Perl where there's a concept of "null
argument",

Would missing argument give you a warmer fuzzier feeling ?

It's a list with null/missing arguments in the list and therefore there
is nothing to insert into the list for the null/missing arguments.

or even where "null" might occur at all (except for null

string and NUL character).

Null \Null\, a. [L. nullus not any, none; ne not + ullus any, a
dim. of unus one; cf. F. nul. See {No}, and {One}, and cf.
{None}.]

3. (Math.) Empty; having no members; as, the null set.

They certainly aren't undef arguments

 

[$Bill]

And null statement.

I shouldn't have gone down that path. Anyway, I don't know of any
"null arguments" anywhere.

$a = $b + ;

is a syntax error (hell, I'd better check that ... yup) as is

$c = >> $d;

It obviously depends on the operator and the context. The only operator
we're talking about is the comma operator and only in a list context
(totally different results in a scalar context).

 

[Janek Schleicher]

Am 18.12.2013 22:53, schrieb Tim McDaniel:

So I figured that allowing one trailing comma was merely special-cased
to allow for neat diffing, like in a revision-control system diffing
old
%a = (
FIELD1 => 23,
FIELD2 => 45,
); ....

Why in the dickens are extra commas permitted in most places?

Beside the syntactic parsing reasons,
it has the benefit of easy maintanance.

Imagine you'd have something in addition like
%b = (
FIELD3 => 81,
FIELD4 => 99,
);

and for w/e reason you want to combine then in source code to

%ab = (
...
...
);

with the extra unnecessairy comma at the end,
you just can copy and paste the data lines,
without them, you'd have to insert a , at some place.
Especially when working with configuration, that gets disturbing and
when dynamic evaluation comes into play, also can lead to strange failures.


Greetings,
Janek

 

[Rainer Weikusat]

$ perl -e 'use strict; use warnings; use Data:umper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
$VAR1 = [
15,
16
];

The exception is that it doesn't allow a leading comma just after the
"(".

Why in the dickens are extra commas permitted in most places?

perlop
...
Comma Operator
...
In list context, it's just the list argument separator, and inserts both its
arguments into the list.

I saw that, but most of these commas are missing an argument on one
side or both.

The key is the "it's just the list argument separator" here which
implies that a comma in list context is not 'the comma operator' (which
takes two arguments, evaluates both left-to-right and returns the
result of the right one[*]) but a separator. This means it is used while
creating a token sequence during syntactical analysis: Once the lexer
encounters a , the current token is finished and it starts accumulating
a new one. Enter another comma --- no token yet, continue looking. And
so on[**].

[*] Technically, the scalar-context comma operator is also not an operator
in the sense that + is, it's rather a compiler directive.

[**] This description might seem inaccurate, eg it is possible to
define a sub

sub digits
{
return 0,,1,2,3,4,5,6,7,8,9;
}

which, when executed in list context, returns a list of digits but
ends up returning only the last element of this list in scalar context.
However, what happens here is just that all list elements are pushed on
the stack from left to right. In scalar context, the element at the top
of the stack kept while all others are removed.

 

[Tim McDaniel]

with the extra unnecessairy comma at the end,
you just can copy and paste the data lines,
without them, you'd have to insert a , at some place.

So I figured that allowing one trailing comma was merely special-cased
to allow for neat diffing, like in a revision-control system diffing

As you wrote, it also helps maintenance.

In a similar way, Perl allows dropping the semicolon before "}", but I
always put one there anyway (with limited exceptions: one-line or
few-line code blocks for sort, grep, and map).

I understand the concept of "null statement" and that it's
occasionally useful, so I have a mental model of why
perl -e '{print "a\n";;;;;;;;;;;;;;; print "b\n"}'
works fine.

What I'm asking is why Perl allows and ignores multiple internal
commas in a list. What benefit could this possibly give? Does that
benefit outweigh being silent on the occasional of leaving off an
operand?

 

[Jürgen Exner]

In a similar way, Perl allows dropping the semicolon before "}",

Actually, that's not quite correct.
In Perl the semicolon is a statement seperator instead of a statement
terminator as in some other programming languages.
Therefore Perl allows you to create an emtpy statement, e.g. in front of
a "}", by adding an additional semicolon if it pleases you.

That's the same age-old question as if
"foo\nbar\n\buz"
is a text with 3 lines or a text with 2 lines plus some garbage after
the second line.

jue

 

[Rainer Weikusat]

On 12/18/2013 13:53, Tim McDaniel wrote:

$ perl -e 'use strict; use warnings; use Data:umper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
$VAR1 = [
15,
16
];

The exception is that it doesn't allow a leading comma just after the
"(".

Why in the dickens are extra commas permitted in most places?

perlop
...
Comma Operator
...
In list context, it's just the list argument separator, and inserts both its
arguments into the list.

I saw that, but most of these commas are missing an argument on one
side or both.

The key is the "it's just the list argument separator" here which
implies that a comma in list context is not 'the comma operator' (which
takes two arguments, evaluates both left-to-right and returns the
result of the right one[*]) but a separator.

This idea is unfortunately unsuitable for describing the behaviour of
perl because commatose expression apparently generally work, cf

perl -e 'print 5 if 0,,,,5'

 

[Rainer Weikusat]

The key is the "it's just the list argument separator" here which
implies that a comma in list context is not 'the comma operator' (which
takes two arguments, evaluates both left-to-right and returns the
result of the right one[*]) but a separator.

There is only one comma operator. It always parses the same, and it
always compiles down to the same opcode (it has to, because context
isn't always known at compile time).

Judging from the 5.10.1 perly.y, there's actually no such thing as 'a
comma operator' which would need to be a 'binary operator between
terms'. The lexer treats it as such but in the parser, it's only (slight
simplification)

/* Expressions are a list of terms joined by commas */
argexpr : argexpr ','
{
#ifdef MAD
OP* op = newNULLLIST();
token_getmad($2,op,',');
$$ = append_elem(OP_LIST, $1, op);
#else
$$ = $1;
#endif
}
| argexpr ',' term
{
OP* term = $3;
DO_MAD(
term = newUNOP(OP_NULL, 0, term);
token_getmad($2,term,',');
)
$$ = append_elem(OP_LIST, $1, term);
}
| term %prec PREC_LOW
;

ie, there is something like a list operator and the , is used as purely
syntactical element for separating terms in a list.

[...]

[*] Technically, the scalar-context comma operator is also not an operator
in the sense that + is, it's rather a compiler directive.

Nonsense. The lexer, parser and optree handle it exactly the same way as
they handle +, except for the fact that OP_LIST takes multiple arguments.

At least in certain cases, the list is expanded inline, with no 'list
operator' anywhere in sight, eg

[rw@sable]~#perl -MO=Concise,-exec,a -e 'sub a { return ($_[0], $_[3], $_[2]); }'
main::a:
1 <;> nextstate(main 1 -e:1) v
2 <0> pushmark s
3 <#> aelemfast[*_] s
4 <#> aelemfast[*_] s/3
5 <#> aelemfast[*_] s/2
6 <@> return KP
7 <1> leavesub[1 ref] K/REFC,1

In constrast to this, an addition compiles to

[rw@sable]~#perl -MO=Concise,-exec,a -e 'sub a { return ($_[0] + $_[3] + $_[2]); }'
main::a:
1 <;> nextstate(main 1 -e:1) v
2 <0> pushmark s
3 <#> aelemfast[*_] s
4 <#> aelemfast[*_] s/3
5 <2> add[t3] sK/2
6 <#> aelemfast[*_] s/2
7 <2> add[t5] sKP/2
8 <@> return K
9 <1> leavesub[1 ref] K/REFC,1

And in any case,

[rw@sable]~#perl -e 'LIST =~ /,|comma/i or print "something else\n"'
something else

It would be interesting to know if 'the comma operator' in C is actually
treated as an operator or if the historical misnomer actually came from
there.

NB: I'm very much obliged to you for posting this because it has greatly
helped my understanding of perl, even though this reply may not sound
like this. I don't have an affirmtative mind ...

 

[Rainer Weikusat]

Ben Morrow <[email protected]> writes:

[...]

Judging from the 5.10.1 perly.y, there's actually no such thing as 'a
comma operator' which would need to be a 'binary operator between
terms'. The lexer treats it as such but in the parser, it's only (slight
simplification)

/* Expressions are a list of terms joined by commas */
argexpr : argexpr ','

[Sometime between 5.10.1 and the blead source I posted an extract from
before, this production was renamed to 'listexpr'...]

| argexpr ',' term
| term %prec PREC_LOW
;

ie, there is something like a list operator and the , is used as purely
syntactical element for separating terms in a list.

OK, you can call it that if you like; it's not really any different from
the ?: operator, which uses '?' and ':' to separate its three
arguments.

If it really wasn't different, you'd refer to 'the question mark
operator' and 'the colon operator' in this case and this would be as
inadequate because they're also just syntactical elements of something
else.

My point was that 'the scalar comma operator' and 'the list context list
construction operator' are actually the same operator, whose behaviour
is context-dependant.

There is no such thing as 'a comma operator' in the Perl grammar (at
least, I didn't find anything like that), commas are used as separators
in a certain kind of compound expression. As Tim McDaniel observed,
despite comma is documented as binary operator, it doesn't behave like
one: It has no arity. The other hypothesis, "it is a binary operator but
one which can have imaginary arguments" is to be rejected on the grounds
that it introduces artifical complications in order to bend reality such
that it fits with terminology (applying Occam's razor).

[*] Technically, the scalar-context comma operator is also not an operator
in the sense that + is, it's rather a compiler directive.

Nonsense. The lexer, parser and optree handle it exactly the same way as
they handle +, except for the fact that OP_LIST takes multiple arguments.

At least in certain cases, the list is expanded inline, with no 'list
operator' anywhere in sight, eg

[rw@sable]~#perl -MO=Concise,-exec,a -e 'sub a { return ($_[0], $_[3],
$_[2]); }'
main::a:
1 <;> nextstate(main 1 -e:1) v
2 <0> pushmark s
3 <#> aelemfast[*_] s
4 <#> aelemfast[*_] s/3
5 <#> aelemfast[*_] s/2
6 <@> return KP
7 <1> leavesub[1 ref] K/REFC,1

That's a slight hack: this is the 'listop' production, which does
actually build an OP_LIST, but then, rather than adding an extra op, it
converts the OP_LIST into an OP_RETURN. It's easier to see what's
happening without -exec:

~/src/perl/perl% perl -MO=Concise,a -e'sub a { $x, $y, $z }'
main::a:
7 <1> leavesub[1 ref] K/REFC,1 ->(end)
- <@> lineseq KP ->7
1 <;> nextstate(main 1 -e:1) v ->2
6 <@> list K ->7
2 <0> pushmark s ->3
- <1> ex-rv2sv sK/1 ->4
3 <$> gvsv(*x) s ->4
- <1> ex-rv2sv sK/1 ->5
4 <$> gvsv(*y) s ->5
- <1> ex-rv2sv sK/1 ->6
5 <$> gvsv(*z) s ->6
-e syntax OK
~/src/perl/perl% perl -MO=Concise,a -e'sub a { return $x, $y, $z }'
main::a:
7 <1> leavesub[1 ref] K/REFC,1 ->(end)
- <@> lineseq KP ->7
1 <;> nextstate(main 1 -e:1) v ->2
6 <@> return K ->7
2 <0> pushmark s ->3
- <1> ex-rv2sv sK/1 ->4
3 <$> gvsv(*x) s ->4
- <1> ex-rv2sv sK/1 ->5
4 <$> gvsv(*y) s ->5
- <1> ex-rv2sv sK/1 ->6
5 <$> gvsv(*z) s ->6
-e syntax OK

The only different between these two is that perl has turned the 'list'
into a 'return'.

The difference I see here is that there has to be some op (not operator)
in this place and if no other was provided, a 'list' op is used. That
'return' happens to be equivalent to that is just an accidental property of
my example (The same phenomenon can be observed for reverse and grep and
presumably, for any other list operator returning a list).

[...]

In C the comma really does have two entirely different meanings: the
'comma operator' which behaves like Perl's comma in scalar context, and
the comma-as-an-argument-separator in the argument list of a function
call, which is not really an operator at all.

The main difference is that , is really a binary operator in C, eg this

 

[Rainer Weikusat]

As I said, you can call it what you like. 'The comma operator' is easier
to say than 'the list-of-terms-separated-by-(possibly-multiple)-commas
operator', but if you'd rather call it OP_LIST or 'the list operator'
that's fine by me. (You might get some argument from people who claim
there's no such thing as a list in scalar context, an attitude that is
unfortunately supported by the documentation.)

I think refering to the complete syntactical construct as 'list' or
'list expression' makes more sense than talking about a 'comma operator'
here, especially considering that ; isn't called 'the semicolon
operator' despite the striking similarities between the two: A list is a
comma-separated list of expression which are evaluated from left to
right and ultimatively end up as arguments of a list operator. If an
explicit list operator doesn't exist, an implicit "return the list in
list context, the last element otherwise" list operator is being used
(which implies that a list expression in scalar context _works_ like the
C 'comma operator', except that the syntax is less restrictive).