Extra commas ignored?

Discussion in 'Perl Misc' started by Tim McDaniel, Dec 18, 2013.

  1. Tim McDaniel

    Tim McDaniel Guest

    So I figured that allowing one trailing comma was merely special-cased
    to allow for neat diffing, like in a revision-control system diffing
    old
    %a = (
    FIELD1 => 23,
    FIELD2 => 45,
    );
    versus new
    %a = (
    FIELD1 => 23,
    FIELD2 => 45,
    FIELD3 => 67,
    );

    But then I accidentally did
    FIELD67 => , # FIXME
    and learned that the extra comma is ignored. In fact, it seems to
    ignore extra commas wherever:

    $ perl -e 'use strict; use warnings; use Data::Dumper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
    $VAR1 = [
    15,
    16
    ];

    The exception is that it doesn't allow a leading comma just after the
    "(".

    Why in the dickens are extra commas permitted in most places?

    --
    Tim McDaniel,
     
    Tim McDaniel, Dec 18, 2013
    #1
    1. Advertising

  2. Tim McDaniel

    $Bill Guest

    On 12/18/2013 13:53, Tim McDaniel wrote:
    >
    > $ perl -e 'use strict; use warnings; use Data::Dumper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
    > $VAR1 = [
    > 15,
    > 16
    > ];
    >
    > The exception is that it doesn't allow a leading comma just after the
    > "(".
    >
    > Why in the dickens are extra commas permitted in most places?


    perlop
    ....
    Comma Operator
    ....
    In list context, it's just the list argument separator, and inserts both its
    arguments into the list. These arguments are also evaluated from left to
    right.

    Basically you just have null arguments to the comma operator after the
    15 and 16.
     
    $Bill, Dec 19, 2013
    #2
    1. Advertising

  3. Tim McDaniel

    Tim McDaniel Guest

    In article <l8te5m$87t$>, $Bill <> wrote:
    >On 12/18/2013 13:53, Tim McDaniel wrote:
    >>
    >> $ perl -e 'use strict; use warnings; use Data::Dumper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
    >> $VAR1 = [
    >> 15,
    >> 16
    >> ];
    >>
    >> The exception is that it doesn't allow a leading comma just after the
    >> "(".
    >>
    >> Why in the dickens are extra commas permitted in most places?

    >
    >perlop
    >...
    > Comma Operator
    >...
    > In list context, it's just the list argument separator, and inserts both its
    > arguments into the list.


    I saw that, but most of these commas are missing an argument on one
    side or both.

    >Basically you just have null arguments to the comma operator after
    >the 15 and 16.


    I don't know of anywhere in Perl where there's a concept of "null
    argument", or even where "null" might occur at all (except for null
    string and NUL character). They certainly aren't undef arguments
    (which I might have expected) or the output would have shown them.

    --
    Tim McDaniel,
     
    Tim McDaniel, Dec 19, 2013
    #3
  4. Tim McDaniel

    Tim McDaniel Guest

    In article <l8tosi$eq4$>,
    Tim McDaniel <> wrote:
    >I don't know of anywhere in Perl where there's a concept of "null
    >argument", or even where "null" might occur at all (except for null
    >string and NUL character).


    And null statement.

    I shouldn't have gone down that path. Anyway, I don't know of any
    "null arguments" anywhere.

    $a = $b + ;

    is a syntax error (hell, I'd better check that ... yup) as is

    $c = >> $d;

    And things that might look like something with an implied operand are
    actually a different operator, like

    $e = * $f; # Can't use an undefined value as a symbol reference at -e line 1.

    $g = - h; # Results in "-h"

    --
    Tim McDaniel,
     
    Tim McDaniel, Dec 19, 2013
    #4
  5. Tim McDaniel

    $Bill Guest

    On 12/18/2013 19:24, Tim McDaniel wrote:
    > In article <l8te5m$87t$>, $Bill <> wrote:
    >> On 12/18/2013 13:53, Tim McDaniel wrote:
    >>>
    >>> $ perl -e 'use strict; use warnings; use Data::Dumper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
    >>> $VAR1 = [
    >>> 15,
    >>> 16
    >>> ];
    >>>
    >>> The exception is that it doesn't allow a leading comma just after the
    >>> "(".
    >>>
    >>> Why in the dickens are extra commas permitted in most places?

    >>
    >> perlop
    >> ...
    >> Comma Operator
    >> ...
    >> In list context, it's just the list argument separator, and inserts both its
    >> arguments into the list.

    >
    > I saw that, but most of these commas are missing an argument on one
    > side or both.
    >
    >> Basically you just have null arguments to the comma operator after
    >> the 15 and 16.

    >
    > I don't know of anywhere in Perl where there's a concept of "null
    > argument",


    Would missing argument give you a warmer fuzzier feeling ? :)

    It's a list with null/missing arguments in the list and therefore there
    is nothing to insert into the list for the null/missing arguments.

    or even where "null" might occur at all (except for null
    > string and NUL character).


    Null \Null\, a. [L. nullus not any, none; ne not + ullus any, a
    dim. of unus one; cf. F. nul. See {No}, and {One}, and cf.
    {None}.]

    3. (Math.) Empty; having no members; as, the null set.

    They certainly aren't undef arguments
    > (which I might have expected) or the output would have shown them.
     
    $Bill, Dec 19, 2013
    #5
  6. Tim McDaniel

    $Bill Guest

    On 12/18/2013 19:31, Tim McDaniel wrote:
    > In article <l8tosi$eq4$>,
    > Tim McDaniel <> wrote:
    >> I don't know of anywhere in Perl where there's a concept of "null
    >> argument", or even where "null" might occur at all (except for null
    >> string and NUL character).

    >
    > And null statement.
    >
    > I shouldn't have gone down that path. Anyway, I don't know of any
    > "null arguments" anywhere.
    >
    > $a = $b + ;
    >
    > is a syntax error (hell, I'd better check that ... yup) as is
    >
    > $c = >> $d;


    It obviously depends on the operator and the context. The only operator
    we're talking about is the comma operator and only in a list context
    (totally different results in a scalar context).

    > And things that might look like something with an implied operand are
    > actually a different operator, like
    >
    > $e = * $f; # Can't use an undefined value as a symbol reference at -e line 1.
    >
    > $g = - h; # Results in "-h"
    >
     
    $Bill, Dec 19, 2013
    #6
  7. Am 18.12.2013 22:53, schrieb Tim McDaniel:
    > So I figured that allowing one trailing comma was merely special-cased
    > to allow for neat diffing, like in a revision-control system diffing
    > old
    > %a = (
    > FIELD1 => 23,
    > FIELD2 => 45,
    > );

    ....
    >
    > Why in the dickens are extra commas permitted in most places?


    Beside the syntactic parsing reasons,
    it has the benefit of easy maintanance.

    Imagine you'd have something in addition like
    %b = (
    FIELD3 => 81,
    FIELD4 => 99,
    );

    and for w/e reason you want to combine then in source code to

    %ab = (
    ...
    ...
    );

    with the extra unnecessairy comma at the end,
    you just can copy and paste the data lines,
    without them, you'd have to insert a , at some place.
    Especially when working with configuration, that gets disturbing and
    when dynamic evaluation comes into play, also can lead to strange failures.


    Greetings,
    Janek
     
    Janek Schleicher, Dec 19, 2013
    #7
  8. (Tim McDaniel) writes:
    > In article <l8te5m$87t$>, $Bill <> wrote:
    >>On 12/18/2013 13:53, Tim McDaniel wrote:
    >>>
    >>> $ perl -e 'use strict; use warnings; use Data::Dumper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
    >>> $VAR1 = [
    >>> 15,
    >>> 16
    >>> ];
    >>>
    >>> The exception is that it doesn't allow a leading comma just after the
    >>> "(".
    >>>
    >>> Why in the dickens are extra commas permitted in most places?

    >>
    >>perlop
    >>...
    >> Comma Operator
    >>...
    >> In list context, it's just the list argument separator, and inserts both its
    >> arguments into the list.

    >
    > I saw that, but most of these commas are missing an argument on one
    > side or both.


    The key is the "it's just the list argument separator" here which
    implies that a comma in list context is not 'the comma operator' (which
    takes two arguments, evaluates both left-to-right and returns the
    result of the right one[*]) but a separator. This means it is used while
    creating a token sequence during syntactical analysis: Once the lexer
    encounters a , the current token is finished and it starts accumulating
    a new one. Enter another comma --- no token yet, continue looking. And
    so on[**].

    [*] Technically, the scalar-context comma operator is also not an operator
    in the sense that + is, it's rather a compiler directive.

    [**] This description might seem inaccurate, eg it is possible to
    define a sub

    sub digits
    {
    return 0,,1,2,3,4,5,6,7,8,9;
    }

    which, when executed in list context, returns a list of digits but
    ends up returning only the last element of this list in scalar context.
    However, what happens here is just that all list elements are pushed on
    the stack from left to right. In scalar context, the element at the top
    of the stack kept while all others are removed.
     
    Rainer Weikusat, Dec 19, 2013
    #8
  9. Tim McDaniel

    Tim McDaniel Guest

    In article <>,
    Janek Schleicher <> wrote:
    >with the extra unnecessairy comma at the end,
    >you just can copy and paste the data lines,
    >without them, you'd have to insert a , at some place.


    Yes, I understand that. As I wrote:
    > So I figured that allowing one trailing comma was merely special-cased
    > to allow for neat diffing, like in a revision-control system diffing

    As you wrote, it also helps maintenance.

    In a similar way, Perl allows dropping the semicolon before "}", but I
    always put one there anyway (with limited exceptions: one-line or
    few-line code blocks for sort, grep, and map).

    I understand the concept of "null statement" and that it's
    occasionally useful, so I have a mental model of why
    perl -e '{print "a\n";;;;;;;;;;;;;;; print "b\n"}'
    works fine.

    What I'm asking is why Perl allows and ignores multiple internal
    commas in a list. What benefit could this possibly give? Does that
    benefit outweigh being silent on the occasional of leaving off an
    operand?

    --
    Tim McDaniel,
     
    Tim McDaniel, Dec 19, 2013
    #9
  10. (Tim McDaniel) wrote:
    >In a similar way, Perl allows dropping the semicolon before "}",


    Actually, that's not quite correct.
    In Perl the semicolon is a statement seperator instead of a statement
    terminator as in some other programming languages.
    Therefore Perl allows you to create an emtpy statement, e.g. in front of
    a "}", by adding an additional semicolon if it pleases you.

    That's the same age-old question as if
    "foo\nbar\n\buz"
    is a text with 3 lines or a text with 2 lines plus some garbage after
    the second line.

    jue
     
    Jürgen Exner, Dec 19, 2013
    #10
  11. Rainer Weikusat <> writes:
    > (Tim McDaniel) writes:
    >> In article <l8te5m$87t$>, $Bill <> wrote:
    >>>On 12/18/2013 13:53, Tim McDaniel wrote:
    >>>>
    >>>> $ perl -e 'use strict; use warnings; use Data::Dumper; my @a = (15,,,,,,16,,,,); print Dumper(\@a), "\n"'
    >>>> $VAR1 = [
    >>>> 15,
    >>>> 16
    >>>> ];
    >>>>
    >>>> The exception is that it doesn't allow a leading comma just after the
    >>>> "(".
    >>>>
    >>>> Why in the dickens are extra commas permitted in most places?
    >>>
    >>>perlop
    >>>...
    >>> Comma Operator
    >>>...
    >>> In list context, it's just the list argument separator, and inserts both its
    >>> arguments into the list.

    >>
    >> I saw that, but most of these commas are missing an argument on one
    >> side or both.

    >
    > The key is the "it's just the list argument separator" here which
    > implies that a comma in list context is not 'the comma operator' (which
    > takes two arguments, evaluates both left-to-right and returns the
    > result of the right one[*]) but a separator.


    This idea is unfortunately unsuitable for describing the behaviour of
    perl because commatose expression apparently generally work, cf

    perl -e 'print 5 if 0,,,,5'
     
    Rainer Weikusat, Dec 19, 2013
    #11
  12. Ben Morrow <> writes:
    > Quoth Rainer Weikusat <>:
    >>
    >> The key is the "it's just the list argument separator" here which
    >> implies that a comma in list context is not 'the comma operator' (which
    >> takes two arguments, evaluates both left-to-right and returns the
    >> result of the right one[*]) but a separator.

    >
    > There is only one comma operator. It always parses the same, and it
    > always compiles down to the same opcode (it has to, because context
    > isn't always known at compile time).


    Judging from the 5.10.1 perly.y, there's actually no such thing as 'a
    comma operator' which would need to be a 'binary operator between
    terms'. The lexer treats it as such but in the parser, it's only (slight
    simplification)

    /* Expressions are a list of terms joined by commas */
    argexpr : argexpr ','
    {
    #ifdef MAD
    OP* op = newNULLLIST();
    token_getmad($2,op,',');
    $$ = append_elem(OP_LIST, $1, op);
    #else
    $$ = $1;
    #endif
    }
    | argexpr ',' term
    {
    OP* term = $3;
    DO_MAD(
    term = newUNOP(OP_NULL, 0, term);
    token_getmad($2,term,',');
    )
    $$ = append_elem(OP_LIST, $1, term);
    }
    | term %prec PREC_LOW
    ;

    ie, there is something like a list operator and the , is used as purely
    syntactical element for separating terms in a list.

    [...]

    >> [*] Technically, the scalar-context comma operator is also not an operator
    >> in the sense that + is, it's rather a compiler directive.

    >
    > Nonsense. The lexer, parser and optree handle it exactly the same way as
    > they handle +, except for the fact that OP_LIST takes multiple arguments.


    At least in certain cases, the list is expanded inline, with no 'list
    operator' anywhere in sight, eg

    [rw@sable]~#perl -MO=Concise,-exec,a -e 'sub a { return ($_[0], $_[3], $_[2]); }'
    main::a:
    1 <;> nextstate(main 1 -e:1) v
    2 <0> pushmark s
    3 <#> aelemfast[*_] s
    4 <#> aelemfast[*_] s/3
    5 <#> aelemfast[*_] s/2
    6 <@> return KP
    7 <1> leavesub[1 ref] K/REFC,1

    In constrast to this, an addition compiles to

    [rw@sable]~#perl -MO=Concise,-exec,a -e 'sub a { return ($_[0] + $_[3] + $_[2]); }'
    main::a:
    1 <;> nextstate(main 1 -e:1) v
    2 <0> pushmark s
    3 <#> aelemfast[*_] s
    4 <#> aelemfast[*_] s/3
    5 <2> add[t3] sK/2
    6 <#> aelemfast[*_] s/2
    7 <2> add[t5] sKP/2
    8 <@> return K
    9 <1> leavesub[1 ref] K/REFC,1

    And in any case,

    [rw@sable]~#perl -e 'LIST =~ /,|comma/i or print "something else\n"'
    something else

    It would be interesting to know if 'the comma operator' in C is actually
    treated as an operator or if the historical misnomer actually came from
    there.

    NB: I'm very much obliged to you for posting this because it has greatly
    helped my understanding of perl, even though this reply may not sound
    like this. I don't have an affirmtative mind ...
     
    Rainer Weikusat, Dec 20, 2013
    #12
  13. Ben Morrow <> writes:
    > Quoth Rainer Weikusat <>:
    >> Ben Morrow <> writes:


    [...]

    >> > There is only one comma operator. It always parses the same, and it
    >> > always compiles down to the same opcode (it has to, because context
    >> > isn't always known at compile time).

    >>
    >> Judging from the 5.10.1 perly.y, there's actually no such thing as 'a
    >> comma operator' which would need to be a 'binary operator between
    >> terms'. The lexer treats it as such but in the parser, it's only (slight
    >> simplification)
    >>
    >> /* Expressions are a list of terms joined by commas */
    >> argexpr : argexpr ','

    >
    > [Sometime between 5.10.1 and the blead source I posted an extract from
    > before, this production was renamed to 'listexpr'...]
    >
    >> | argexpr ',' term
    >> | term %prec PREC_LOW
    >> ;
    >>
    >> ie, there is something like a list operator and the , is used as purely
    >> syntactical element for separating terms in a list.

    >
    > OK, you can call it that if you like; it's not really any different from
    > the ?: operator, which uses '?' and ':' to separate its three
    > arguments.


    If it really wasn't different, you'd refer to 'the question mark
    operator' and 'the colon operator' in this case and this would be as
    inadequate because they're also just syntactical elements of something
    else.

    > My point was that 'the scalar comma operator' and 'the list context list
    > construction operator' are actually the same operator, whose behaviour
    > is context-dependant.


    There is no such thing as 'a comma operator' in the Perl grammar (at
    least, I didn't find anything like that), commas are used as separators
    in a certain kind of compound expression. As Tim McDaniel observed,
    despite comma is documented as binary operator, it doesn't behave like
    one: It has no arity. The other hypothesis, "it is a binary operator but
    one which can have imaginary arguments" is to be rejected on the grounds
    that it introduces artifical complications in order to bend reality such
    that it fits with terminology (applying Occam's razor).

    >
    >> >> [*] Technically, the scalar-context comma operator is also not an operator
    >> >> in the sense that + is, it's rather a compiler directive.
    >> >
    >> > Nonsense. The lexer, parser and optree handle it exactly the same way as
    >> > they handle +, except for the fact that OP_LIST takes multiple arguments.

    >>
    >> At least in certain cases, the list is expanded inline, with no 'list
    >> operator' anywhere in sight, eg
    >>
    >> [rw@sable]~#perl -MO=Concise,-exec,a -e 'sub a { return ($_[0], $_[3],
    >> $_[2]); }'
    >> main::a:
    >> 1 <;> nextstate(main 1 -e:1) v
    >> 2 <0> pushmark s
    >> 3 <#> aelemfast[*_] s
    >> 4 <#> aelemfast[*_] s/3
    >> 5 <#> aelemfast[*_] s/2
    >> 6 <@> return KP
    >> 7 <1> leavesub[1 ref] K/REFC,1

    >
    > That's a slight hack: this is the 'listop' production, which does
    > actually build an OP_LIST, but then, rather than adding an extra op, it
    > converts the OP_LIST into an OP_RETURN. It's easier to see what's
    > happening without -exec:
    >
    > ~/src/perl/perl% perl -MO=Concise,a -e'sub a { $x, $y, $z }'
    > main::a:
    > 7 <1> leavesub[1 ref] K/REFC,1 ->(end)
    > - <@> lineseq KP ->7
    > 1 <;> nextstate(main 1 -e:1) v ->2
    > 6 <@> list K ->7
    > 2 <0> pushmark s ->3
    > - <1> ex-rv2sv sK/1 ->4
    > 3 <$> gvsv(*x) s ->4
    > - <1> ex-rv2sv sK/1 ->5
    > 4 <$> gvsv(*y) s ->5
    > - <1> ex-rv2sv sK/1 ->6
    > 5 <$> gvsv(*z) s ->6
    > -e syntax OK
    > ~/src/perl/perl% perl -MO=Concise,a -e'sub a { return $x, $y, $z }'
    > main::a:
    > 7 <1> leavesub[1 ref] K/REFC,1 ->(end)
    > - <@> lineseq KP ->7
    > 1 <;> nextstate(main 1 -e:1) v ->2
    > 6 <@> return K ->7
    > 2 <0> pushmark s ->3
    > - <1> ex-rv2sv sK/1 ->4
    > 3 <$> gvsv(*x) s ->4
    > - <1> ex-rv2sv sK/1 ->5
    > 4 <$> gvsv(*y) s ->5
    > - <1> ex-rv2sv sK/1 ->6
    > 5 <$> gvsv(*z) s ->6
    > -e syntax OK
    >
    > The only different between these two is that perl has turned the 'list'
    > into a 'return'.


    The difference I see here is that there has to be some op (not operator)
    in this place and if no other was provided, a 'list' op is used. That
    'return' happens to be equivalent to that is just an accidental property of
    my example (The same phenomenon can be observed for reverse and grep and
    presumably, for any other list operator returning a list).

    [...]

    >> It would be interesting to know if 'the comma operator' in C is actually
    >> treated as an operator or if the historical misnomer actually came from
    >> there.

    >
    > In C the comma really does have two entirely different meanings: the
    > 'comma operator' which behaves like Perl's comma in scalar context, and
    > the comma-as-an-argument-separator in the argument list of a function
    > call, which is not really an operator at all.


    The main difference is that , is really a binary operator in C, eg this

    ------
    int main(void)
    {
    return 1,, 2;
    }
    ------

    doesn't compile because of a missing operand.
     
    Rainer Weikusat, Dec 22, 2013
    #13
  14. Ben Morrow <> writes:
    > Quoth Rainer Weikusat <>:
    >> Ben Morrow <> writes:
    >> > Quoth Rainer Weikusat <>:
    >> >>
    >> >> ie, there is something like a list operator and the , is used as purely
    >> >> syntactical element for separating terms in a list.
    >> >
    >> > OK, you can call it that if you like; it's not really any different from
    >> > the ?: operator, which uses '?' and ':' to separate its three
    >> > arguments.

    >>
    >> If it really wasn't different, you'd refer to 'the question mark
    >> operator' and 'the colon operator' in this case and this would be as
    >> inadequate because they're also just syntactical elements of something
    >> else.

    >
    > As I said, you can call it what you like. 'The comma operator' is easier
    > to say than 'the list-of-terms-separated-by-(possibly-multiple)-commas
    > operator', but if you'd rather call it OP_LIST or 'the list operator'
    > that's fine by me. (You might get some argument from people who claim
    > there's no such thing as a list in scalar context, an attitude that is
    > unfortunately supported by the documentation.)


    I think refering to the complete syntactical construct as 'list' or
    'list expression' makes more sense than talking about a 'comma operator'
    here, especially considering that ; isn't called 'the semicolon
    operator' despite the striking similarities between the two: A list is a
    comma-separated list of expression which are evaluated from left to
    right and ultimatively end up as arguments of a list operator. If an
    explicit list operator doesn't exist, an implicit "return the list in
    list context, the last element otherwise" list operator is being used
    (which implies that a list expression in scalar context _works_ like the
    C 'comma operator', except that the syntax is less restrictive).
     
    Rainer Weikusat, Dec 30, 2013
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. GIMME
    Replies:
    2
    Views:
    877
    GIMME
    Feb 11, 2004
  2. Jon Maz
    Replies:
    3
    Views:
    1,657
    Jon Maz
    Feb 13, 2004
  3. Mad Scientist Jr
    Replies:
    3
    Views:
    787
    bloomfield
    Jun 21, 2004
  4. AviraM
    Replies:
    2
    Views:
    6,399
    Manish Pandit
    Sep 28, 2006
  5. mathieu
    Replies:
    3
    Views:
    602
    Bo Persson
    Sep 4, 2009
Loading...

Share This Page