s/// parametrized with backreferences

Discussion in 'Perl Misc' started by msciwoj, Mar 11, 2009.

  1. msciwoj

    msciwoj Guest

    Is it possible to use substitution with parameters as arguments -
    especially the replacement part?
    Works for me to some extent but not with backreferences...

    Let's say I'm interested in prompting user for arguments: pattern and
    replacement and he would like to use equivalent of:
    s/(.)\n(.)/\2\n\1/mg

    Continuing, I could have two vars defined, accordingly:

    $pattern='(.)\n(.)'; #$ARGV[1];
    $replacement='\2\n\1'; #$ARGV[2]

    and my idea is to use here:
    $_ =~ s/$pattern/$replacement/mg

    but it doesn't work - (instead using backreferendes, gives '\1' and
    '\2' in the output, what makes sens after all). The question is how
    (if only possible) to use s/// with a replacement part defined in a
    variable (ie. run-time user defined string)?

    Any ideas?
    Thanks!
    m.
    msciwoj, Mar 11, 2009
    #1
    1. Advertising

  2. msciwoj <> writes:

    > Let's say I'm interested in prompting user for arguments: pattern and
    > replacement and he would like to use equivalent of:
    > s/(.)\n(.)/\2\n\1/mg


    If you had run you script with 'use warnings' it would have told you
    that '\1 better written as $1'.

    > Continuing, I could have two vars defined, accordingly:
    >
    > $pattern='(.)\n(.)'; #$ARGV[1];
    > $replacement='\2\n\1'; #$ARGV[2]
    >
    > and my idea is to use here:
    > $_ =~ s/$pattern/$replacement/mg


    By using $1 instead of \1 and by using the correct number of /e
    modifiers you could get something to work:

    $replacement = '\2\n\1'; # $ARGV[2];

    # Massage replacement into something workable:
    $replacement =~ s/\\(\d)/'.\$$1.'/g; # Magic - huh?
    $replacement = qq('$replacement');

    # Do the substitution:
    d/(.*)-(.*)/$replacement/ee;


    But the more I look at it the uglier it seems to become.

    //Makholm
    Peter Makholm, Mar 11, 2009
    #2
    1. Advertising

  3. msciwoj <> wrote:
    > Is it possible to use substitution with parameters as arguments -
    > especially the replacement part?



    The replacement part of s/// is a double quotish string, so your
    question reduces to this FAQ:

    How can I expand variables in text strings?


    > Works for me to some extent but not with backreferences...
    >
    > Let's say I'm interested in prompting user for arguments: pattern and
    > replacement and he would like to use equivalent of:
    > s/(.)\n(.)/\2\n\1/mg



    You should always enable warnings when developing Perl code. That should be:

    s/(.)\n(.)/$2\n$1/mg

    However the s///m modifier only affects the ^ and $ anchors, and since
    you don't use those anchors, you don't need that modifier:

    s/(.)\n(.)/$2\n$1/g


    > Continuing, I could have two vars defined, accordingly:
    >
    > $pattern='(.)\n(.)'; #$ARGV[1];
    > $replacement='\2\n\1'; #$ARGV[2]
    >
    > and my idea is to use here:
    > $_ =~ s/$pattern/$replacement/mg



    If you are reading $_ from a filehandle, then see also:

    perldoc -q match

    I'm having trouble matching over more than one line. What's wrong?


    > The question is how
    > (if only possible) to use s/// with a replacement part defined in a
    > variable (ie. run-time user defined string)?



    -------------------------
    #!/usr/bin/perl
    use warnings;
    use strict;

    my $pattern = '(.)\n(.)';
    my $replacement = '"$2\n$1"'; # note the double quotes for the 1st eval to see

    $_ = "first line\nsecond line\nthird line\nfourth line";

    s/$pattern/$replacement/eeg;

    print;
    -------------------------


    --
    Tad McClellan
    email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
    Tad J McClellan, Mar 11, 2009
    #3
  4. msciwoj

    msciwoj Guest

    Works!
    Thanks Tom!

    I'm not an expert and don't understand why we need double quotes with
    one more eval step (ee modifier) while one eval step without quotes
    should also work in my opinion.
    In other words having:

    my $pattern = '(.)\n(.)';
    my $replacement1 = '"$2\n$1"';
    my $replacement2 = '$2\n$1';

    why:
    s/$pattern/$replacement1/eeg;
    WORKS, while:
    s/$pattern/$replacement2/eg;
    NOT?

    Thanks
    m.
    msciwoj, Mar 11, 2009
    #4
  5. msciwoj <> wrote:

    > Thanks Tom!



    My name is Tad.


    > I'm not an expert and don't understand why we need double quotes with
    > one more eval step (ee modifier)



    I made a thinko.

    The double quotes are for the *second* eval, not for the first eval.


    > while one eval step without quotes
    > should also work in my opinion.
    > In other words having:
    >
    > my $pattern = '(.)\n(.)';
    > my $replacement1 = '"$2\n$1"';
    > my $replacement2 = '$2\n$1';
    >
    > why:
    > s/$pattern/$replacement1/eeg;



    What does the 1st eval see?

    $replacement1

    so it evaluates it, and the 2nd eval sees the value of $replacement1

    "$2\n$1"

    so it interpolates the values for $2 and $1 into the replacement string.

    If you don't put the double quotes in $replacement1, then the 2nd eval sees:

    $2\n$1

    which is a syntax error, just as

    #!/usr/bin/perl
    use strict;
    use warnings;
    $2\n$1

    results in a syntax error.


    > WORKS, while:
    > s/$pattern/$replacement2/eg;



    What does the eval see?

    $replacement2

    so it evaluates it and the _data_ (not code)

    $2\n$1

    is used as the replacement text.


    > NOT?



    Because that is what it is supposed to do.


    --
    Tad McClellan
    email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
    Tad J McClellan, Mar 11, 2009
    #5
  6. msciwoj

    msciwoj Guest

    Tad,

    Sorry, I was bit distracted
    Apologies

    I see the reason why single eval doesnt work with variable (without
    double quotes).
    If double eval works with double quotes then this also should work
    (just a check):
    s/$pattern/"$2\n$1"/ge;
    and it works indeed!

    The mystery for me is this 'second' evaluation actually, like you
    said, "interpolates the values for $2 and $1 into the replacement
    string" and it seems it's the only way that does the job.
    A bit of mystery for me but at the end it works and only this counts,
    I guess.

    Thanks again,
    m.
    msciwoj, Mar 11, 2009
    #6
  7. Tad J McClellan <> writes:

    > my $pattern = '(.)\n(.)';
    > my $replacement = '"$2\n$1"'; # note the double quotes for the 1st eval to see


    Doh. Didn't think of that.

    Much nicer than my workaround...

    //Makholm
    Peter Makholm, Mar 12, 2009
    #7
  8. msciwoj

    Guest

    On Wed, 11 Mar 2009 16:24:17 -0700 (PDT), msciwoj <> wrote:

    >Tad,
    >
    >Sorry, I was bit distracted
    >Apologies
    >
    >I see the reason why single eval doesnt work with variable (without
    >double quotes).
    >If double eval works with double quotes then this also should work
    >(just a check):
    >s/$pattern/"$2\n$1"/ge;
    >and it works indeed!
    >
    >The mystery for me is this 'second' evaluation actually, like you
    >said, "interpolates the values for $2 and $1 into the replacement
    >string" and it seems it's the only way that does the job.
    >A bit of mystery for me but at the end it works and only this counts,
    >I guess.
    >
    >Thanks again,
    >m.


    Indeed, the question is valid.

    In the nut of it, evaling a variable with a value '$str'
    will work, but with a value of '$str\n$str' will not.

    But if you notice, the vaule of '$str\n$str' and '$str\\n$str'
    are equal. In escence, there is a conflict with the parser on some
    level that is not quite understood as pertaining to eval.

    Whitness the below code. All works fine. Replace "replacement2" variable
    with this value: '$2\n' and watch the error's.

    There are other constructs that work when it shouldn't and visa versa,
    so the Tad explanation is not even close to being correct. About the only
    thing he got correct is /ee only works when there is double interpretation,
    but then it works when the evals are broken apart in steps, as a single eval,
    but then only on some constructs.

    Beware of simple explanations!

    -sln

    =========================================================

    use strict;
    use warnings;

    my ($str1,$str2,$e);
    my $data = <<EOT;
    A
    B
    C
    EOT

    print "\n0: -------------\n\n";

    $str1 = 'hello';
    $str2 = '$str1';
    print $str2,"\n";
    print eval($str2),"\n";

    print "\n1: -------------\n\n";

    my $replacement2 = '$2';
    print $replacement2,"\n";

    print "\n2: -------------\n\n";

    $data =~ /(.)\n(.)/;
    $e = eval ($replacement2);
    print "->",$e,"\n";

    print "\n3: -------------\n\n";

    $data =~ s/(.)\n(.)/eval($replacement2)/eg;
    print $data."\n";
    , Mar 13, 2009
    #8
  9. msciwoj <> wrote:

    > I see the reason why single eval doesnt work with variable (without
    > double quotes).
    > If double eval works with double quotes then this also should work
    > (just a check):
    > s/$pattern/"$2\n$1"/ge;
    > and it works indeed!



    That is because the replacement part is "double quotish" (without s///e).

    Once you add the s///e modifier, it is now an expression to be evaluated.

    This does the same thing too:

    s/$pattern/ $2 . "\n" . $1 /ge;


    > The mystery for me is this 'second' evaluation actually, like you
    > said, "interpolates the values for $2 and $1 into the replacement
    > string" and it seems it's the only way that does the job.



    Any expression that results in the string you want to use as
    the replacement text will do the job.

    In my original code, you could use any of the below to get the
    same results as the original did:

    my $replacement = '$2 . "\n" . $1';
    or
    my $replacement = '$2 . $/ . $1';
    or even (if on *nix)
    my $replacement = '$2 . chr(10) . $1';


    --
    Tad McClellan
    email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
    Tad J McClellan, Mar 13, 2009
    #9
  10. <> wrote:
    > On Wed, 11 Mar 2009 16:24:17 -0700 (PDT), msciwoj <> wrote:


    >>The mystery for me is this 'second' evaluation actually, like you
    >>said, "interpolates the values for $2 and $1 into the replacement
    >>string" and it seems it's the only way that does the job.
    >>A bit of mystery for me but at the end it works and only this counts,
    >>I guess.



    > In the nut of it, evaling a variable with a value '$str'
    > will work,



    Because the value is a valid Perl expression.


    > but with a value of '$str\n$str' will not.



    Because the value is not a valid Perl expression.


    > But if you notice, the vaule of '$str\n$str' and '$str\\n$str'
    > are equal.



    That is how single quoted strings work.

    The "Scalar value constructors" section in perldata points out that
    there are only 2 special characters in a single quoted string.

    Single quote is special (it marks the end of the string) unless
    it is preceded by a backslash.

    Backslash is special if it preceeds a single quote or a backslash.

    A backslash that does not preceed 1 of those 2 characters is literal.

    So the 1st string above has a literal backslash and an "n" in it.

    The 2nd string above has a backslashed backslash (which becomes
    a backslash) and an "n" in it.


    > In escence, there is a conflict with the parser on some
    > level that is not quite understood



    Don't let that stop you from pretending to answer questions about it.


    > as pertaining to eval.



    It has nothing to do with eval.

    It has to do with single-quoted strings.


    > Whitness the below code. All works fine. Replace "replacement2" variable
    > with this value: '$2\n' and watch the error's.



    That is because in the code below, the value of $replacement2
    is a valid expression:

    perl -e '$2'

    But the value you suggest above is not a valid expression:

    perl -e '$2\n'

    Throws a syntax error just as this does:

    #!/usr/bin/perl
    use warnings;
    use strict;
    $2\n


    > There are other constructs that work when it shouldn't and visa versa,



    No there aren't.

    Those are the result of you not understanding how things work.

    When you discover one, post it here and learn what you are missing.


    > so the Tad explanation is not even close to being correct.



    My explanation is much more correct than your misguided attempt.


    > About the only
    > thing he got correct is /ee only works when there is double interpretation,

    ^^^^^^^^^^

    I did not say that.


    > but then it works when the evals are broken apart in steps, as a single eval,



    No it doesn't.


    > but then only on some constructs.



    Show us one.


    > Beware of simple explanations!



    Beware of simpletons providing "explanations"!



    >=========================================================
    >
    > use strict;
    > use warnings;
    >
    > my ($str1,$str2,$e);
    > my $data = <<EOT;
    > A
    > B
    > C
    > EOT
    >
    > print "\n0: -------------\n\n";
    >
    > $str1 = 'hello';
    > $str2 = '$str1';
    > print $str2,"\n";
    > print eval($str2),"\n";
    >
    > print "\n1: -------------\n\n";
    >
    > my $replacement2 = '$2';
    > print $replacement2,"\n";
    >
    > print "\n2: -------------\n\n";
    >
    > $data =~ /(.)\n(.)/;
    > $e = eval ($replacement2);
    > print "->",$e,"\n";
    >
    > print "\n3: -------------\n\n";
    >
    > $data =~ s/(.)\n(.)/eval($replacement2)/eg;
    > print $data."\n";
    >
    >



    --
    Tad McClellan
    email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
    Tad J McClellan, Mar 13, 2009
    #10
  11. msciwoj

    Guest

    On Fri, 13 Mar 2009 17:42:34 -0500, Tad J McClellan <> wrote:

    > <> wrote:
    >> On Wed, 11 Mar 2009 16:24:17 -0700 (PDT), msciwoj <> wrote:

    >
    >>>The mystery for me is this 'second' evaluation actually, like you
    >>>said, "interpolates the values for $2 and $1 into the replacement
    >>>string" and it seems it's the only way that does the job.
    >>>A bit of mystery for me but at the end it works and only this counts,
    >>>I guess.

    >
    >
    >> In the nut of it, evaling a variable with a value '$str'
    >> will work,

    >
    >
    >Because the value is a valid Perl expression.
    >
    >
    >> but with a value of '$str\n$str' will not.

    >
    >
    >Because the value is not a valid Perl expression.
    >
    >
    >> But if you notice, the vaule of '$str\n$str' and '$str\\n$str'
    >> are equal.

    >
    >
    >That is how single quoted strings work.
    >
    >The "Scalar value constructors" section in perldata points out that
    >there are only 2 special characters in a single quoted string.
    >
    >Single quote is special (it marks the end of the string) unless
    >it is preceded by a backslash.
    >
    >Backslash is special if it preceeds a single quote or a backslash.
    >
    >A backslash that does not preceed 1 of those 2 characters is literal.
    >
    >So the 1st string above has a literal backslash and an "n" in it.
    >
    >The 2nd string above has a backslashed backslash (which becomes
    >a backslash) and an "n" in it.
    >
    >
    >> In escence, there is a conflict with the parser on some
    >> level that is not quite understood

    >
    >
    >Don't let that stop you from pretending to answer questions about it.
    >
    >
    >> as pertaining to eval.

    >
    >
    >It has nothing to do with eval.
    >
    >It has to do with single-quoted strings.
    >
    >
    >> Whitness the below code. All works fine. Replace "replacement2" variable
    >> with this value: '$2\n' and watch the error's.

    >
    >
    >That is because in the code below, the value of $replacement2
    >is a valid expression:
    >
    > perl -e '$2'
    >
    >But the value you suggest above is not a valid expression:
    >
    > perl -e '$2\n'
    >


    Oh, I get it. Its all about "valid expression" now and not eval in regex's.
    And now since you have stated '$2' is a valid expression and '$2\n' isn't,
    it raises the question of the ilogic in your nothing, so nothing argument.

    Now, given the "valid expression" of '$2', this should be equivalent in terms of eval:

    $replacement2 = '$2';
    $data =~ /(.)\n(.)/;
    $e = eval ($replacement2);
    print "->",$e,"\n";

    $data =~ s/(.)\n(.)/$replacement2/eg;
    print $data."\n";

    Unfortunately, you forgot your brain today Tad.

    -sln
    , Mar 14, 2009
    #11
  12. <> wrote:
    > On Fri, 13 Mar 2009 17:42:34 -0500, Tad J McClellan <> wrote:
    >


    >>> In the nut of it, evaling a variable with a value '$str'
    >>> will work,

    >>
    >>
    >>Because the value is a valid Perl expression.
    >>
    >>
    >>> but with a value of '$str\n$str' will not.

    >>
    >>
    >>Because the value is not a valid Perl expression.



    > Oh, I get it. Its all about "valid expression"



    You're welcome.


    --
    Tad McClellan
    email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
    Tad J McClellan, Mar 14, 2009
    #12
  13. msciwoj

    Guest

    On Fri, 13 Mar 2009 21:48:55 -0500, Tad J McClellan <> wrote:

    > <> wrote:
    >> On Fri, 13 Mar 2009 17:42:34 -0500, Tad J McClellan <> wrote:
    >>

    >
    >>>> In the nut of it, evaling a variable with a value '$str'
    >>>> will work,
    >>>
    >>>
    >>>Because the value is a valid Perl expression.
    >>>
    >>>
    >>>> but with a value of '$str\n$str' will not.
    >>>
    >>>
    >>>Because the value is not a valid Perl expression.

    >
    >
    >> Oh, I get it. Its all about "valid expression"

    >
    >
    >You're welcome.


    No thanks! But you are welcome.

    Here's some valid expression for ya Tad:

    $replacement2 = '$2';
    $data =~ /(.)\n(.)/;
    $e = eval ($replacement2);
    print "->",$e,"\n";

    $data =~ s/(.)\n(.)/$replacement2/eg;
    print $data."\n";
    , Mar 14, 2009
    #13
  14. <> wrote:
    > On Fri, 13 Mar 2009 21:48:55 -0500, Tad J McClellan <> wrote:
    >
    >> <> wrote:
    >>> On Fri, 13 Mar 2009 17:42:34 -0500, Tad J McClellan <> wrote:
    >>>

    >>
    >>>>> In the nut of it, evaling a variable with a value '$str'
    >>>>> will work,
    >>>>
    >>>>
    >>>>Because the value is a valid Perl expression.
    >>>>
    >>>>
    >>>>> but with a value of '$str\n$str' will not.
    >>>>
    >>>>
    >>>>Because the value is not a valid Perl expression.

    >>
    >>
    >>> Oh, I get it. Its all about "valid expression"

    >>
    >>
    >>You're welcome.

    >
    > No thanks! But you are welcome.
    >
    > Here's some valid expression for ya Tad:
    >
    > $replacement2 = '$2';
    > $data =~ /(.)\n(.)/;
    > $e = eval ($replacement2);
    > print "->",$e,"\n";
    >
    > $data =~ s/(.)\n(.)/$replacement2/eg;
    > print $data."\n";



    That works as expected. Since there is only one eval with s///e,
    the result after evaluating $replacement2 is the string '$2'.

    What were you expecting it to do instead?

    If you were expecting the last print to print the value of the $2
    variable, then you don't have enough levels of eval to get to where
    '$2' is evaluated.

    The 2nd level of eval can be gotten with either

    $data =~ s/(.)\n(.)/$replacement2/eeg;

    or

    $data =~ s/(.)\n(.)/$e/eg;


    --
    Tad McClellan
    email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
    Tad J McClellan, Mar 14, 2009
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark Fletcher
    Replies:
    1
    Views:
    477
    Mark Fletcher
    May 19, 2004
  2. Chris Nolte
    Replies:
    9
    Views:
    4,259
    Jeff Schwab
    May 25, 2004
  3. dhek bhun kho

    java.util.regex: Backreferences?

    dhek bhun kho, Jul 9, 2003, in forum: Java
    Replies:
    2
    Views:
    774
    dhek bhun kho
    Jul 9, 2003
  4. Amy Dillavou

    backreferences

    Amy Dillavou, Sep 28, 2005, in forum: Python
    Replies:
    4
    Views:
    427
    Peter
    Sep 28, 2005
  5. Pankaj

    Backreferences in python ?

    Pankaj, Jan 23, 2006, in forum: Python
    Replies:
    7
    Views:
    3,252
    Sion Arrowsmith
    Jan 24, 2006
Loading...

Share This Page