s/// parametrized with backreferences

[msciwoj]

Is it possible to use substitution with parameters as arguments -
especially the replacement part?
Works for me to some extent but not with backreferences...

Let's say I'm interested in prompting user for arguments: pattern and
replacement and he would like to use equivalent of:
s/(.)\n(.)/\2\n\1/mg

Continuing, I could have two vars defined, accordingly:

$pattern='(.)\n(.)'; #$ARGV[1];
$replacement='\2\n\1'; #$ARGV[2]

and my idea is to use here:
$_ =~ s/$pattern/$replacement/mg

but it doesn't work - (instead using backreferendes, gives '\1' and
'\2' in the output, what makes sens after all). The question is how
(if only possible) to use s/// with a replacement part defined in a
variable (ie. run-time user defined string)?

Any ideas?
Thanks!
m.

 

[Peter Makholm]

Let's say I'm interested in prompting user for arguments: pattern and
replacement and he would like to use equivalent of:
s/(.)\n(.)/\2\n\1/mg

If you had run you script with 'use warnings' it would have told you
that '\1 better written as $1'.

Continuing, I could have two vars defined, accordingly:

$pattern='(.)\n(.)'; #$ARGV[1];
$replacement='\2\n\1'; #$ARGV[2]

and my idea is to use here:
$_ =~ s/$pattern/$replacement/mg

By using $1 instead of \1 and by using the correct number of /e
modifiers you could get something to work:

$replacement = '\2\n\1'; # $ARGV[2];

# Massage replacement into something workable:
$replacement =~ s/\\(\d)/'.\$$1.'/g; # Magic - huh?
$replacement = qq('$replacement');

# Do the substitution:
d/(.*)-(.*)/$replacement/ee;


But the more I look at it the uglier it seems to become.

//Makholm

 

[Tad J McClellan]

Is it possible to use substitution with parameters as arguments -
especially the replacement part?

The replacement part of s/// is a double quotish string, so your
question reduces to this FAQ:

How can I expand variables in text strings?

Works for me to some extent but not with backreferences...

Let's say I'm interested in prompting user for arguments: pattern and
replacement and he would like to use equivalent of:
s/(.)\n(.)/\2\n\1/mg

You should always enable warnings when developing Perl code. That should be:

s/(.)\n(.)/$2\n$1/mg

However the s///m modifier only affects the ^ and $ anchors, and since
you don't use those anchors, you don't need that modifier:

s/(.)\n(.)/$2\n$1/g

Continuing, I could have two vars defined, accordingly:

$pattern='(.)\n(.)'; #$ARGV[1];
$replacement='\2\n\1'; #$ARGV[2]

and my idea is to use here:
$_ =~ s/$pattern/$replacement/mg

If you are reading $_ from a filehandle, then see also:

perldoc -q match

I'm having trouble matching over more than one line. What's wrong?

The question is how
(if only possible) to use s/// with a replacement part defined in a
variable (ie. run-time user defined string)?

-------------------------
#!/usr/bin/perl
use warnings;
use strict;

my $pattern = '(.)\n(.)';
my $replacement = '"$2\n$1"'; # note the double quotes for the 1st eval to see

$_ = "first line\nsecond line\nthird line\nfourth line";

s/$pattern/$replacement/eeg;

print;

 

[msciwoj]

Works!
Thanks Tom!

I'm not an expert and don't understand why we need double quotes with
one more eval step (ee modifier) while one eval step without quotes
should also work in my opinion.
In other words having:

my $pattern = '(.)\n(.)';
my $replacement1 = '"$2\n$1"';
my $replacement2 = '$2\n$1';

why:
s/$pattern/$replacement1/eeg;
WORKS, while:
s/$pattern/$replacement2/eg;
NOT?

Thanks
m.

 

[Tad J McClellan]

Thanks Tom!

My name is Tad.

I'm not an expert and don't understand why we need double quotes with
one more eval step (ee modifier)

I made a thinko.

The double quotes are for the *second* eval, not for the first eval.

while one eval step without quotes
should also work in my opinion.
In other words having:

my $pattern = '(.)\n(.)';
my $replacement1 = '"$2\n$1"';
my $replacement2 = '$2\n$1';

why:
s/$pattern/$replacement1/eeg;

What does the 1st eval see?

$replacement1

so it evaluates it, and the 2nd eval sees the value of $replacement1

"$2\n$1"

so it interpolates the values for $2 and $1 into the replacement string.

If you don't put the double quotes in $replacement1, then the 2nd eval sees:

$2\n$1

which is a syntax error, just as

#!/usr/bin/perl
use strict;
use warnings;
$2\n$1

results in a syntax error.

WORKS, while:
s/$pattern/$replacement2/eg;

What does the eval see?

$replacement2

so it evaluates it and the _data_ (not code)

$2\n$1

is used as the replacement text.

NOT?

Because that is what it is supposed to do.

 

[msciwoj]

Tad,

Sorry, I was bit distracted
Apologies

I see the reason why single eval doesnt work with variable (without
double quotes).
If double eval works with double quotes then this also should work
(just a check):
s/$pattern/"$2\n$1"/ge;
and it works indeed!

The mystery for me is this 'second' evaluation actually, like you
said, "interpolates the values for $2 and $1 into the replacement
string" and it seems it's the only way that does the job.
A bit of mystery for me but at the end it works and only this counts,
I guess.

Thanks again,
m.

 

[Peter Makholm]

my $pattern = '(.)\n(.)';
my $replacement = '"$2\n$1"'; # note the double quotes for the 1st eval to see

Doh. Didn't think of that.

Much nicer than my workaround...

//Makholm

 

[sln]

Tad,

Sorry, I was bit distracted
Apologies

I see the reason why single eval doesnt work with variable (without
double quotes).
If double eval works with double quotes then this also should work
(just a check):
s/$pattern/"$2\n$1"/ge;
and it works indeed!

The mystery for me is this 'second' evaluation actually, like you
said, "interpolates the values for $2 and $1 into the replacement
string" and it seems it's the only way that does the job.
A bit of mystery for me but at the end it works and only this counts,
I guess.

Thanks again,
m.

Indeed, the question is valid.

In the nut of it, evaling a variable with a value '$str'
will work, but with a value of '$str\n$str' will not.

But if you notice, the vaule of '$str\n$str' and '$str\\n$str'
are equal. In escence, there is a conflict with the parser on some
level that is not quite understood as pertaining to eval.

Whitness the below code. All works fine. Replace "replacement2" variable
with this value: '$2\n' and watch the error's.

There are other constructs that work when it shouldn't and visa versa,
so the Tad explanation is not even close to being correct. About the only
thing he got correct is /ee only works when there is double interpretation,
but then it works when the evals are broken apart in steps, as a single eval,
but then only on some constructs.

Beware of simple explanations!

-sln

=========================================================

use strict;
use warnings;

my ($str1,$str2,$e);
my $data = <<EOT;
A
B
C
EOT

print "\n0: -------------\n\n";

$str1 = 'hello';
$str2 = '$str1';
print $str2,"\n";
print eval($str2),"\n";

print "\n1: -------------\n\n";

my $replacement2 = '$2';
print $replacement2,"\n";

print "\n2: -------------\n\n";

$data =~ /(.)\n(.)/;
$e = eval ($replacement2);
print "->",$e,"\n";

print "\n3: -------------\n\n";

$data =~ s/(.)\n(.)/eval($replacement2)/eg;
print $data."\n";

 

[Tad J McClellan]

I see the reason why single eval doesnt work with variable (without
double quotes).
If double eval works with double quotes then this also should work
(just a check):
s/$pattern/"$2\n$1"/ge;
and it works indeed!

That is because the replacement part is "double quotish" (without s///e).

Once you add the s///e modifier, it is now an expression to be evaluated.

This does the same thing too:

s/$pattern/ $2 . "\n" . $1 /ge;

The mystery for me is this 'second' evaluation actually, like you
said, "interpolates the values for $2 and $1 into the replacement
string" and it seems it's the only way that does the job.

Any expression that results in the string you want to use as
the replacement text will do the job.

In my original code, you could use any of the below to get the
same results as the original did:

my $replacement = '$2 . "\n" . $1';
or
my $replacement = '$2 . $/ . $1';
or even (if on *nix)
my $replacement = '$2 . chr(10) . $1';

 

[Tad J McClellan]

In the nut of it, evaling a variable with a value '$str'
will work,

Because the value is a valid Perl expression.

but with a value of '$str\n$str' will not.

Because the value is not a valid Perl expression.

But if you notice, the vaule of '$str\n$str' and '$str\\n$str'
are equal.

That is how single quoted strings work.

The "Scalar value constructors" section in perldata points out that
there are only 2 special characters in a single quoted string.

Single quote is special (it marks the end of the string) unless
it is preceded by a backslash.

Backslash is special if it preceeds a single quote or a backslash.

A backslash that does not preceed 1 of those 2 characters is literal.

So the 1st string above has a literal backslash and an "n" in it.

The 2nd string above has a backslashed backslash (which becomes
a backslash) and an "n" in it.

In escence, there is a conflict with the parser on some
level that is not quite understood

Don't let that stop you from pretending to answer questions about it.

as pertaining to eval.

It has nothing to do with eval.

It has to do with single-quoted strings.

Whitness the below code. All works fine. Replace "replacement2" variable
with this value: '$2\n' and watch the error's.

That is because in the code below, the value of $replacement2
is a valid expression:

perl -e '$2'

But the value you suggest above is not a valid expression:

perl -e '$2\n'

Throws a syntax error just as this does:

#!/usr/bin/perl
use warnings;
use strict;
$2\n

There are other constructs that work when it shouldn't and visa versa,

No there aren't.

Those are the result of you not understanding how things work.

When you discover one, post it here and learn what you are missing.

so the Tad explanation is not even close to being correct.

My explanation is much more correct than your misguided attempt.

About the only
thing he got correct is /ee only works when there is double interpretation,

^^^^^^^^^^

I did not say that.

but then it works when the evals are broken apart in steps, as a single eval,

No it doesn't.

but then only on some constructs.

Show us one.

Beware of simple explanations!

Beware of simpletons providing "explanations"!

 

[sln]

Because the value is a valid Perl expression.




Because the value is not a valid Perl expression.




That is how single quoted strings work.

The "Scalar value constructors" section in perldata points out that
there are only 2 special characters in a single quoted string.

Single quote is special (it marks the end of the string) unless
it is preceded by a backslash.

Backslash is special if it preceeds a single quote or a backslash.

A backslash that does not preceed 1 of those 2 characters is literal.

So the 1st string above has a literal backslash and an "n" in it.

The 2nd string above has a backslashed backslash (which becomes
a backslash) and an "n" in it.




Don't let that stop you from pretending to answer questions about it.




It has nothing to do with eval.

It has to do with single-quoted strings.




That is because in the code below, the value of $replacement2
is a valid expression:

perl -e '$2'

But the value you suggest above is not a valid expression:

perl -e '$2\n'

Oh, I get it. Its all about "valid expression" now and not eval in regex's.
And now since you have stated '$2' is a valid expression and '$2\n' isn't,
it raises the question of the ilogic in your nothing, so nothing argument.

Now, given the "valid expression" of '$2', this should be equivalent in terms of eval:

$replacement2 = '$2';
$data =~ /(.)\n(.)/;
$e = eval ($replacement2);
print "->",$e,"\n";

$data =~ s/(.)\n(.)/$replacement2/eg;
print $data."\n";

Unfortunately, you forgot your brain today Tad.

-sln

 

[Tad J McClellan]

Oh, I get it. Its all about "valid expression"

You're welcome.

 

[sln]

You're welcome.

No thanks! But you are welcome.

Here's some valid expression for ya Tad:

$replacement2 = '$2';
$data =~ /(.)\n(.)/;
$e = eval ($replacement2);
print "->",$e,"\n";

$data =~ s/(.)\n(.)/$replacement2/eg;
print $data."\n";

 

[Tad J McClellan]

No thanks! But you are welcome.

Here's some valid expression for ya Tad:

$replacement2 = '$2';
$data =~ /(.)\n(.)/;
$e = eval ($replacement2);
print "->",$e,"\n";

$data =~ s/(.)\n(.)/$replacement2/eg;
print $data."\n";

That works as expected. Since there is only one eval with s///e,
the result after evaluating $replacement2 is the string '$2'.

What were you expecting it to do instead?

If you were expecting the last print to print the value of the $2
variable, then you don't have enough levels of eval to get to where
'$2' is evaluated.

The 2nd level of eval can be gotten with either

$data =~ s/(.)\n(.)/$replacement2/eeg;

or

$data =~ s/(.)\n(.)/$e/eg;