Challenge: CPU-optimized byte-wise or-equals (for a meter of beer)

M

Michele Dondi

Just copying here from: <http://perlmonks.org/?node_id=638552>
(Please check the thread for other interventions. BTW: I'm *not* the
author of the post...)


I have two very long (>64k) strings of equal lengths - $s1 and $s2.
They are strings of bytes, meaning that any value from chr(0) to
chr(255) is legal. $s2, however, will not have any chr(0). $s1 may or
may not have any. What I need to do is look at each byte in $s1 and if
it is chr(0), replace it with the corresponding byte in $s2. So,
something like the following code:

sub foo {
my ($s1, $s2) = @_;

my @s1 = split //, $s1;
my @s2 = split //, $s2;

foreach my $idx ( 0 .. $#s1 ) {
if ( $s1[$idx] eq chr(0) ) {
$s1[$idx] = $s2[$idx];
}
}

return join '', @s1;
}

foo() could return the resulting string or it could modify $s1 in
place. If foo() returns $s1, I'm going to be doing $s1 = foo( $s1, $s2
); in all cases.

Here's what I've got so far, including Benchmark harness. Whoever
comes up with the fastest version earns a meter of beer from me
whenever we see each other.

#!/usr/bin/perl

use 5.6.0;

use strict;
use warnings FATAL => 'all';

use Benchmark qw( cmpthese );

my $s1 = join '', (do_rand(1) x 100_000);
my $s2 = join '', (do_rand(0) x 100_000);

cmpthese( -2, {
'split1' => sub { my $s3 = split1( $s1, $s2 ) },
'substr1' => sub { my $s3 = substr1( $s1, $s2 ) },
});

sub split1 {
my ($s1, $s2) = @_;

my @s1 = split //, $s1;
my @s2 = split //, $s2;

foreach my $idx ( 0 .. $#s1 ) {
if ( $s1[$idx] eq chr(0) ) {
$s1[$idx] = $s2[$idx];
}
}

return join '', @s1;
}

sub substr1 {
my ($s1, $s2) = @_;

for my $idx ( 0 .. length($s1) ) {
if ( substr($s1,$idx,1) eq chr(0) ) {
substr($s1, $idx, 1) = substr($s2, $idx, 1);
}
}

return $s1;
}

# This makes sure that $s1 has chr(0)'s in it and $s2 does not.
sub do_rand {
my $n = (shift) ? int(rand(255)) : int(rand(254)) + 1;
return chr( $n );
}

__END__

Update: It looks like there is a 2-way tie between avar and moritz. I
went ahead and wrote an in-place version of moritz's code. Thanks to
SuicideJunkie for fixing my stupidity in the test data. The script now
looks like:

#!/usr/bin/perl

use 5.6.0;

use strict;
use warnings FATAL => 'all';

#use Test::More no_plan => 1;
use Benchmark qw( cmpthese );

my $s1 = do_rand(0, 100_000);
my $s2 = do_rand(1, 100_000);
my $expected = split1( \$s1, \$s2 );

cmpthese( -3, {
'avar2' => sub {
my $s3 = $s1; avar2( \$s3, \$s2 );
# is( $s3, $expected, "avar2" );
},
'moritz' => sub {
my $s3 = $s1; moritz( \$s3, \$s2 );
# is( $s3, $expected, "moritz" );
},
});

sub split1 {
my ($s1, $s2) = @_;

my @s1 = split //, $$s1;
my @s2 = split //, $$s2;

foreach my $idx ( 0 .. $#s1 ) {
if ( $s1[$idx] eq chr(0) ) {
$s1[$idx] = $s2[$idx];
}
}

$$s1 = join '', @s1;
}

sub avar2 {
my ($s1, $s2) = @_;
use bytes;
$$s1 =~ s/\0/substr $$s2, pos($$s1), 1/eg;
}

sub moritz {
my ($s1, $s2) = @_;

my $pos = 0;
while ( 0 < ( $pos = index $$s1, "\000", $pos ) ) {
substr( $$s1, $pos, 1 ) = substr( $$s2, $pos, 1 );
}
}

sub do_rand {
my ($min, $len) = @_;
my $n = "";
for (1 .. $len) {
$n .= chr( rand(255-$min)+$min )
}
return $n;
}

__END__

I'm going to keep it open until 24 hours have passed from the initial
posting of this node. If no-one gets any faster, both moritz and avar
have a meter of beer from me.


Michele
 
M

Michele Dondi

:: Just copying here from: <http://perlmonks.org/?node_id=638552>
:: (Please check the thread for other interventions. BTW: I'm *not* the
:: author of the post...)

Is there a point of reposting perlmonks threads to Usenet? People
who are interested in perlmonks already read it. People who don't,
well, they don't and there's no need to repost for them.

I *do* think that there's a point, for people interested in *Perl* who
may like one interface and dislike the other one. And I think there's
a point reposting stuff from one place to the other if it contributes
to Perl knowledge or is otherwise intriguing. For the future I'll
stick to include a [PM] "tag" in the subject for those like you who
will want to filter such posts out a priori.


Michele
 
C

Charlton Wilbur

MD> I *do* think that there's a point, for people interested in
MD> *Perl* who may like one interface and dislike the other
MD> one. And I think there's a point reposting stuff from one
MD> place to the other if it contributes to Perl knowledge or is
MD> otherwise intriguing.

I'm afraid I concur with Abigail here. If I wanted to read Perlmonks,
I'd read Perlmonks.

If Perlmonks is lacking in competent and knowledgeable posters, then
perhaps it's time to reexamine their choice of interfaces. And if
they're not so lacking, reposting Perlmonks threads here and clpm
threads there serves only to annoy.

Charlton
 
U

Uri Guttman

MD> for my $idx ( 0 .. length($s1) ) {

MD> if ( substr($s1,$idx,1) eq chr(0) ) {

you can loop over the $s1 chars with a regex of /./g which should/could be
faster than a substr call

MD> substr($s1, $idx, 1) = substr($s2, $idx, 1);

use 4 arg substr which is much faster than lvalue substr

MD> Update: It looks like there is a 2-way tie between avar and moritz. I
MD> went ahead and wrote an in-place version of moritz's code. Thanks to
MD> SuicideJunkie for fixing my stupidity in the test data. The script now
MD> looks like:


MD> sub avar2 {
MD> my ($s1, $s2) = @_;
MD> use bytes;
MD> $$s1 =~ s/\0/substr $$s2, pos($$s1), 1/eg;
MD> }

i was thinking along that line before i realized you had gotten answers.

MD> sub moritz {
MD> my ($s1, $s2) = @_;

MD> my $pos = 0;
MD> while ( 0 < ( $pos = index $$s1, "\000", $pos ) ) {
MD> substr( $$s1, $pos, 1 ) = substr( $$s2, $pos, 1 );

again, 4 arg substr will be much faster than lvalue substr.

another crazy idea is to use something like $s1 =~ /(\0)/g to find and
grab all the zero spots. then use @+ (or @- whichever works right) as a
list of indexes to substr out of $s2 and into $s1. that line can be done
in a for modifier. something like this (very untested):

@zeroes = $s1 =~ /(\0)/g ;
substr( $s1, $_, 1, substr( $s2, $_, 1 ) ) for @- ;

one last idea is maybe bit::vector has such a feature or methods to
build one. it is very fast and optimized for this sort of thing.

uri
 
X

xhoster

Michele Dondi said:
Update: It looks like there is a 2-way tie between avar and moritz. I
went ahead and wrote an in-place version of moritz's code. Thanks to
SuicideJunkie for fixing my stupidity in the test data. The script now
looks like:

#!/usr/bin/perl

use 5.6.0;

use strict;
use warnings FATAL => 'all';

#use Test::More no_plan => 1;
use Benchmark qw( cmpthese );

my $s1 = do_rand(0, 100_000);
my $s2 = do_rand(1, 100_000);
my $expected = split1( \$s1, \$s2 );

Because split1 modifies $s1 in place, $s1 is equal to $expected
even before your tests start. Therefore, a no-op routine would still
satisfy your "is" assertions (were they not commented out). Once I fix
that, moritz seems to start giving the wrong answers.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
 
X

xhoster

Because split1 modifies $s1 in place, $s1 is equal to $expected
even before your tests start. Therefore, a no-op routine would still
satisfy your "is" assertions (were they not commented out). Once I fix
that, moritz seems to start giving the wrong answers.

The problem with moritz is that it compares index(...) to >0, rather than
either >-1 or >=0, for success, so it fails when the first char is \000. I
fixed that, and changed to 4-argument substr.

For my own effort, I perhaps cheated by using Inline C, and depending on
the C representations of both strings being null terminated.

Rate split1 substr1 avar2 moritz moritz2 foo2
split1 5.35/s -- -82% -100% -100% -100% -100%
substr1 30.3/s 467% -- -97% -98% -99% -100%
avar2 1160/s 21595% 3725% -- -39% -44% -86%
moritz 1907/s 35576% 6190% 64% -- -8% -77%
moritz2 2080/s 38806% 6759% 79% 9% -- -75%
foo2 8318/s 155488% 27330% 617% 336% 300% --




use Inline C=> 'DATA';
....
__DATA__
__C__
void foo2 (unsigned char * s1, unsigned char * s2) {
while ( *s2 != NULL ) {
while ( *s1 != NULL ) {
s1++; s2++;
};
*s1=*s2;
};
};

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
 
J

Josef Moellers

Keith said:
Beer is measured in volume, not length. The OP (at PM) should really be
offering a cubic meter of beer, and moritz and avar should demand such.

It certainly depends on where you live: "a meter of beer" is quit common
in some areas here in Germany: just place glasses next to each other for
a length of 1 meter! There is even a competition: two drinkers start at
each end and whoever reaches the middle first, wins.

Josef
 
M

Michele Dondi

If Perlmonks is lacking in competent and knowledgeable posters, then
perhaps it's time to reexamine their choice of interfaces. And if

Well, of course if I see that many people agree with you and Abigail,
then I will stop doing so. Not that I do it *routinely* nor that I've
done that so many times...

However PM is *not* lacking in competent and knowledgeable posters, in
fact did the OP in this particular example receive quite a lot of
*sensible* answers.

Yet the problem seemed interesting enough to be shared: that it was
posted there is somewhat of circumstance. If I had stumbled in it by
randomly browsing the web and still found it interesting, then I would
have posted it here as well.
they're not so lacking, reposting Perlmonks threads here and clpm
threads there serves only to annoy.

Well it may be and may not be. I copied there the content of a thread
posted here some months ago (with proper attributions) and at
<http://perlmonks.org/?node_id=638535> you can read:

: ++ for bringing that here, so I read it :)
:
: I didn't know about @CARP_NOT, so I learned something new today.

Also, Uri Guttman and xhoster took a shoot at the actual problem, so
they must have found it interesting.

Honestly, I don't want to bother anyone. Do you think that
occasionally posting here interesting stuff from PM with a suitable
tag in the Subject for you and others to easily filter it out would be
so bad?


Michele
 
C

Charlton Wilbur

MD> Honestly, I don't want to bother anyone. Do you think that
MD> occasionally posting here interesting stuff from PM with a
MD> suitable tag in the Subject for you and others to easily
MD> filter it out would be so bad?

I think that both Usenet and Perlmonks are interactive media, and both
are best served by an occasional post saying something like "there's
an interesting conversation going on about $subject over on
Perlmonks." People who want Perlmonks know where to find it, by and
large, and it's not as if clpm is hurting for traffic.

I don't think "Post it, and if people find it unwelcome they can
filter it out" is a polite strategy to pursue. But I'm not the
arbiter of clpm behavior, except in that I'm fairly quick to score
subjects and posters down.

Charlton
 
U

Uri Guttman

MD> On 13 Sep 2007 11:39:28 -0400, Charlton Wilbur

MD> Well, of course if I see that many people agree with you and Abigail,
MD> then I will stop doing so. Not that I do it *routinely* nor that I've
MD> done that so many times...

i am not there because of the web api (why no news/email gateway?) and
because i have never was into the monk culture. and yes i know plenty of
people there but i have seen too many idiots. there are idiots on usenet
too but emacs makes it easier to ignore them.

uri
 
T

Tad McClellan

Charlton Wilbur said:
MD> I *do* think that there's a point, for people interested in
MD> *Perl* who may like one interface and dislike the other
MD> one. And I think there's a point reposting stuff from one
MD> place to the other if it contributes to Perl knowledge or is
MD> otherwise intriguing.

I'm afraid I concur with Abigail here.


<aol> Me too! </aol>
 
B

brian d foy

Well, of course if I see that many people agree with you and Abigail,
then I will stop doing so. Not that I do it *routinely* nor that I've
done that so many times...

I agree with them. It's a lame thing to do, reposting other people's
stuff (heh, from the guy who runs the perlfaq server :).

Posting your own stuff is fine, or your own thoughts on someone else's
ideas is fine, but just reposting something with no transformative
effect is a pillar of copyright infringement, and it's rude to the
author.
 
B

brian d foy

Michele Dondi said:
Honestly, I don't want to bother anyone. Do you think that
occasionally posting here interesting stuff from PM with a suitable
tag in the Subject for you and others to easily filter it out would be
so bad?

How about posting original content of which you are the author? Talking
about a problem that you think is interesting, expounding on it, and so
on would be a lot better.

Otherwise, let the original authors decide where to post their own work.
 
J

jgraber

Michele Dondi said:
I have two very long (>64k) strings of equal lengths - $s1 and $s2.
They are strings of bytes, meaning that any value from chr(0) to
chr(255) is legal. $s2, however, will not have any chr(0). $s1 may or
may not have any. What I need to do is look at each byte in $s1 and if
it is chr(0), replace it with the corresponding byte in $s2.

Benchmark this:

(my $s1m = $s1) =~ tr/\000-\377/\377\000/;
$s1 |= ($s2 & $s1m);
 
M

Martijn Lievaart

For my own effort, I perhaps cheated by using Inline C, and depending on
the C representations of both strings being null terminated.

But s1 contains null characters, so this won't work.

M4
 
X

xhoster

Martijn Lievaart said:
But s1 contains null characters, so this won't work.

Did you try it?

If s1 did *not* contain null characters, then there would be no point.
The fact that s1 does contain null characters is what this is all about.
The point is that it also needs to end with a null character, regardless
of what null it has in the middle.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
 
M

Martijn Lievaart

Did you try it?

If s1 did *not* contain null characters, then there would be no point.
The fact that s1 does contain null characters is what this is all about.
The point is that it also needs to end with a null character, regardless
of what null it has in the middle.

No I did not try it. I don't have to. this C implementation stops at the
first null character. Why do you think it'll behave otherwise?

M4
 
P

Peter J. Holzer

No I did not try it. I don't have to.

You may not have to try it, but you have to read the code more
carefully.
this C implementation stops at the first null character. Why do you
think it'll behave otherwise?

Because that's how he programmed it?

(BTW, the use of NULL is wrong - that should be 0 or '\0').

hp
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,755
Messages
2,569,534
Members
45,008
Latest member
Rahul737

Latest Threads

Top