Problem with my perl script - old records getting over written by new $completedData

V

Vivek.M

Hi, Background: bttrack.py is a open src torrent tracker, that uses
the option --dfile fileName, to save state information in. The file is
bencoded and has parts of it's content in binary. The encoding specs
and file format are described here:
http://wiki.theory.org/BitTorrentSpecification#lists

I am trying to write my own Perl script to create my own "dfile".

Data from a valid dstateFile (binary data has been converted to
ASCII):

d9:completedd20:.i.?.vk..&.7.....g..i0ee5:peersd20:.i.?.vk..&.
7.....g..
d20:-AZ2504-4PGqC1Tza6WDd2:ip9:127.0.0.13:key8:vkYZ3mwZ4:lefti12415246
5e3:nati0e4:porti35ee20:-
UT1740-..........#.d2:ip9:127.0.0.13:key8:29FA
43C64:lefti37137041e3:nati0e4:porti35eeeee


The above data when you interpret correctly (based on the
TorrentSpecs)
forms a data structure like so:

{ 'completed' => { $shaHashBin => '0' }
'peers' => { $shaHashBin => { $clientID { ip => $ipAddr, key=>
$Key, left=>'0', nat=>'0', port=>$port}
$clientID2 { ip =>
$ipAddr, key=>$Key, left=>'0', nat=>'0', port=>$port}
}
}
}


Unfortunately for some reason $clientID is not being expanded properly
and instead of getting ClientID1/ClientID2/ClientID3 etc and their
associated ip/key/left/nat/port entries, all i get is a
single entry! I'm sure that the old records/entries are being
overwritten by the new entries - so I'm left with a perfectly valid
dstateFile with the last ip:port pair that was read from the file!

I also tried something like so:
$_=<IPFILE>;
chomp;
($ipAddr, $port) = split(/:/);

$clientID = RandomString(20);
$key = RandomString(8);

# dflist file structure.
<SNIP>

while (<IPFILE>) {
chomp;
($ipAddr, $port) = split(/:/);

$clientID = RandomString(20);
$key = RandomString(8);

$completedData->{'peers'}->{$clientID} =
MakeClientHash($ipAddr, $key, $port);
}

This works and the records don;t get overwritten BUT the file format
is broken and i get data like so: d9:completedd20:.i.?.vk..&.
7.....g..i0ee5:peersd20:.i.?.vk..&.7.....g..
d20:-AZ2504-4PGqC1Tza6WDd2:ip9:127.0.0.13:key8:vkYZ3mwZ4:lefti12415246
5e3:nati0e4:porti35eee20:-
UT1740-..........#.d2:ip9:127.0.0.13:key8:29FA
43C64:lefti37137041e3:nati0e4:porti35eeee

Note the (eeee) That's }}}}. There should be 5 of them :( also note
eee20 - that should read ee20 :( Oh, and note: bencode breaks if the
torrentfile is complicated (just in case you actually debug
that).


------------------------------------------------------------------------------------------------------------------------------------------
#!/usr/bin/perl
#
# Program to create a file called dflist that bttrack.py can
understand.
#use strict;
#use warnings;

use POSIX qw(strftime);

use Getopt::Std;
use URI::Escape 'uri_escape_utf8';
use Digest::SHA1 qw(sha1 sha1_hex sha1_base64);
use Convert::Bencode qw(bencode bdecode);

my ($torrentFile, $ipFile, $dflistFile);
my $debug=0;

(my $progname = $0) =~ s|.*/||;


sub UsageError() {
print STDERR "\nusage: $progname torrentFile ipFile dfileName\n";
exit(2);
}

sub FatalError( $ ) {
print STDERR "$progname: @_\n";
exit(1);
}

sub GetRunConfig() {
# If we have more than two arguments signal a error.
UsageError if (@ARGV != 3);

if (@ARGV == 3) { $torrentFile = $ARGV[0];
$ipFile = $ARGV[1];
$dflistFile = $ARGV[2]; }
}

sub ReadFile( $ ) {
my ($pathname) = @_;
local $/;
my $FH;
open($FH, '<', $pathname)
or FatalError("cannot open $pathname for reading: $!");
my $contents = <$FH>;
close $FH;
return $contents;
}

sub WriteFile( $$ ) {
my $FH;
my ($pathname, $contents) = @_;
open($FH, '>', $pathname)
or FatalError("cannot open $pathname for writing: $!");
# Required because we are writing binary data
binmode($FH);
print $FH $contents;
close($FH)
or FatalError("error writing to file $pathname: $!");
}

sub RandomString( $ ) {
my ($lenOfRanStr) = @_;
my @chars=('a'..'z','A'..'Z','0'..'9','_');
my $ranStr;

foreach (1..$lenOfRanStr) {
# rand @chars will generate a random
# number between 0 and scalar @chars
$ranStr .= $chars[rand @chars];
}
return $ranStr;
}


sub MakeClientHash( $$$ ) {
my ($ipAddr, $key, $port) = @_;
return { ip => $ipAddr,
key => $key,
left => '0',
nat => '0',
port => $port };
}

sub CreateDflist {
# Creates a dflist file based on the torrentfile and the
# ipaddress file you provide.
my ($contents, $contentsRef, $shaHashBin, $completedData);

# Read the supplied torrentfile, extract the info values, and compute
# the sha1 hash in binary form ($shaHashBin).
$contents = ReadFile($torrentFile);
$contentsRef = bdecode($contents);
$shaHashBin = sha1(bencode($contentsRef->{'info'}));
my ( $clientID, $ipAddr, $key, $port);

# Read $ipFile and get the $ipAddr:$port pairs for the $torrentFile.
open(IPFILE, $ipFile)
or die "can't open $ipFile: $!";

while(<IPFILE>) {
chomp $_;
($ipAddr, $port) = split(/:/);

$clientID = RandomString(20);
$key = RandomString(8);

# dflist file structure.
$completedData = {
'completed' => { $shaHashBin => '0' },
'peers' => { $shaHashBin => {$clientID =>

MakeClientHash($ipAddr, $key, $port)} }
};
}

# Write dflist data after encoding.
my $encString = bencode($completedData);
WriteFile("$dflistFile", $encString);

print $encString;
}

GetRunConfig;
CreateDflist;
 
V

Vivek.M

Hi, i shortened it. This is the kind of data structure i need to
assign to a anonymous
hash called $completedData. $completedData is passed to bencode to
encode
it and the encoded data is written to disk. (I am a noob at Perl! The
problem just looks complicated..)

Template Data Structure i need to create, that $completedData refers
to: (clientID1,2,3,4 etc..)
This data structure is then encoded by bencode, and put to disk thus
creating a valid file that
bttrack.oy can understand.

{ 'completed' => { $shaHashBin => '0' }
'peers' => { $shaHashBin => { $clientID1 {
ip => $ipAddr, key =>
$Key,
left=>'0', nat=>'0',
port=>$port
}
$clientID2 {
ip =>$ipAddr, key=>
$Key,
left=>'0', nat=>'0',
port=>$port
}
}
}
}

#Perl code i am using (suitably edited for clarity):

sub MakeClientHash( $$$ ) {
my ($ipAddr, $key, $port) = @_;
return { ip => $ipAddr,
key => $key,
left => '0',
nat => '0',
port => $port };

}

sub CreateDflist {

|<SNIP>|

while(<IPFILE>) {
chomp $_;
($ipAddr, $port) = split(/:/);

$clientID = RandomString(20);
$key = RandomString(8);

# dflist file structure.
$completedData = {
'completed' => { $shaHashBin => '0' },
'peers' => { $shaHashBin => {$clientID =>
MakeClientHash($ipAddr, $key, $port)} }
};
}

# Write dflist data after encoding.
my $encString = bencode($completedData);
WriteFile("$dflistFile", $encString);

print $encString;

}

CreateDflist;

The code runs without any errors BUT i get only one valid record on
file - $clientIDXX where XX is the last ip:port pair in file.
 
J

John W. Krahn

Vivek.M said:
Hi, i shortened it. This is the kind of data structure i need to
assign to a anonymous hash called $completedData. $completedData
is passed to bencode to encode it and the encoded data is written
to disk. (I am a noob at Perl! The problem just looks complicated..)

Template Data Structure i need to create, that $completedData refers
to: (clientID1,2,3,4 etc..)
This data structure is then encoded by bencode, and put to disk thus
creating a valid file that bttrack.oy can understand.

[ SNIP ]

#Perl code i am using (suitably edited for clarity):

[ SNIP ]

sub CreateDflist {

|<SNIP>|

while(<IPFILE>) {
chomp $_;
($ipAddr, $port) = split(/:/);

$clientID = RandomString(20);
$key = RandomString(8);

# dflist file structure.
$completedData = {
'completed' => { $shaHashBin => '0' },
'peers' => { $shaHashBin => {$clientID =>
MakeClientHash($ipAddr, $key, $port)} }
};

You are assigning a single (anonymous) hash for every record in the file which
is why only the last one is saved.

You need to assign values to hash keys instead:

$completedData->{ completed }{ $shaHashBin } = 0;
$completedData->{ peers }{ $shaHashBin }{ $clientID } = MakeClientHash(
$ipAddr, $key, $port );

}

# Write dflist data after encoding.
my $encString = bencode($completedData);
WriteFile("$dflistFile", $encString);

print $encString;

}

CreateDflist;

The code runs without any errors BUT i get only one valid record on
file - $clientIDXX where XX is the last ip:port pair in file.


John
 
V

Vivek.M

Hey John, it's working :) :) Yeah! Ugh!! I did it the right way the
first time but i did this:
$completedData->{'peers'}->{$clientID} =
MakeClientHash($ipAddr, $key, $port);
instead of this:
$completedData->{'peers'}->{'shaHashBin'}->{$clientID} =
MakeClientHash($ipAddr, $key, $port);


So my structure got mangled and then i did the other garbage! Many
thanks! Also, a huge thanks to the guys on freenodes #perl who rock!
They told me the same thing :) Needless to say i am very please <g>
Bye
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top