shadabh said:
shadabh wrote:
Hi all,
Just wanted to run this through Perl gurus to see if fit is
correct?. I have a file that could possibly be 1GB in variable
length EBCDIC data. I will read the file as EBCDIC data and based
on some criteria split it into 100 different files which will add
up to the 1GB. This way a particular copy book can be applied to
easy of the split files. The approach I am using is a filehandle
( IO::FileHandle and $Strings), substr and write out to 100
different files after applying the 'logic'. I will use two
routine, one to read and one to write, I have tested this out
with 100MB file and it works fine. The question though is there a
memory limit to this, as we are using strings to break the files.
Or is there an alternative way to do this? Comments, suggestions,
improvements and alternatives will really help to design the
code. thanks
You have to show us what "this" is first. Â It is kind of hard to
make comments, suggestions, improvements and alternatives to
something that we cannot see.
John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order. Â Â Â Â Â Â Â Â Â Â Â Â Â Â -- Larry Wall- Hide
quoted text -
Sure. Here is what I meant by 'this way'. Please comment. Thanks
You don't modify $raw_data inside the loop so there is no point in
testing it every time through the loop.
$var_len=$INIT_LEN+$AC_NUM_LENGTH;
my $var_len = $INIT_LEN + $AC_NUM_LENGTH;
$val = substr $raw_data,$var_len,4;
$asc_string = $val;
my $asc_string = substr $raw_data, $var_len, 4;
eval '$asc_string =~ tr/\000-\377/' . $cp_037 . '/';
# open(OUTF, '>>ebcdic_ID.txt');
# #print OUTF $var_len, $asc_string, "\n";
# close OUTF;
open(PARM, '<TSY2_PARM.par') || die("Could not open the
parammeter file $PARM_FILE ! File read failed ..check iF file
exits");
You are opening 'TSY2_PARM.par' but your error message says you are
opening $PARM_FILE. Â You should include the $! variable in the error
message so you know *why* it failed to open.
open my $PARM, '<', 'TSY2_PARM.par' or die "Could not open
'TSY2_PARM.par' $!";
You assign the same data to $parm_data every time so...
if (($parm_data =~ m!($asc_string)!g) eq 1) {
your pattern will always match at the same place every time through
the loop so if the pattern is present you have an infinite loop. Â You
are not using the contents of $1 so the parentheses are superfluous.
The comparison test to the string '1' is superfluous.
if ( $parm_data =~ /$asc_string/g ) {
$COPYBOOK_LEN = substr $parm_data,length($`)+4,4;
The use of $` is discouraged as it slows down *all* regular
expressions in your program.
my $COPYBOOK_LEN = substr $parm_data, $-[ 0 ] + 4, 4;
print $COPYBOOK_LEN;
close (PARM);
$OUT_DATAFILE = 'EBCDIC_'.$asc_string.'.txt';
$RECORD_LEN= $COPYBOOK_LEN+$HEADER_DATA;
open(OUTF, ">>$OUT_DATAFILE")|| die("Could not open file. File
read failed ..check id file exits");
open my $OUTF, '>>', $OUT_DATAFILE or die "Could not open
'$OUT_DATAFILE' $!";
print OUTF substr $raw_data, $INIT_LEN, $RECORD_LEN;
close OUTF;
$INIT_LEN = $INIT_LEN + $RECORD_LEN;
print $INIT_LEN;
print $var_len;
}
else {
print 'End of file reached or copy book is not a part of the
loading process', "\n";
exit 0;
}
}
So, to summarise:
open my $PARM, '<', 'TSY2_PARM.par' or die "Could not open the
parammeter file 'TSY2_PARM.par' $!";
my $parm_data = <$PARM>;
close $PARM;
while ( 1 ) {
my $var_len = $INIT_LEN + $AC_NUM_LENGTH;
my $asc_string = substr $raw_data, $var_len, 4;
eval '$asc_string =~ tr/\000-\377/' . $cp_037 . '/';
if ( $parm_data =~ /\Q$asc_string\E/g ) {
my $COPYBOOK_LEN = substr $parm_data, $-[ 0 ] + 4, 4;
my $OUT_DATAFILE = "EBCDIC_$asc_string.txt";
my $RECORD_LEN Â = $COPYBOOK_LEN + $HEADER_DATA;
open my $OUTF, '>>', $OUT_DATAFILE or die "Could not open
'$OUT_DATAFILE' $!";
print $OUTF substr $raw_data, $INIT_LEN, $RECORD_LEN;
close $OUTF;
$INIT_LEN += $RECORD_LEN;
print $COPYBOOK_LEN, $INIT_LEN, $var_len;
}
else {
print "End of file reached or copy book is not a part of the
loading process\n";
last;
}
}
exit 0;
__END__
John
