repeated calculations everyday

[Shalini Joshi]

Hey.

The following is the logic-structure of the code I am attempting to
write:

if (today is the first of the month)
{

Read all files with name containing SYSMT03.xxxxx.txt
#x denotes arbitrary value

Each file contains information pertaining to a particular owner. So
I parse the data no1, y for each user . Now this entire data from all
these files is to be sorted on the basis of this no1. That is more
than one user can have the same value for no1.

#The question I ask at this point is, how would I go about doing this
in perl? #What data structure would I use given this ?

I have to sum the total of y(from various user files) for each no1. I
call this value BASE1

}


else if (today is not the first day of the week, then for each
remaining day(recurring calculations)

{
Read another set of files names containing SYSMT02Axxxx.txt

Again similar information. Now what I will do here is calculate the
change in BASE and add it to BASE1 to get at BASE2. SImilarly for the
remaining days.(eg BASE3= BASE2+change3)..

}

How will I be able to go about designing the data structures etc in
perl?


I would appreciate any tips/help/reading references...


I am getting stuck right in the beginning itself...where I try and
match the regular expressions:

I say :


open (ACCOUNT_POSITION, '/home/perlstuff' ) or die "Can't open file:
$!\n";

<ACCOUNT_POSITION> =~ /.+03\..{5}\.txt


It doesnt work for some reason although i have the said file in my
directory. Where am I going wrong on this one???


Thanka again for all the help in past and in advance for this one!

Regards

Shalini

 

[Ben Morrow]

Quoth (e-mail address removed) (Shalini Joshi):

The following is the logic-structure of the code I am attempting to
write:

if (today is the first of the month)
{

Read all files with name containing SYSMT03.xxxxx.txt
#x denotes arbitrary value

perldoc -f glob

Each file contains information pertaining to a particular owner. So
I parse the data no1, y for each user . Now this entire data from all
these files is to be sorted on the basis of this no1. That is more
than one user can have the same value for no1.

#The question I ask at this point is, how would I go about doing this
in perl? #What data structure would I use given this ?

I have to sum the total of y(from various user files) for each no1. I
call this value BASE1

I would call it BASE0, as Perl starts array numbers and day-of-month
numbers (indeed, ordinals generally) at 0.

my @BASE;

for (<SYSMT03.*.txt>) {
my ($no1, $y) = read_data_from $_;

no warnings 'uninitialized';
$BASE[0]{$no1} += $y;
}

}

else if (today is not the first day of the week, then for each
remaining day(recurring calculations)

{
Read another set of files names containing SYSMT02Axxxx.txt

Again similar information. Now what I will do here is calculate the
change in BASE and add it to BASE1 to get at BASE2. SImilarly for the
remaining days.(eg BASE3= BASE2+change3)..

my $mday = (localtime)[3];
for (...) {
my ($no1, $my) = read_data_from $_;

$BASE[$mday]{$no1} = $BASE[$mday - 1]{$no1}
unless exists $BASE[$mday]{$no1};

$BASE[$mday]{$no1} += $y;
}

You can make this structure permanent between runs using the Storable
module.

}

I am getting stuck right in the beginning itself...where I try and
match the regular expressions:

I say :

open (ACCOUNT_POSITION, '/home/perlstuff' ) or die "Can't open file:
$!\n";

Use three-arg open.
Use lexical filehandles.
Don't use unnecessary parens.
Include the filename in the error message.
Probably don't put "\n" on the end of die messages.
I would use a shorter variable name...

open my $ACCT_POS, '<', '/home/perlstuff'
or die "can't open /home/perlstuff: $!";

<ACCOUNT_POSITION> =~ /.+03\..{5}\.txt

This is a syntax error.

It doesnt work for some reason although i have the said file in my
directory. Where am I going wrong on this one???

Define 'doesn't work'. What are you expecting to happen, and what
actually happens?

Ben

 

[Joe Smith]

Quoth (e-mail address removed) (Shalini Joshi):

I would call it BASE0, as Perl starts array numbers and day-of-month
numbers (indeed, ordinals generally) at 0.

The day-of-week and month-in-year are based at zero (making it easy to
index into an array that translates the array index into a text string),
but day-of-month starts at 1.
-Joe

 

[Shalini Joshi]

So here is what I have of the code uptil now.
The thing that has me stumped is how do access the BASE array that I
stored using the storable module. When i try and print out the
particular element, it works but doesnt work ..gives me a HASH{xxxx}
error...also as I mention right at the end I store it as BASE[0][$i]
where $i is an element of an array of unique 9 digit numbers. (not
necessarily sorted etc)...Is there any way I can get it to print based
on the length of the hard-linked array..as opposed to the index or
something???

Thanks for the help!

Regards,

Shalini


if ((localtime)[3] ==1)
{
print "Today is the first day of the month!\n";
print "We are going to read all Account Position Files!\n";

#For each client we read the Account Position File: thus all account
position files on the server.

opendir DIR, '/home/sj2122/perlstuff' or die "$!";
my @files = grep {/[^(03)]+03\.[^\.]+\.txt$/} readdir(DIR);
closedir(DIR);

foreach $i (@files)
{
# Read every position file
print "$i\n";
open ACCOUNT_POSITION, "<$i" or die "Can't open $i $!";
local $_ = <ACCOUNT_POSITION>;
if (/^APR/)
{
push @position_records,$_;
}

while (<ACCOUNT_POSITION>)
{ #get all lines of the record
# stop if we encounter an APR line

if (/^APR/)
{
push @position_records, $_;

}

}
close ACCOUNT_POSITION;
}

foreach $i (@position_records)
{

%parameters_owner= (
dealer_number => substr($i,6,7),
CUSIP => substr($i,22,9),
cust_accnt_number => substr($i,38,20),
total_shares => substr($i,59,15),
);

if ($parameters_owner{dealer_number}==7654321)
{

print "Yahoo! We need to add this CUSIP number to selected records
in Price File!\n";


push @arrayofparams, {%parameters_owner};
push @cusip, $parameters_owner{CUSIP};

push @customers, $parameters_owner{cust_accnt_number};

}

}
my %hash = map {$_ => 1} @cusip;
@cusip_new = keys %hash; # to get unique CUSIP numbers

my %hash2 = map {$_ => 1} @customers;
@customers_new = keys %hash2;

foreach $i (@cusip_new)
{

my $sharetotal=0;
for $y (0 .. $#arrayofparams)
{
for $valuesonly (values %{ $arrayofparams[$y]} )
{
if ($valuesonly eq $i)
{
print "$valuesonly\n";
print "$arrayofparams[$y]{total_shares}\n";
$sharetotal += $arrayofparams[$y]{total_shares};
}
}
}
print "Total shares for $i is $sharetotal\n";
$BASE[0]{$i}= $sharetotal; # For day 1, total for the various
CUSIPs
print "$BASE[0]{$i}\n";

}
store(\@BASE, 'base_values'); # For later access.

open RECORDS, "< AETNA1.txt " or die "Can't read file: $!";

local $_ = <RECORDS>;

while (<RECORDS>)
{ #get all lines of the record

# stop if we encounter an FPR line

if (/^FPR/)
{
push @records, $_;
}

}

close RECORDS;



my $date= substr(@records[0],22,8);
my $cpday= &UnixDate($date,"%d");
my $cpmonth=&UnixDate($date,"%m");
my $cpyear=&UnixDate($date,"%y");
my $cpdate=$cpmonth.$cpday.$cpyear;

my $cpfilename= "AD".$cpyear.$cpmonth.$cpday."\.PRI";
my $wanted_handle = IO::File->new($cpfilename, O_WRONLY|O_CREAT)
or die "Couldn't open $cpfilename for writing:$!\n";


foreach $i (@records)
{

%parameters= (

CUSIP_P => substr($i,6,9),
fund_name => substr($i,59,38),
processing_date => $cpdate,
NAV_P => substr($i,30,9),
Fund_Type_P => "MU",
Public_Offering_Price => "000".substr($i,39,9),
);


foreach $selectedcusip (@cusip_new)
{
if ($selectedcusip eq $parameters{CUSIP_P})
{
push @arraypriceparams, {%parameters};

}

}
}


for $x (0 .. $#arraypriceparams)

{
print $wanted_handle
"$arraypriceparams[$x]{CUSIP_P},$arraypriceparams[$x]{Fund_Type_P},$arraypriceparams[$x]{processing_date},$arraypriceparams[$x]{Public_Offering_Price}\n";
}


}

#If not the first day of the month, read DIRECT FINANCIAL ACTIVITY
FILE

else
{
my $mday = (localtime)[3];
print "Today we are going to read the Direct Financial Acitivity
File and see the status of our CUSIPs!\n";

# Retrieving all base values

my $base_ref = retrieve('base_values');

#my @BASE = @{$base_ref};
#foreach $i (@{$base_ref})
#{
# print "$i{AET001A70}";
#}
print "$$base_ref[0]{AET001A20}\n";

 

[Ben Morrow]

Quoth (e-mail address removed) (Shalini Joshi):

So here is what I have of the code uptil now.

if ((localtime)[3] ==1)
{
print "Today is the first day of the month!\n";
print "We are going to read all Account Position Files!\n";

Set $\="\n" rather than putting it on the end of every print.

#For each client we read the Account Position File: thus all account
position files on the server.

opendir DIR, '/home/sj2122/perlstuff' or die "$!";
my @files = grep {/[^(03)]+03\.[^\.]+\.txt$/} readdir(DIR);
closedir(DIR);

Use lexical dirhandles.
Use scopes to close them.
Include the filename in the error message.

my @files = grep /.../, do {
my $dir = '/home/sj2122/perlstuff';
opendir my $DIR, $dir or die "can't opendir $dir: $!";
readdir $DIR;
};

foreach $i (@files)

Why aren't you using strictures?

for my $i (@files) {

{
# Read every position file
print "$i\n";
open ACCOUNT_POSITION, "<$i" or die "Can't open $i $!";

Use lexical filehandles.
Use scopes to close them.
Use 3-arg open.

open my $ACCOUNT_POSITION, '<', $i or die "can't open $i: $!";

local $_ = <ACCOUNT_POSITION>;
if (/^APR/)
{
push @position_records,$_;
}

Why are you manually going through one iteration of the loop below?

while (<ACCOUNT_POSITION>)
{ #get all lines of the record
# stop if we encounter an APR line

This doesn't stop: it records the line and carries on. If you want to
stop you will need a 'last' inside the if.

if (/^APR/)
{
push @position_records, $_;

}

I would have written this

/^APR/ and push @position_records, $_;

}
close ACCOUNT_POSITION;

There is no need for an explicit close with lexical FHs.

}

foreach $i (@position_records)

^^ my

{

%parameters_owner= (

^^ my

dealer_number => substr($i,6,7),
CUSIP => substr($i,22,9),
cust_accnt_number => substr($i,38,20),
total_shares => substr($i,59,15),
);

if ($parameters_owner{dealer_number}==7654321)

I presume there is a good reason for this value being hard-coded? If
nothing else I would put

use constant SELECTED_DEALER => 7654321;

at the top and use that instead.

{

print "Yahoo! We need to add this CUSIP number to selected records
in Price File!\n";


push @arrayofparams, {%parameters_owner};

If you'd properly created the %parameters_owner as a lexical the you
could simply push a reference to it

push @arrayofparams, \%parameters_owner;

and save a copy.

push @cusip, $parameters_owner{CUSIP};

push @customers, $parameters_owner{cust_accnt_number};

}

}
my %hash = map {$_ => 1} @cusip;
@cusip_new = keys %hash; # to get unique CUSIP numbers

my %hash2 = map {$_ => 1} @customers;
@customers_new = keys %hash2;

I would put each if these in a block: there is no need for the temporary
hashes to exist outside of those two statements:

my @cusip_new = do {
my %tmp = map { $_ => 1 } @cusip;
keys %tmp;
};

foreach $i (@cusip_new)
{

my $sharetotal=0;
for $y (0 .. $#arrayofparams)

for my $y (@arrayofparams) {

{
for $valuesonly (values %{ $arrayofparams[$y]} )

for my $valuesonly (values %$y) {

Are you sure this is correct? Do you not simply want to test if
$y->{CUSIP} eq $i? (I am asking 'is there an occasion where
dealer_number, cust_acct_number or total_shares will match a cusip, and
do you want to catch those matches or not?')

{
if ($valuesonly eq $i)
{
print "$valuesonly\n";
print "$arrayofparams[$y]{total_shares}\n";
....$y->{total_shares}...

$sharetotal += $arrayofparams[$y]{total_shares};
}
}
}
print "Total shares for $i is $sharetotal\n";
$BASE[0]{$i}= $sharetotal; # For day 1, total for the various
CUSIPs

Why not just build your total in $BASE[0]{$i} in the first place?

print "$BASE[0]{$i}\n";

}
store(\@BASE, 'base_values'); # For later access.

open RECORDS, "< AETNA1.txt " or die "Can't read file: $!";

local $_ = <RECORDS>;

while (<RECORDS>)
{ #get all lines of the record

# stop if we encounter an FPR line

if (/^FPR/)
{
push @records, $_;
}

}

close RECORDS;



my $date= substr(@records[0],22,8);

^ $
Why don't you have warnings turned on?

my $cpday= &UnixDate($date,"%d");

Don't call subs with & unless you know what it does and why you need it.

my $cpmonth=&UnixDate($date,"%m");
my $cpyear=&UnixDate($date,"%y");
my $cpdate=$cpmonth.$cpday.$cpyear;

my $cpfilename= "AD".$cpyear.$cpmonth.$cpday."\.PRI";

my $cpfilename = UnixDate $date, 'AD%y%m%d.PRI';

my $wanted_handle = IO::File->new($cpfilename, O_WRONLY|O_CREAT)
or die "Couldn't open $cpfilename for writing:$!\n";

Use lexical FHs instead of IO::File; or certainly, be consistent.
It is usual to use caps for filehandles.
Don't put "\n" on the end of die messages.

foreach $i (@records)
{

%parameters= (

CUSIP_P => substr($i,6,9),
fund_name => substr($i,59,38),
processing_date => $cpdate,

processing_data => UnixData $data, '%m%d%y';

Except don't use m-d-y date formats, they're just too brain-damaged for
words...

NAV_P => substr($i,30,9),
Fund_Type_P => "MU",
Public_Offering_Price => "000".substr($i,39,9),
);


foreach $selectedcusip (@cusip_new)
{
if ($selectedcusip eq $parameters{CUSIP_P})
{
push @arraypriceparams, {%parameters};

}

}
}


for $x (0 .. $#arraypriceparams)

{
print $wanted_handle
"$arraypriceparams[$x]{CUSIP_P},$arraypriceparams[$x]{Fund_Type_P},$arraypriceparams[$x]{processing_date},$arraypriceparams[$x]{Public_Offering_Price}\n";
}


}

#If not the first day of the month, read DIRECT FINANCIAL ACTIVITY
FILE

else
{
my $mday = (localtime)[3];
print "Today we are going to read the Direct Financial Acitivity
File and see the status of our CUSIPs!\n";

# Retrieving all base values

my $base_ref = retrieve('base_values');

#my @BASE = @{$base_ref};
#foreach $i (@{$base_ref})
#{
# print "$i{AET001A70}";
#}
print "$$base_ref[0]{AET001A20}\n";

The thing that has me stumped is how do access the BASE array that I
stored using the storable module. When i try and print out the
particular element, it works but doesnt work ..gives me a HASH{xxxx}
error...

Please explain...your $base_ref is here a ref to an array of hashes, as
you seem to have figured out; I would have written the deref above

$base_ref->[0]{AET001A20}

.. If you want to loop, you will need @$base_ref:

for my $i (@$base_ref) {
print $i->{AET001A70};
}

also as I mention right at the end I store it as BASE[0][$i]
where $i is an element of an array of unique 9 digit numbers. (not
necessarily sorted etc)...Is there any way I can get it to print based
on the length of the hard-linked array..as opposed to the index or
something???

Please explain more clearly what you want... what do you mean by 'print
based on'? Do you mean 'print a list sorted by'? And there is no such
thing as a 'hard-linked array' in Perl: the only link between
$BASE[0]{$i} and @cusp_ip is the value they hold in common.

Ben

 

[Shalini Joshi]

Hey Ben!

Thanks again for the help. What I am getting totally confused here is
to what data structures to use given the problem. I need to get at the
information across days so I store a reference to the array
BASE[$mday-1]{$i} for $i (@cusips) . NOw i would like to get at all
the elements of the array for all $i for a particular value of $mday.
I can't seem to do that.


Also could you advise me on the choice of a data structure. Right now
I read in all the records,and store them as array of hashes and
compare these.

Do let me know if it makes any sense or should I re-state my problem a
bit more clearly..

Thanks again ...

Regards,

Shalini

Quoth (e-mail address removed) (Shalini Joshi):

So here is what I have of the code uptil now.

if ((localtime)[3] ==1)
{
print "Today is the first day of the month!\n";
print "We are going to read all Account Position Files!\n";

Set $\="\n" rather than putting it on the end of every print.

#For each client we read the Account Position File: thus all account
position files on the server.

opendir DIR, '/home/sj2122/perlstuff' or die "$!";
my @files = grep {/[^(03)]+03\.[^\.]+\.txt$/} readdir(DIR);
closedir(DIR);

Use lexical dirhandles.
Use scopes to close them.
Include the filename in the error message.

my @files = grep /.../, do {
my $dir = '/home/sj2122/perlstuff';
opendir my $DIR, $dir or die "can't opendir $dir: $!";
readdir $DIR;
};

foreach $i (@files)

Why aren't you using strictures?

for my $i (@files) {

{
# Read every position file
print "$i\n";
open ACCOUNT_POSITION, "<$i" or die "Can't open $i $!";

Use lexical filehandles.
Use scopes to close them.
Use 3-arg open.

open my $ACCOUNT_POSITION, '<', $i or die "can't open $i: $!";

local $_ = <ACCOUNT_POSITION>;
if (/^APR/)
{
push @position_records,$_;
}

Why are you manually going through one iteration of the loop below?

while (<ACCOUNT_POSITION>)
{ #get all lines of the record
# stop if we encounter an APR line

This doesn't stop: it records the line and carries on. If you want to
stop you will need a 'last' inside the if.

if (/^APR/)
{
push @position_records, $_;

}

I would have written this

/^APR/ and push @position_records, $_;

}
close ACCOUNT_POSITION;

There is no need for an explicit close with lexical FHs.

}

foreach $i (@position_records)

^^ my

{

%parameters_owner= (

^^ my

dealer_number => substr($i,6,7),
CUSIP => substr($i,22,9),
cust_accnt_number => substr($i,38,20),
total_shares => substr($i,59,15),
);

if ($parameters_owner{dealer_number}==7654321)

I presume there is a good reason for this value being hard-coded? If
nothing else I would put

use constant SELECTED_DEALER => 7654321;

at the top and use that instead.

{

print "Yahoo! We need to add this CUSIP number to selected records
in Price File!\n";


push @arrayofparams, {%parameters_owner};

If you'd properly created the %parameters_owner as a lexical the you
could simply push a reference to it

push @arrayofparams, \%parameters_owner;

and save a copy.

push @cusip, $parameters_owner{CUSIP};

push @customers, $parameters_owner{cust_accnt_number};

}

}
my %hash = map {$_ => 1} @cusip;
@cusip_new = keys %hash; # to get unique CUSIP numbers

my %hash2 = map {$_ => 1} @customers;
@customers_new = keys %hash2;

I would put each if these in a block: there is no need for the temporary
hashes to exist outside of those two statements:

my @cusip_new = do {
my %tmp = map { $_ => 1 } @cusip;
keys %tmp;
};

foreach $i (@cusip_new)
{

my $sharetotal=0;
for $y (0 .. $#arrayofparams)

for my $y (@arrayofparams) {

{
for $valuesonly (values %{ $arrayofparams[$y]} )

for my $valuesonly (values %$y) {

Are you sure this is correct? Do you not simply want to test if
$y->{CUSIP} eq $i? (I am asking 'is there an occasion where
dealer_number, cust_acct_number or total_shares will match a cusip, and
do you want to catch those matches or not?')

{
if ($valuesonly eq $i)
{
print "$valuesonly\n";
print "$arrayofparams[$y]{total_shares}\n";
...$y->{total_shares}...

$sharetotal += $arrayofparams[$y]{total_shares};
}
}
}
print "Total shares for $i is $sharetotal\n";
$BASE[0]{$i}= $sharetotal; # For day 1, total for the various
CUSIPs

Why not just build your total in $BASE[0]{$i} in the first place?

print "$BASE[0]{$i}\n";

}
store(\@BASE, 'base_values'); # For later access.

open RECORDS, "< AETNA1.txt " or die "Can't read file: $!";

local $_ = <RECORDS>;

while (<RECORDS>)
{ #get all lines of the record

# stop if we encounter an FPR line

if (/^FPR/)
{
push @records, $_;
}

}

close RECORDS;



my $date= substr(@records[0],22,8);

^ $
Why don't you have warnings turned on?

my $cpday= &UnixDate($date,"%d");

Don't call subs with & unless you know what it does and why you need it.

my $cpmonth=&UnixDate($date,"%m");
my $cpyear=&UnixDate($date,"%y");
my $cpdate=$cpmonth.$cpday.$cpyear;

my $cpfilename= "AD".$cpyear.$cpmonth.$cpday."\.PRI";

my $cpfilename = UnixDate $date, 'AD%y%m%d.PRI';

my $wanted_handle = IO::File->new($cpfilename, O_WRONLY|O_CREAT)
or die "Couldn't open $cpfilename for writing:$!\n";

Use lexical FHs instead of IO::File; or certainly, be consistent.
It is usual to use caps for filehandles.
Don't put "\n" on the end of die messages.

foreach $i (@records)
{

%parameters= (

CUSIP_P => substr($i,6,9),
fund_name => substr($i,59,38),
processing_date => $cpdate,

processing_data => UnixData $data, '%m%d%y';

Except don't use m-d-y date formats, they're just too brain-damaged for
words...

NAV_P => substr($i,30,9),
Fund_Type_P => "MU",
Public_Offering_Price => "000".substr($i,39,9),
);


foreach $selectedcusip (@cusip_new)
{
if ($selectedcusip eq $parameters{CUSIP_P})
{
push @arraypriceparams, {%parameters};

}

}
}


for $x (0 .. $#arraypriceparams)

{
print $wanted_handle
"$arraypriceparams[$x]{CUSIP_P},$arraypriceparams[$x]{Fund_Type_P},$arraypriceparams[$x]{processing_date},$arraypriceparams[$x]{Public_Offering_Price}\n";
}


}

#If not the first day of the month, read DIRECT FINANCIAL ACTIVITY
FILE

else
{
my $mday = (localtime)[3];
print "Today we are going to read the Direct Financial Acitivity
File and see the status of our CUSIPs!\n";

# Retrieving all base values

my $base_ref = retrieve('base_values');

#my @BASE = @{$base_ref};
#foreach $i (@{$base_ref})
#{
# print "$i{AET001A70}";
#}
print "$$base_ref[0]{AET001A20}\n";

The thing that has me stumped is how do access the BASE array that I
stored using the storable module. When i try and print out the
particular element, it works but doesnt work ..gives me a HASH{xxxx}
error...

Please explain...your $base_ref is here a ref to an array of hashes, as
you seem to have figured out; I would have written the deref above

$base_ref->[0]{AET001A20}

. If you want to loop, you will need @$base_ref:

for my $i (@$base_ref) {
print $i->{AET001A70};
}

also as I mention right at the end I store it as BASE[0][$i]
where $i is an element of an array of unique 9 digit numbers. (not
necessarily sorted etc)...Is there any way I can get it to print based
on the length of the hard-linked array..as opposed to the index or
something???

Please explain more clearly what you want... what do you mean by 'print
based on'? Do you mean 'print a list sorted by'? And there is no such
thing as a 'hard-linked array' in Perl: the only link between
$BASE[0]{$i} and @cusp_ip is the value they hold in common.

Ben

 

[Shalini Joshi]

Hey Ben!

Thanks again for the help. What I am getting totally confused here is
to what data structures to use given the problem. I need to get at the
information across days so I store a reference to the array
BASE[$mday-1]{$i} for $i (@cusips) . NOw i would like to get at all
the elements of the array for all $i for a particular value of $mday.
I can't seem to do that.


Also could you advise me on the choice of a data structure. Right now
I read in all the records,and store them as array of hashes and
compare these.

Do let me know if it makes any sense or should I re-state my problem a
bit more clearly..

Thanks again ...

Regards,

Shalini

Quoth (e-mail address removed) (Shalini Joshi):

So here is what I have of the code uptil now.

if ((localtime)[3] ==1)
{
print "Today is the first day of the month!\n";
print "We are going to read all Account Position Files!\n";

Set $\="\n" rather than putting it on the end of every print.

#For each client we read the Account Position File: thus all account
position files on the server.

opendir DIR, '/home/sj2122/perlstuff' or die "$!";
my @files = grep {/[^(03)]+03\.[^\.]+\.txt$/} readdir(DIR);
closedir(DIR);

Use lexical dirhandles.
Use scopes to close them.
Include the filename in the error message.

my @files = grep /.../, do {
my $dir = '/home/sj2122/perlstuff';
opendir my $DIR, $dir or die "can't opendir $dir: $!";
readdir $DIR;
};

foreach $i (@files)

Why aren't you using strictures?

for my $i (@files) {

{
# Read every position file
print "$i\n";
open ACCOUNT_POSITION, "<$i" or die "Can't open $i $!";

Use lexical filehandles.
Use scopes to close them.
Use 3-arg open.

open my $ACCOUNT_POSITION, '<', $i or die "can't open $i: $!";

local $_ = <ACCOUNT_POSITION>;
if (/^APR/)
{
push @position_records,$_;
}

Why are you manually going through one iteration of the loop below?

while (<ACCOUNT_POSITION>)
{ #get all lines of the record
# stop if we encounter an APR line

This doesn't stop: it records the line and carries on. If you want to
stop you will need a 'last' inside the if.

if (/^APR/)
{
push @position_records, $_;

}

I would have written this

/^APR/ and push @position_records, $_;

}
close ACCOUNT_POSITION;

There is no need for an explicit close with lexical FHs.

}

foreach $i (@position_records)

^^ my

{

%parameters_owner= (

^^ my

dealer_number => substr($i,6,7),
CUSIP => substr($i,22,9),
cust_accnt_number => substr($i,38,20),
total_shares => substr($i,59,15),
);

if ($parameters_owner{dealer_number}==7654321)

I presume there is a good reason for this value being hard-coded? If
nothing else I would put

use constant SELECTED_DEALER => 7654321;

at the top and use that instead.

{

print "Yahoo! We need to add this CUSIP number to selected records
in Price File!\n";


push @arrayofparams, {%parameters_owner};

If you'd properly created the %parameters_owner as a lexical the you
could simply push a reference to it

push @arrayofparams, \%parameters_owner;

and save a copy.

push @cusip, $parameters_owner{CUSIP};

push @customers, $parameters_owner{cust_accnt_number};

}

}
my %hash = map {$_ => 1} @cusip;
@cusip_new = keys %hash; # to get unique CUSIP numbers

my %hash2 = map {$_ => 1} @customers;
@customers_new = keys %hash2;

I would put each if these in a block: there is no need for the temporary
hashes to exist outside of those two statements:

my @cusip_new = do {
my %tmp = map { $_ => 1 } @cusip;
keys %tmp;
};

foreach $i (@cusip_new)
{

my $sharetotal=0;
for $y (0 .. $#arrayofparams)

for my $y (@arrayofparams) {

{
for $valuesonly (values %{ $arrayofparams[$y]} )

for my $valuesonly (values %$y) {

Are you sure this is correct? Do you not simply want to test if
$y->{CUSIP} eq $i? (I am asking 'is there an occasion where
dealer_number, cust_acct_number or total_shares will match a cusip, and
do you want to catch those matches or not?')

{
if ($valuesonly eq $i)
{
print "$valuesonly\n";
print "$arrayofparams[$y]{total_shares}\n";
...$y->{total_shares}...

$sharetotal += $arrayofparams[$y]{total_shares};
}
}
}
print "Total shares for $i is $sharetotal\n";
$BASE[0]{$i}= $sharetotal; # For day 1, total for the various
CUSIPs

Why not just build your total in $BASE[0]{$i} in the first place?

print "$BASE[0]{$i}\n";

}
store(\@BASE, 'base_values'); # For later access.

open RECORDS, "< AETNA1.txt " or die "Can't read file: $!";

local $_ = <RECORDS>;

while (<RECORDS>)
{ #get all lines of the record

# stop if we encounter an FPR line

if (/^FPR/)
{
push @records, $_;
}

}

close RECORDS;



my $date= substr(@records[0],22,8);

^ $
Why don't you have warnings turned on?

my $cpday= &UnixDate($date,"%d");

Don't call subs with & unless you know what it does and why you need it.

my $cpmonth=&UnixDate($date,"%m");
my $cpyear=&UnixDate($date,"%y");
my $cpdate=$cpmonth.$cpday.$cpyear;

my $cpfilename= "AD".$cpyear.$cpmonth.$cpday."\.PRI";

my $cpfilename = UnixDate $date, 'AD%y%m%d.PRI';

my $wanted_handle = IO::File->new($cpfilename, O_WRONLY|O_CREAT)
or die "Couldn't open $cpfilename for writing:$!\n";

Use lexical FHs instead of IO::File; or certainly, be consistent.
It is usual to use caps for filehandles.
Don't put "\n" on the end of die messages.

foreach $i (@records)
{

%parameters= (

CUSIP_P => substr($i,6,9),
fund_name => substr($i,59,38),
processing_date => $cpdate,

processing_data => UnixData $data, '%m%d%y';

Except don't use m-d-y date formats, they're just too brain-damaged for
words...

NAV_P => substr($i,30,9),
Fund_Type_P => "MU",
Public_Offering_Price => "000".substr($i,39,9),
);


foreach $selectedcusip (@cusip_new)
{
if ($selectedcusip eq $parameters{CUSIP_P})
{
push @arraypriceparams, {%parameters};

}

}
}


for $x (0 .. $#arraypriceparams)

{
print $wanted_handle
"$arraypriceparams[$x]{CUSIP_P},$arraypriceparams[$x]{Fund_Type_P},$arraypriceparams[$x]{processing_date},$arraypriceparams[$x]{Public_Offering_Price}\n";
}


}

#If not the first day of the month, read DIRECT FINANCIAL ACTIVITY
FILE

else
{
my $mday = (localtime)[3];
print "Today we are going to read the Direct Financial Acitivity
File and see the status of our CUSIPs!\n";

# Retrieving all base values

my $base_ref = retrieve('base_values');

#my @BASE = @{$base_ref};
#foreach $i (@{$base_ref})
#{
# print "$i{AET001A70}";
#}
print "$$base_ref[0]{AET001A20}\n";

The thing that has me stumped is how do access the BASE array that I
stored using the storable module. When i try and print out the
particular element, it works but doesnt work ..gives me a HASH{xxxx}
error...

Please explain...your $base_ref is here a ref to an array of hashes, as
you seem to have figured out; I would have written the deref above

$base_ref->[0]{AET001A20}

. If you want to loop, you will need @$base_ref:

for my $i (@$base_ref) {
print $i->{AET001A70};
}

also as I mention right at the end I store it as BASE[0][$i]
where $i is an element of an array of unique 9 digit numbers. (not
necessarily sorted etc)...Is there any way I can get it to print based
on the length of the hard-linked array..as opposed to the index or
something???

Please explain more clearly what you want... what do you mean by 'print
based on'? Do you mean 'print a list sorted by'? And there is no such
thing as a 'hard-linked array' in Perl: the only link between
$BASE[0]{$i} and @cusp_ip is the value they hold in common.

Ben

 

[Josef Moellers]

Hey Ben!

Thanks again for the help. What I am getting totally confused here is
to what data structures to use given the problem. I need to get at the
information across days so I store a reference to the array
BASE[$mday-1]{$i} for $i (@cusips) . NOw i would like to get at all
the elements of the array for all $i for a particular value of $mday.
I can't seem to do that.

The advantage with computers is that you can try, experiment.
What have you tried so far and where have you failed?
What error messagesa have you got and where do you not understand them?
What is BASE[$mday-1]{$i}? It's not Perl, AFAIK, $BASE[$mday-1]{$i}
would be, it would be a hash reference.

[ useless quoting deleted ]