How do I do this without database?

[Public Interest]

I am trying to scan stock price and volumes on a watch list. I have 1100
stocks on the list and I want to save them in a single file then use excel.
Here is the format I want:

symbol|price on day1|volume on day1|price on day2|price on day2|....
IBM|98.02|203092|97.01|1212300
SUNW|3.45|1112232|3.72|1212219

I will scan the price and update add the new price and volume to the end of
the line of each stock. The only thing I came up is to read the whole file
in one array and add things to each element of the array then rewrite the
whole array to the file. I know it is kind of breaking the RULE of not
reading the whole file but only line by line. And it takes me 1 hours for
scan all the data due to my dial-up connections. So I want to have something
that I can start and pause then restart.But if I only read line by line, how
do I append only to the end of the line not to the end of the file? It came
back to the old question which is how to delete/modify lines in a text file.

I am using active perl on windows.

 

[Public Interest]

Ok, I did not know that Tie:File is now in place starting with 5.8.0

Tie:File super

 

[Eric J. Roode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ok, I did not know that Tie:File is now in place starting with 5.8.0

Tie:File super

Tie::File is super, but you misunderstand it. It will not make your
program any faster. What Tie::File does is (more or less) what you have
been trying to do -- it has to do the same amount of reading and writing.
Tie::File makes it easier to update files on a line-by-line basis; it does
not make it any faster, in general.

- --
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBP5KPGWPeouIeTNHoEQJE+ACg1OkZ0/lQety2EJmsomal0UzWEuIAn0rt
+q/EjmONqmuppofN5Lj7ShKv
=sNXV
-----END PGP SIGNATURE-----

 

[James Willmore]

I am trying to scan stock price and volumes on a watch list. I have
1100 stocks on the list and I want to save them in a single file
then use excel. Here is the format I want:

symbol|price on day1|volume on day1|price on day2|price on day2|....
IBM|98.02|203092|97.01|1212300
SUNW|3.45|1112232|3.72|1212219

I will scan the price and update add the new price and volume to the
end of the line of each stock. The only thing I came up is to read
the whole file in one array and add things to each element of the
array then rewrite the whole array to the file. I know it is kind of
breaking the RULE of not reading the whole file but only line by
line. And it takes me 1 hours for scan all the data due to my
dial-up connections. So I want to have something that I can start
and pause then restart.But if I only read line by line, how do I
append only to the end of the line not to the end of the file? It
came back to the old question which is how to delete/modify lines in
a text file.

I am using active perl on windows.

You _could_ use DBD::CSV. In this case, the file could be treated as
a database. So, you could use SQL to manipulate the file. You _may_
not gain much as far as speed using this method.

If you're more interested in speed, use a RDBMS - such as MySQL,
PostgreSQL, etc.

You could also, since you're using Windows, M$ Access and use the
DBD::ODBC module to access the database. However, it's not an
efficent or portable solution.

Bottom line, IMHO, is give serious consideration to finding a way to
do this in a RDBMS.

HTH

--
Jim

Copyright notice: all code written by the author in this post is
released under the GPL. http://www.gnu.org/licenses/gpl.txt
for more information.

a fortune quote ...
"We are on the verge: Today our program proved Fermat's
next-to-last theorem." -- Epigrams in Programming, ACM SIGPLAN
Sept. 1982

 

[Bob Walton]

I am trying to scan stock price and volumes on a watch list. I have 1100
stocks on the list and I want to save them in a single file then use excel.
Here is the format I want:

symbol|price on day1|volume on day1|price on day2|price on day2|....
IBM|98.02|203092|97.01|1212300
SUNW|3.45|1112232|3.72|1212219

I will scan the price and update add the new price and volume to the end of
the line of each stock. The only thing I came up is to read the whole file
in one array and add things to each element of the array then rewrite the
whole array to the file. I know it is kind of breaking the RULE of not
reading the whole file but only line by line. And it takes me 1 hours for
scan all the data due to my dial-up connections. So I want to have something
that I can start and pause then restart.But if I only read line by line, how
do I append only to the end of the line not to the end of the file? It came
back to the old question which is how to delete/modify lines in a text file.

I am using active perl on windows.

You should check out tie'ing a hash to a DBM-type file.

That will give very quick access to individual records
without the necessity to read the entire file each time.
See:

perldoc -f dbmopen
perldoc -f tie

to get started with that.

 

[Eric Bohlman]

You should check out tie'ing a hash to a DBM-type file.

That will give very quick access to individual records
without the necessity to read the entire file each time.

I think everybody here has been misreading the OP's requirements. He's not
looking to speed up the scanning process, which as he points out is slow
because the data is being pulled in over a slow connection. The most
optimized database engine in the world would probably give a 2%-3% runtime
savings.

What makes the most sense to me is to grab the watch list from the Web and
build from it a hash whose keys are the stock symbols and whose values are
the prices. Then open the pipe-delimited file and read it line by line; if
there's an entry in the hash for the symbol on the line, tack the value on
to the end. In any case, write out the line.