How do I do this without database?

Discussion in 'Perl Misc' started by Public Interest, Oct 19, 2003.

  1. I am trying to scan stock price and volumes on a watch list. I have 1100
    stocks on the list and I want to save them in a single file then use excel.
    Here is the format I want:

    symbol|price on day1|volume on day1|price on day2|price on day2|....
    IBM|98.02|203092|97.01|1212300
    SUNW|3.45|1112232|3.72|1212219

    I will scan the price and update add the new price and volume to the end of
    the line of each stock. The only thing I came up is to read the whole file
    in one array and add things to each element of the array then rewrite the
    whole array to the file. I know it is kind of breaking the RULE of not
    reading the whole file but only line by line. And it takes me 1 hours for
    scan all the data due to my dial-up connections. So I want to have something
    that I can start and pause then restart.But if I only read line by line, how
    do I append only to the end of the line not to the end of the file? It came
    back to the old question which is how to delete/modify lines in a text file.

    I am using active perl on windows.
    Public Interest, Oct 19, 2003
    #1
    1. Advertising

  2. Ok, I did not know that Tie:File is now in place starting with 5.8.0

    Tie:File super
    Public Interest, Oct 19, 2003
    #2
    1. Advertising

  3. -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    "Public Interest" <> wrote in news:cQokb.6701$Ec1.610254
    @bgtnsc05-news.ops.worldnet.att.net:

    > Ok, I did not know that Tie:File is now in place starting with 5.8.0
    >
    > Tie:File super


    Tie::File is super, but you misunderstand it. It will not make your
    program any faster. What Tie::File does is (more or less) what you have
    been trying to do -- it has to do the same amount of reading and writing.
    Tie::File makes it easier to update files on a line-by-line basis; it does
    not make it any faster, in general.

    - --
    Eric
    $_ = reverse sort $ /. r , qw p ekca lre uJ reh
    ts p , map $ _. $ " , qw e p h tona e and print

    -----BEGIN PGP SIGNATURE-----
    Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

    iQA/AwUBP5KPGWPeouIeTNHoEQJE+ACg1OkZ0/lQety2EJmsomal0UzWEuIAn0rt
    +q/EjmONqmuppofN5Lj7ShKv
    =sNXV
    -----END PGP SIGNATURE-----
    Eric J. Roode, Oct 19, 2003
    #3
  4. On Sun, 19 Oct 2003 04:11:32 GMT
    "Public Interest" <> wrote:

    > I am trying to scan stock price and volumes on a watch list. I have
    > 1100 stocks on the list and I want to save them in a single file
    > then use excel. Here is the format I want:
    >
    > symbol|price on day1|volume on day1|price on day2|price on day2|....
    > IBM|98.02|203092|97.01|1212300
    > SUNW|3.45|1112232|3.72|1212219
    >
    > I will scan the price and update add the new price and volume to the
    > end of the line of each stock. The only thing I came up is to read
    > the whole file in one array and add things to each element of the
    > array then rewrite the whole array to the file. I know it is kind of
    > breaking the RULE of not reading the whole file but only line by
    > line. And it takes me 1 hours for scan all the data due to my
    > dial-up connections. So I want to have something that I can start
    > and pause then restart.But if I only read line by line, how do I
    > append only to the end of the line not to the end of the file? It
    > came back to the old question which is how to delete/modify lines in
    > a text file.
    >
    > I am using active perl on windows.


    You _could_ use DBD::CSV. In this case, the file could be treated as
    a database. So, you could use SQL to manipulate the file. You _may_
    not gain much as far as speed using this method.

    If you're more interested in speed, use a RDBMS - such as MySQL,
    PostgreSQL, etc.

    You could also, since you're using Windows, M$ Access and use the
    DBD::ODBC module to access the database. However, it's not an
    efficent or portable solution.

    Bottom line, IMHO, is give serious consideration to finding a way to
    do this in a RDBMS.

    HTH

    --
    Jim

    Copyright notice: all code written by the author in this post is
    released under the GPL. http://www.gnu.org/licenses/gpl.txt
    for more information.

    a fortune quote ...
    "We are on the verge: Today our program proved Fermat's
    next-to-last theorem." -- Epigrams in Programming, ACM SIGPLAN
    Sept. 1982
    James Willmore, Oct 19, 2003
    #4
  5. Public Interest

    Bob Walton Guest

    Public Interest wrote:

    > I am trying to scan stock price and volumes on a watch list. I have 1100
    > stocks on the list and I want to save them in a single file then use excel.
    > Here is the format I want:
    >
    > symbol|price on day1|volume on day1|price on day2|price on day2|....
    > IBM|98.02|203092|97.01|1212300
    > SUNW|3.45|1112232|3.72|1212219
    >
    > I will scan the price and update add the new price and volume to the end of
    > the line of each stock. The only thing I came up is to read the whole file
    > in one array and add things to each element of the array then rewrite the
    > whole array to the file. I know it is kind of breaking the RULE of not
    > reading the whole file but only line by line. And it takes me 1 hours for
    > scan all the data due to my dial-up connections. So I want to have something
    > that I can start and pause then restart.But if I only read line by line, how
    > do I append only to the end of the line not to the end of the file? It came
    > back to the old question which is how to delete/modify lines in a text file.
    >
    > I am using active perl on windows.


    You should check out tie'ing a hash to a DBM-type file.

    That will give very quick access to individual records
    without the necessity to read the entire file each time.
    See:

    perldoc -f dbmopen
    perldoc -f tie

    to get started with that.


    --
    Bob Walton
    Email: http://bwalton.com/cgi-bin/emailbob.pl
    Bob Walton, Oct 19, 2003
    #5
  6. Public Interest

    Eric Bohlman Guest

    Bob Walton <> wrote in
    news::

    > Public Interest wrote:
    >
    >> I am trying to scan stock price and volumes on a watch list. I have
    >> 1100 stocks on the list and I want to save them in a single file then
    >> use excel. Here is the format I want:
    >>
    >> symbol|price on day1|volume on day1|price on day2|price on day2|....
    >> IBM|98.02|203092|97.01|1212300
    >> SUNW|3.45|1112232|3.72|1212219
    >>
    >> I will scan the price and update add the new price and volume to the
    >> end of the line of each stock. The only thing I came up is to read
    >> the whole file in one array and add things to each element of the
    >> array then rewrite the whole array to the file. I know it is kind of
    >> breaking the RULE of not reading the whole file but only line by
    >> line. And it takes me 1 hours for scan all the data due to my dial-up
    >> connections. So I want to have something that I can start and pause
    >> then restart.But if I only read line by line, how do I append only to
    >> the end of the line not to the end of the file? It came back to the
    >> old question which is how to delete/modify lines in a text file.
    >>
    >> I am using active perl on windows.

    >
    > You should check out tie'ing a hash to a DBM-type file.
    >
    > That will give very quick access to individual records
    > without the necessity to read the entire file each time.


    I think everybody here has been misreading the OP's requirements. He's not
    looking to speed up the scanning process, which as he points out is slow
    because the data is being pulled in over a slow connection. The most
    optimized database engine in the world would probably give a 2%-3% runtime
    savings.

    What makes the most sense to me is to grab the watch list from the Web and
    build from it a hash whose keys are the stock symbols and whose values are
    the prices. Then open the pipe-delimited file and read it line by line; if
    there's an entry in the hash for the symbol on the line, tack the value on
    to the end. In any case, write out the line.
    Eric Bohlman, Oct 19, 2003
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dan
    Replies:
    8
    Views:
    8,254
  2. =?Utf-8?B?U1M=?=
    Replies:
    3
    Views:
    3,830
    Juan T. Llibre
    May 15, 2005
  3. Paul W
    Replies:
    4
    Views:
    7,805
    Nick Gilbert
    Jun 2, 2005
  4. Replies:
    5
    Views:
    420
    Steven Bethard
    Mar 3, 2006
  5. Tony Johansson
    Replies:
    7
    Views:
    499
    Gregory A. Beamer
    Dec 23, 2009
Loading...

Share This Page