RE: Python does not take up available physical memory

Discussion in 'Python' started by Prasad, Ramit, Oct 19, 2012.

  1. Chris Angelico wrote:

    > On Sat, Oct 20, 2012 at 4:08 AM, Pradipto Banerjee
    > <> wrote:

    > > I amtrying to read a file into memory. The size of the file is around 1 GB.
    > > I have a 3GB memory PC and the Windows Task Manager shows 2.3 GB available
    > > physical memory when I was trying to read the file. I tried to read the file
    > > as follows:
    > >
    > >
    > >

    > >>>> fdata = open(filename, 'r').read()

    >
    > Is this Python 2 or Python 3? Just throwing a random possibility out
    > there, could it be that reading it in and converting it to Unicode
    > text requires more memory than you have?
    >
    > My recommendation: Unless you actually need to search the whole file
    > as a single string, iterate over the file instead:
    >
    > for line in open(filename):
    > # do something with line


    If you (OP) are in Python 2.5+ I would do the following instead.

    with open(filename) as f:
    for line in f:
    # do something with line

    This will automatically close the file when it is done. I doubt
    it will help with memory issues, but closing files after you
    are done with them is a Good practice.


    Ramit Prasad


    This email is confidential and subject to important disclaimers and
    conditions including on offers for the purchase or sale of
    securities, accuracy and completeness of information, viruses,
    confidentiality, legal privilege, and legal entity disclaimers,
    available at http://www.jpmorgan.com/pages/disclosures/email.
    Prasad, Ramit, Oct 19, 2012
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Emile van Sebille
    Replies:
    0
    Views:
    232
    Emile van Sebille
    Oct 20, 2012
  2. Ian Kelly
    Replies:
    0
    Views:
    145
    Ian Kelly
    Oct 19, 2012
  3. Pradipto Banerjee
    Replies:
    1
    Views:
    131
    Steven D'Aprano
    Oct 19, 2012
  4. Pradipto Banerjee
    Replies:
    6
    Views:
    225
    Pradipto Banerjee
    Oct 21, 2012
  5. MRAB
    Replies:
    0
    Views:
    116
Loading...

Share This Page