avoding the accumulation of array when using loop.

Discussion in 'Python' started by Isaac Won, Jan 2, 2013.

  1. Isaac Won

    Isaac Won Guest

    Hi all,

    Thanks to Hans, I have had a good progress on my problem.

    Followings are Hans's Idea:

    import numpy as np

    b = []
    c = 4
    f = open("text.file", "r")

    while c < 10:
    c = c + 1


    f.seek(0,0)

    for columns in ( raw.strip().split() for raw in f ):
    b.append(columns[c])

    y = np.array(b, float)
    print c, y


    It's a bit inefficient to read the same file several times.
    You might consider reading it just once. For example:


    import numpy as np

    b = []



    f = open("text.file", "r")

    data = [ line.strip().split() for line in f ]
    f.close()

    for c in xrange(5, 11):
    for row in data:
    b.append(row[c])


    y = np.array(b, float)
    print c, y
    -------------------------------------------------------------------------------

    It is a great idea, but I found some problems. I want each individual arrayof y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initialize forloop for each time to produce array. This effort has not been very successful.
    Do you guys have any idea? I will really appreciate ant help and idea.

    Thanks,

    Isaac
     
    Isaac Won, Jan 2, 2013
    #1
    1. Advertising

  2. Isaac Won

    Dave Angel Guest

    On 01/02/2013 05:21 PM, Isaac Won wrote:
    > Hi all,
    >
    > Thanks to Hans, I have had a good progress on my problem.
    >
    > Followings are Hans's Idea:
    >
    > import numpy as np
    >
    > b = []
    > c = 4
    > f = open("text.file", "r")
    >
    > while c < 10:
    > c = c + 1
    >
    >
    > f.seek(0,0)
    >
    > for columns in ( raw.strip().split() for raw in f ):
    > b.append(columns[c])
    >
    > y = np.array(b, float)
    > print c, y
    >
    >
    > It's a bit inefficient to read the same file several times.


    Don't bet on it. The OS and the libraries and Python each do some
    buffering, so it might be nearly as fast to just reread if it's a small
    file. And if it's a huge one, the list would be even bigger. So the
    only sizes where the second approach is likely better is the mid-size file.

    > You might consider reading it just once. For example:
    >
    >
    > import numpy as np
    >
    > b = []
    >
    >
    >
    > f = open("text.file", "r")
    >
    > data = [ line.strip().split() for line in f ]
    > f.close()
    >
    > for c in xrange(5, 11):
    > for row in data:
    > b.append(row[c])
    >
    >
    > y = np.array(b, float)
    > print c, y
    > -------------------------------------------------------------------------------
    >
    > It is a great idea, but I found some problems. I want each individual array of y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initialize for loop for each time to produce array. This effort has not been very successful.
    > Do you guys have any idea? I will really appreciate ant help and idea.


    Your description is very confusing. But i don't see why you just don't
    just set b=[] inside the outer loop, rather than doing it at the begin
    of the program.

    for c in xrange(5, 11):
    b = []
    for row in data:
    b.append(row[c])



    --

    DaveA
     
    Dave Angel, Jan 2, 2013
    #2
    1. Advertising

  3. Isaac Won

    Isaac Won Guest

    On Wednesday, January 2, 2013 5:54:18 PM UTC-6, Dave Angel wrote:
    > On 01/02/2013 05:21 PM, Isaac Won wrote:
    >
    > > Hi all,

    >
    > >

    >
    > > Thanks to Hans, I have had a good progress on my problem.

    >
    > >

    >
    > > Followings are Hans's Idea:

    >
    > >

    >
    > > import numpy as np

    >
    > >

    >
    > > b = []

    >
    > > c = 4

    >
    > > f = open("text.file", "r")

    >
    > >

    >
    > > while c < 10:

    >
    > > c = c + 1

    >
    > >

    >
    > >

    >
    > > f.seek(0,0)

    >
    > >

    >
    > > for columns in ( raw.strip().split() for raw in f ):

    >
    > > b.append(columns[c])

    >
    > >

    >
    > > y = np.array(b, float)

    >
    > > print c, y

    >
    > >

    >
    > >

    >
    > > It's a bit inefficient to read the same file several times.

    >
    >
    >
    > Don't bet on it. The OS and the libraries and Python each do some
    >
    > buffering, so it might be nearly as fast to just reread if it's a small
    >
    > file. And if it's a huge one, the list would be even bigger. So the
    >
    > only sizes where the second approach is likely better is the mid-size file.
    >
    >
    >
    > > You might consider reading it just once. For example:

    >
    > >

    >
    > >

    >
    > > import numpy as np

    >
    > >

    >
    > > b = []

    >
    > >

    >
    > >

    >
    > >

    >
    > > f = open("text.file", "r")

    >
    > >

    >
    > > data = [ line.strip().split() for line in f ]

    >
    > > f.close()

    >
    > >

    >
    > > for c in xrange(5, 11):

    >
    > > for row in data:

    >
    > > b.append(row[c])

    >
    > >

    >
    > >

    >
    > > y = np.array(b, float)

    >
    > > print c, y

    >
    > > -------------------------------------------------------------------------------

    >
    > >

    >
    > > It is a great idea, but I found some problems. I want each individual array of y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initializefor loop for each time to produce array. This effort has not been very successful.

    >
    > > Do you guys have any idea? I will really appreciate ant help and idea.

    >
    >
    >
    > Your description is very confusing. But i don't see why you just don't
    >
    > just set b=[] inside the outer loop, rather than doing it at the begin
    >
    > of the program.
    >
    >
    >
    > for c in xrange(5, 11):
    >
    > b = []
    >
    > for row in data:
    >
    > b.append(row[c])
    >
    >
    >
    >
    >
    >
    >
    > --
    >
    >
    >
    > DaveA


    Hi Dave,

    I really appreciate your advice. It was really helpful.

    Isaac
     
    Isaac Won, Jan 3, 2013
    #3
  4. Isaac Won

    Isaac Won Guest

    On Wednesday, January 2, 2013 5:54:18 PM UTC-6, Dave Angel wrote:
    > On 01/02/2013 05:21 PM, Isaac Won wrote:
    >
    > > Hi all,

    >
    > >

    >
    > > Thanks to Hans, I have had a good progress on my problem.

    >
    > >

    >
    > > Followings are Hans's Idea:

    >
    > >

    >
    > > import numpy as np

    >
    > >

    >
    > > b = []

    >
    > > c = 4

    >
    > > f = open("text.file", "r")

    >
    > >

    >
    > > while c < 10:

    >
    > > c = c + 1

    >
    > >

    >
    > >

    >
    > > f.seek(0,0)

    >
    > >

    >
    > > for columns in ( raw.strip().split() for raw in f ):

    >
    > > b.append(columns[c])

    >
    > >

    >
    > > y = np.array(b, float)

    >
    > > print c, y

    >
    > >

    >
    > >

    >
    > > It's a bit inefficient to read the same file several times.

    >
    >
    >
    > Don't bet on it. The OS and the libraries and Python each do some
    >
    > buffering, so it might be nearly as fast to just reread if it's a small
    >
    > file. And if it's a huge one, the list would be even bigger. So the
    >
    > only sizes where the second approach is likely better is the mid-size file.
    >
    >
    >
    > > You might consider reading it just once. For example:

    >
    > >

    >
    > >

    >
    > > import numpy as np

    >
    > >

    >
    > > b = []

    >
    > >

    >
    > >

    >
    > >

    >
    > > f = open("text.file", "r")

    >
    > >

    >
    > > data = [ line.strip().split() for line in f ]

    >
    > > f.close()

    >
    > >

    >
    > > for c in xrange(5, 11):

    >
    > > for row in data:

    >
    > > b.append(row[c])

    >
    > >

    >
    > >

    >
    > > y = np.array(b, float)

    >
    > > print c, y

    >
    > > -------------------------------------------------------------------------------

    >
    > >

    >
    > > It is a great idea, but I found some problems. I want each individual array of y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initializefor loop for each time to produce array. This effort has not been very successful.

    >
    > > Do you guys have any idea? I will really appreciate ant help and idea.

    >
    >
    >
    > Your description is very confusing. But i don't see why you just don't
    >
    > just set b=[] inside the outer loop, rather than doing it at the begin
    >
    > of the program.
    >
    >
    >
    > for c in xrange(5, 11):
    >
    > b = []
    >
    > for row in data:
    >
    > b.append(row[c])
    >
    >
    >
    >
    >
    >
    >
    > --
    >
    >
    >
    > DaveA


    Hi Dave,

    I really appreciate your advice. It was really helpful.

    Isaac
     
    Isaac Won, Jan 3, 2013
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?VGltOjouLg==?=

    Loop the loop...

    =?Utf-8?B?VGltOjouLg==?=, Feb 16, 2005, in forum: ASP .Net
    Replies:
    2
    Views:
    1,420
    Karl Seguin
    Feb 16, 2005
  2. Ragnar

    Memory accumulation in web service

    Ragnar, Nov 5, 2003, in forum: ASP .Net Web Services
    Replies:
    0
    Views:
    347
    Ragnar
    Nov 5, 2003
  3. Replies:
    12
    Views:
    218
    Robert Dober
    Mar 16, 2007
  4. Erwin S. Andreasen
    Replies:
    2
    Views:
    146
    Martin Honnen
    Jun 22, 2004
  5. Isaac Won
    Replies:
    9
    Views:
    460
    Ulrich Eckhardt
    Mar 4, 2013
Loading...

Share This Page