Looping through File Question

Discussion in 'Python' started by planetmatt, Sep 5, 2007.

  1. planetmatt

    planetmatt Guest

    I am a Python beginner. I am trying to loop through a CSV file which
    I can do. What I want to change though is for the loop to start at
    row 2 in the file thus excluding column headers.

    At present I am using this statement to initiate a loop though the
    records:

    for line in f.readlines():

    How do I start this at row 2?
     
    planetmatt, Sep 5, 2007
    #1
    1. Advertising

  2. planetmatt

    vijayca Guest

    On Sep 5, 3:58 pm, planetmatt <> wrote:
    > I am a Python beginner. I am trying to loop through a CSV file which
    > I can do. What I want to change though is for the loop to start at
    > row 2 in the file thus excluding column headers.
    >
    > At present I am using this statement to initiate a loop though the
    > records:
    >
    > for line in f.readlines():
    >
    > How do I start this at row 2?


    just use readline() method...this will move the file pointer to the
    next line....
    example:
    fd=open("/tmp.txt","r")
    fd.readline() ##THIS MOVES THE FILE POINTER TO THE
    SECOND LINE

    now carry on with...
    for line in f.readlines(): #THIS WILL BEGIN WITH THE SECOND ROW AS
    THE FILE POINTER IS IN 2NDLINE
    ....
     
    vijayca, Sep 5, 2007
    #2
    1. Advertising

  3. planetmatt

    Amit Khemka Guest

    On 9/5/07, Francesco Guerrieri <> wrote:
    > On 9/5/07, planetmatt <> wrote:
    > > I am a Python beginner. I am trying to loop through a CSV file which
    > > I can do. What I want to change though is for the loop to start at
    > > row 2 in the file thus excluding column headers.
    > >
    > > At present I am using this statement to initiate a loop though the
    > > records:
    > >
    > > for line in f.readlines():
    > >
    > > How do I start this at row 2?
    > >

    >
    > you can simply call (and maybe throw away) f.readline() a single time before
    > looping.
    > If the lenght of the first line is fixed, you can also use f.seek to start
    > reading from the second row.
    >
    > francesco


    Btw, if you are trying to read a csv file and parse it, you can save
    some work .. have a look at "csv" module !

    --
    ----
    Amit Khemka
    website: www.onyomo.com
    wap-site: www.owap.in
     
    Amit Khemka, Sep 5, 2007
    #3
  4. planetmatt

    John Machin Guest

    On Sep 5, 8:58 pm, planetmatt <> wrote:
    > I am a Python beginner. I am trying to loop through a CSV file which
    > I can do. What I want to change though is for the loop to start at
    > row 2 in the file thus excluding column headers.
    >
    > At present I am using this statement to initiate a loop though the
    > records:
    >
    > for line in f.readlines():
    >
    > How do I start this at row 2?


    The quick answer to your literal question is:
    for line in f.readlines()[1:]:
    or, with extreme loss of elegance, this:
    for lino, line in enumerate(f.readlines()):
    if not lino:
    continue

    But readline and readlines are old hat, and you wouldn't want to read
    a file of a few million lines into a big list, so a better answer is:

    _unused = f.next()
    for line in f:

    But you did say you were reading a CSV file, and you don't really want
    to do your own CSV parsing, even if you think you know how to get it
    right, so best is:

    import csv
    rdr = csv.reader(f)
    heading_row = rdr.next()
    for data_row in rdr:

    HTH,
    John
     
    John Machin, Sep 5, 2007
    #4
  5. planetmatt

    planetmatt Guest

    On 5 Sep, 12:34, John Machin <> wrote:
    > On Sep 5, 8:58 pm, planetmatt <> wrote:
    >
    > > I am a Python beginner. I am trying to loop through a CSV file which
    > > I can do. What I want to change though is for the loop to start at
    > > row 2 in the file thus excluding column headers.

    >
    > > At present I am using this statement to initiate a loop though the
    > > records:

    >
    > > for line in f.readlines():

    >
    > > How do I start this at row 2?

    >
    > The quick answer to your literal question is:
    > for line in f.readlines()[1:]:
    > or, with extreme loss of elegance, this:
    > for lino, line in enumerate(f.readlines()):
    > if not lino:
    > continue
    >
    > But readline and readlines are old hat, and you wouldn't want to read
    > a file of a few million lines into a big list, so a better answer is:
    >
    > _unused = f.next()
    > for line in f:
    >
    > But you did say you were reading a CSV file, and you don't really want
    > to do your own CSV parsing, even if you think you know how to get it
    > right, so best is:
    >
    > import csv
    > rdr = csv.reader(f)
    > heading_row = rdr.next()
    > for data_row in rdr:
    >
    > HTH,
    > John


    Thanks so much for the quick response. All working now.

    I had looked at the CSV module but when I ran into another problem of
    trying to loop through all columns, I was given a solution in another
    forum which used the readlines() method.

    I have looked at the CSV documentation but didn't see any mention of
    heading_row or data_row. Is there a definitive Python documentation
    site with code examples like MS's MSDN?
     
    planetmatt, Sep 5, 2007
    #5
  6. planetmatt

    John Machin Guest

    On Sep 5, 10:26 pm, planetmatt <> wrote:
    > On 5 Sep, 12:34, John Machin <> wrote:
    >
    >
    >
    > > On Sep 5, 8:58 pm, planetmatt <> wrote:

    >
    > > > I am a Python beginner. I am trying to loop through a CSV file which
    > > > I can do. What I want to change though is for the loop to start at
    > > > row 2 in the file thus excluding column headers.

    >
    > > > At present I am using this statement to initiate a loop though the
    > > > records:

    >
    > > > for line in f.readlines():

    >
    > > > How do I start this at row 2?

    >
    > > The quick answer to your literal question is:
    > > for line in f.readlines()[1:]:
    > > or, with extreme loss of elegance, this:
    > > for lino, line in enumerate(f.readlines()):
    > > if not lino:
    > > continue

    >
    > > But readline and readlines are old hat, and you wouldn't want to read
    > > a file of a few million lines into a big list, so a better answer is:

    >
    > > _unused = f.next()
    > > for line in f:

    >
    > > But you did say you were reading a CSV file, and you don't really want
    > > to do your own CSV parsing, even if you think you know how to get it
    > > right, so best is:

    >
    > > import csv
    > > rdr = csv.reader(f)
    > > heading_row = rdr.next()
    > > for data_row in rdr:

    >
    > > HTH,
    > > John

    >
    > Thanks so much for the quick response. All working now.
    >
    > I had looked at the CSV module but when I ran into another problem of
    > trying to loop through all columns, I was given a solution in another
    > forum which used the readlines() method.


    Antique advice which still left you doing the CSV parsing :)

    >
    > I have looked at the CSV documentation but didn't see any mention of
    > heading_row or data_row.


    Ummm ... 'heading_row' and 'data_row' are identifiers of the kind that
    you or I would need to make up in any language; why did you expect to
    find them in the documentation?

    > Is there a definitive Python documentation
    > site with code examples like MS's MSDN?


    The definitive Python documentation site is (I suppose) http://www.python.org/doc/

    I don't know what "code examples like MS's MSDN" means. I avoid MSDN
    like I'd avoid a nurse carrying a bottle of Dettol and a wire
    brush :)

    Here's an example of sucking your data into a list of lists and
    accessing it columnwise:

    rdr = csv.reader(f)
    whatever_you_want_to_call_the_heading_row = rdr.next()
    data = list(rdr)
    sum_col_5 = sum(float(row[5]) for row in data)
    print "The value in row 3, column 4 is", data[3][4]

    HTH,
    John
     
    John Machin, Sep 5, 2007
    #6
  7. > > > On Sep 5, 8:58 pm, planetmatt <> wrote:
    >
    > > > > I am a Python beginner. I am trying to loop through a CSV file which
    > > > > I can do. What I want to change though is for the loop to start at
    > > > > row 2 in the file thus excluding column headers.


    The DictReader object automatically does this for you:

    >>> data = """Name,age,job

    .... Tom,21,programmer
    .... Sara,22,chef
    .... Molly,23,doctor"""
    >>> from csv import DictReader
    >>> myStuff = DictReader(data.split('\n'))
    >>> for line in myStuff:

    .... print line
    ....
    {'job': 'programmer', 'age': '21', 'Name': 'Tom'}
    {'job': 'chef', 'age': '22', 'Name': 'Sara'}
    {'job': 'doctor', 'age': '23', 'Name': 'Molly'}

    HTH,
    Tom
     
    Thomas Nelson, Sep 5, 2007
    #7
  8. John Machin wrote:
    > On Sep 5, 10:26 pm, planetmatt <> wrote:
    >> On 5 Sep, 12:34, John Machin <> wrote:
    >>
    >>
    >>
    >>> On Sep 5, 8:58 pm, planetmatt <> wrote:
    >>>> I am a Python beginner. I am trying to loop through a CSV file which
    >>>> I can do. What I want to change though is for the loop to start at
    >>>> row 2 in the file thus excluding column headers.
    >>>> At present I am using this statement to initiate a loop though the
    >>>> records:
    >>>> for line in f.readlines():
    >>>> How do I start this at row 2?
    >>> The quick answer to your literal question is:
    >>> for line in f.readlines()[1:]:
    >>> or, with extreme loss of elegance, this:
    >>> for lino, line in enumerate(f.readlines()):
    >>> if not lino:
    >>> continue
    >>> But readline and readlines are old hat, and you wouldn't want to read
    >>> a file of a few million lines into a big list, so a better answer is:
    >>> _unused = f.next()
    >>> for line in f:
    >>> But you did say you were reading a CSV file, and you don't really want
    >>> to do your own CSV parsing, even if you think you know how to get it
    >>> right, so best is:
    >>> import csv
    >>> rdr = csv.reader(f)
    >>> heading_row = rdr.next()
    >>> for data_row in rdr:
    >>> HTH,
    >>> John

    >> Thanks so much for the quick response. All working now.
    >>
    >> I had looked at the CSV module but when I ran into another problem of
    >> trying to loop through all columns, I was given a solution in another
    >> forum which used the readlines() method.

    >
    > Antique advice which still left you doing the CSV parsing :)
    >
    >> I have looked at the CSV documentation but didn't see any mention of
    >> heading_row or data_row.

    >
    > Ummm ... 'heading_row' and 'data_row' are identifiers of the kind that
    > you or I would need to make up in any language; why did you expect to
    > find them in the documentation?
    >
    >> Is there a definitive Python documentation
    >> site with code examples like MS's MSDN?

    >
    > The definitive Python documentation site is (I suppose) http://www.python.org/doc/
    >
    > I don't know what "code examples like MS's MSDN" means. I avoid MSDN
    > like I'd avoid a nurse carrying a bottle of Dettol and a wire
    > brush :)
    >
    > Here's an example of sucking your data into a list of lists and
    > accessing it columnwise:
    >
    > rdr = csv.reader(f)
    > whatever_you_want_to_call_the_heading_row = rdr.next()
    > data = list(rdr)
    > sum_col_5 = sum(float(row[5]) for row in data)
    > print "The value in row 3, column 4 is", data[3][4]
    >
    > HTH,
    > John
    >


    Given that you have a header row you could also do (untested):

    rdr = csv.DictReader(f)
    data = list(rdr)
    sum_Value = sum(float(row['Value']) for row in data)
    print "The value in row 3, column 'ColName' is", data[3]['ColName']

    DictReader generates a list of dictionaries which take the keys from
    your header row.
     
    =?ISO-8859-1?Q?Ricardo_Ar=E1oz?=, Sep 6, 2007
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Bruno Desthuilliers

    Re: looping through a file

    Bruno Desthuilliers, Jul 25, 2003, in forum: Python
    Replies:
    3
    Views:
    361
    Steven Taschuk
    Jul 28, 2003
  2. Rosario Morgan
    Replies:
    3
    Views:
    344
    bruno at modulix
    Jun 14, 2006
  3. AJ Hartley

    Looping through a log file

    AJ Hartley, Mar 4, 2005, in forum: Perl Misc
    Replies:
    5
    Views:
    88
    Eric Bohlman
    Mar 5, 2005
  4. Aaron
    Replies:
    2
    Views:
    530
    dhtml
    Apr 10, 2011
  5. Replies:
    5
    Views:
    279
Loading...

Share This Page