CSV reader and unique ids

Discussion in 'Python' started by Mike P, Sep 1, 2008.

  1. Mike P

    Mike P Guest

    Hi All,

    I'm trying to use the CSV module to read in some data and then use a
    hashable method (as there are millions of records) to find unique ids
    and push these out to another file,

    can anyone advise? Below is the code so far


    fin = open(CSV_INPUT, "rb")
    fout = open(CSV_OUTPUT, "wb")
    reader = csv.reader(fin, delimiter=chr(254))
    writer = csv.writer(fout)

    headerList = reader.next()
    UID = {}

    #For help
    #print headerList
    # ['Time', 'User-ID', 'IP']

    try:
    for row in reader[1]:
    UID[row] = 1
    else:
    List= UID.keys()
    writer.writerows(List)
    fin.close()
    fout.close()

    Mike
     
    Mike P, Sep 1, 2008
    #1
    1. Advertisements

  2. Mike P

    Tim Golden Guest

    Mike P wrote:
    > I'm trying to use the CSV module to read in some data and then use a
    > hashable method (as there are millions of records) to find unique ids
    > and push these out to another file,


    You could either zip with a counter or use the uuid module,
    depending on just how unique you want your ids to be.

    <code>
    import os, sys
    import csv
    import itertools
    import uuid

    stuff = "the quick brown fox jumps over the lazy dog".split ()

    f = open ("output.csv", "wb")
    writer = csv.writer (f)

    #
    # Style 1 - numeric counter
    #
    writer.writerows (zip (itertools.count (), stuff))

    #
    # Style 2 - uuid
    #
    writer.writerows ((uuid.uuid1 (), s) for s in stuff)

    f.close ()
    os.startfile ("output.csv")

    </code>

    TJG
     
    Tim Golden, Sep 1, 2008
    #2
    1. Advertisements

  3. Mike P

    Guest

    Benchmark differences between 32 and 64 bit Python?

    Anyone have any benchmarks on the difference in performance between 32
    and 64 bit versions of Python for specific categories of operation, eg.
    math, file, string, etc. operations?

    My question is OS neutral so feel free to share your experience with
    either Windows or Linux OS's.

    Thank you,
    Malcolm
     
    , Sep 1, 2008
    #3
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jeff Tolman
    Replies:
    0
    Views:
    3,237
    Jeff Tolman
    Nov 4, 2003
  2. Darrel

    manually creating unique IDs

    Darrel, Oct 30, 2004, in forum: ASP .Net
    Replies:
    3
    Views:
    589
    Guadala Harry
    Oct 31, 2004
  3. SouthSpawn
    Replies:
    1
    Views:
    3,250
    Scott Allen
    Apr 22, 2005
  4. Dizzy Haze

    Namespaces and unique IDs

    Dizzy Haze, Sep 27, 2005, in forum: XML
    Replies:
    6
    Views:
    749
    Henry S. Thompson
    Sep 30, 2005
  5. Replies:
    2
    Views:
    939
    Beat Bolli
    Jun 21, 2005
  6. Replies:
    1
    Views:
    1,144
    Mufasa
    May 19, 2008
  7. Tim
    Replies:
    1
    Views:
    474
    Peter Otten
    Jul 5, 2010
  8. anoop
    Replies:
    1
    Views:
    239
    Dominick Baier
    Mar 19, 2007
Loading...