CSV reader and unique ids

Discussion in 'Python' started by Mike P, Sep 1, 2008.

  1. Mike P

    Mike P Guest

    Hi All,

    I'm trying to use the CSV module to read in some data and then use a
    hashable method (as there are millions of records) to find unique ids
    and push these out to another file,

    can anyone advise? Below is the code so far


    fin = open(CSV_INPUT, "rb")
    fout = open(CSV_OUTPUT, "wb")
    reader = csv.reader(fin, delimiter=chr(254))
    writer = csv.writer(fout)

    headerList = reader.next()
    UID = {}

    #For help
    #print headerList
    # ['Time', 'User-ID', 'IP']

    try:
    for row in reader[1]:
    UID[row] = 1
    else:
    List= UID.keys()
    writer.writerows(List)
    fin.close()
    fout.close()

    Mike
    Mike P, Sep 1, 2008
    #1
    1. Advertising

  2. Mike P

    Tim Golden Guest

    Mike P wrote:
    > I'm trying to use the CSV module to read in some data and then use a
    > hashable method (as there are millions of records) to find unique ids
    > and push these out to another file,


    You could either zip with a counter or use the uuid module,
    depending on just how unique you want your ids to be.

    <code>
    import os, sys
    import csv
    import itertools
    import uuid

    stuff = "the quick brown fox jumps over the lazy dog".split ()

    f = open ("output.csv", "wb")
    writer = csv.writer (f)

    #
    # Style 1 - numeric counter
    #
    writer.writerows (zip (itertools.count (), stuff))

    #
    # Style 2 - uuid
    #
    writer.writerows ((uuid.uuid1 (), s) for s in stuff)

    f.close ()
    os.startfile ("output.csv")

    </code>

    TJG
    Tim Golden, Sep 1, 2008
    #2
    1. Advertising

  3. Mike P

    Guest

    Benchmark differences between 32 and 64 bit Python?

    Anyone have any benchmarks on the difference in performance between 32
    and 64 bit versions of Python for specific categories of operation, eg.
    math, file, string, etc. operations?

    My question is OS neutral so feel free to share your experience with
    either Windows or Linux OS's.

    Thank you,
    Malcolm
    , Sep 1, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dizzy Haze

    Namespaces and unique IDs

    Dizzy Haze, Sep 27, 2005, in forum: XML
    Replies:
    6
    Views:
    538
    Henry S. Thompson
    Sep 30, 2005
  2. Replies:
    2
    Views:
    690
    Beat Bolli
    Jun 21, 2005
  3. Replies:
    1
    Views:
    838
    Mufasa
    May 19, 2008
  4. Tim
    Replies:
    1
    Views:
    309
    Peter Otten
    Jul 5, 2010
  5. anoop
    Replies:
    1
    Views:
    175
    Dominick Baier
    Mar 19, 2007
Loading...

Share This Page