Best way to create a Dictionary from file of Key-Value pairs?

[Equis Uno]

Hi,

Assume I'm given a 100k file full of key-value pairs:

date,value
26-Feb-04,36.47
25-Feb-04,36.43
24-Feb-04,36.30
23-Feb-04,37.00
20-Feb-04,37.00
19-Feb-04,37.87

What is the best way to copy this data into a Dictionary object?

-moi

 

[Bob Ippolito]

Hi,

Assume I'm given a 100k file full of key-value pairs:

date,value
26-Feb-04,36.47
25-Feb-04,36.43
24-Feb-04,36.30
23-Feb-04,37.00
20-Feb-04,37.00
19-Feb-04,37.87

What is the best way to copy this data into a Dictionary object?

Getting that data into a dictionary is the easy part.. deciding how
you want to use it is much harder.

Python 2.3 comes with a module designed specifically to read files that
look like that, so here's a start:

# none of this code is tested
import csv
myfile = file('myfilename')
myfile.readline() # skip "date,value" line
datevaluemap = dict(csv.reader(myfile))

But your dates and values would end up as strings.. so you might want
to do something like
datevaluemap = dict([(key, float(value)) for (key, value) in
csv.reader(myfile)])

Which would get your values in as floats, not strings. The dates would
still be strings though, and you might want to use a datetime.date
object for that. I don't want to demonstrate how to convert those too,
since it's longer, uglier, and I don't know strftime format codes off
the top of my head.. but you can look into datetime.date and
time.strptime.

-bob

 

[Sean Ross]

Hi,

Assume I'm given a 100k file full of key-value pairs:

date,value
26-Feb-04,36.47
25-Feb-04,36.43
24-Feb-04,36.30
23-Feb-04,37.00
20-Feb-04,37.00
19-Feb-04,37.87

What is the best way to copy this data into a Dictionary object?

-moi

Here's one way:

d = dict([l.strip().split(',') for l in open(fname)])

This opens the file, reads one line at a time, trims the beginning and end
of the line, splits the line at the comma delimiter - your key-value pairs -
and builds a list of these key-value pairs which is then used by dict() to
make a new dictionary. This is not really a robust solution, but you get the
idea. Your solution may require more processing if you want the values to be
floats (for example) or if there might be blank lines or extra white space
in the data, or whatever other issues you may want to guard against. Plus,
you may want to move the open() operation outside of the list comprehension
so you can check/handle possible errors (like file not found, for instance),
as well as close the file, that sort of thing ...

HTH,
Sean

 

[vincent wehren]

| Hi,
|
| Assume I'm given a 100k file full of key-value pairs:
|
| date,value
| 26-Feb-04,36.47
| 25-Feb-04,36.43
| 24-Feb-04,36.30
| 23-Feb-04,37.00
| 20-Feb-04,37.00
| 19-Feb-04,37.87
|
| What is the best way to copy this data into a Dictionary object?

If it suffices ro generate a "string-to-string mapping", you could do:
k, v = line.rstrip().split(",")
d[k] = v

This will give you:

{'20-Feb-04': '37.00', '19-Feb-04': '37.87', '26-Feb-04': '36.47',
'23-Feb-04': '37.00', '25-Feb-04': '36.43', '24-Feb-04': '36.30'}

HTH,
Vincent Wehren



|
| -moi

 

[Terry Reedy]

Hi,

Assume I'm given a 100k file full of key-value pairs:

date,value
26-Feb-04,36.47
25-Feb-04,36.43
24-Feb-04,36.30
23-Feb-04,37.00
20-Feb-04,37.00
19-Feb-04,37.87

What is the best way to copy this data into a Dictionary object?

If you really have one value for each of numerous sequential dates, you
might be better off putting values in list that is attribute of class also
with start and stop date attributes.

TJR

 

[Skip Montanaro]

Equis> Assume I'm given a 100k file full of key-value pairs:

Equis> date,value
Equis> 26-Feb-04,36.47
Equis> 25-Feb-04,36.43
Equis> 24-Feb-04,36.30
Equis> 23-Feb-04,37.00
Equis> 20-Feb-04,37.00
Equis> 19-Feb-04,37.87

Equis> What is the best way to copy this data into a Dictionary object?

Maybe overkill, but you have a csv file. I'd use the csv module (new in
2.3):

import csv

d = {}
rdr = csv.DictReader(file("source-file"))
for row in rdr:
d[row["date"]] = row["value"]

Skip

 

[Equis Uno]

I must say, CLP rocks.

I added some syntax to convert the key from string to a time data
type:

import time
# Convert the values to float and the keys to time
myfile = file('csvData.txt')
myfile.readline() # skip "date,value" line
datevaluemap = dict([( time.strptime(key, '%d-%b-%y'), float(value))
for (key, value) in csv.reader(myfile)])
print datevaluemap

output:

{(2004, 2, 24, 0, 0, 0, 1, 55, -1): 36.299999999999997, (2004, 2, 19,
0, 0, 0, 3, 50, -1): 37.869999999999997, (2004, 2, 26, 0, 0, 0, 3, 57,
-1): 36.469999999999999, (2004, 2, 20, 0, 0, 0, 4, 51, -1): 37.0,
(2004, 2, 23, 0, 0, 0, 0, 54, -1): 37.0, (2004, 2, 25, 0, 0, 0, 2, 56,
-1): 36.43}

I'm way happy!

-moi