File to dict

[david]

(e-mail address removed) ha scritto:

Hello everyone,

I have written this small utility function for transforming legacy file
to Python dict:


def lookupdmo(domain):
lines = open('/etc/virtual/domainowners','r').readlines() lines
= [ [y.lstrip().rstrip() for y in x.split(':')] for x in
lines]
lines = [ x for x in lines if len(x) == 2 ] d = dict()
for line in lines:
d[line[0]]=line[1]
return d[domain]

cache = None

def lookup( domain ):
if not cache:
cache = dict( [map( lambda x: x.strip(), x.split(':')) for x in
open('/etc/virtual/domainowners','r').readlines()])
return cache.get(domain)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in lookup
UnboundLocalError: local variable 'cache' referenced before assignment

You miss the:
def lookup(domain):
global cache
...

bye

 

[Glauco]

david ha scritto:

(e-mail address removed) ha scritto:

Hello everyone,

I have written this small utility function for transforming legacy file
to Python dict:


def lookupdmo(domain):
lines = open('/etc/virtual/domainowners','r').readlines() lines
= [ [y.lstrip().rstrip() for y in x.split(':')] for x in
lines]
lines = [ x for x in lines if len(x) == 2 ] d = dict()
for line in lines:
d[line[0]]=line[1]
return d[domain]

cache = None

def lookup( domain ):
if not cache:
cache = dict( [map( lambda x: x.strip(), x.split(':')) for x in
open('/etc/virtual/domainowners','r').readlines()])
return cache.get(domain)

lookup(1)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in lookup
UnboundLocalError: local variable 'cache' referenced before assignment

You miss the:
def lookup(domain):
global cache
...

bye

yezzz!

you can use global or static


Gla

 

[mrkafk]

cache = None

def lookup( domain ):
if not cache:
cache = dict( [map( lambda x: x.strip(), x.split(':')) for x in
open('/etc/virtual/domainowners','r').readlines()])
return cache.get(domain)

Neat solution! It just needs small correction for empty or badly
formed lines:

dict([map( lambda x: x.strip(), x.split(':')) for x in open('/etc/
virtual/domainowners','r') if ':' in x])

 

[J. Clifford Dyer]

Hello everyone,

I have written this small utility function for transforming legacy
file to Python dict:


def lookupdmo(domain):
lines = open('/etc/virtual/domainowners','r').readlines()
lines = [ [y.lstrip().rstrip() for y in x.split(':')] for x in
lines]
lines = [ x for x in lines if len(x) == 2 ]
d = dict()
for line in lines:
d[line[0]]=line[1]
return d[domain]

The /etc/virtual/domainowners file contains double-colon separated
entries:
domain1.tld: owner1
domain2.tld: own2
domain3.another: somebody
...

Now, the above lookupdmo function works. However, it's rather tedious
to transform files into dicts this way and I have quite a lot of such
files to transform (like custom 'passwd' files for virtual email
accounts etc).

Don't do more lookup than you have to to get the answer you need.

(untested code)

class DictFromFile(object):
def __init__(self, filename):
self.f = open(filename)
self.d = dict()

def __getitem__(self, key):
while key not in self.d:
try:
k, v = self._retrieve_next()
self.d[k] = v
if k == key:
break
except StopIteration:
break
return self.d[key]

def _retrieve_next(self):
line = self.f.next()
k, v = line.split(':',1)
return k.strip(), v.strip()


This will act like a dictionary, but only look as far into the file as
it needs to. If you've got a 10,000 line file, and all your results
come out of the first 10 lines, this will save you some processing.
There's more operator overloading you could do, of course.

Cheers,
Cliff

 

[Jeremy C B Nicoll]

For the first one you are parsing the entire file everytime you want
to lookup just one domain...

Is the file sorted? If so wouldn't it be easier either to read the whole
thing and then binary-chop search it, or if the file is vast to use seek
creatively to binary chop it but only read particular records from disk?

(One could chop by using seek to locate to a particular byte location in the
file then read to end of that record then read the next complete record.)

If the file isn't sorted, then .... why not? Also if the file is vast
surely there's some or a lot of point in breaking it up into a group of
smaller files?