imap vs map

[mk]

Hello everyone,

I re-wrote more "slowly" an example at the end of
http://wordaligned.org/articles/essential-python-reading-list


This example finds anagrams in the text file.

====
from itertools import groupby, imap
from operator import itemgetter

from string import ascii_lowercase, ascii_uppercase, punctuation,
maketrans, translate

data = open(r"c:\temp\words.txt", "rt").read()

trtable = maketrans(ascii_uppercase, ascii_lowercase)

words = translate(data, trtable, deletions = punctuation)

words = list(set(words.split()))

sw = sorted(words, key=sorted)

gb = groupby(sw, sorted)

print map(list, imap(itemgetter(1), gb))
===

words.txt:
===
Word Aligned
three
space sensitive programming
Feed Logo tins
Essential Python post Reading List
stop course there
times isnt
capes
===

Now, when I execute above, it works:

[['capes', 'space'], ['aligned'], ['reading'], ['essential'],
['programming'], ['course'], ['feed'], ['word'], ['there', 'three'],
['sensitive'], ['times'], ['logo'], ['python'], ['list'], ['isnt',
'tins'], ['stop', 'post']]


However, when I change the last line to:

print map(list, map(itemgetter(1), gb))

It stops working:

[[], [], [], [], [], [], [], [], [], [], [], [], [], [], [], ['post']]

Why? I was under impression that the only difference between map and
imap is that imap returns iterator allowing to produce a list, while map
returns equivalent list?

Regards,
mk

 

[Arnaud Delobelle]

Hello everyone,

I re-wrote more "slowly" an example at the end of
http://wordaligned.org/articles/essential-python-reading-list


This example finds anagrams in the text file.

====
from itertools import groupby, imap
from operator import itemgetter

from string import ascii_lowercase, ascii_uppercase, punctuation,
maketrans, translate

data = open(r"c:\temp\words.txt", "rt").read()

trtable = maketrans(ascii_uppercase, ascii_lowercase)

words = translate(data, trtable, deletions = punctuation)

words = list(set(words.split()))

sw = sorted(words, key=sorted)

gb = groupby(sw, sorted)

print map(list, imap(itemgetter(1), gb))
===

words.txt:
===
Word Aligned
three
space sensitive programming
Feed Logo tins
Essential Python post Reading List
stop course there
times isnt
capes
===

Now, when I execute above, it works:

[['capes', 'space'], ['aligned'], ['reading'], ['essential'],
['programming'], ['course'], ['feed'], ['word'], ['there', 'three'],
['sensitive'], ['times'], ['logo'], ['python'], ['list'], ['isnt',
tins'], ['stop', 'post']]


However, when I change the last line to:

print map(list, map(itemgetter(1), gb))

It stops working:

[[], [], [], [], [], [], [], [], [], [], [], [], [], [], [], ['post']]

Why? I was under impression that the only difference between map and
imap is that imap returns iterator allowing to produce a list, while
map returns equivalent list?

Regards,
mk

From the itertools docs [1]:

The returned group is itself an iterator that shares the underlying
iterable with groupby(). Because the source is shared, when the
groupby() object is advanced, the previous group is no longer
visible. So, if that data is needed later, it should be stored as a
list:

(example follows)

So when you use:

print map(list, imap(itemgetter(1), gb))

each group is stored as a list *before* the groupby() object is
advanced, so that's ok.

However, when you use:

print map(list, map(itemgetter(1), gb))

the groupby() object is advanced to the end *before* each group is
stored as a list. According the the docs quoted above, this renders
each group no longer visible.

It is for the same reason that the following are different:
.... for i in range(5):
.... yield lambda: i
....

map(lambda f: f(), imap(lambda x: x, foo())) [0, 1, 2, 3, 4]
map(lambda f: f(), map(lambda x: x, foo()))

[4, 4, 4, 4, 4]

Although looking back at it, I don't know if this will help you

[1] http://docs.python.org/library/itertools.html#itertools.groupby