correct usage of a generator?

[Tim]

Hi, I need to generate a list of file names that increment, like this:
fname1
fname2
fname3 and so on.

I don't know how many I'll need until runtime so I figure a generator is called for.

def fname_gen(stem):
i = 0
while True:
i = i+1
yield '%s%d' % (stem,i)

blarg = fname_gen('blarg')
boo = fname_gen('boo')

n = 3
for w in range(0,n):
in_name = blarg.next()
out_name = boo.next()


This works, but is it the 'normal' way to accomplish the task when you don't know 'n' until run-time?
thanks,
--Tim

 

[Steven D'Aprano]

Hi, I need to generate a list of file names that increment, like this:
fname1
fname2
fname3 and so on.

I don't know how many I'll need until runtime so I figure a generator is
called for.

def fname_gen(stem):
i = 0
while True:
i = i+1
yield '%s%d' % (stem,i)

blarg = fname_gen('blarg')
boo = fname_gen('boo')

n = 3
for w in range(0,n):
in_name = blarg.next()
out_name = boo.next()


This works, but is it the 'normal' way to accomplish the task when you
don't know 'n' until run-time? thanks,

Looks perfectly fine to me. In Python 2.6 onwards, I'd write next(blarg)
rather than blarg.next(), and similarly for boo. In Python 3.x, you have
to use next(blarg).

If I were to nit-pick, I'd write the last part as:

for _ in range(3):
in_name = blarg.next()
out_name = boo.next()

where _ is the traditional "don't care" name in Python.

 

[Peter Otten]

Hi, I need to generate a list of file names that increment, like this:
fname1
fname2
fname3 and so on.

I don't know how many I'll need until runtime so I figure a generator is
called for.

def fname_gen(stem):
i = 0
while True:
i = i+1
yield '%s%d' % (stem,i)

blarg = fname_gen('blarg')
boo = fname_gen('boo')

n = 3
for w in range(0,n):
in_name = blarg.next()
out_name = boo.next()


This works, but is it the 'normal' way to accomplish the task when you
don't know 'n' until run-time? thanks,

As a fan of the itertools module I would spell it (untested)

from itertools import count, islice, izip

def fname_gen(stem):
return ("%s%d" % (stem, i) for i in count(1))
# ...
for in_name, out_name in islice(izip(blarg, boo), n):
#...

but your way is perfectly OK.

 

[Neil Cerutti]

I don't see anything wrong with that. I've written similar code
in terms of itertools.count, myself. If I may make a suggestion,
it would be nice to zero-pad the number to achieve
lexicographical ordering of filenames. However, if you really
have no idea of the magnitude you might need that won't work. The
following assumes you'll need less than 1000.

counter = itertools.count()
....
with open("%03d" % counter.next(), "w") as the_next_file:
...

My Python < 3.0 is rusty, so sorry if I messed that up.

 

[Mel Wilson]

Hi, I need to generate a list of file names that increment, like this:
fname1
fname2
fname3 and so on.

I don't know how many I'll need until runtime so I figure a generator is
called for.

def fname_gen(stem):
i = 0
while True:
i = i+1
yield '%s%d' % (stem,i)

blarg = fname_gen('blarg')
boo = fname_gen('boo')

n = 3
for w in range(0,n):
in_name = blarg.next()
out_name = boo.next()


This works, but is it the 'normal' way to accomplish the task when you
don't know 'n' until run-time? thanks,

It's kind of overkill in the toy demo example, but if the main loop is
a little more abstract, e.g.

for task in task_supplier():
in_name = blarg.next()
out_name = boo.next()
handle_task (task, in_name, out_name)

then it's obviously a good thing.

One quibble (that Peter Otten also suggested): if your business rules
expect that files blarg25 and boo25 (for example) work together, then
you'd be better off generating them together as a pair in a single
generator call. As it is, there's a chance of the blarg and boo
generators getting out of step and supplying mismatched names.

Mel.

 

[Tim]

thanks everyone. I thought blarg.next() looked a little strange--I'm just learning generators now and I'm glad too see that next(blarg) is the way to go.

The example really was just a toy or I would use an iterator. I do need the two (well, the several) generators to not be coupled together so they increment independently. I'll keep the zero-padding advice in mind--I didn't think of that one.

what a great group this is.
thanks again!
--Tim