How to iterate the input over a particular size?

[joy99]

Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
rest_words=string_to_word[9:]
len_rest_word=len(rest_words)
if len_rest_word>9:
remaining_words=rest_words[9:]


In this program, I am trying to extract first 9 words from an
indefinitely long string, until it reaches 0.
Am I writing it ok, or should I use while, or lambda?
If any one can suggest.

Hope you are enjoying a nice vacation of Merry Christmas. If any one
is bit free and may guide me up bit.

Wishing you a happy day ahead,
Best Regards,
Subhabrata.

 

[Shawn Milochik]

A couple of notes:

Your code is too packed together. That makes it hard to read. Put in some whitespace (and comments, where necessary), and put spaces around your equal signs (x = 23 instead of x=23).

That said, here's something I threw together:

#!/usr/bin/env python

def chop_list(words, size = 9):

"""
Take a list, returning values
from the end of the list, and
the original list minus those values.

>>> chop_list(['test', 'with', 'four', 'words'], 2)

(['test', 'with'], ['four', 'words'])

>>> chop_list(['test', 'with', 'four', 'words'], 3)

(['test'], ['with', 'four', 'words'])

>>> chop_list(['test', 'with', 'four', 'words'], 4)

([], ['test', 'with', 'four', 'words'])


"""

chopped = words[-size:]
old_to_return = len(words) - len(chopped)

return words[ld_to_return], chopped


if __name__ == '__main__':

import doctest
doctest.testmod()

sample_string = '''There are more than nine words here.
I know because I wrote them.'''

word_list = sample_string.split()

while word_list:

word_list, new = chop_list(word_list)
print "Pulled off:\t%s\n\tRemaining: %s" % (new, word_list)

 

[Benjamin Kaplan]

Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
            rest_words=string_to_word[9:]
            len_rest_word=len(rest_words)
            if len_rest_word>9:
                     remaining_words=rest_words[9:]


In this program, I am trying to extract first 9 words from an
indefinitely long string, until it reaches 0.
Am I writing it ok, or should I use while, or lambda?
If any one can suggest.

Hope you are enjoying a nice vacation of Merry Christmas. If any one
is bit free and may guide me up bit.

Wishing you a happy day ahead,
Best Regards,
Subhabrata.
--

You want the first 9 words? string_to_word[:9]
You want the last 9 words? string_to_word[-9:]

If you want the groups of words, use a loop- that's the only way to
get all of them for any length list.

 

[joy99]

Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
            rest_words=string_to_word[9:]
            len_rest_word=len(rest_words)
            if len_rest_word>9:
                     remaining_words=rest_words[9:]

In this program, I am trying to extract first 9 words from an
indefinitely long string, until it reaches 0.
Am I writing it ok, or should I use while, or lambda?
If any one can suggest.

Hope you are enjoying a nice vacation of Merry Christmas. If any one
is bit free and may guide me up bit.

Wishing you a happy day ahead,
Best Regards,
Subhabrata.
--

You want the first 9 words? string_to_word[:9]
You want the last 9 words? string_to_word[-9:]

If you want the groups of words, use a loop- that's the only way to
get all of them for any length list.

http://mail.python.org/mailman/listinfo/python-list- Hide quoted text -

- Show quoted text -- Hide quoted text -

- Show quoted text -

Dear Group,
Answers were good. But I am looking for a smarter solution like:

for i[:2] in list:
.....

etc. or by doing some looping over loop.
Do not worry I'll work out the answer.

Wishing you a happy day ahead,
Regards,
Subhabrata.

 

[Francesco Bochicchio]

Dear Group,
I am encountering a small question.
Suppose, I write the following code,
input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
            rest_words=string_to_word[9:]
            len_rest_word=len(rest_words)
            if len_rest_word>9:
                     remaining_words=rest_words[9:]
In this program, I am trying to extract first 9 words from an
indefinitely long string, until it reaches 0.
Am I writing it ok, or should I use while, or lambda?
If any one can suggest.
Hope you are enjoying a nice vacation of Merry Christmas. If any one
is bit free and may guide me up bit.
Wishing you a happy day ahead,
Best Regards,
Subhabrata.
--

You want the first 9 words? string_to_word[:9]
You want the last 9 words? string_to_word[-9:]

If you want the groups of words, use a loop- that's the only way to
get all of them for any length list.

- Show quoted text -- Hide quoted text -

- Show quoted text -

Dear Group,
Answers were good. But I am looking for a smarter solution like:

for i[:2] in list:
....

etc. or by doing some looping over loop.
Do not worry I'll work out the answer.

Wishing you a happy day ahead,
Regards,
Subhabrata.

Not sure I understood your question, but if you need just to plit a
big list in sublists of no more than 9 elements, then you can do
someting like:

def sublists(biglist, n ):
"Splits a big list in sublists of max n elements"
prev_idx = 0; res = []
for idx in xrange(n, len(biglist)+n, n ):
res.append( biglist[prev_idx:idx] )
prev_idx = idx
return res

I would not be surprised if somewhere in python standard library there
is something like this (possibly better), but
could not find anything.

Another solution could be this smarter-looking but less readeable one
liner:

sublists = [ big_list[ii+9)] for i in xrange( 0, len
(big_list)+9, 9) if i < len(big_list) ]


P.S : if your biglist is huge (but being typed in I don't think so)
then you better convert the "sublists" function in a
generator.


HTH

Ciao

 

[Peter Otten]

Not sure I understood your question, but if you need just to plit a
big list in sublists of no more than 9 elements, then you can do
someting like:

def sublists(biglist, n ):
"Splits a big list in sublists of max n elements"
prev_idx = 0; res = []
for idx in xrange(n, len(biglist)+n, n ):
res.append( biglist[prev_idx:idx] )
prev_idx = idx
return res

I would not be surprised if somewhere in python standard library there
is something like this (possibly better), but
could not find anything.

A lazy alternative:
.... items = iter(items)
.... for first in items:
.... chunk = [first]
.... chunk.extend(islice(items, n-1))
.... yield chunk
....[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

One with lazy chunks:
.... items = iter(items)
.... for first in items:
.... yield chain((first,), islice(items, n-1))
....

[list(chunk) for chunk in chunks(range(10), 3)]

[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

Prior art by Duncan Booth:
http://mail.python.org/pipermail/python-list/2007-November/517993.html

Peter

 

[Vlastimil Brom]

Dear Group,
Answers were good. But I am looking for a smarter solution like:

for i[:2] in list:
....

etc. or by doing some looping over loop.
Do not worry I'll work out the answer.

Wishing you a happy day ahead,
Regards,
Subhabrata.

Hi,
maybe just something like:

input_list = range(10)
n = 3
[input_list[i:i+n] for i in range(0, len(input_list), n)] [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

?
(possibly using xrange and generator expression instead of the list
comprehension)

vbr

 

[Lie Ryan]

Francesco Bochicchio wrote:
One with lazy chunks:
.... items = iter(items)
.... for first in items:
.... yield chain((first,), islice(items, n-1))
....

[list(chunk) for chunk in chunks(range(10), 3)]

[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

better:
import itertools
def chunks(items, n):
ctr = (x // n for x in itertools.count())
return itertools.groupby(items, lambda _: next(ctr))

input_list = range(10)
list(list(x[1]) for x in chunks(input_list, 3))
# [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

 

[joy99]

Dear Group,
Answers were good. But I am looking for a smarter solution like:

for i[:2] in list:
....

etc. or by doing some looping over loop.
Do not worry I'll work out the answer.

Wishing you a happy day ahead,
Regards,
Subhabrata.

Hi,
maybe just something like:

input_list = range(10)
n = 3
[input_list[i:i+n] for i in range(0, len(input_list), n)]

[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

?
(possibly using xrange and generator expression instead of the list
comprehension)

 vbr

vbr,
Just Brilliant.
I manipulated it bit but I got the key, but thanks to all for giving
me lots of solutions and lots of insights over Python. Everytime I
visit this room I learn so much. Thank you all.

Wishing a Great Day ahead,

Best Regards,
Subhabrata.

 

[Peter Otten]

Francesco Bochicchio wrote:
One with lazy chunks:

from itertools import chain, islice
def chunks(items, n):

.... items = iter(items)
.... for first in items:
.... yield chain((first,), islice(items, n-1))
....

[list(chunk) for chunk in chunks(range(10), 3)]

[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

better:
import itertools
def chunks(items, n):
ctr = (x // n for x in itertools.count())
return itertools.groupby(items, lambda _: next(ctr))

input_list = range(10)
list(list(x[1]) for x in chunks(input_list, 3))
# [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

I disagree. It may be of little practical relevance that this fails after
about sys.maxint items (Python 2.5) or at least slows down (Python 2.6) when
Python long integer arithmetic kicks in, but I still feel uneasy about it.
If I were to use groupby() I'd probably code the "counter" as

ctr = cycle([False]*n + [True]*n)

Peter

 

[John Posner]

Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
rest_words=string_to_word[9:]
len_rest_word=len(rest_words)
if len_rest_word>9:
remaining_words=rest_words[9:]

Here's an issue that has not, I think, been addressed in this thread. The
OP's problem is:

1. Start with an indefinitely long string.

2. Convert the string to a list, splitting on whitespace.

3. Repeatedly return subslices of the list, until the list is exhausted.

This thread has presented one-chunk-at-a-time (e.g. generator/itertools)
approaches to Step #3, but what about Step #2? I've looked in the Python
documentation, and I've done some Googling, but I haven't found a
generator version of the string function split(). Am I missing something?

Tx,
John

 

[Steven D'Aprano]

Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
rest_words=string_to_word[9:]
len_rest_word=len(rest_words)
if len_rest_word>9:
remaining_words=rest_words[9:]

Here's an issue that has not, I think, been addressed in this thread.
The OP's problem is:

1. Start with an indefinitely long string.

2. Convert the string to a list, splitting on whitespace.

3. Repeatedly return subslices of the list, until the list is exhausted.

This thread has presented one-chunk-at-a-time (e.g. generator/itertools)
approaches to Step #3, but what about Step #2? I've looked in the Python
documentation, and I've done some Googling, but I haven't found a
generator version of the string function split(). Am I missing
something?

"Indefinitely long" doesn't mean you can't use split.

But if you want a lazy splitter, here's a version which should do what
you want:


def lazy_split(text):
accumulator = []
for c in text:
if c in string.whitespace:
if accumulator:
yield ''.join(accumulator)
accumulator = []
else:
accumulator.append(c)
if accumulator:
yield ''.join(accumulator)


Other alternatives are to use a regex to find runs of whitespace
characters, then yield everything else; or to use the itertools.groupby
function.

 

[John Posner]

Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
rest_words=string_to_word[9:]
len_rest_word=len(rest_words)
if len_rest_word>9:
remaining_words=rest_words[9:]

Here's an issue that has not, I think, been addressed in this thread.
The OP's problem is:

1. Start with an indefinitely long string.

2. Convert the string to a list, splitting on whitespace.

3. Repeatedly return subslices of the list, until the list is exhausted.

This thread has presented one-chunk-at-a-time (e.g. generator/itertools)
approaches to Step #3, but what about Step #2? I've looked in the Python
documentation, and I've done some Googling, but I haven't found a
generator version of the string function split(). Am I missing
something?

"Indefinitely long" doesn't mean you can't use split.

But if you want a lazy splitter, here's a version which should do what
you want:


def lazy_split(text):
accumulator = []
for c in text:
if c in string.whitespace:
if accumulator:
yield ''.join(accumulator)
accumulator = []
else:
accumulator.append(c)
if accumulator:
yield ''.join(accumulator)

Thanks -- I coded a similar generator, too. Yours handles the corner cases
and end-of-string more elegantly, so I won't share mine.

Other alternatives are to use a regex to find runs of whitespace
characters, then yield everything else; or to use the itertools.groupby
function.

Yup, that approach occurred to me as I was falling asleep last night (OMG,
get a life, buddy!):

def groupby_split(text):
whitespace_grouper = itertools.groupby(
text,
lambda(c): c not in string.whitespace)
for boolval, group_iter in itertools.ifilter(
lambda pair: pair[0] == True,
whitespace_grouper):
yield "".join(group_iter)

Tx,
John

 

[John Posner]

Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
rest_words=string_to_word[9:]
len_rest_word=len(rest_words)
if len_rest_word>9:
remaining_words=rest_words[9:]

Here's an issue that has not, I think, been addressed in this thread.
The OP's problem is:

1. Start with an indefinitely long string.

2. Convert the string to a list, splitting on whitespace.

3. Repeatedly return subslices of the list, until the list is exhausted.

This thread has presented one-chunk-at-a-time (e.g. generator/itertools)
approaches to Step #3, but what about Step #2? I've looked in the Python
documentation, and I've done some Googling, but I haven't found a
generator version of the string function split(). Am I missing
something?

"Indefinitely long" doesn't mean you can't use split.

But if you want a lazy splitter, here's a version which should do what
you want:


def lazy_split(text):
accumulator = []
for c in text:
if c in string.whitespace:
if accumulator:
yield ''.join(accumulator)
accumulator = []
else:
accumulator.append(c)
if accumulator:
yield ''.join(accumulator)

Thanks -- I coded a similar generator, too. Yours handles the corner cases
and end-of-string more elegantly, so I won't share mine.

Other alternatives are to use a regex to find runs of whitespace
characters, then yield everything else; or to use the itertools.groupby
function.

Yup, that approach occurred to me as I was falling asleep last night (OMG,
get a life, buddy!):

def groupby_split(text):
whitespace_grouper = itertools.groupby(
text,
lambda(c): c not in string.whitespace)
for boolval, group_iter in itertools.ifilter(
lambda pair: pair[0] == True,
whitespace_grouper):
yield "".join(group_iter)

Tx,
John

 

[John Posner]

Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
rest_words=string_to_word[9:]
len_rest_word=len(rest_words)
if len_rest_word>9:
remaining_words=rest_words[9:]

Here's an issue that has not, I think, been addressed in this thread.
The OP's problem is:

1. Start with an indefinitely long string.

2. Convert the string to a list, splitting on whitespace.

3. Repeatedly return subslices of the list, until the list is exhausted.

This thread has presented one-chunk-at-a-time (e.g. generator/itertools)
approaches to Step #3, but what about Step #2? I've looked in the Python
documentation, and I've done some Googling, but I haven't found a
generator version of the string function split(). Am I missing
something?

"Indefinitely long" doesn't mean you can't use split.

But if you want a lazy splitter, here's a version which should do what
you want:


def lazy_split(text):
accumulator = []
for c in text:
if c in string.whitespace:
if accumulator:
yield ''.join(accumulator)
accumulator = []
else:
accumulator.append(c)
if accumulator:
yield ''.join(accumulator)

Thanks -- I coded a similar generator, too. Yours handles the corner cases
and end-of-string more elegantly, so I won't share mine.

Other alternatives are to use a regex to find runs of whitespace
characters, then yield everything else; or to use the itertools.groupby
function.

Yup, that approach occurred to me as I was falling asleep last night (OMG,
get a life, buddy!):

def groupby_split(text):
whitespace_grouper = itertools.groupby(
text,
lambda(c): c not in string.whitespace)
for boolval, group_iter in itertools.ifilter(
lambda pair: pair[0] == True,
whitespace_grouper):
yield "".join(group_iter)

Tx,
John

 

[Aahz]

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
rest_words=string_to_word[9:]
len_rest_word=len(rest_words)
if len_rest_word>9:
remaining_words=rest_words[9:]

input_string.split(None, 9) does something like what you want, but it
does require recopying the tail each iteration. Overall, if you have
enough RAM and you don't need to preserve input_string after splitting,
your approach probably is the one I'd use.