Strange re behavior: normal?

Discussion in 'Python' started by Robin Munn, Aug 14, 2003.

  1. Robin Munn

    Robin Munn Guest

    How is re.split supposed to work? This wasn't at all what I expected:

    [rmunn@localhost ~]$ python
    Python 2.2.2 (#1, Jan 12 2003, 12:07:20)
    [GCC 3.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import re
    >>> re.split(r'\W+', 'a b c d')

    ['a', 'b', 'c', 'd']
    >>> # Expected result.

    ....
    >>> re.split(r'\b', 'a b c d')

    ['a b c d']
    >>> # Huh?


    Since \b matches the empty string, but only at the beginning and end of
    a word, I would have expected re.split(r'\b', 'a b c d' to produce
    either:

    ['', 'a', ' ', 'b', ' ', 'c', ' ', 'd', '']

    or:

    ['a', ' ', 'b', ' ', 'c', ' ', 'd']

    But I didn't expect that re.split(r'\b', 'a b c d') would yield no splits
    whatsoever. The module doc says "split(pattern, string[, maxsplit = 0]):
    split string by the occurrences of pattern". re.findall() seems to think
    that \b occurs eight times in 'a b c d':

    >>> re.findall(r'\b', 'a b c d')

    ['', '', '', '', '', '', '', '']

    So why doesn't re.split() think so? I'm puzzled.

    --
    Robin Munn <> | http://www.rmunn.com/ | PGP key 0x6AFB6838
    -----------------------------+-----------------------+----------------------
    "Remember, when it comes to commercial TV, the program is not the product.
    YOU are the product, and the advertiser is the customer." - Mark W. Schumann
     
    Robin Munn, Aug 14, 2003
    #1
    1. Advertising

  2. Robin Munn

    David C. Fox Guest

    Robin Munn wrote:
    > How is re.split supposed to work? This wasn't at all what I expected:
    >
    > [rmunn@localhost ~]$ python
    > Python 2.2.2 (#1, Jan 12 2003, 12:07:20)
    > [GCC 3.2] on linux2
    > Type "help", "copyright", "credits" or "license" for more information.
    >
    >>>>import re
    >>>>re.split(r'\W+', 'a b c d')

    >
    > ['a', 'b', 'c', 'd']
    >
    >>>># Expected result.

    >
    > ...
    >
    >>>>re.split(r'\b', 'a b c d')

    >
    > ['a b c d']
    >
    >>>># Huh?

    >
    >
    > Since \b matches the empty string, but only at the beginning and end of
    > a word, I would have expected re.split(r'\b', 'a b c d' to produce
    > either:
    >
    > ['', 'a', ' ', 'b', ' ', 'c', ' ', 'd', '']
    >
    > or:
    >
    > ['a', ' ', 'b', ' ', 'c', ' ', 'd']
    >
    > But I didn't expect that re.split(r'\b', 'a b c d') would yield no splits
    > whatsoever. The module doc says "split(pattern, string[, maxsplit = 0]):
    > split string by the occurrences of pattern". re.findall() seems to think
    > that \b occurs eight times in 'a b c d':
    >
    >
    >>>>re.findall(r'\b', 'a b c d')

    >
    > ['', '', '', '', '', '', '', '']
    >
    > So why doesn't re.split() think so? I'm puzzled.
    >


    It looks like re.split is not splitting on zero-length matches. I get
    similar behavior (in Python 2.2.2) if I try:

    re.split('x*', 'a b c d')

    or

    re.split('(?=c)', 'a b c d')

    I don't have the Python source handy to verify this hypothesis, though.

    If this is correct, it should at least be documented.

    David
     
    David C. Fox, Aug 15, 2003
    #2
    1. Advertising

  3. Robin Munn

    Mike Rovner Guest

    Fredrik Lundh wrote:
    > Mike Rovner wrote:
    >> IMHO that split behavior is a bug although technicaly it is not.
    >> (From re manual:
    >> "This module provides regular expression matching operations similar
    >> to those found in Perl.")

    >
    > is "split" really a matching operation?
    >
    > fact is, all methods have Python-specific behaviour. it's just the RE
    > language itself that's based on Perl.


    With all due respect to Python and not trying to bend it to any other
    language
    I believe it trys to do what user expects.

    string.split() splits on (clearly documented nonempty) substring.
    re.split() splits on RE.
    splut(r'\b',...) clearly means (at least for me) 'split on word boundary'
    It doesn't reject it (as in r'\b?') nor provide expected behavior.

    I understand why it does what it does, but don't agree with it.

    Regards,
    Mike
     
    Mike Rovner, Aug 15, 2003
    #3
  4. Mike Rovner wrote:

    > With all due respect to Python and not trying to bend it to any other
    > language I believe it trys to do what user expects.


    it does exactly what I expect it to do.

    </F>
     
    Fredrik Lundh, Aug 15, 2003
    #4
  5. Robin Munn

    John J. Lee Guest

    "Fredrik Lundh" <> writes:

    > Mike Rovner wrote:
    >
    > > With all due respect to Python and not trying to bend it to any other
    > > language I believe it trys to do what user expects.

    >
    > it does exactly what I expect it to do.
    >
    > </F>


    You don't count. ;-)


    John
     
    John J. Lee, Aug 15, 2003
    #5
  6. Robin Munn

    Robin Munn Guest

    Michael Janssen <-frankfurt.de> wrote:
    >
    > What's the good of splitting by boundaries? Someone else wanted this a
    > few days ago on tutor and I can't figure out a reason by now.


    Heh. I bet I know the name of the person who was asking about this on
    the tutor list. He's a friend of mine, and I've been helping him learn
    Python. He E-mailed me about trying to split on word boundaries with
    re.split(r'\b', 'some text'), and it was his E-mail that caused me to
    discover that splitting by boundaries didn't do what I expected.

    What's the good of it? As someone else pointed out, it allows you to
    fetch the words and the separating text, yielding:

    ['See', ' ', 'Spot', '. ', 'See', ' ', 'Spot', ' ', 'run', '.']

    which may be useful in certain English-language-parsing situations,
    since it would allow you to look "ahead" or "back" from a word to see
    what punctuation precedes or follows it.

    Anyway, the re.split behavior I described isn't particularly bothering
    me, but I do think it should be better documented. Time to submit a doc
    patch, methinks...

    --
    Robin Munn <> | http://www.rmunn.com/ | PGP key 0x6AFB6838
    -----------------------------+-----------------------+----------------------
    "Remember, when it comes to commercial TV, the program is not the product.
    YOU are the product, and the advertiser is the customer." - Mark W. Schumann
     
    Robin Munn, Aug 18, 2003
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Rusty Shackleford

    normal aspnet_wp.exe behavior?

    Rusty Shackleford, Oct 8, 2005, in forum: ASP .Net
    Replies:
    8
    Views:
    412
    Rusty Shackleford
    Oct 11, 2005
  2. baobaoba
    Replies:
    4
    Views:
    430
    Rob Williscroft
    Nov 19, 2003
  3. Aldric Giacomoni

    Ruby 1.8.6 - normal behavior ?

    Aldric Giacomoni, Dec 16, 2008, in forum: Ruby
    Replies:
    5
    Views:
    136
    Aldric Giacomoni
    Dec 18, 2008
  4. Walle Wallen
    Replies:
    3
    Views:
    91
    Walle Wallen
    Apr 28, 2010
  5. Doug Lerner
    Replies:
    5
    Views:
    103
    Thomas 'PointedEars' Lahn
    Jan 16, 2006
Loading...

Share This Page