Regular expressions and the global interpreter lock

Discussion in 'Python' started by Duncan Grisby, Nov 18, 2005.

  1. Hi,

    I have encountered a problem with the re module. I have a
    multi-threaded program that does lots of regular expression searching,
    with some relatively complex regular expressions. Occasionally, events
    can conspire to mean that the re search takes minutes. That's bad
    enough in and of itself, but the real problem is that the re engine
    does not release the interpreter lock while it is running. All the
    other threads are therefore blocked for the entire time it takes to do
    the regular expression search.

    Is there any fundamental reason why the re module cannot release the
    interpreter lock, for at least some of the time it is running? The
    ideal situation for me would be if it could do most of its work with
    the lock released, since the software is running on a multi processor
    machine that could productively do other work while the re is being
    processed. Failing that, could it at least periodically release the
    lock to give other threads a chance to run?

    A quick look at the code in _sre.c suggests that for most of the time,
    no Python objects are being manipulated, so the interpreter lock could
    be released. Has anyone tried to do that?

    Thanks,

    Duncan.

    --
    -- Duncan Grisby --
    -- --
    -- http://www.grisby.org --
     
    Duncan Grisby, Nov 18, 2005
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Aahz
    Replies:
    0
    Views:
    966
  2. Fuzzyman
    Replies:
    3
    Views:
    532
    Andrew MacIntyre
    Dec 5, 2003
  3. Tomas Christiansen

    Global Interpreter Lock

    Tomas Christiansen, Sep 24, 2004, in forum: Python
    Replies:
    3
    Views:
    324
    Michael Hoffman
    Sep 24, 2004
  4. Paul Rubin

    global interpreter lock

    Paul Rubin, Aug 19, 2005, in forum: Python
    Replies:
    59
    Views:
    1,328
    Michael Sparks
    Sep 15, 2005
  5. Noman Shapiro
    Replies:
    0
    Views:
    258
    Noman Shapiro
    Jul 17, 2013
Loading...

Share This Page