RE: Iterating generator from C (PostgreSQL's pl/python RETUNSETOF/RECORD iterator support broken on

Discussion in 'Python' started by Hannu Krosing, May 15, 2006.

  1. Sorry for cross-posting, but this IS a cross-platform issue.

    Christian Tismer tismer at stackless.com wrote:
    > Sven Suursoho wrote:
    >
    > >>> Is there any way to rewrite following program to handle returned
    > >>> generator without hitting this bug?

    >
    > The only way I can think of getting around this is to make
    > sure that there is always a running python frame.
    > This would be possible if you can control how the
    > extension is called.


    What would be the easiest way to hold a "always running" python frame ?

    The actual calls to iterator come through a pl/python framework, which can hold some
    state - like some inter-call dictionaries - so keeping also a simple outer frame
    there may be possible.

    What danger (apart from general uglyness) may lurk there in keeping this frame ?

    > > Unfortunately, this is not an option because I can't control used
    > > environment: I'm trying to improve PostgreSQL's stored procedure
    > > language PL/Python and this software can be used everywhere.

    >
    > Then there is no other way than to report the bug, get the
    > fix back-ported and nagging the PL/Python maintainers
    > to update things after the fix.


    Unfortunately there is no 'after the fix', as the fix must happen outside
    postgresql/plpython development and should also run on oldish systems.

    The reason we want to get some workaround for that bug is the need
    to overcome resistance from core postgreSQL developers to inclusion of our
    plpython enchancements to postgreSQLs bundled plpython due to one specific use
    of our generic enchancement (using a built-in generator, usually a function with yield)
    on buggy RedHat's bundled plpython.so causing crashes.

    Also, PL/Python has been in minimal maintenance mode for many years, with
    basically only immediate bugfixing going on.

    We at Skype (that is currently Sven Suursoho) are the first ones doing
    active development on it after Andrew Bosma dropped development many years
    ago once he got just the very basic stuff working.

    > Also a test should be added which is probably missing since a while.


    Test where ? In python.so build process, so that RedHat will spot it and
    fix their RPM builds ?

    As for testing in actual pl/python build environment, we had objections from
    leading postgresql Tom Lane that even if we do test it at build time,
    a determined DBA may substitute a buggy python.so later and still crash her DB instance.

    Do you have any ideas, how to test for buggy asserts in python runtime
    environment without actually triggering the assert ?

    Then we could introduce some (uglish) special handling for generators in
    pl/pythons iterator.

    > I'd put a warning somewhere that generators are broken in
    > debug mode, file an issue as well, but avoid trying to hack
    > around this. It would make the bug even more resistent :)


    We have been trying to advocate such approach, but so far with modest results :(

    --
    ----------------
    Hannu Krosing
    Database Architect
    Skype Technologies OÜ
    Akadeemia tee 21 F, Tallinn, 12618, Estonia

    Skype me: callto:hkrosing
    Get Skype for free: http://www.skype.com
     
    Hannu Krosing, May 15, 2006
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. carl
    Replies:
    5
    Views:
    2,424
    James Kanze
    Nov 25, 2009
  2. Meszaros, Stacy

    Inserting record into postgresql database

    Meszaros, Stacy, Mar 11, 2011, in forum: Python
    Replies:
    0
    Views:
    196
    Meszaros, Stacy
    Mar 11, 2011
  3. hajer
    Replies:
    0
    Views:
    293
    hajer
    Oct 20, 2011
  4. ded
    Replies:
    3
    Views:
    178
  5. Francis Hwang
    Replies:
    0
    Views:
    139
    Francis Hwang
    Mar 22, 2006
Loading...

Share This Page