Embedding Python in Python

Discussion in 'Python' started by Darryl, Oct 8, 2003.

  1. Darryl

    Darryl Guest

    Hi,

    I've been programming in Python for a couple of weeks now and am quite
    impressed with the language - it was very easy to learn (it actually
    reminds me a lot of programming in BASIC many many years ago - it's
    very fast to hack together a couple lines of code in interactive
    mode). I'm throwing together a simple little game (sort of a MUD-type
    thing) as my first 'big' project, and I thought it would be cool to be
    able to script the game in Python.

    Basically, I want to be able to type scripts in from within the game
    and attach them to objects, monsters, rooms, whatever. Now, I -could-
    use exec to run the code, but in an imaginary world where I intended
    to actually release this game to the public, I'd need to make sure
    that the user won't break anything, either accidentally or
    maliciously. So what I'd like to be able to do is essentially sandbox
    the code: expose a set of global
    variables/functions/modules/classes/whatever to the script for
    interaction with the game world, and make sure that the script doesn't
    do silly things like file access. A little research has indicated that
    previous versions of Python had sandboxing capabilities but these have
    broken in new releases and don't look like they're getting fixed any
    time soon. So is there any (safe) way to embed Python within itself
    for application scripting?

    Thanks,
    - Darryl Shpak
    Darryl, Oct 8, 2003
    #1
    1. Advertising

  2. Darryl

    JCM Guest

    Darryl <> wrote:
    ....
    > Basically, I want to be able to type scripts in from within the game
    > and attach them to objects, monsters, rooms, whatever. Now, I -could-
    > use exec to run the code, but in an imaginary world where I intended
    > to actually release this game to the public, I'd need to make sure
    > that the user won't break anything, either accidentally or
    > maliciously.


    I worked on a similar project. What we did was parse the users' code
    (take a look at the compiler module) and statically analyze it for
    anything "dangerous". We took a very conservative approach--if I
    remember correctly, we disallowed import statements, exec statements,
    and a bunch of identifiers including eval, globals, and anything of
    the form __ident__ (and maybe some other things). Then we coded up
    some machinery to let users safely import their own modules and a
    subset of the Python library modules. We also provided special
    classes from which they could inherit which didn't require defining or
    calling any functions named like __this__.

    We could have done some dataflow analysis to be smarter about letting
    users use some identifiers which we'd prohibited, but we never got
    that far.
    JCM, Oct 8, 2003
    #2
    1. Advertising

  3. Darryl

    Darryl Guest

    JCM <> wrote in message news:<bm1mt6$maa$>...
    > (...) What we did was parse the users' code
    > (take a look at the compiler module) and statically analyze it for
    > anything "dangerous".


    Ugh...I had considered this approach but was hoping I didn't have to
    do it :) I've only been hacking Python for a couple weeks, so I don't
    know all the ins and outs of what I'd need to watch for...in my
    experience, most languages seem to have obscure ways of doing things
    that you wouldn't think of normally, and Python seems no exception.
    From a more pragmatic perspective though, this is just a toy project,
    so it's not too important if I miss something.

    > (...) anything of the form __ident__


    Now that's an interesting idea...from what I've seen of Python, that
    should catch most of the really obscure ways of doing things. I
    haven't looked, but I can only assume that Python has regexp matching
    in it somewhere, so some of the simple checks should be only a few
    lines of code. A blacklist of keywords seems like a good first start
    (it's a fairly restrictive context, so even common things like def and
    lambda can probably be blocked).

    I also just tested and noticed that syntax errors seem to be thrown as
    exceptions (rather than causing the interpreter to gasp and die), so I
    shouldn't have to worry about a badly-written script crashing the
    whole app.

    Thanks for the input,
    - Darryl
    Darryl, Oct 9, 2003
    #3
  4. In article <>,
    Darryl <> wrote:
    >JCM <> wrote in message
    >news:<bm1mt6$maa$>...
    >> (...) What we did was parse the users' code
    >> (take a look at the compiler module) and statically analyze it for
    >> anything "dangerous".

    >
    >Ugh...I had considered this approach but was hoping I didn't have to
    >do it :) I've only been hacking Python for a couple weeks, so I don't
    >know all the ins and outs of what I'd need to watch for...in my
    >experience, most languages seem to have obscure ways of doing things
    >that you wouldn't think of normally, and Python seems no exception.
    >From a more pragmatic perspective though, this is just a toy project,
    >so it's not too important if I miss something.
    >
    >> (...) anything of the form __ident__

    >
    >Now that's an interesting idea...from what I've seen of Python, that
    >should catch most of the really obscure ways of doing things. I
    >haven't looked, but I can only assume that Python has regexp matching
    >in it somewhere, so some of the simple checks should be only a few
    >lines of code. A blacklist of keywords seems like a good first start
    >(it's a fairly restrictive context, so even common things like def and
    >lambda can probably be blocked).
    >
    >I also just tested and noticed that syntax errors seem to be thrown as
    >exceptions (rather than causing the interpreter to gasp and die), so I
    >shouldn't have to worry about a badly-written script crashing the
    >whole app.

    .
    .
    .
    I'm surprised--astounded, in fact--that those more expert
    with Python than I haven't already jumped in to correct
    errors that seem to be arising in this thread.

    Yes, we all count on the Python interpreter to toss excep-
    tions when it's unhappy, *not* "to gasp and die". There
    are a few situations it can't handle, but only few.

    There's a lot to say on the subject of interpretation of
    code supplied by users. In particular, the three Python
    projects
    rexec
    Bastion
    RestrictedPython
    all address this requirement on a technical level. If
    your goal is a useful working application, I strongly
    urge you to read up on these. Correct construction on
    your own of a "blacklist" is ... difficult.
    --

    Cameron Laird <>
    Business: http://www.Phaseit.net
    Cameron Laird, Oct 10, 2003
    #4
  5. Darryl

    JCM Guest

    Cameron Laird <> wrote:
    ....
    > I'm surprised--astounded, in fact--that those more expert
    > with Python than I haven't already jumped in to correct
    > errors that seem to be arising in this thread.


    What errors?

    > Yes, we all count on the Python interpreter to toss excep-
    > tions when it's unhappy, *not* "to gasp and die". There
    > are a few situations it can't handle, but only few.


    The original question was about ensuring users write code which is,
    by some definition, safe. For example, you don't want users calling
    "os.system('rm -rf /')". You also don't want them overriding
    builtins.

    > There's a lot to say on the subject of interpretation of
    > code supplied by users. In particular, the three Python
    > projects
    > rexec
    > Bastion
    > RestrictedPython
    > all address this requirement on a technical level. If
    > your goal is a useful working application, I strongly
    > urge you to read up on these. Correct construction on
    > your own of a "blacklist" is ... difficult.


    I'm under the impression that the rexec and Bastion modules do not
    work with Python 2.3+. I'm not familiar with RestrictedPython.
    JCM, Oct 10, 2003
    #5
  6. JCM wrote:
    ...
    > I'm under the impression that the rexec and Bastion modules do not
    > work with Python 2.3+.


    Right -- alas -- they don't give 100% security, so they were disabled
    (also in 2.2.3 -- they had just the same exploitable weaknesses in
    earlier 2.2 releases).

    > I'm not familiar with RestrictedPython.


    It's part of Zope. I have no experience of how well or badly it works.


    Alex
    Alex Martelli, Oct 10, 2003
    #6
  7. Darryl

    Darryl Guest

    (Cameron Laird) wrote in message news:<>...
    > There's a lot to say on the subject of interpretation of
    > code supplied by users. In particular, the three Python
    > projects
    > rexec
    > Bastion
    > RestrictedPython
    > all address this requirement on a technical level. If
    > your goal is a useful working application, I strongly
    > urge you to read up on these. Correct construction on
    > your own of a "blacklist" is ... difficult.


    Given my current level of Python knowledge, I don't consider myself
    capable of writing such a blacklist - I might be able to pull it off
    in other languages, but not Python. I've looked at rexec and Bastion,
    and (as another poster mentioned) got the impression they were
    defunct. I've also been pointed at Zope, but haven't had time to
    investigate it.

    Since this is a casual project that I'm not spending too much time on,
    I'll probably ignore the issue for now. Since I often see Python
    promoted as an application scripting language, I had hoped that it had
    built-in restrictions of this nature, akin to a Java applet's sandbox.
    I'm now getting the impression that if I embedded Python as the
    scripting language in any app, it would be similar to using VBA - an
    unrestricted environment that gives the script the power to do
    whatever it wants.

    I think I'll go ahead with the project with only a few simple
    restrictions right now (i.e. no 'import') and only allow trusted users
    to write scripts. I'll also run the application in a chroot jail
    (OS-specific, I know) to limit the amount of damage that can be done.

    Thanks for all the advice,
    - Darryl
    Darryl, Oct 14, 2003
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Phil Frost

    Embedding Python in Python

    Phil Frost, Aug 18, 2004, in forum: Python
    Replies:
    23
    Views:
    750
    Paul Rubin
    Aug 19, 2004
  2. Maurice LING

    embedding python in python

    Maurice LING, Sep 29, 2004, in forum: Python
    Replies:
    8
    Views:
    361
    Jeff Shannon
    Oct 1, 2004
  3. Kakacek
    Replies:
    1
    Views:
    370
    Kakacek
    Jan 12, 2006
  4. abhinav
    Replies:
    1
    Views:
    431
    Armin Steinhoff
    Feb 19, 2006
  5. Shankar
    Replies:
    1
    Views:
    696
    Serge Orlov
    May 5, 2006
Loading...

Share This Page