Embedding Python in Python

D

Darryl

Hi,

I've been programming in Python for a couple of weeks now and am quite
impressed with the language - it was very easy to learn (it actually
reminds me a lot of programming in BASIC many many years ago - it's
very fast to hack together a couple lines of code in interactive
mode). I'm throwing together a simple little game (sort of a MUD-type
thing) as my first 'big' project, and I thought it would be cool to be
able to script the game in Python.

Basically, I want to be able to type scripts in from within the game
and attach them to objects, monsters, rooms, whatever. Now, I -could-
use exec to run the code, but in an imaginary world where I intended
to actually release this game to the public, I'd need to make sure
that the user won't break anything, either accidentally or
maliciously. So what I'd like to be able to do is essentially sandbox
the code: expose a set of global
variables/functions/modules/classes/whatever to the script for
interaction with the game world, and make sure that the script doesn't
do silly things like file access. A little research has indicated that
previous versions of Python had sandboxing capabilities but these have
broken in new releases and don't look like they're getting fixed any
time soon. So is there any (safe) way to embed Python within itself
for application scripting?

Thanks,
- Darryl Shpak
 
J

JCM

Darryl said:
Basically, I want to be able to type scripts in from within the game
and attach them to objects, monsters, rooms, whatever. Now, I -could-
use exec to run the code, but in an imaginary world where I intended
to actually release this game to the public, I'd need to make sure
that the user won't break anything, either accidentally or
maliciously.

I worked on a similar project. What we did was parse the users' code
(take a look at the compiler module) and statically analyze it for
anything "dangerous". We took a very conservative approach--if I
remember correctly, we disallowed import statements, exec statements,
and a bunch of identifiers including eval, globals, and anything of
the form __ident__ (and maybe some other things). Then we coded up
some machinery to let users safely import their own modules and a
subset of the Python library modules. We also provided special
classes from which they could inherit which didn't require defining or
calling any functions named like __this__.

We could have done some dataflow analysis to be smarter about letting
users use some identifiers which we'd prohibited, but we never got
that far.
 
D

Darryl

JCM said:
(...) What we did was parse the users' code
(take a look at the compiler module) and statically analyze it for
anything "dangerous".

Ugh...I had considered this approach but was hoping I didn't have to
do it :) I've only been hacking Python for a couple weeks, so I don't
know all the ins and outs of what I'd need to watch for...in my
experience, most languages seem to have obscure ways of doing things
that you wouldn't think of normally, and Python seems no exception.
From a more pragmatic perspective though, this is just a toy project,
so it's not too important if I miss something.
(...) anything of the form __ident__

Now that's an interesting idea...from what I've seen of Python, that
should catch most of the really obscure ways of doing things. I
haven't looked, but I can only assume that Python has regexp matching
in it somewhere, so some of the simple checks should be only a few
lines of code. A blacklist of keywords seems like a good first start
(it's a fairly restrictive context, so even common things like def and
lambda can probably be blocked).

I also just tested and noticed that syntax errors seem to be thrown as
exceptions (rather than causing the interpreter to gasp and die), so I
shouldn't have to worry about a badly-written script crashing the
whole app.

Thanks for the input,
- Darryl
 
C

Cameron Laird

Ugh...I had considered this approach but was hoping I didn't have to
do it :) I've only been hacking Python for a couple weeks, so I don't
know all the ins and outs of what I'd need to watch for...in my
experience, most languages seem to have obscure ways of doing things
that you wouldn't think of normally, and Python seems no exception.
From a more pragmatic perspective though, this is just a toy project,
so it's not too important if I miss something.


Now that's an interesting idea...from what I've seen of Python, that
should catch most of the really obscure ways of doing things. I
haven't looked, but I can only assume that Python has regexp matching
in it somewhere, so some of the simple checks should be only a few
lines of code. A blacklist of keywords seems like a good first start
(it's a fairly restrictive context, so even common things like def and
lambda can probably be blocked).

I also just tested and noticed that syntax errors seem to be thrown as
exceptions (rather than causing the interpreter to gasp and die), so I
shouldn't have to worry about a badly-written script crashing the
whole app.
.
.
.
I'm surprised--astounded, in fact--that those more expert
with Python than I haven't already jumped in to correct
errors that seem to be arising in this thread.

Yes, we all count on the Python interpreter to toss excep-
tions when it's unhappy, *not* "to gasp and die". There
are a few situations it can't handle, but only few.

There's a lot to say on the subject of interpretation of
code supplied by users. In particular, the three Python
projects
rexec
Bastion
RestrictedPython
all address this requirement on a technical level. If
your goal is a useful working application, I strongly
urge you to read up on these. Correct construction on
your own of a "blacklist" is ... difficult.
 
J

JCM

Cameron Laird said:
I'm surprised--astounded, in fact--that those more expert
with Python than I haven't already jumped in to correct
errors that seem to be arising in this thread.

What errors?
Yes, we all count on the Python interpreter to toss excep-
tions when it's unhappy, *not* "to gasp and die". There
are a few situations it can't handle, but only few.

The original question was about ensuring users write code which is,
by some definition, safe. For example, you don't want users calling
"os.system('rm -rf /')". You also don't want them overriding
builtins.
There's a lot to say on the subject of interpretation of
code supplied by users. In particular, the three Python
projects
rexec
Bastion
RestrictedPython
all address this requirement on a technical level. If
your goal is a useful working application, I strongly
urge you to read up on these. Correct construction on
your own of a "blacklist" is ... difficult.

I'm under the impression that the rexec and Bastion modules do not
work with Python 2.3+. I'm not familiar with RestrictedPython.
 
A

Alex Martelli

JCM wrote:
...
I'm under the impression that the rexec and Bastion modules do not
work with Python 2.3+.

Right -- alas -- they don't give 100% security, so they were disabled
(also in 2.2.3 -- they had just the same exploitable weaknesses in
earlier 2.2 releases).
I'm not familiar with RestrictedPython.

It's part of Zope. I have no experience of how well or badly it works.


Alex
 
D

Darryl

There's a lot to say on the subject of interpretation of
code supplied by users. In particular, the three Python
projects
rexec
Bastion
RestrictedPython
all address this requirement on a technical level. If
your goal is a useful working application, I strongly
urge you to read up on these. Correct construction on
your own of a "blacklist" is ... difficult.

Given my current level of Python knowledge, I don't consider myself
capable of writing such a blacklist - I might be able to pull it off
in other languages, but not Python. I've looked at rexec and Bastion,
and (as another poster mentioned) got the impression they were
defunct. I've also been pointed at Zope, but haven't had time to
investigate it.

Since this is a casual project that I'm not spending too much time on,
I'll probably ignore the issue for now. Since I often see Python
promoted as an application scripting language, I had hoped that it had
built-in restrictions of this nature, akin to a Java applet's sandbox.
I'm now getting the impression that if I embedded Python as the
scripting language in any app, it would be similar to using VBA - an
unrestricted environment that gives the script the power to do
whatever it wants.

I think I'll go ahead with the project with only a few simple
restrictions right now (i.e. no 'import') and only allow trusted users
to write scripts. I'll also run the application in a chroot jail
(OS-specific, I know) to limit the amount of damage that can be done.

Thanks for all the advice,
- Darryl
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top